The CircleCI Data Breach: The TLDR

The CircleCI Data Breach: The TLDR

CircleCI, a developer product focused on Continuous Integration (CI) and Continuous Deployment (CD), with over one million users, published an advisory this week urging its customers to immediately rotate all secrets following a breach of the company’s systems. The blog was published by their CTO and the details can be found on their website. Though the exact details of the breach are not available, the recommendations from CircleCI have been specific –

• Please rotate any and all secrets stored in CircleCI.
• Users need to review internal logs for signs of any unauthorized access starting from December 21, 2022, to January 4, 2023, or until when the secrets are rotated.
• Tokens that need to be rotated include 
  • OAuth tokens
  • Project API tokens 
  • User API tokens
  • Runner Tokens
  • Project environment and context variables

CircleCI has not disclosed any further specifics about the breach, but said it has also invalidated all Project API tokens and that they need to be replaced. For a CI/CD company whose customers rely on continuous software delivery this results in disruption to the automation, and a lot of remediation work, as tokens form the fundamental basis which enables the CI/CD pipeline to connect and automate the tools and scripts which make upthe pipeline. 

By nature, a CI/CD solution needs to connect securely to a lot of software development and deployment tools across the SDLC, such as source code management systems, package repositories, build tools, testing tools, infrastructure tools, IaaS, and PaaS.

This means each pipeline will contain or have access to a lot of credentials. In organizations using DevOps methods this also likely means the pipeline has credentials to production systems.  

Auth Tokens: Not the First Security Issue Rodeo

Unfortunately, this is not the only recent instance where Auth tokens have been a security concern as a similar security incident was published by Slack in the last week of December. In that case Slack employee tokens were stolen and misused to gain access to their GitHub repository from where the threat actor downloaded private code repositories on December 27. 

So what is the best recourse here given these incidents will continue to come up given the widespread usage of Auth Tokens? A few common techniques include – 

1. Avoid using long-lived credentials in any of the tools, this can be achieved by constant key rotation. Here are a few examples of how to do that in AWS, Auth0 and Github
2. Use OIDC to authenticate against your cloud environment or automated tools like CI/CD pipelines that support it, instead of storing static tokens.
3. Use IP ranges to limit inbound connections to your apps or APIs to known IP addresses. This is especially possible in controlled environments where the automated tools are running in AWS, Azure etc and the IPAM is allocating addresses from a predefined IP block.
4. Use a secrets manager that supports key rotation so that they can be periodically changed or in response to a potential leak or compromise. HashiCorp Vault, AWS Secrets manager etc support this functionality and also 
5. In tools which allow sharing of environment variables across projects use contexts to  rotate keys automatically via API.

At Traceable, protecting against AuthN, AuthZ vulnerabilities and attacks forms the basis of our API Threat Protection and Active API Security Testing modules.

We protect against common vulnerabilities such as Token Expiration, Algorithmic confusion, Invalid signatures all the way to Broken Object level Authorization and Broken Function level authorization.

Since we use Token attributes to identify client connections we can protect against malicious requests coming from compromised tokens at runtime in addition to other controls for Malicious sources accessing your APIs.

About Traceable

Traceable is the industry’s leading API security platform that identifies APIs, evaluates API risk posture, stops API attacks, and provides deep analytics for threat hunting and forensic research. With visual depictions of API paths at the core of its technology, its platform applies the power of distributed tracing and machine learning models for API security across the entire development lifecycle. Visual depictions provide insight into user and API behaviors to understand anomalies and block API attacks, enabling organizations to be more secure and resilient. Learn more at traceable.ai.