DigitalOcean Partners with Traceable to help Advance Fraud Detection

Life at DigitalOcean Before Traceable

Limited API Telemetry

DigitalOcean's platform is powered by thousands of external (north-south) API endpoints between their Core API and UI layer supported by thousands more internal APIs due to their microservices architecture. While the engineering team had tools in place to help monitor API telemetry from a performance perspective, DigitalOcean's security team realized that they lacked API telemetry that may be helpful from a security standpoint. With API attacks on the rise, they wanted to ensure that they had the ability to help detect, investigate, and respond to more sophisticated threats. They had a WAF in front of their APIs but it was not  enough. To understand the mechanics of an incident, they needed a solution that could provide visibility into the business logic of their application. This meant gathering telemetry from both their external-facing North-South APIs and internally connected East-West APIs.

We had all the appropriate tools available at the time– WAF, DDoS protection, traditional API telemetry that focused on API performance– but we didn’t have anything that could provide a security perspective lens: attack patterns, cohorts of abuse, etc. - Tim Lisko, Senior Director of Security Engineering

Fraudulent Account Creation and Resource Abuse

DigitalOcean offers self-service account creation and a $200 credit for many new users, making it easy for startups and small businesses to get up and running with the platform and launch their applications. The incentive of free compute resources makes DigitalOcean’s account creation process a potential target for bad actors. DigitalOcean’s security team has  observed large-scale scripted attacks attempting to create a high volume of fraudulent accounts. The aim of the attacks is ostensibly to abuse free compute resources for nefarious purposes including cryptomining, cryptojacking, and URLs used in phishing campaigns.

We offboard at least 10k accounts a month, the vast majority of which are not clickfarm generated sign ups but rather highly automated workflows where they’re effectively using code to harvest our resources to then burn down huge amounts of compute and not pay for it. - Will Lefevers, Senior Director Security Operations at DigitalOcean

DigitalOcean’s security team is continuously refining how to help identify and block fraudulent account creation and resource abuse on the platform. The major challenge is accuracy - they need to identify and block bad actors without stopping legitimate users from creating accounts. They recognized that API telemetry data could provide additional signals to help them identify fraudulent activity with higher fidelity.

“I knew the art of the possible with API security. It’s super cool for detecting typical attacks like the OWASP Top Ten, but it also has this huge potential to help us solve a problem that’s not being solved at scale in this space right now, which is fraud and abuse detection.” - Will Lefevers

Comprehensive API Security with Traceable

DigitalOcean looked for an API security solution that could provide rich API security telemetry to power product security, security operations, and fraud and abuse detection. They evaluated several tools and selected Traceable’s API security platform. Traceable was able to deploy out-of-band, automatically discover all API endpoints in DigitalOcean’s environment, and deliver rich data in an explorable API security data lake. Beyond the product capabilities, DigitalOcean also saw Traceable as a partner in the fight against fraud and abuse. DigitalOcean has collaborated closely with Traceable’s data science and product teams to explore new ML-driven techniques for fraud detection. The design partnership resulted in detection of clusters of abusive/fraudulent/inauthentic actors  who were attempting to gain free compute resources on DigitalOcean’s platform.

Comprehensive API Telemetry for Detection and Investigation

With Traceable, DigitalOcean has better visibility into transactions across all API endpoints on their platform. While other API security tools only provide visibility into external facing North-South APIs, Traceable is also able to see internal East-West APIs. This was a critical requirement for the DigitalOcean team.

A lot of products could only see North-South traffic. For us, we know the complexity of our environment. A successful attack could obviously occur on an unsafe North-South endpoint, but a complex, chained, attack might take advantage of something happening East-West. We wanted to see the services talking to services to understand what was happening and where something could potentially break down. - Tim Lisko

Traceable collects and stores every API transaction, providing functionality to filter for secrets and other customer sensitive information, in a queryable API security data lake, enabling DigitalOcean’s security teams to help detect and investigate suspicious activity on the platform. This gives the team better confidence that if a bad actor gets past their defensive tools at the edge, Traceable will proactively alert them to suspicious activity and provide the context they need to more effectively respond. This full visibility into transaction data also gives them greater confidence that blocked attacks are really blocked.

With the data in Traceable we can watch attacks on our APIs unfold in real-time, whether it’s actual nefarious activity or fuzzing from our bug bounty program. If someone is trying to do an enumeration attack or a SQL injection attack, or shell command stuffing against the API that’s easily visible. What’s cool is we can see that they’re just attempts - they’re not successful. That gives us a ton of confidence.” - Tim Lisko

Design Partnership in Advancing Fraud Detection

DigitalOcean’s security data science team works to identify indicators of fraud and abuse on the DigitalOcean platform. The research involves looking across all the security telemetry that DigitalOcean collects and understanding what signals they can use to identify cohorts of potential bad actors and patterns of malicious activity on the platform. Traceable’s API transaction data gives the data science team the ability to analyze the sequences of API calls used in an attack or abuse campaign, and better understand the playbook used by bad actors.

We are excited about the work the Traceable team is doing to sequence API activity for cohorts of bad actors. When we identify these cohorts, we’ve been working with Traceable on techniques to analyze the activity to figure out what steps are taken. Being able to reverse engineer the attacker’s playbook is massive.” - Zara Ahmad-Post

The DigitalOcean team has been working closely in a design partnership with Traceable’s data science and product teams to innovate and apply new techniques to help detect  fraud and abuse. One successful innovation was using graph ML to help detect fraud rings operating on the DigitalOcean platform. Traceable was able to detect clusters of fraudulent accounts controlled by the same fraud attacker by identifying suspicious accounts based on multiple signals. Using this analysis technique, the Traceable and DigitalOcean teams were able to detect 16 high-confidence fraud rings and 1000+ active fraudulent accounts in one week. This detection further enables DigitalOcean to take fraudulent accounts off the platform, and leverage intelligence on fraud ring IPs to shut down additional fraudulent account creation.

Future Plans

DigitalOcean’s team is continuing their design partnership with Traceable to advance detection of fraud and abuse on their platform. By continuing to improve the fidelity of detections using rich API transaction data and sophisticated ML, DigitalOcean can help to make their platform more resilient against both security attacks and potential fraud actors attempting to abuse the platform. This helps lead to a cleaner platform overall, better resiliency, and enhanced product availability for legitimate customers.