The primary currency of the cloud is convenience, yet it’s that very convenience that can trip up developers, particularly those working for a large, regulated institution like a bank. As a company pushes into the cloud, often developers start by writing one-off scripts to tackle security and compliance but, in the process, they discover that this route is hard to audit, monitor, and operationalize. And that’s just to start. If a company really gets serious about going cloud native, those “one-offs” can soon number in the thousands. 

This was Kapil Thangavelu’s problem at Capital One, and it prompted him to create an open source project in 2016 to fix it: Cloud Custodian. Cloud Custodian makes it easy for enterprises to define rules through a YAML DSL to create well-managed, secure, and cost-optimized cloud infrastructure. More recently, Thangavelu joined with Capital One colleague Travis Stanfield to start Stacklet to help fund additional development of Cloud Custodian.

Since its launch, Cloud Custodian has attracted over 300 contributors and broad adoption within big enterprises like Ticketmaster and Verizon Media. But for me, the most impressive thing about Cloud Custodian may well be the open source converts it’s creating.

SEE: Cheat sheet: The most important cloud advances of the decade (free PDF) (TechRepublic) 

Open source starts here

“Twenty percent of our GitHub interactions are with people that created their GitHub account just to interact with Cloud Custodian,” Thangavelu noted. That’s right, of the thousands of enterprises that use Cloud Custodian, and potentially tens of thousands of people within those enterprises, roughly a fifth of them are starting their open source journey because of Cloud Custodian. That’s amazing.

It’s also not surprising, in a way. 

After all, think of the Cloud Custodian user. While they might be a developer, they’re perhaps just as likely to be an operations professional–and, in particular, someone focused on security compliance. Tooling for this space has tended to be vendor-driven, without much of an open source footprint. Suddenly, though, “They see open source [Cloud Custodian] as something that’s both viable and critical to their business,” said Thangavelu. It’s also the case, added Stanfield, that developers are taking on more of the security/governance functions, while those functions are becoming more developer-friendly. At the confluence of the two sits Cloud Custodian.

As these individuals benefit from Cloud Custodian, they’re increasingly contributing. Cloud Custodian gets contributions from different kinds of organizations (large cloud companies, consulting companies, and end-user enterprises like Cox Automotive and Capital One). Contributors might need support for a particular enterprise service, and contribute that addition, or they are more regular contributors (among the 1,300 users Cloud Custodian has on a Slack channel). However it happens, the community for Cloud Custodian continues to swell.

To the cloud!

It’s likely that Cloud Custodian’s popularity will continue to grow. When Thangavelu started Cloud Custodian, he was motivated by a need that pretty much every organization has, or soon will: 

To really unblock develop productivity in the cloud, we needed to have a better way of [managing compliance at scale]. Cloud Custodian emerged as a side project that recognized that all these scripts were effectively doing the same thing. They were querying the cloud control plane….By making filters and actions really fine-grained, marrying it up to a YAML DSL, and then embracing some of the serverless capabilities in the clouds, we were able to effectively do a policy as code tooling to enforce policies in real time across the infrastructure. 

What does this mean in practice? It means that if a developer does something wrong, they get real-time feedback (email, Slack, etc.) saying, “Hey, you just launched an instance unencrypted on the internet, but that’s okay. We shut it down for you. Here’s the corporate policy to do it right the next time in the future.” Not only does this protect the company now, but it fosters behavioral change within the organization so fewer such problems are created in the future, no matter their preferred technology stack (Ansible, Kubernetes, Terraform, whatever). 

Cloud Custodian, in short, is a huge productivity boost for developers and others tasked with cloud transformation, in a safe, predictable way. It’s also yielding new open source converts as security and compliance professionals interact with open source for the first time. That’s two big wins for one relatively young open source project.

Disclosure: I work for AWS, but the views expressed herein are mine.

Open Source Weekly Newsletter

You don’t want to miss our tips, tutorials, and commentary on the Linux OS and open source applications. Delivered Tuesdays

Sign up today

Also see