Back in 2013 Mike Olson made a bold claim: “No dominant platform-level software infrastructure has emerged in the last ten years in closed-source, proprietary form.” Olson is a smart guy, and he was nearly correct except for one small exception to his rule: Splunk. Splunk thrived in spite of its proprietary nature, and leading that success was Gaurav Gupta, then vice president of product at Splunk, and now a partner with Lightspeed Venture Partners. It was a “different time,” he said in an interview, both for the industry and for him.

Ever since then he’s been building infrastructure the open source way, whether running product at Elastic or later investing in companies like Grafana as a VC. As successful as Splunk was, however, Gupta believes that the “incredible amounts of trust” that open source fosters, coupled with low friction to experimentation, make it the smart investment for today, whether you’re a VC or an enterprise trying to innovate your way through a pandemic.

Splunk was different

It’s worth dwelling for a moment on Gupta’s Splunk experience. Splunk, after all, exploded in adoption at a time when much of the infrastructure world went open source. According to Gupta, Splunk may have slipped into the market just in time. After all, he noted, “Open source didn’t exist back then [2004] for the most part.” Yes, Linux was around and, yes, things like MySQL and Drupal were taking root, but open source had yet to command the market like it does today.

Splunk was also helped by the fact it catered to a customer (system administrators and similar roles analyzing log data) that was perhaps neither capable nor interested in digging into source code. What this audience did appreciate, by contrast, was an “incredible end-to-end [product] that really focused on great user experience, and traditionally open source hasn’t done a great job on user experience [for] less technical audiences.” It didn’t hurt that “We were the only one in the market for years,” Gupta continued.

SEE: How to build a successful developer career (free PDF) (TechRepublic)

By Gupta’s reckoning, despite years of VCs trying to fund “copycat” competitors to Splunk, no one successfully did so…until Elastic managed the feat by accident. “Elastic wasn’t designed to be a logging company at all, it was a search company.” Having left Splunk for Elastic, Gupta and team saw that users were starting to use the search tool for logging use cases, and hired the developers behind Logstash and Kibana to help build out Elastic’s log management capabilities. Unlike open source companies before it, Elastic determined to “not be super generic” and instead “create an integrated stack” to target specific use cases like search and logging. 

All of which helps to explain how Splunk emerged as a hugely successful proprietary software company in an area of software (infrastructure) that increasingly skewed open source. It also explains how Gupta jumped from proprietary software to open source. But in a world where cloud delivers and, perhaps, perfects many of the benefits of open source (“ultimately people want to consume open source as a service,” he said), what is it about open source that makes it fertile ground for investments, decades after open source stopped being novel?

Incredible amounts of trust

Cloud gives enterprises a “get-out-of-the-burden-of-maintaining-open-source free” card, but savvy engineering teams still want open source so as to “not lock themselves in and to not create a bunch of technical debt.” How does open source help to alleviate lock-in? Engineering teams can build “a very modular system so that they can swap in and out components as technology improves,” something that is “very hard to do with the turnkey cloud service.” 

SEE: Linux commands for user management (TechRepublic Premium)

That’s the technical side of open source, but there’s more to it than that, Gupta noted. Referring to how Elastic ate away at Splunk’s installed base, Gupta said, “The biggest reason…is there is a deep amount of developer love and appreciation and almost like an addiction to the [open source] product.” This developer love is deeper than just liking to use a given technology: “You develop [it] by being able to feel it and understand the open source technology and be part of a community.” 

Is it impossible to achieve this community love with a proprietary product? No, but “It’s a lot easier to build if you’re open source.” He went on, “When you’re a black box cloud service and you have an API, that’s great. People like Twilio, but do they love it?” With open source projects like Grafana and Elasticsearch, by contrast, developers really love the project, he said, because it’s more than a project, more than a technology: “As a developer, you want to be part of that movement.”

One key aspect of such developer movements isn’t a matter of open source code, though that helps. No, it’s really about trust. 

A lot of it comes from the fact that things are very transparent in these open source companies, their Github repositories, their issues, their roadmaps. [The] majority of the code may be written by the company, but they do a pretty good job of explaining why every single decision is being made, how it’s been made, how it’s architected. 

It’s about trust. When developers have to make a big decision, they’re making a bet. Maybe they’re embedding Elasticsearch, or they’re banking their entire operations team on Grafana. They think, ‘This is something [we’re] going to be stuck with for a while. I’m actually putting my neck on the line to do this.’ And so, good open source companies build incredible amounts of trust.

Such trust is paying dividends for open source companies now, with so many companies struggling to do more with less, and so many developers who are “busy, but they also have time on their hands. They’re exploring,” suggested Gupta, and open source is the lowest-cost software with the least amount of friction to start experimenting…and falling in love with their software. 

Disclosure: I work for AWS, but the views herein are mine and don’t necessarily reflect those of my employer. 

Open Source Weekly Newsletter

You don’t want to miss our tips, tutorials, and commentary on the Linux OS and open source applications. Delivered Tuesdays

Sign up today

Also see