Of the different kinds of databases, nothing has been hotter of late than time series databases. One of those time series databases, QuestDB, is a relative newcomer, but it comes ready with a bold claim: To be the fastest open source time series database. Faster than InfluxDB, TimescaleDB, and the other 30 or so alternatives on the DB-Engines database popularity index?

Yes, said QuestDB cofounders Nicolas Hourcard and Vlad Ilyushchenko in an interview. Fast enough to query billions of rows in milliseconds. 

The key to this performance, the cofounders noted, is “developing the entire database from scratch” to be lock-free, thereby ensuring minimal impact on concurrent reads. In practice, this means that QuestDB can enable multi-tenant use such that multiple sources can populate the database or multiple sources can query it…lock-free. As an added bonus, Ilyushchenko said, QuestDB can manage this with less hardware. 

But this isn’t the only thing QuestDB is trying to do differently to set itself apart. The company also depends upon permissive open-source licensing (Apache 2.0) to give developers unfettered access to the database. Will this combination of speed and developer freedom work? 

SEE: How to launch a successful developer career (TechRepublic Premium)

Years of effort to get millisecond response times

As with much of the best open source software, Ilyushchenko started writing the QuestDB database simply to get work done. He was working at an energy trading company and couldn’t afford a week’s worth of production downtime to use the XML database the company had licensed. He needed to rebuild their real-time vessel tracking system, and needed something that could ingest the data in milliseconds to minutes, not days. 

Stymied by the tools available to him, Ilyushchenko started to write his own storage engine. He left his job, lived off savings and contract jobs, and spent seven years building what would become QuestDB. Later Hourcard joined him, and together they set out to build a different sort of time series database, as Hourcard wrote in a blog:

We have heard a large number of companies complaining about the performance limitations of open source time-series databases. Most of those re-use existing libraries or are an extension of a well-known database which was not designed to process time-series data efficiently in the first place. Instead, we chose an alternative route which took more than 7 years of R&D. Our vision from day 1 was to challenge the norm and build software that uses new approaches and leverages the techniques learned in low-latency trading floors. An important aspect was to study and understand the evolution of hardware to build database software that could extract more performance from CPUs, memory, and modern hard disks.

QuestDB arguably does many things well, but it’s this respect for hardware that I find particularly interesting. “Some databases take hardware for granted,” Ilyushchenko stressed. “They’re saying, ‘We’re going to solve the problem by throwing more hardware at it. We’ll just wait for CPUs to get faster.'” By contrast, he continued, QuestDB is “trying to understand how hardware works, and we’re trying to build software that is friendly to the hardware.” This approach means that QuestDB is generally able to accomplish more with less hardware. The company has published test results of this approach that seem promising. 

Delivering such performance is impressive, but it becomes doubly so if the company can figure out how to sustainably develop the database. 

Open source is a feature, not a bug

After 20 years in open source, much of that time working for startups like MongoDB, I understand why so many open source companies turn to proprietary licensing models to nudge customers to pay. It turns out it’s hard to charge for something people can get for free. So companies have gone for so-called open core models whereby they make much of their code available under an open source license and then close off some features with a proprietary license. Or they might use a license that isn’t open source but also isn’t proprietary in the traditional sense–licenses like the SSPL. 

For QuestDB, the company is leaning toward an open core model, but it has eschewed copyleft licenses like the AGPL, as well as “shared source” licenses like the BSL, for roughly the same reason: Customers don’t like them. “If we were to use AGPL, it would really cause a problem for companies to start using QuestDB and deploying it,” acknowledged Hourcard. Why? Because such restrictive licenses are a no go for many enterprises. The BSL and SSPL, similarly, “comes with baggage,” he went on. 

What’s best for customers, by contrast, and best for QuestDB is to remove these barriers to adoption: “We want to lower the friction as much as possible. All of our product is based on getting started really, really quickly….Our goal right now is to fuel adoption as much as possible. We want to put in as much as we can into the open source version, not to block people from using it.” 

That’s smart business, it turns out, because a license that guarantees people pay but inhibits them from adopting your product is a really bad license. For startups and, really, any company, the first order problem is adoption. Monetization only becomes an issue if enterprises actually want to use your product. 

The one seemingly surefire way to route around this open/closed conundrum is simply to operate the open source software as a service. This is the approach that Yugabyte has taken, and it’s the direction that QuestDB seems to be moving. In a cloud service, the proprietary value is the service, not necessarily the code; in this way, vendor and customer interests are near perfectly aligned. A QuestDB managed cloud service allows the company to invest deeply in making it an ever more useful, popular database while also monetizing that popularity in a highly efficient manner. 

This is the bet the QuestDB cofounders seem to be making. Could they change their minds? Of course. But for now, they appear to be matching their lock-free approach to database architecture with a lock-free approach to database monetization. 

Disclosure: I work for AWS, but the views expressed herein are mine.

Open Source Weekly Newsletter

You don’t want to miss our tips, tutorials, and commentary on the Linux OS and open source applications. Delivered Tuesdays

Sign up today

Also see