In our previous post about the world of crypto we mentioned that, beneath the sound and fury, blockchains are simply a new kind of database. The data they contain tells the story of the collective actions of hundreds of millions of crypto accounts. 

Alas: making sense of that story isn’t as straightforward as we’d like. From a developer’s perspective, blockchains aren’t typical easy-to-query databases. Yes, they’re purpose-built to capture and retain history, but asking certain questions about that history — particularly questions that aggregate lots of data — can be hard. For instance, while it’s trivial to directly ask a blockchain for an account’s current balance, it’s much more difficult to ask for an account’s average balance over the past month. It’s easy to ask who “owns” a specific NFT but it’s much more work to get that token’s full transfer history. There’s an entire class of questions that simply can’t be answered directly by blockchains: for instance, computing last week’s transaction volumes on a popular decentralized exchange is a non-starter. And watching for new blocks as they're mined is a whole other ball of wax.

To solve this problem, developers must build pipelines that extract data from blockchains (typically by running resource-intensive archival nodes) and load that data into traditional queryable databases. Doing this well is hard. There’s a lot of historical data to extract and new data is added to blockchains at a blistering pace. Often, transaction data must be interpreted in the context of one or more smart contracts. There’s also an incredible amount of data that’s referenced by blockchains but lives elsewhere, famously including NFT images and metadata.

Here at PSL, when we see a new engineering challenge, we tend to dive right in and build prototypes to help us understand the contours. It didn’t take long for us to find a toy to build to teach ourselves about working with blockchain data at scale.

Introducing ChainDisco…

In our opinion, there’s nothing more bracingly bizarre on blockchains today than NFTs. We built ChainDisco to highlight this madness by showing a Matrix-esque waterfall of the latest NFT transactions across the entire Ethereum blockchain:

We hope that ChainDisco makes it fun to explore the weird world of NFTs. This said, ChainDisco has no direct facility for purchasing. That’s because NFTs — like most crypto assets today — are purely speculative. Yes, they’re amusing and, frankly, more than a little disturbing to look at. (One of our kids calls ChainDisco “the creepy head picture site”.) But nothing good comes of speculation at scale; we’re content to look, not buy.

…and the BeQue API

Building ChainDisco taught us that working with blockchain data at scale — identifying relevant smart contracts, teasing apart transactions, mucking with metadata, and processing new blocks rapidly enough so that even 30-second-old events appear on the ChainDisco website  — is challenging.  

In the past few months, we’ve said hello to several other developers who have been forced to build their own homegrown blockchain data pipelines. This looks to us like undifferentiated heavy lifting. If (if!) blockchains are part of the future, we’re going to need better tools.

Under the hood, ChainDisco is built on top of a new API that we’re calling BeQue. We’re excited to share a bit about it today. BeQue’s goal is to make it absurdly simple for developers to issue complex queries against data in blockchains and related systems, and to get updates when new data is available. Like this:

Want to get the top 100 NFT trades this week?

Or get notified when an NFT trades for more than a million bucks?

Get the total SOL ever sent from one account to another?

Or compute yesterday’s Uniswap volumes?

BeQue is still in its infancy. The amount and variety of data generated on-chain today is staggering; the road ahead is long. If (again, if!) blockchains are around for the long term, we might expect them to contain plenty of interesting (and hopefully non-speculative) data. As a result, we think that an API that covers a wide terrain with a unified approach has potential.

So that’s it: we hope you have disturbing fun exploring with ChainDisco and enjoy taking a look at the nascent BeQue API. If you’d like to build with BeQue, please get in touch with us; we’d love to hear more about what you’re making.