Perplexity Migrates Core Search Service from DynamoDB to In-House CobbleDB

Perplexity moved its core search service layer from Amazon DynamoDB to CobbleDB, an in-house distributed key-value store written in Rust. The stated motivation for the migration is very specific: under high query volume, delivering document batches of thousands of bytes to large language models created severe latency and cost bottlenecks.

The root of this bottleneck is that the read pattern differs from before. Each query sent to Perplexity generates 100 to 120 target page keys, and the retrieval service splits them into multiple batches of 10 to 20 keys processed in parallel. Traditional search engines return short metadata snippets, whereas retrieval oriented toward language models needs to extract complete chunked passages and dense vector embeddings, bringing the average record payload to about 50 KB.

Putting these two sets of numbers together shows where the pressure comes from: many keys, many batches, large individual payloads, and all these reads occur on the path where users are waiting for answers. DynamoDB's problem is not that it cannot store the data, but that at a production traffic scale exceeding 200,000 requests per second, the usage-based billing model that charges for every byte transferred is economically unsustainable.

Cost is only half the problem. The source points out that DynamoDB's internal partition placement, in-memory caching strategy, and replica routing are all invisible, so engineers cannot avoid tail latency spikes caused by uncached reads, cross-availability-zone network hops, or replica lag.

Another kind of interference comes from writes. Reprocessing jobs triggered by chunking algorithm updates or the adoption of new embedding models push large volumes of writes directly into DynamoDB, creating noisy neighbor contention with real-time user requests. In other words, a single model or algorithm upgrade can simultaneously raise the latency of online reads.

Together these two points explain why Perplexity chose to build its own rather than continue tuning: when the system's internal behavior is unobservable, the optimizations engineers can make are limited. This is an inference from the source facts, not a conclusion directly stated by the source.

Perplexity's solution was not to swap in another database, but to split the storage architecture into three specialized systems: Pillar for persistent state management, Lorry for batch aggregation, and CobbleDB for low-latency serving.

Pillar is based on YTsaurus running on high-capacity mechanical hard drives, maintaining versioned table families of web metadata, passages, and vector representations, with YTsaurus atomic transactions ensuring that crawl updates, state changes, and export queues commit together. Lorry acts as a stateless queue consumer, combining Pillar's exports into partition-aligned batch files, storing payloads in Amazon S3, and sending metadata notifications to CobbleDB. CobbleDB worker nodes independently pull and ingest these S3 batches, fully isolating the hot data serving nodes from the write-intensive crawl pipeline.

The key to this division of labor is that write-intensive reprocessing no longer directly hits the online read path; instead it flows through Pillar to Lorry to S3 batches, and only then is ingested by CobbleDB. The persistence layer trades mechanical hard drives for capacity and cost, while the serving layer trades NVMe for latency.

CobbleDB is a distributed key-value store optimized specifically for batch lookups. Each partition maintains three replicas distributed across mutually independent compute nodes; the core daemon uses RocksDB as its embedded storage engine, combined with memory-mapped caches and local NVMe solid-state drives.

On the read path, a stateless query router maps hashed page identifiers to partitions, preferentially sending requests to node replicas in the same availability zone to reduce network overhead. If the target replica's response time rises, the router performs speculative hedging, simultaneously issuing a read request to a standby replica on another node.

The most notable trade-off is that CobbleDB abandons standard distributed transaction protocols and synchronous consensus algorithms; replicas apply updates asynchronously at their own pace, because the search service can tolerate slight replication lag. This buys lower operational overhead, but it also means applications must be able to tolerate eventual consistency.

In production measurements, CobbleDB reduced batch read latency from a median of 31.4 milliseconds to 5.60 milliseconds, p90 from 56.7 milliseconds to 9.77 milliseconds, and p99 tail latency from 123 milliseconds to 24.2 milliseconds. Synthetic benchmarks handling payloads up to 100 KB show that throughput remains stable at scales of up to 500,000 requests per second.

The nature of these numbers needs to be made clear: they come from Perplexity's own production measurements and synthetic benchmarks, and no third-party independent verification appears in the source. The synthetic benchmark's 500,000 RPS is a test condition, not equivalent to actual production capacity.

Also part of the company's self-reporting is the development scale. The company's CEO, Aravind Srinivas, said CobbleDB consists of about 40,000 lines of Rust code, built in two months by two systems engineers working with a cluster of autonomous AI coding agents, which handled integration testing, build monitoring, and runbooks. The source provides no verifiable details about the actual degree and effectiveness of these agents' participation.

Replacing a fully managed cloud database with an in-house system means node lifecycle management, backup verification, and partition rebalancing fall entirely to internal site reliability engineers. This is the other side of the migration, and it is a trade-off explicitly listed by the source.

At the same time, because replicas ingest batch files at different intervals, applications must be able to tolerate eventual consistency. Search scenarios can accept this, but it is not a premise every system can copy.

The source also mentions that Perplexity plans to open-source the CobbleDB codebase in an upcoming release, but the specific version and timing are not yet determined. The source provides no data on the long-term stability of the system in real production environments after migration, or on the impact of eventual consistency on user experience.