Announcing The FoundationDB Record Layer

Published January 14, 2019

Today, we are excited to announce the open source release of the FoundationDB Record Layer!

From its inception, FoundationDB was designed as a highly scalable key-value store with a simple API. Layers extend the functionality of the database by adding features and data models to allow new storage access patterns. Today we are releasing the FoundationDB Record Layer, which provides relational database semantics on top of FoundationDB. This layer features schema management, indexing facilities, and a rich set of query capabilities. The Record Layer is used in production at Apple to support applications and services for hundreds of millions of users.

A record-oriented database on top of FoundationDB

The Record Layer stores structured data, just like a relational database. Databases managed by the Record Layer support records with fields and types, an evolving schema, complex primary and secondary indexes, and declarative query execution. The Record Layer also includes features not typically found in a traditional relational database, such as support for complex nested data types, indexes on the commit-time of records, and indexes and queries that span different types of records.

Built on top of FoundationDB, the Record Layer inherits FoundationDB's strong ACID semantics, reliability, and performance in a distributed setting. The Record Layer also uses FoundationDB's transactional semantics to provide features similar to a traditional relational database, but in a distributed setting. For example, the Record Layer's secondary indexes are maintained transactionally, so they're always up-to-date with the latest changes to the data. Transactions reduce the number of bugs in application code and greatly simplify application development.

Built for Scale

The Record Layer is built for a massive scale, allowing millions of discrete database instances to be managed within a single FoundationDB cluster. Its design and core feature set were built to scale to many millions of concurrent users and a diverse ecosystem of client applications each with unique data models and query access patterns.

To make operations simple, the Record Layer is stateless, so scaling up is as simple as launching more instances. It's also built from the ground up for massive multi-tenancy, encapsulating and isolating all of a tenant's state. The Record Layer can also tightly constrain and balance resource consumption across users in a predictable fashion, even in the face of a wide variety of workloads and activity. Together, these properties enable the Record Layer to scale elastically as the demands of its clients grow.

Together, the Record Layer and FoundationDB form the backbone of Apple's CloudKit. We wrote a paper describing how we built the Record Layer to run at massive scale and how CloudKit uses it. Today, you can read the ~~preprint~~ SIGMOD'19 to learn more.


There's a lot more to say about the Record Layer. At a very high level, the Record Layer:

All of this leverages the full breadth and power of FoundationDB functionality, including strong ACID semantics and advanced features such as controllable isolation levels.

Get Started

The getting started guide will walk you through creating a simple application that uses the Record Layer. If you'd like to dig deeper, we wrote an overview that goes into greater detail.

We encourage your participation in a new forum section for the Record Layer where you can ask discuss development, ask questions, and get involved in the open source project. We're excited to see what the community will build on top of the Record Layer.

More information can be found at:

Updated 2020/10/24: Link to SIGMOD'19 paper instead or preprint.