Lakehouse overview

The gap it is meant to fill

D1 is the transactional tier, and it is the wrong shape for event data.

Gameplay events, request telemetry and ledger history are append-only, high volume, and queried by range and aggregate rather than by id. Putting them in D1 works until it does not, and the failure is gradual: a table that grows without bound, a query that gets slower every month, and eventually a decision you should have made a year earlier.

Today those have nowhere in the kit to go.

The shape it is planned to take

One Zod object is the whole table definition, exactly as it is for D1. From it, the capability would derive the ingest stream schema, the table, the write codec and the query types — the same schemas-are-the-documentation rule the rest of the kit follows.

It would sit on three Cloudflare products that already exist: R2 Data Catalog for storage, Pipelines for ingest, and R2 SQL to query across them.

The wiring is the missing part, and wiring is what Pithy is for. Two facts about those products shape the design:

There is no write binding for the catalog, so writes would reach it as stream, then pipeline, then sink — and the sink is what creates the table. So the capability has to own that whole lifecycle, because a sink cannot adopt a table something else made.

And the read path is read-only: no inserts, no updates, no DDL. A query builder for it would have no mutation half at all, by design.

Why it is not here yet

Because the honest version of a capability is a package, a manifest, namespaced migrations with tested rollbacks, routes with declared verification, a security review, and docs — and until all of that exists, shipping the name would be worse than shipping nothing.

The rest of the catalog is what that standard produces. This page is what the standard looks like when something has not met it yet.

What to do in the meantime

If your event volume is small, D1 with a retention policy and a scheduled aggregate is a real answer, and it is one you can build today. A daily rollup into a summary table keeps the query fast and the raw table bounded.

If it is not small, Cloudflare’s three products work without this capability. You would be writing the wiring this is planned to remove — the stream, the pipeline, the sink, the schema in three places — which is exactly the work that makes the case for the capability.

If you want to follow it, the issue is linked from the marketing page for this module, and it is where the design is being argued out.

What the rest of the section looks like

Every shipping capability has four pages — Overview, Add it, Use it, Reference — and deep-dives beneath them where there is model to explain.

This one has this page, and it will keep having only this page until there is a package behind it. The capability catalog lists everything that does ship.

ESC