Skip to content

Commit 6a53823

Browse files
KyleAMathewsKyleAMathews
committed
Add tanstack-db 0.5 blog post
1 parent e34b794 commit 6a53823

1 file changed

Lines changed: 352 additions & 0 deletions

File tree

‎src/blog/tanstack-db-0.5-query-driven-sync.md‎

Lines changed: 352 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,352 @@
1+
---
2+
title: TanStack DB 0.5 — Query-Driven Sync
3+
published: 2025-11-12
4+
authors:
5+
- Sam Willis
6+
- Kevin De Porre
7+
- Kyle Mathews
8+
---
9+
10+
You don't need a new API for every component. With 0.5, the component's query _is_ the API call.
11+
12+
```tsx
13+
// Your component's query...
14+
const { data: projectTodos } = useLiveQuery((q) =>
15+
q
16+
.from({ todos })
17+
.join({ projects }, (t, p) => eq(t.projectId, p.id))
18+
.where(({ todos }) => eq(todos.status, 'active')),
19+
)
20+
21+
// ...becomes these precise API calls automatically:
22+
// GET /api/projects/123
23+
// GET /api/todos?projectId=123&status=active
24+
```
25+
26+
No custom endpoint. No GraphQL resolver. No backend change. Just write your query and TanStack DB figures out exactly what to fetch.
27+
28+
We're releasing TanStack DB 0.5 today with Query-Driven Sync—a feature that fundamentally changes how you think about loading data.
29+
30+
## Pure queries over data
31+
32+
React's breakthrough was making components pure functions of state: `UI = f(state)`. You describe what you want to render, React handles the how.
33+
34+
TanStack DB brings the same philosophy to data: `view = query(collections)`. You describe what data you need. **DB handles the fetching, caching, and updating**—even across 100k+ row datasets.
35+
36+
```tsx
37+
// Pure view over state
38+
function TodoList({ todos, filter }) {
39+
return todos
40+
.filter((t) => t.status === filter)
41+
.map((t) => <TodoItem todo={t} />)
42+
}
43+
44+
// Pure query over data
45+
function TodoList({ filter }) {
46+
const { data: todos } = useLiveQuery(
47+
(q) =>
48+
q
49+
.from({ todos: todoCollection })
50+
.where(({ todos }) => eq(todos.status, filter)),
51+
[filter],
52+
)
53+
return todos.map((t) => <TodoItem todo={t} />)
54+
}
55+
```
56+
57+
The difference? React recomputes the view when state changes. TanStack DB recomputes the _query_ when data changes—and optimizes the network calls automatically.
58+
59+
## The reactive client-first store for your API
60+
61+
TanStack DB is a client-first store for your API powered by [differential dataflow](https://github.com/TimelyDataflow/differential-dataflow)—a technique that recomputes only what changed. When you mark a todo complete, DB updates query results in <1ms on a modern laptop, even with 100,000+ rows in memory.
62+
63+
This isn't just fast filtering. It's **live queries** that incrementally maintain themselves as data changes. **Effortless optimistic mutations** that instantly update all affected queries, then reconcile with the server. And a **normalized collection store** that eliminates duplicate data and keeps everything coherent.
64+
65+
[When we released 0.1](/blog/tanstack-db-0.1), we described three options teams face and that TanStack DB enables a new Option, C:
66+
67+
> **Option A. View-specific APIs** (fast render, slow network, endless endpoint sprawl)
68+
>
69+
> **Option B. Load-everything-and-filter** (simple backend, sluggish client)
70+
>
71+
> **Option C. Normalized collections + differential dataflow** (load once, query instantly, no jitter)
72+
73+
## The problem we kept hearing about
74+
75+
Since [we released TanStack DB 0.1](/blog/tanstack-db-0.1) in July, we've gotten the same question over and over:
76+
77+
> "This looks great for loading normalized data once, but what if my `users` table has 100,000 rows? I can't load everything."
78+
79+
They're right. Before 0.5, collections loaded their entire dataset upfront. That works beautifully for many apps with datasets in the thousands of rows, but it's not a one-size-fits-all solution.
80+
81+
Here's what we realized: **a collection shouldn't dictate what data loads. Your queries should.**
82+
83+
A collection defines the _schema_ and _security boundaries_ for a data domain. Your live queries define _which subset_ of that domain to load right now.
84+
85+
## Three sync modes: Pick the right loading strategy
86+
87+
This led to three sync modes, each optimized for different use cases:
88+
89+
**Eager mode (default & only mode before v0.5):** Load entire collection upfront. Best for <10k rows of mostly static data—user preferences, small reference tables.
90+
91+
**On-demand mode:** Load only what queries request. Best for large datasets (>50k rows), search interfaces, catalogs where most data won't be accessed.
92+
93+
**Progressive mode:** Load query subset immediately, sync full dataset in background. Best for collaborative apps where you want instant first paint AND sub-millisecond queries for everything else.
94+
95+
Most apps use a mix. Your user profile? Eager. Your products catalog? On-demand. Your shared project workspace? Progressive.
96+
97+
Let's see how each works.
98+
99+
## On-demand sync: Your query becomes the API call
100+
101+
With 0.5, you add one line to your collection:
102+
103+
```tsx
104+
const productsCollection = createCollection(
105+
queryCollection({
106+
queryKey: ['products'],
107+
queryFn: async (ctx) => {
108+
// Parse your query predicates into API parameters
109+
const params = parseLoadSubsetOptions(ctx.meta?.loadSubsetOptions)
110+
111+
// GET /api/products with query-specific filters
112+
return api.getProducts(params)
113+
},
114+
syncMode: 'on-demand', // ← New!
115+
}),
116+
)
117+
```
118+
119+
Now when you write this query:
120+
121+
```tsx
122+
const { data: electronics } = useLiveQuery((q) =>
123+
q
124+
.from({ product: productsCollection })
125+
.where(({ product }) =>
126+
and(eq(product.category, 'electronics'), lt(product.price, 100)),
127+
)
128+
.orderBy(({ product }) => product.price, 'asc')
129+
.limit(10),
130+
)
131+
```
132+
133+
TanStack DB automatically calls your `queryFn` with:
134+
135+
```
136+
GET /api/products?category=electronics&price_lt=100&sort=price:asc&limit=10
137+
```
138+
139+
**No custom API endpoint.** **No GraphQL schema changes.** Just a general-purpose products API that accepts filter parameters.
140+
141+
Your component's query becomes the API call.
142+
143+
If you're familiar with Relay or Apollo, this should feel familiar: components declare their data needs, and the framework optimizes fetching and updates. The difference? You get Relay-style normalized caching and automatic updates without GraphQL. Your REST, GraphQL, or tRPC API stays simple, your queries stay powerful, and differential dataflow keeps everything fast client-side.
144+
145+
## Request Economics: Smarter than it looks
146+
147+
"Wait," you're thinking, "doesn't this create N+1 query problems?"
148+
149+
No—and here's why the performance story is actually _better_ than custom APIs.
150+
151+
### **Automatic request collapsing**
152+
153+
Multiple components requesting the same data trigger exactly one network call:
154+
155+
```tsx
156+
// Component A
157+
const { data: active } = useLiveQuery((q) =>
158+
q.from({ todos }).where(({ todos }) => eq(todos.status, 'active')),
159+
)
160+
161+
// Component B (same query, different component)
162+
const { data: active } = useLiveQuery((q) =>
163+
q.from({ todos }).where(({ todos }) => eq(todos.status, 'active')),
164+
)
165+
166+
// Result: ONE network request
167+
// GET /api/todos?status=active
168+
```
169+
170+
TanStack DB compares predicates across all live queries and deduplicates requests automatically.
171+
172+
### **Subset matching and delta loading**
173+
174+
When you navigate from viewing 10 products to viewing 20, DB doesn't reload everything:
175+
176+
```tsx
177+
// Initial query: loads 10 products
178+
const { data } = useLiveQuery((q) => q.from({ products }).limit(10))
179+
180+
// User clicks "load more": loads ONLY the next 10
181+
fetchNextPage()
182+
```
183+
184+
```
185+
# Page 1
186+
GET /api/products?limit=10&offset=0
187+
188+
# Page 2
189+
GET /api/products?limit=10&offset=10
190+
# NOT: GET /api/products?limit=20
191+
```
192+
193+
Already-loaded rows are reused; only the new window crosses the wire. The collection tracks which predicates it has already satisfied and only fetches the delta.
194+
195+
### **Join optimization**
196+
197+
Complex joins don't cause request explosions. They trigger a minimal set of filtered requests:
198+
199+
```tsx
200+
// Join todos with their projects
201+
const { data } = useLiveQuery((q) =>
202+
q
203+
.from({ todos })
204+
.join({ projects }, (t, p) => eq(t.projectId, p.id))
205+
.where(({ todos }) => eq(todos.status, 'active')),
206+
)
207+
208+
// Network calls:
209+
// GET /api/todos?status=active (returns 10 todos)
210+
// GET /api/projects?id_in=123,124,125 (only the 3 unique project IDs)
211+
//
212+
// NOT 10 separate project requests!
213+
```
214+
215+
DB analyzes the join to determine exactly which related records are needed, then fetches them in a single batched request.
216+
217+
### **Respects your cache policies**
218+
219+
Query Collection integrates with TanStack Query's `staleTime` and `gcTime`:
220+
221+
```tsx
222+
const productsCollection = createCollection(queryCollection({
223+
queryKey: ['products'],
224+
queryFn: fetchProducts,
225+
staleTime: 5 * 60 * 1000, // 5 minutes
226+
syncMode: 'on-demand'
227+
}))
228+
229+
// First query: network request
230+
useLiveQuery(q => q.from({ products }).where(...))
231+
232+
// Same query within 5 minutes: instant, no network
233+
useLiveQuery(q => q.from({ products }).where(...))
234+
235+
// Different query within 5 minutes: only fetches the diff
236+
useLiveQuery(q => q.from({ products }).where(...).limit(20))
237+
```
238+
239+
You get TanStack Query's sophisticated caching plus DB's intelligent subset tracking.
240+
241+
**The result:** Fewer total network requests than custom view-specific APIs, with better cache utilization and zero endpoint sprawl.
242+
243+
## Progressive sync: Fast initial paint + instant client queries
244+
245+
On-demand mode is great for search interfaces and catalogs where you'll never touch most of the data. But what about collaborative apps where you *want* the full dataset client-side for instant queries and offline access, but also want fast first paint?
246+
247+
That's progressive mode: load what you need immediately, sync everything in the background.
248+
249+
```tsx
250+
const todoCollection = createCollection(
251+
electricCollection({
252+
table: 'todos',
253+
syncMode: 'progressive',
254+
}),
255+
)
256+
257+
// First query loads immediately (on-demand)
258+
const { data: urgentTodos } = useLiveQuery((q) =>
259+
q
260+
.from({ todos: todoCollection })
261+
.where(({ todos }) => eq(todos.priority, 'urgent')),
262+
)
263+
// ~100ms: Network request for urgent todos only
264+
265+
// Meanwhile, collection syncs full dataset in background
266+
// After sync completes: all queries run in <1ms client-side
267+
```
268+
269+
Now your first query loads in ~100ms with a targeted network request. While the user interacts with that data, the full dataset syncs in the background. Once complete, all subsequent queries—even complex joins and filters—run in sub-millisecond time purely client-side.
270+
271+
**Progressive mode shines with sync engines** like Electric, Trailbase, and PowerSync. With traditional fetch approaches, loading more data means re-fetching everything, which gets expensive fast. But sync engines only send deltas—the actual changed rows—making it cheap to maintain large client-side datasets. You get instant queries over 10,000s of rows without the network cost of repeatedly fetching all that data.
272+
273+
With REST APIs, progressive mode is less common since updates generally require full re-fetches. But for sync engines, it's often the sweet spot: fast first paint + instant everything else.
274+
275+
## Works today with REST. Gets better with sync engines.
276+
277+
Query-Driven Sync is designed to work with your existing REST, GraphQL, or tRPC APIs. No backend migration required—just map your predicates to your API's parameters (as shown below) and you're done.
278+
279+
For teams using sync engines like [Electric](https://electric-sql.com), [Trailbase](https://trailbase.io/), or [PowerSync](https://www.powersync.com/), you get additional benefits:
280+
281+
- **Real-time updates** via streaming (no polling required)
282+
- **Automatic predicate translation** (no manual mapping needed)
283+
- **Delta-only syncing** (only changed rows cross the wire)
284+
285+
For example, Electric translates your client query directly into Postgres queries, applies authorization rules, and streams updates. Your component's query becomes a secure, real-time, authorized Postgres query—no API endpoint needed.
286+
287+
Collections abstract the data source. Start with REST. Upgrade to sync when you need real-time.
288+
289+
## How Query Collection predicate parsing works
290+
291+
Query Collection is designed for REST, GraphQL, tRPC, and any other API-based backend. When you enable `syncMode: 'on-demand'`, TanStack DB automatically passes your query predicates (where clauses, orderBy, limit) to your `queryFn` as expression trees in `ctx.meta.loadSubsetOptions`. You write the mapping logic once to translate these into your API's format.
292+
293+
We provide helper functions to make this straightforward:
294+
295+
```tsx
296+
queryFn: async (ctx) => {
297+
// Parse expression trees into a simple format
298+
const { filters, sorts, limit } = parseLoadSubsetOptions(ctx.meta?.loadSubsetOptions)
299+
300+
// Map to your REST API's query parameters
301+
const params = new URLSearchParams()
302+
filters.forEach(({ field, operator, value }) => {
303+
if (operator === 'eq') params.set(field.join('.'), String(value))
304+
else if (operator === 'lt') params.set(`${field.join('.')}_lt`, String(value))
305+
// Map other operators as needed
306+
})
307+
if (limit) params.set('limit', String(limit))
308+
309+
return fetch(`/api/products?${params}`).then(r => r.json())
310+
}
311+
```
312+
313+
For APIs with custom formats (like GraphQL), use `parseWhereExpression` with custom handlers:
314+
315+
```tsx
316+
queryFn: async (ctx) => {
317+
const { where, orderBy, limit } = ctx.meta?.loadSubsetOptions
318+
319+
// Map to GraphQL's where clause format
320+
const whereClause = parseWhereExpression(where, {
321+
handlers: {
322+
eq: (field, value) => ({ [field.join('_')]: { _eq: value } }),
323+
lt: (field, value) => ({ [field.join('_')]: { _lt: value } }),
324+
and: (...conditions) => ({ _and: conditions }),
325+
}
326+
})
327+
328+
// Use whereClause in your GraphQL query...
329+
}
330+
```
331+
332+
You write this mapping once per collection. After that, every query automatically generates the right API calls.
333+
334+
**Can't modify your API?** Your mapping doesn't need to be precise. Many queries can map to a single broad API call—for example, any product search query could map to `GET /api/products` which returns all products. TanStack DB filters client-side. As your API evolves to support more predicates, your client code doesn't change—just update the mapping to push down more filters. Start broad, optimize incrementally.
335+
336+
[Full Query Collection predicate parsing documentation →](https://tanstack.com/db/latest/docs/collections/query-collection#queryFn-and-predicate-push-down)
337+
338+
## Shipping toward 1.0
339+
340+
Query-Driven Sync (0.5) completes the core vision: intelligent loading that adapts to your queries, instant client-side updates via differential dataflow, and seamless persistence back to your backend. We're targeting 1.0 for December 2025, focusing on API stability and comprehensive docs.
341+
342+
**This is new—we need early adopters.** Query-Driven Sync works and ships today, but it's fresh. If you try it, we'd love your feedback on rough edges or API improvements. Join us in [Discord](https://discord.gg/tanstack) or open [GitHub issues](https://github.com/TanStack/db/issues).
343+
344+
### Try it today
345+
346+
```bash
347+
npm install @tanstack/react-db@0.5.0
348+
```
349+
350+
---
351+
352+
Collections define schemas and security boundaries. Queries define what loads and when. Your components define UIs. Finally, each concern is separate—and your data layer adapts to how you actually use it.

0 commit comments

Comments
 (0)