# Mergeable Sketches For Analytics
A mergeable sketch is a compact summary that can be combined with other summaries without going back to raw events.
That property is why sketches are so useful in distributed analytics.
## The Pattern
```mermaid
flowchart LR
A[shard A events] --> SA[sketch A]
B[shard B events] --> SB[sketch B]
C[shard C events] --> SC[sketch C]
SA --> Merge[merge]
SB --> Merge
SC --> Merge
Merge --> Query[approx answer]
```
Examples:
- HLL sketches merge for unique counts.
- Count-min sketches merge for approximate frequencies.
- Quantile sketches merge for latency distributions.
- Roaring bitmaps can compactly represent exact integer sets and support fast set operations.
The merge operation is the contract. If two summaries cannot be combined safely, they are just local caches, not distributed analytics building blocks.
## Why It Matters
Without mergeability, global analytics requires shipping raw events to one place before answering the query. With mergeability, each region, shard, or worker can keep a local summary and merge later.
This enables:
- regional aggregation,
- windowed dashboards,
- precomputed rollups,
- backfills without full raw scans,
- cheap "last N buckets" queries.
Example: each region builds `hll:unique_viewers:video_9:2026-07-18`. The global dashboard merges the regional sketches and asks for one approximate count. The raw events can stay in regional storage until audit or recomputation is needed.
## Sketches Are Contracts
Each sketch answers a narrow question.
| Question | Candidate |
|---|---|
| How many unique users? | HyperLogLog, Theta sketch |
| What are the top items? | frequent-items sketch |
| What is p95 latency? | quantile sketch |
| Which exact integer IDs are in this segment? | Roaring bitmap |
Do not use one sketch because it is fashionable. Use the one whose merge, error, and query semantics match the product.
## Design Dimensions
| Dimension | Why It Matters |
|---|---|
| Error model | Users and alerts need to know how wrong the answer can be. |
| Merge semantics | Union, frequency addition, and quantile merge are different. |
| Serialization | Sketches often cross languages, services, and storage systems. |
| Versioning | Old buckets must remain readable after library upgrades. |
| Raw fallback | Approximate projections need a repair path. |
Store metadata next to the sketch: type, library/version, parameters, time bucket, dimensions, sample count if relevant, and source offsets or watermarks.
## Operational Pattern
```text
raw events -> per-shard sketches -> durable bucket snapshots -> query-time merge
```
Hot sketches may live in memory or Redis. Durable snapshots can live in object storage, a warehouse, Cassandra, DynamoDB, or Postgres depending on query shape.
Query-time merge should be bounded. If a dashboard merges 50,000 sketches for one chart, precompute rollups such as hourly to daily or region to global.
## Failure Modes
- Merging sketches built with different parameters or hash functions.
- Treating approximate values as exact billing numbers.
- Losing sample count context for percentiles and tail metrics.
- Letting high-cardinality dimensions create millions of tiny sketches.
- Upgrading a sketch library without a compatibility plan.
## Senior-Level Checks
- Does the sketch support union only, or also intersection/difference?
- Is error bounded and visible to users?
- Can the sketch be serialized across services/languages?
- Can old sketch versions be read after library upgrades?
- What is the raw-event fallback?
## Related Pages
- [[wiki/hyperloglog-cardinality-estimation]]
- [[wiki/bucketed-time-window-aggregation]]
- [[wiki/raw-events-vs-derived-analytics]]