scaling-query-volume

Scaling for Query Volume

Problem: When a query has a large limit (e.g. 1000) and there are multiple shards (e.g. 10), naively each shard must return the full 1000 results — totaling 10,000 scored points transferred and merged. This is wasteful since data is expected to be roughly randomly distributed across auto-shards.

Core idea

Instead of asking every shard for the full limit, ask each shard for a smaller limit computed via Poisson distribution statistics, then merge. This relies on auto-sharding spreading data across shards in an approximately random, independent way, so results are probabilistic rather than guaranteed complete.

When it activates

• More than 1 shard
• Auto-sharding is in use (all queried shards share the same shard key)
• The request's limit + offset >= SHARD_QUERY_SUBSAMPLING_LIMIT (128)
• The query is not exact

These conditions and the threshold value reflect Qdrant internals at the time of writing and may change between Qdrant versions. Check current Qdrant documentation or source before giving tuning advice. Treat the threshold, the safety factor and the Poisson threshold as internal server behavior, not as user settings.

Key tradeoff

The strategy trades a small probability of slightly incomplete results for a large reduction in inter-shard data transfer, especially for high-limit queries across many shards. The 1.2x safety factor and the 99.9% Poisson threshold are intended to keep the error rate low — comparable to inaccuracies already introduced by approximate vector indices like HNSW.

Practical guidance

Results are probabilistic, not guaranteed complete. When completeness matters, use an exact query, which this optimization does not apply to, or verify the results. Keeping limit plus offset below the threshold also avoids subsampling. Scroll performance and pagination tuning are out of scope for this skill.