pgvector Filtered Search Benchmark: p99 Latency & Cost 2026

A vector search benchmark can look excellent until a tenant filter or ACL predicate enters the query.

A system that returns the nearest neighbors in 12 milliseconds on a warm, unfiltered corpus may struggle when it must search only one customer’s documents, exclude revoked content, enforce permissions, and compete with concurrent inserts. In production RAG, those conditions are normal—not edge cases.

The right comparison is therefore not “Which database has the fastest ANN query?” It’s whether Aurora PostgreSQL with pgvector can deliver the required recall and p99 latency while keeping relational metadata, authorization, and ingestion in one consistent system.

Key takeaways

• Raw vector size is only the starting point. One hundred million 1,536-dimensional float32 embeddings require 614.4 GB of raw decimal storage.
• Filters can change both recall and tail latency, particularly when the index and candidate strategy don’t match the data distribution.
• A benchmark should report recall, p50/p95/p99 latency, throughput, write interference, hardware, and cost assumptions—not just a vendor’s headline query time.
• pgvector is especially attractive when authorization and metadata joins matter. A dedicated vector database becomes more compelling when retrieval traffic and corpus size need to scale independently.

Why filtered search changes the benchmark

A typical RAG query looks more like this than a bare nearest-neighbor lookup:

SELECT id, content, embedding <=> $1 AS distance
FROM document_chunks
WHERE tenant_id = $2
  AND document_status = 'published'
  AND access_level <= $3
  AND updated_at > $4
ORDER BY embedding <=> $1
LIMIT 20;

The database must find semantically similar chunks, apply tenant isolation, enforce access rules, honor freshness requirements, and return results while another process may be inserting, updating, or deleting documents.

That creates several failure modes:

• An ANN index may find good global neighbors that are mostly outside the permitted tenant or ACL scope.
• A highly selective filter may leave too few valid candidates after approximate search.
• A hot tenant can consume disproportionate CPU, cache, and connection capacity.
• Index maintenance and WAL traffic can push otherwise acceptable p99 latency beyond the application budget.
• A separate vector service may return results quickly but require an additional authorization query—or worse, rely on a stale copy of permissions.

A fair test must measure the complete authorized retrieval path, not just vector distance calculation.

A benchmark you can reproduce

Use the production embedding dimension and a representative corpus. If the application uses 1,536-dimensional embeddings, don’t benchmark only 768 dimensions because they are easier to fit in memory.

A useful test matrix includes these corpus sizes:

The dataset should contain more than vectors. Include document IDs, tenant IDs, publication status, timestamps, ACL attributes, source metadata, and realistic text lengths. Generate tenant sizes from a skewed distribution rather than assigning every tenant the same number of rows. Production systems often have a small number of very large tenants and many small ones.

Generate queries from held-out documents or real application traffic with sensitive content removed. For each query, record:

• Tenant and ACL scope
• Requested top_k
• Freshness cutoff
• Expected filter selectivity
• Whether the query targets a hot or cold tenant
• The exact nearest-neighbor result set used for evaluation

Exact search provides the ground truth:

SELECT id
FROM document_chunks
WHERE tenant_id = $1
  AND document_status = 'published'
  AND access_level <= $2
  AND updated_at > $3
ORDER BY embedding <=> $4
LIMIT 100;

Run that query on a sampled workload, or on the full corpus if the dataset is small enough. Compare ANN results with the exact authorized neighbors and report recall@10 and recall@100. Measure quality at matched operating points—for example, 90%, 95%, and 99% recall—rather than comparing arbitrary index settings.

Required test conditions

Document the conditions alongside every result:

Run each scenario long enough to capture steady state. A five-minute warm-cache test is not enough if index builds, autovacuum, checkpointing, or ingestion batches create periodic latency spikes.

At minimum, test:

1. Global search with no metadata filter.
2. Tenant search matching 1–5% of the corpus.
3. ACL-heavy search matching 0.01–1%.
4. Freshness-constrained search during continuous ingestion.
5. A hot tenant generating a disproportionate share of traffic.
6. Recovery or index maintenance while reads continue.

Report throughput in queries per second, but keep it beside p50, p95, and p99 latency. Throughput without tail latency can hide a system that serves most requests quickly and stalls the rest.

Memory: 614.4 GB is not a RAM recommendation

For 100 million 1,536-dimensional float32 embeddings:

100,000,000 × 1,536 × 4 bytes = 614.4 GB of raw decimal vector storage

That is the payload only. It is not the amount of RAM required, and it is not the expected size of a production database.

Actual sizing must include table and tuple overhead, metadata, ANN graph or posting structures, indexes on filter columns, WAL, replicas, backups, temporary build space, PostgreSQL shared buffers, connection memory, and operating-system cache.

Index residency matters more than the raw vector arithmetic. If the working set cannot stay reasonably warm, storage latency and cache churn may dominate query performance. Conversely, keeping the entire corpus in RAM may be unnecessary if traffic is concentrated in a smaller hot set.

Plan capacity for maintenance as well. HNSW builds can require substantial CPU, memory, and temporary storage. Updates and deletes create dead tuples that vacuum must reclaim. A replica may double or multiply storage and write costs even when read traffic is unchanged.

pgvector supports exact search and approximate indexes such as HNSW and IVFFlat. It also supports reduced-precision representations, including half-precision and binary vectors, in versions that provide those types. Don’t describe “IVFFlat with flat compression” as a single feature: IVFFlat is an ANN index type, while the vector’s representation—full precision, half precision, binary, or another format—is a separate choice. Confirm the exact combination supported by the Aurora PostgreSQL and pgvector versions in your deployment using the AWS Aurora PostgreSQL extension matrix and the pgvector documentation.

Compression can improve storage and cache behavior, but it may reduce recall. Tune it against the application’s quality target, not against storage cost alone.

pgvector versus a dedicated vector service

Aurora PostgreSQL’s advantage is data locality. Vectors, document state, tenant metadata, and permissions can remain under one transaction boundary. A filtered query doesn’t need a second system or a synchronization pipeline to determine whether a result is still authorized.

A dedicated vector database typically offers more specialized ANN features and makes it easier to scale retrieval independently from OLTP. The trade-off is the surrounding data path: dual writes, delete propagation, re-embedding jobs, lag monitoring, failure handling, and possibly a second authorization lookup.

Vendor benchmarks can provide useful leads, but they are not interchangeable evidence. Results from Azure Cosmos DB’s vector search documentation or [Google AlloyDB’s vector search guidance](https://cloud.google.com/alloydb/docs/ai/using ScaNN) may use different hardware, datasets, filters, recall definitions, and concurrency. Treat published “up to” improvements as configuration-specific claims until the test can be reproduced.

The same caution applies to pricing. Compare complete deployments, including replicas, storage, backups, network transfer, synchronization, and engineering time. The useful unit is not cost per ANN query:

Cost per successful authorized answer = total infrastructure and operating cost ÷ completed answers

Choosing at 10 million, 100 million, and beyond

At 10 million vectors, pgvector is often the simplest choice when documents and permissions already live in PostgreSQL. Eliminating synchronization can be more valuable than shaving a few milliseconds from an unfiltered ANN query.

At 100 million, measure whether the index working set fits the available memory and whether ingestion pushes database p99 beyond its budget. A separate retrieval tier becomes easier to justify when search traffic has a different scaling pattern from transactions.

At larger scales, a tiered design may keep authoritative metadata and hot vectors in Aurora while placing colder embeddings in a specialized or object-storage-backed system. That only works if the team can define routing, freshness guarantees, deletion behavior, and what happens when the secondary index falls behind.

Frequently asked questions

Is pgvector fast enough for production RAG?

Often, yes—especially when queries require tenant, ACL, status, or freshness predicates. Suitability depends on dimension, index configuration, filter selectivity, concurrency, cache state, and the p99 target. A benchmark using your workload is more useful than a universal vector-count limit.

How much RAM does 100 million 1,536-dimensional embeddings require?

The raw float32 payload is 614.4 GB of decimal storage. Actual capacity planning must add ANN structures, table and metadata overhead, filter indexes, buffers, replicas, WAL, temporary build space, and maintenance headroom. Raw vector size alone is not a RAM recommendation.

When should a team move to a dedicated vector database?

Consider it when retrieval must scale independently, the corpus no longer fits comfortably within the database’s storage and memory model, or ANN indexing and query load are degrading transactional p99. Stay with pgvector when transactional authorization, relational joins, and a single source of truth are more important than maximum specialized ANN throughput.