FastAPI + Qdrant: Scaling Semantic Search Beyond pgvector

pgvector is the right starting point for almost every RAG project — it's zero extra infrastructure and fully ACID-compliant. But it has real limits, and knowing when you've hit them (rather than fighting pgvector past its comfort zone) saves a lot of wasted tuning time.

Signs it's time to move to a dedicated vector store

• Your collection is approaching or past a few million vectors and HNSW index build/query times are degrading
• You need complex metadata filtering (multiple conditions, nested fields) at query time with consistent low latency
• You're running true multi-tenant RAG and want hard isolation between tenant collections, not just a WHERE tenant_id = ? clause
• Your write throughput (ingesting new documents) is high enough that it's contending with read query performance on the same Postgres instance

If none of these apply, migrating is very likely premature optimization. If two or more apply, Qdrant (or a comparable dedicated vector database) is worth the added infrastructure.

What actually changes in your code

If your RAG layer is built against a store-agnostic interface — which is how FastAPI AI Kit structures it — the application code that calls rag.ingest() and rag.query() doesn't change at all. Only the backing adapter does:

# .env — this line is the entire application-level change
VECTOR_STORE=qdrant
QDRANT_URL=http://localhost:6333

# Everything else is unchanged
await rag.ingest(source="doc.pdf", collection="my-kb")
result = await rag.query(question="...", collection="my-kb", top_k=5)

If your RAG code directly calls SQLAlchemy with raw pgvector queries scattered through your endpoints, this is the point where that design costs you — you'd need to find and rewrite every call site. Worth abstracting behind an interface even before you think you'll need Qdrant.

Mapping collections and filtering

pgvector filtering is just SQL — any WHERE clause works. Qdrant uses payload filters, which are structurally different but map cleanly onto the same logical filters:

# pgvector: filtering is plain SQL
select(Document).where(Document.tenant_id == tenant_id, Document.doc_type == "contract")

# Qdrant: payload filter on the same fields
from qdrant_client.models import Filter, FieldCondition, MatchValue

Filter(must=[
    FieldCondition(key="tenant_id", match=MatchValue(value=tenant_id)),
    FieldCondition(key="doc_type", match=MatchValue(value="contract")),
])

The migration work is mostly translating filter logic like this, plus a one-time backfill of existing vectors into Qdrant collections.

Running a dual-write migration

Don't cut over in one deploy. Dual-write to both stores for a period, read from pgvector until you've verified Qdrant results match, then flip the read path:

async def ingest_dual(source: str, collection: str):
    chunks = await process_document(source)
    await pgvector_store.upsert(collection, chunks)
    await qdrant_store.upsert(collection, chunks)  # write to both during migration

# Read path stays on pgvector until verified
result = await pgvector_store.query(collection, query_embedding, top_k=5)
# result_qdrant = await qdrant_store.query(...)  # shadow-read for comparison, log divergence

Log any divergence between the two stores' top-k results during the shadow-read period. If results consistently match, you're safe to cut the read path over and eventually stop dual-writing.

Collection design for multi-tenancy

Qdrant collections-per-tenant give you clean isolation — delete a tenant, delete a collection, no cross-tenant query risk. This is a real advantage over a single-table-with-tenant-id approach in pgvector, at the cost of more collections to manage operationally (backup, index configuration) per tenant.

async def get_or_create_tenant_collection(tenant_id: str):
    collection_name = f"tenant_{tenant_id}"
    if not await qdrant_client.collection_exists(collection_name):
        await qdrant_client.create_collection(
            collection_name,
            vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
        )
    return collection_name

What FastAPI AI Kit includes

The kit's RAG pipeline is built against a store-agnostic interface from the start — rag.ingest() and rag.query() work identically against pgvector or Qdrant. Switching is a VECTOR_STORE environment variable, with both adapters already implemented, tested, and handling the filter/collection translation described above.