Vector Databases
When to Use This Skill
Use this skill when:
- Choosing between vector database options
- Designing semantic/similarity search systems
- Optimizing vector search performance
- Understanding ANN algorithm trade-offs
- Scaling vector search infrastructure
- Implementing hybrid search (vectors + filters)
Keywords: vector database, embeddings, vector search, similarity search, ANN, approximate nearest neighbor, HNSW, IVF, FAISS, Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, cosine similarity, semantic search
Vector Database Comparison
Managed Services
| Database | Strengths | Limitations | Best For |
|---|
| Pinecone | Fully managed, easy scaling, enterprise | Vendor lock-in, cost at scale | Enterprise production |
| Weaviate Cloud | GraphQL, hybrid search, modules | Complexity | Knowledge graphs |
| Zilliz Cloud | Milvus-based, high performance | Learning curve | High-scale production |
| MongoDB Atlas Vector | Existing MongoDB users | Newer feature | MongoDB shops |
| Elastic Vector | Existing Elastic stack | Resource heavy | Search platforms |
Self-Hosted Options
| Database | Strengths | Limitations | Best For |
|---|
| Milvus | Feature-rich, scalable, GPU support | Operational complexity | Large-scale production |
| Qdrant | Rust performance, filtering, easy | Smaller ecosystem | Performance-focused |
| Weaviate | Modules, semantic, hybrid | Memory usage | Knowledge applications |
| Chroma | Simple, Python-native | Limited scale | Development, prototyping |
| pgvector | PostgreSQL extension | Performance limits | Postgres shops |
| FAISS | Library, not DB, fastest | No persistence, no filtering | Research, embedded |
Selection Decision Tree
Need managed, don't want operations?
├── Yes → Pinecone (simplest) or Weaviate Cloud
└── No (self-hosted)
└── Already using PostgreSQL?
├── Yes, <1M vectors → pgvector
└── No
└── Need maximum performance at scale?
├── Yes → Milvus or Qdrant
└── No
└── Prototyping/development?
├── Yes → Chroma
└── No → Qdrant (balanced choice)
ANN Algorithms
Algorithm Overview
Exact KNN:
• Search ALL vectors
• O(n) time complexity
• Perfect accuracy
• Impractical at scale
Approximate NN (ANN):
• Search SUBSET of vectors
• O(log n) to O(1) complexity
• Near-perfect accuracy
• Practical at any scale
HNSW (Hierarchical Navigable Small World)
Layer 3: ○───────────────────────○ (sparse, long connections)
│ │
Layer 2: ○───○───────○───────○───○ (medium density)
│ │ │ │ │
Layer 1: ○─○─○─○─○─○─○─○─○─○─○─○─○ (denser)
│││││││││││││││││││││││
Layer 0: ○○○○○○○○○○○○○○○○○○○○○○○○○ (all vectors)
Search: Start at top layer, greedily descend
• Fast: O(log n) search time
• High recall: >95% typically
• Memory: Extra graph storage
HNSW Parameters:
| Parameter | Description | Trade-off |
|---|
M | Connections per node | Memory vs. recall |
ef_construction | Build-time search width | Build time vs. recall |
ef_search | Query-time search width | Latency vs. recall |
IVF (Inverted File Index)
Clustering Phase:
┌─────────────────────────────────────────┐
│ Cluster vectors into K centroids │
│ │
│ ● ● ● ● │
│ /│\ /│\ /│\ /│\ │
│ ○○○○○ ○○○○○ ○○○○○ ○○○○○ │
│ Cluster 1 Cluster 2 Cluster 3 Cluster 4│
└─────────────────────────────────────────┘
Search Phase:
1. Find nprobe nearest centroids
2. Search only those clusters
3. Much faster than exhaustive
IVF Parameters:
| Parameter | Description | Trade-off |
|---|
nlist | Number of clusters | Build time vs. search quality |
nprobe | Clusters to search | Latency vs. recall |
IVF-PQ (Product Quantization)
Original Vector (128 dim):
[0.1, 0.2, ..., 0.9] (128 × 4 bytes = 512 bytes)
PQ Compressed (8 subvectors, 8-bit codes):
[23, 45, 12, 89, 56, 34, 78, 90] (8 bytes)
Memory reduction: 64x
Accuracy trade-off: ~5% recall drop
Algorithm Comparison
| Algorithm | Search Speed | Memory | Build Time | Recall |
|---|
| Flat/Brute | Slow (O(n)) | Low | None | 100% |
| IVF | Fast | Low | Medium | 90-95% |
| IVF-PQ | Very fast | Very low | Medium | 85-92% |
| HNSW | Very fast | High | Slow | 95-99% |
| HNSW+PQ | Very fast | Medium | Slow | 90-95% |
When to Use Which
< 100K vectors:
└── Flat index (exact search is fast enough)
100K - 1M vectors:
└── HNSW (best recall/speed trade-off)
1M - 100M vectors:
├── Memory available → HNSW
└── Memory constrained → IVF-PQ or HNSW+PQ
> 100M vectors:
└── Sharded IVF-PQ or distributed HNSW
Distance Metrics
Common Metrics
| Metric | Formula | Range | Best For |
|---|
| Cosine Similarity | A·B / (||A|| ||B||) | [-1, 1] | Normalized embeddings |
| Dot Product | A·B | (-∞, ∞) | When magnitude matters |
| Euclidean (L2) | √Σ(A-B)² | [0, ∞) | Absolute distances |
| Manhattan (L1) | Σ|A-B| | [0, ∞) | High-dimensional sparse |
Metric Selection
Embeddings pre-normalized (unit vectors)?
├── Yes → Cosine = Dot Product (use Dot, faster)
└── No
└── Magnitude meaningful?
├── Yes → Dot Product
└── No → Cosine Similarity
Note: Most embedding models output normalized vectors
→ Dot product is usually the best choice
Filtering and Hybrid Search
Pre-filtering vs Post-filtering
Pre-filtering (Filter → Search):
┌─────────────────────────────────────────┐
│ 1. Apply metadata filter │
│ (category = "electronics") │
│ Result: 10K of 1M vectors │
│ │
│ 2. Vector search on 10K vectors │
│ Much faster, guaranteed filter match │
└─────────────────────────────────────────┘
Post-filtering (Search → Filter):
┌─────────────────────────────────────────┐
│ 1. Vector search on 1M vectors │
│ Return top-1000 │
│ │
│ 2. Apply metadata filter │
│ May return < K results! │
└─────────────────────────────────────────┘
Hybrid Search Architecture
Query: "wireless headphones under $100"
│
┌─────┴─────┐
▼ ▼
┌───────┐ ┌───────┐
│Vector │ │Filter │
│Search │ │ Build │
│"wire- │ │price │
│less │ │< 100 │
│head- │ │ │
│phones"│ │ │
└───────┘ └───────┘
│ │
└─────┬─────┘
▼
┌───────────┐
│ Combine │
│ Results │
└───────────┘
Metadata Index Design
| Metadata Type | Index Strategy | Query Example |
|---|
| Categorical | Bitmap/hash index | category = "books" |
| Numeric range | B-tree | price BETWEEN 10 AND 50 |
| Keyword search | Inverted index | tags CONTAINS "sale" |
| Geospatial | R-tree/geohash | location NEAR (lat, lng) |
Scaling Strategies
Sharding Approaches
Naive Sharding (by ID):
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Shard 1 │ │ Shard 2 │ │ Shard 3 │
│ IDs 0-N │ │IDs N-2N │ │IDs 2N-3N│
└─────────┘ └─────────┘ └─────────┘
Query → Search ALL shards → Merge results
Semantic Sharding (by cluster):
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Shard 1 │ │ Shard 2 │ │ Shard 3 │
│ Tech │ │ Health │ │ Finance │
│ docs │ │ docs │ │ docs │
└─────────┘ └─────────┘ └─────────┘
Query → Route to relevant shard(s) → Faster!
Replication
┌─────────────────────────────────────────┐
│ Load Balancer │
└─────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Replica 1│ │Replica 2│ │Replica 3│
│ (Read) │ │ (Read) │ │ (Read) │
└─────────┘ └─────────┘ └─────────┘
│ │ │
└───────────┼───────────┘
│
┌─────────┐
│ Primary │
│ (Write) │
└─────────┘
Scaling Decision Matrix
| Scale (vectors) | Architecture | Replication |
|---|
| < 1M | Single node | Optional |
| 1-10M | Single node, more RAM | For HA |
| 10-100M | Sharded, few nodes | Required |
| 100M-1B | Sharded, many nodes | Required |
| > 1B | Sharded + tiered | Required |
Performance Optimization
Index Build Optimization
| Optimization | Description | Impact |
|---|
| Batch insertion | Insert in batches of 1K-10K | 10x faster |
| Parallel build | Multi-threaded index construction | 2-4x faster |
| Incremental index | Add to existing index | Avoids rebuild |
| GPU acceleration | Use GPU for training (IVF) | 10-100x faster |
Query Optimization
| Optimization | Description | Impact |
|---|
| Warm cache | Keep index in memory | 10x latency reduction |
| Query batching | Batch similar queries | Higher throughput |
| Reduce dimensions | PCA, random projection | 2-4x faster |
| Early termination | Stop when "good enough" | Lower latency |
Memory Optimization
Memory per vector:
┌────────────────────────────────────────┐
│ 1536 dims × 4 bytes = 6KB per vector │
│ │
│ 1M vectors: │
│ Raw: 6GB │
│ + HNSW graph: +2-4GB (M-dependent) │
│ = 8-10GB total │
│ │
│ With PQ (64 subquantizers): │
│ 1M vectors: ~64MB │
│ = 100x reduction │
└────────────────────────────────────────┘
Operational Considerations
Backup and Recovery
| Strategy | Description | RPO/RTO |
|---|
| Snapshots | Periodic full backup | Hours |
| WAL replication | Write-ahead log streaming | Minutes |
| Real-time sync | Synchronous replication | Seconds |
Monitoring Metrics
| Metric | Description | Alert Threshold |
|---|
| Query latency p99 | 99th percentile latency | > 100ms |
| Recall | Search accuracy | < 90% |
| QPS | Queries per second | Capacity dependent |
| Memory usage | Index memory | > 80% |
| Index freshness | Time since last update | Domain dependent |
Index Maintenance
┌─────────────────────────────────────────┐
│ Index Maintenance Tasks │
├─────────────────────────────────────────┤
│ • Compaction: Merge small segments │
│ • Reindex: Rebuild degraded index │
│ • Vacuum: Remove deleted vectors │
│ • Optimize: Tune parameters │
│ │
│ Schedule during low-traffic periods │
└─────────────────────────────────────────┘
Common Patterns
Multi-Tenant Vector Search
Option 1: Namespace/Collection per tenant
┌─────────────────────────────────────────┐
│ tenant_1_collection │
│ tenant_2_collection │
│ tenant_3_collection │
└─────────────────────────────────────────┘
Pro: Complete isolation
Con: Many indexes, operational overhead
Option 2: Single collection + tenant filter
┌─────────────────────────────────────────┐
│ shared_collection │
│ metadata: { tenant_id: "..." } │
│ Pre-filter by tenant_id │
└─────────────────────────────────────────┘
Pro: Simpler operations
Con: Requires efficient filtering
Real-Time Updates
Write Path:
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Write │ │ Buffer │ │ Merge │
│ Request │───▶│ (Memory) │───▶│ to Index │
└─────────────┘ └─────────────┘ └─────────────┘
Strategy:
1. Buffer writes in memory
2. Periodically merge to main index
3. Search: main index + buffer
4. Compact periodically
Embedding Versioning
Version 1 embeddings ──┐
│
Version 2 embeddings ──┼──▶ Parallel indexes during migration
│
│ ┌─────────────────────┐
└───▶│ Gradual reindexing │
│ Blue-green switch │
└─────────────────────┘
Cost Estimation
Storage Costs
Cost = (vectors × dimensions × bytes × replication) / GB × $/GB/month
Example:
10M vectors × 1536 dims × 4 bytes × 3 replicas = 184 GB
At $0.10/GB/month = $18.40/month storage
Note: Memory (for serving) costs more than storage
Compute Costs
Factors:
• QPS (queries per second)
• Latency requirements
• Index type (HNSW needs more RAM)
• Filtering complexity
Rule of thumb:
• 1M vectors, HNSW, <50ms latency: 16GB RAM
• 10M vectors, HNSW, <50ms latency: 64-128GB RAM
• 100M vectors: Distributed system required
Related Skills
rag-architecture - Using vector databases in RAG systemsllm-serving-patterns - LLM inference with vector retrievalml-system-design - End-to-end ML pipeline designestimation-techniques - Capacity planning for vector systems
Version History
- v1.0.0 (2025-12-26): Initial release - Vector database patterns for systems design
Last Updated
Date: 2025-12-26