Estimation Techniques

This skill provides frameworks for back-of-envelope calculations essential for system design and capacity planning.

When to Use This Skill

Keywords: back-of-envelope, estimation, QPS, storage calculation, bandwidth, latency, capacity planning, scale estimation

Use this skill when:

• Estimating system capacity requirements
• Calculating storage needs for a feature
• Determining bandwidth requirements
• Sizing infrastructure for expected load
• Justifying architectural decisions with numbers
• Preparing for system design interviews

Core Principle

Estimation is not about precision, it's about order of magnitude.

Getting within 10x is usually good enough for architectural decisions. The goal is to identify if you need:

• 1 server or 100 servers
• 1 GB or 1 TB of storage
• 10 ms or 1 second latency

Essential Numbers to Know

Powers of 2

Power	Value	Approximate
2^10	1,024	~1 Thousand (KB)
2^20	1,048,576	~1 Million (MB)
2^30	1,073,741,824	~1 Billion (GB)
2^40	1,099,511,627,776	~1 Trillion (TB)

Time Conversions

Unit	Seconds	Useful For
1 minute	60	Short operations
1 hour	3,600	Batch jobs
1 day	86,400 (~100K)	Daily aggregations
1 month	2,592,000 (~2.5M)	Monthly calculations
1 year	31,536,000 (~30M)	Annual projections

Availability Targets

Availability	Downtime/Year	Downtime/Month	Downtime/Day
99% (two 9s)	3.65 days	7.31 hours	14.4 min
99.9% (three 9s)	8.76 hours	43.8 min	1.44 min
99.99% (four 9s)	52.6 min	4.38 min	8.64 sec
99.999% (five 9s)	5.26 min	26.3 sec	864 ms

Latency Numbers Every Programmer Should Know

See full reference: references/latency-numbers.md

Quick reference:

Operation	Latency	Relative
L1 cache reference	0.5 ns	1x
L2 cache reference	7 ns	14x
Main memory reference	100 ns	200x
SSD random read	16 us	32,000x
HDD seek	2 ms	4,000,000x
Round trip same datacenter	0.5 ms	1,000,000x
Round trip CA to Netherlands	150 ms	300,000,000x

Estimation Patterns

Pattern 1: QPS (Queries Per Second)

Formula

QPS = (Number of Users) x (Actions per User per Day) / (Seconds per Day)

Example: Twitter-like service

Given:
- 300 million monthly active users
- 50% are daily active = 150M DAU
- Average user reads 20 tweets/day

QPS = 150M * 20 / 86,400
    = 3 billion / 100,000
    = 30,000 QPS

Peak load (typically 2-3x average):
Peak QPS = 30,000 * 3 = 90,000 QPS

Pattern 2: Storage Estimation

Formula

Storage = (Number of Items) x (Size per Item) x (Replication Factor) x (Time Period)

Example: Photo storage service

Given:
- 100 million users
- 10% upload daily = 10M uploads/day
- Average photo size = 2 MB
- Keep 5 years of data
- Replication factor = 3

Daily storage = 10M * 2 MB = 20 TB
Yearly storage = 20 TB * 365 = 7.3 PB
5-year storage = 7.3 * 5 = 36.5 PB
With replication = 36.5 * 3 = ~110 PB

Pattern 3: Bandwidth Estimation

Formula

Bandwidth = (QPS) x (Request Size or Response Size)

Example: Video streaming service

Given:
- 1 million concurrent viewers
- Average bitrate = 5 Mbps
- Peak hours: 8 PM - 11 PM

Bandwidth = 1M * 5 Mbps = 5 Tbps

With 20% overhead: ~6 Tbps

CDN egress cost (rough):
$0.02/GB * 6 Tbps * 3 hours * 3600 sec/hour / 8 bits/byte
= massive cost (hence why Netflix built their own CDN)

Pattern 4: Cache Size Estimation

Formula

Cache Size = (QPS) x (Cache TTL) x (Response Size) x (Unique Ratio)

Example: API response cache

Given:
- 10,000 QPS
- Cache TTL = 5 minutes = 300 seconds
- Average response = 10 KB
- 20% of requests are unique

Cache entries = 10,000 * 300 * 0.20 = 600,000 entries
Cache size = 600,000 * 10 KB = 6 GB

With overhead (keys, metadata): ~10 GB

Pattern 5: Database Sizing

Formula

DB Size = (Number of Rows) x (Row Size) x (Index Overhead) x (Replication)

Example: User profile database

Given:
- 500 million users
- Average profile = 1 KB (name, email, settings, etc.)
- Index overhead = 30%
- Primary + 2 replicas = 3x

Data size = 500M * 1 KB = 500 GB
With indexes = 500 GB * 1.3 = 650 GB
With replication = 650 GB * 3 = ~2 TB

Memory for hot data (20%): ~400 GB

Common Estimation Scenarios

Scenario 1: URL Shortener

Requirements:
- 100M new URLs/month
- 10:1 read:write ratio

Writes:
- 100M / (30 * 24 * 3600) = ~40 writes/second
- Peak: ~100 writes/second

Reads:
- 40 * 10 = 400 reads/second
- Peak: ~1000 reads/second

Storage (5 years):
- 100M URLs/month * 60 months = 6 billion URLs
- Average URL = 100 bytes (short) + 500 bytes (long) = 600 bytes
- 6B * 600 bytes = 3.6 TB
- With indexes and overhead: ~5 TB

Scenario 2: Chat Application

Requirements:
- 10M daily active users
- Average 50 messages sent/day
- Average 200 messages received/day

Message throughput:
- Sends: 10M * 50 / 86,400 = ~6,000 messages/second
- Peak: ~20,000 messages/second

Connections:
- Each user maintains 1-3 connections (phone, laptop, tablet)
- Peak concurrent: 10M * 0.1 (10% online) * 2 = 2M connections

Storage (1 year):
- 10M users * 50 msgs/day * 365 days = 182B messages/year
- Average message = 200 bytes
- 182B * 200 bytes = 36.4 TB/year

Scenario 3: Video Streaming

Requirements:
- 100M monthly active users
- 30% watch daily = 30M DAU
- Average 1 hour/day viewing

Concurrent viewers (peak):
- 30M DAU / 24 hours * 3 (peak factor) = ~4M concurrent

Bandwidth:
- Average stream: 5 Mbps
- 4M * 5 Mbps = 20 Tbps peak bandwidth

Storage (library of 10K titles):
- Average video = 2 hours
- Multiple qualities: 480p (1GB), 720p (3GB), 1080p (5GB), 4K (20GB)
- Per title: ~30 GB
- Library: 10K * 30 GB = 300 TB

Estimation Tips

Round Aggressively

Instead of:      Use:
86,400 seconds   ~100,000 (10^5)
2.5 million      ~3 million
7.3 petabytes    ~10 petabytes

Use Orders of Magnitude

Think in powers of 10:

• Thousands (10^3)
• Millions (10^6)
• Billions (10^9)
• Trillions (10^12)

State Your Assumptions

Always verbalize:

• "I'm assuming 10% of users are active at peak"
• "I'm estimating average message size at 200 bytes"
• "I'm using a 3x replication factor"

Sanity Check Results

After calculating, ask:

• "Does this make sense?"
• "Is this in the right order of magnitude?"
• "What would change if my assumption is off by 10x?"

Common Mistakes

Mistake 1: Ignoring Peak vs Average

Problem

Sizing for average load.

Average QPS: 10,000
Peak QPS: 30,000 (often 2-3x average)

If you size for 10,000, you'll fail at peak.

Mistake 2: Forgetting Replication

Problem

Calculating raw storage without copies.

Data: 1 TB
With 3 replicas: 3 TB
With backups: 4-5 TB

Mistake 3: Not Accounting for Growth

Problem

Sizing for current, not future.

Current users: 10M
Expected growth: 50%/year
Year 3: 10M * 1.5^3 = 34M users

Size for at least 2x current to avoid near-term issues.

Mistake 4: Over-Precision

Problem

Calculating to 3 decimal places.

Bad: "We need exactly 3,456,789 IOPS"
Good: "We need roughly 3-4 million IOPS"

Quick Reference Calculations

Need	Formula
QPS	users * actions/day / 86400
Storage/day	items/day * size/item
Bandwidth	QPS * response_size
Cache hit rate	1 - (DB_QPS / total_QPS)
Servers needed	QPS / QPS_per_server
Shards needed	data_size / max_shard_size

Related Skills

• design-interview-methodology - Overall interview framework
• quality-attributes-taxonomy - NFR definitions (scalability, performance)
• database-scaling - Database capacity planning (Phase 3)
• caching-strategies - Cache sizing and hit rates (Phase 3)

Related Commands

• /sd:estimate <scenario> - Calculate capacity interactively

Related Agents

• capacity-planner - Guided estimation with calculations

References

• references/latency-numbers.md - Complete latency reference table

---

Version History

• v1.0.0 (2025-12-26): Initial release

---

Last Updated

Date: 2025-12-26 Model: claude-opus-4-5-20251101