From fullstack-mkt-skills
Guides A/B test setup for ads, landing pages, emails, products: selects variables, calculates sample sizes, sets up tracking, analyzes statistical significance. Triggers on 'A/B test', 'split test' keywords.
npx claudepluginhub minhnv0807/fullstack-mkt-skills --plugin fullstack-mkt-skillsThis skill uses the workspace's default tool permissions.
Doc `.agents/product-marketing-context.md` neu co.
Guides planning, design, and implementation of A/B tests, split tests, and experiments using hypothesis frameworks, test types, and sample size calculations.
Guides A/B test planning and design with hypothesis frameworks, sample size calculations, test types, and statistical principles for valid, actionable results. Useful for experiments or split tests.
Designs statistically valid A/B tests with hypotheses, sample sizes, test types, and metrics. Activates on experiment keywords like A/B test or hypothesis.
Share bugs, ideas, or general feedback.
Doc .agents/product-marketing-context.md neu co.
Quy tac cot loi. Neu test 2 bien cung luc → khong biet bien nao tac dong.
Format: "Neu lam [X], metric [Y] se tang [Z%]"
Khong duoc dung test som.
Test tung tuan tron (khong 3 ngay, khong 10 ngay) — khach mua hanh vi khac nhau cac ngay.
Khong check ket qua moi gio. Dung ket qua cho den khi het thoi han + du sample.
Chi tin khi p-value < 0.05. Neu khong: ngau nhien, khong phai do bien cua ban.
Ghi lai:
→ De sau nay ban khong test lai cai da test.
Sample size/variant =
16 × p × (1-p) / MDE²
Trong do:
p = Baseline conversion rate hien tai (vi du 0.02 = 2%)
MDE = Minimum Detectable Effect (muc do thay doi toi thieu ban muon phat hien, vi du 0.2 = 20% tang)
Landing page hien co conv rate 3%, muon phat hien thay doi 20%+:
=> Can 25,866 visit total, neu 500 visit/ngay → 52 ngay.
Landing page hien conv rate 10%, muon phat hien 20%+:
=> 7,200 visit total, 500/ngay → 14 ngay (kha thi).
| Traffic/ngay | Conv rate | Thoi gian can | Co nen test? |
|---|---|---|---|
| < 100 visit | Bat ky | 2+ thang | Khong — traffic thap |
| 100-500 | 2-5% | 3-6 tuan | Co, nhung kien nhan |
| 500-2K | 2-5% | 2-3 tuan | Co — ly tuong |
| 2K+ | Bat ky | 1-2 tuan | Co — nhanh |
Neu traffic thap (< 100/ngay) → khong test, thay vao do tap trung tang traffic truoc.
Tac dong cao nhat. 80% nguoi doc headline, 20% doc body → headline thay doi toi uu = bien nho nhat.
Idea:
Do thay doi duoc de test. Thuong tang 5-25%.
Test:
Test:
Test:
Test:
Test:
Test:
Test:
| Tool | Phu hop | Chi phi |
|---|---|---|
| Meta Ads A/B test built-in | Test ads (creative/audience/placement) | Free |
| TikTok Ads Split test | Test TikTok ads | Free |
| Google Optimize (deprecated) | Da ngung — dung thay the: VWO, Convert.com | Tu $199/thang |
| PostHog | Test product features, landing | Free tier OK |
| Unbounce / Instapage | Landing page built-in A/B | $90+/thang |
| Manual (split URL) | Don gian, 50/50 URL redirect | Free |
1. Tao 2 phien ban landing page: /page-a va /page-b
2. Split traffic 50/50:
- Qua Meta Ads: 2 ad set cung audience, khac link dich
- Qua Google Ads: 2 ad cung traffic, khac link
- Qua organic: dung URL rewrite (rare)
3. Tracking conversion rieng tung page:
- Meta Pixel: event "Lead" voi custom parameter "page_version" = A/B
- GA4: UTM campaign=test-header, utm_content=a/b
4. Sau 14 ngay + du sample, export data
Dung calculator online: https://www.evanmiller.org/ab-testing/chi-squared.html
Input:
Output:
| p-value | Uplift | Ket luan |
|---|---|---|
| < 0.05 | > 5% | B thang — implement B |
| < 0.05 | < 5% | Significant nhung nho — can nhac cost |
| 0.05-0.10 | > 10% | Ket qua borderline — keo dai test |
| > 0.10 | Bat ky | Khong significant — giu A hoac test tiep |
# A/B Test: [Ten test]
Ngay tao: [YYYY-MM-DD]
Chu so huu: [Ten nguoi]
## 1. Gia thuyet
"Neu [thay doi X], [metric Y] se tang [Z%]"
## 2. Bien test
- Variant A (Control): [Mo ta hien tai]
- Variant B (Challenger): [Mo ta thay doi]
- Chi thay doi: [1 thing]
## 3. Metric
- Primary: [VD: Conversion rate]
- Secondary: [VD: Bounce rate, Time on page]
## 4. Sample size & Duration
- Baseline: [p = %]
- MDE: [% change muon phat hien]
- Sample/variant: [so]
- Traffic/ngay: [so]
- Thoi gian can: [so ngay]
## 5. Setup
- Tool: [Ten]
- URL A: [...]
- URL B: [...]
- Tracking: [Meta Pixel / GA4 / PostHog events]
- Split: 50/50 (hoac khac ghi ro)
## 6. Timeline
- Start: [ngay]
- End: [ngay]
- Review: [ngay]
## 7. Ket qua (dien sau khi ket thuc)
| Variant | Visitors | Conversions | Rate | Uplift vs A |
|---------|----------|-------------|------|-------------|
| A | | | | — |
| B | | | | +X% |
p-value: [x]
Significant (p<0.05): [Yes/No]
## 8. Ket luan
[B thang / A thang / Khong significant]
## 9. Hanh dong
[Implement / Test tiep / Rollback]
## 10. Bai hoc
[Dieu hoc duoc → ap dung cho test sau]