Imageboard research tool

Browse, search, and export threads across 20 imageboard and forum sites.

Prerequisites

• Python 3.10+ with requests installed (pip install requests)

No authentication needed — all supported APIs are public and read-only.

Supported sites

Western

Site	Engine	Notes
4chan.org	vichan	Boards API, CDN images
leftypol.org	vichan	Politics
wizchan.org	vichan	Male-only community
smuglo.li	vichan	Anime/manga community
uboachan.net	vichan	Yume Nikki/RPGMaker community
kissu.moe	vichan	Otaku culture
sushigirl.us	vichan	Comfy-posting community
8kun.top	vichan	Boards API (328 boards)
kohlchan.net	lynxchan	German-language imageboard
endchan.net	lynxchan	Free-speech focused
zzzchan.xyz	jschan	Misc boards
trashchan.xyz	jschan	Comfy/retro boards
2ch.hk	makaba	Largest Russian imageboard
desuarchive.org	foolfuuka	4chan archive (search API)
archive.4plebs.org	foolfuuka	4chan archive (search API)

Japanese

Site	Engine	Notes
5ch.net	dat	Largest Japanese BBS. cp932 encoding. May return HTTP 451 from non-JP IPs
jbbs.shitaraba.net	shitaraba	Hosted boards. Board format: category/id (e.g., computer/43680). EUC-JP
machi.to	machibbs	Regional Japanese boards. JSON API, UTF-8

Korean

Site	Engine	Notes
dcinside.com	dcinside	Korean imageboard (283M monthly visits). Web scraping. Auto-detects gallery type (major/minor/mini)
arca.live	arcalive	Korean community platform (145M monthly visits). Mobile JSON API

Quick start

CHANS="python ${CLAUDE_PLUGIN_ROOT}/skills/chans-research/scripts/chans.py"

# List all supported sites
$CHANS sites

# List boards on a site
$CHANS boards --site 4chan.org
$CHANS boards --site 5ch.net

# Browse a board catalog
$CHANS catalog --site 4chan.org -b g
$CHANS catalog --site 5ch.net -b news4vip
$CHANS catalog --site machi.to -b tokyo
$CHANS catalog --site dcinside.com -b programming

# Export a thread to markdown
$CHANS thread --site 4chan.org -b biz -t 12345678
$CHANS thread --site machi.to -b tokyo -t 12345 --stdout

# Export with local images (downloads full + thumb to assets/)
$CHANS thread --site 4chan.org -b biz -t 12345678 -D -o ./export

# Download images for a thread already in the DB
$CHANS download --site 4chan.org -b biz -t 12345678 -o ./export

# Shitaraba uses category/id board format
$CHANS catalog --site jbbs.shitaraba.net -b computer/43680

# Bulk ingest boards into local SQLite
$CHANS ingest --site 4chan.org -B g,biz,sci
$CHANS ingest --site 5ch.net -B news4vip,poverty

# Search across all ingested data (FTS5) — works with CJK text
$CHANS search -q "artificial intelligence"
$CHANS search -q "テスト"
$CHANS search -q "bitcoin" --site 4chan.org --board biz

# Search FoolFuuka archives (live API, no ingest needed)
$CHANS search -q "bitcoin" --site desuarchive.org --board g

# Export search results to markdown
$CHANS export -q "machine learning" -o ./exports

# Show DB statistics
$CHANS stats

Workflow

1. Discover sites and boards

$CHANS sites                           # all 20 supported sites
$CHANS boards --site 4chan.org         # all boards on 4chan
$CHANS boards --site 5ch.net          # all boards on 5ch (from bbsmenu)

2. Ingest data

$CHANS ingest --site 4chan.org -B g,biz,sci
$CHANS ingest --site 5ch.net -B news4vip,poverty

Ingestion fetches the full catalog for each board and stores all OP posts in a local SQLite database (~/.claude/cache/chans.db) with FTS5 indexing.

3. Search

$CHANS search -q "home server"
$CHANS search -q "rust programming" --site 4chan.org --board g
$CHANS search -q "テスト"              # CJK search works with FTS5

For archive sites (desuarchive, 4plebs), search hits the live FoolFuuka API and also ingests results locally.

4. Export threads

$CHANS thread --site 4chan.org -b g -t 12345678 --stdout
$CHANS thread --site machi.to -b tokyo -t 12345 --stdout

5. Download images locally

The -D / --download-images flag on the thread command downloads all full-size and thumbnail images into an assets/ subdirectory and rewrites the markdown export to use local assets/ paths instead of CDN URLs. Files are named {post_id}_full.{ext} and {post_id}_thumb.{ext}.

# Export thread with images saved locally
$CHANS thread --site 4chan.org -b biz -t 12345678 -D -o ./research

# Standalone: download images for a thread already in the DB
$CHANS download --site 4chan.org -b biz -t 12345678 -o ./research

Image-capable engines: vichan (4chan, leftypol, etc.), lynxchan (kohlchan, endchan), makaba (2ch.hk), foolfuuka (desuarchive, 4plebs). Japanese and Korean engines do not extract image URLs.

Always use -D when exporting threads for presentations or visual research. Imageboard CDN URLs are ephemeral — threads get pruned and images disappear. Local downloads ensure images survive for use with /frontend-slides or any other downstream tool. The exported markdown with local assets/ paths can be read directly as image source material.

Architecture

scripts/
├── chans.py        # CLI entrypoint (argparse + commands)
├── adapters.py     # Site registry + 10 engine adapters
├── fetch.py        # HTTP: rate limits, If-Modified-Since, retries
├── db.py           # SQLite + FTS5 storage
├── html_strip.py   # HTML → plaintext (vichan, lynxchan, dat, dcinside)
└── dat_parse.py    # 2ch/5ch dat format parser (subject.txt + thread.dat)

10 engine adapter families normalize all sites to a canonical post schema:

• VichanAdapter — 4chan, leftypol, wizchan, smuglo.li, uboachan, kissu.moe, sushigirl.us, 8kun.top
• LynxchanAdapter / JschanAdapter — kohlchan, endchan, zzzchan, trashchan
• MakabaAdapter — 2ch.hk (Russian)
• FoolFuukaAdapter — desuarchive, 4plebs (search-only archives)
• DatAdapter — 5ch.net (Japanese dat format, server discovery via bbsmenu)
• ShitarabaAdapter — jbbs.shitaraba.net (EUC-JP, category/id boards)
• MachiBBSAdapter — machi.to (JSON API, regional Japanese boards)
• DCInsideAdapter — dcinside.com (Korean, web HTML scraping, auto-detects gallery type)
• ArcaLiveAdapter — arca.live (Korean, mobile JSON API)

API rate limits

Western sites default to 1 req/s. Japanese sites use 1.5-2.0s. Korean sites use 1.5s. The fetch layer enforces per-domain rate limiting automatically. Repeat requests to the same URL use If-Modified-Since to avoid re-downloading unchanged data.

Important notes

• All APIs are read-only — no posting capability
• Threads on live boards are ephemeral — export promptly before archival
• Content is unmoderated — expect raw language and NSFW content
• Archive sites (FoolFuuka) have permanent storage but search-only access
• Data is stored locally in ~/.claude/cache/chans.db (SQLite with WAL mode)
• 5ch.net may return HTTP 451 from non-Japanese IPs — the adapter handles this gracefully
• Shitaraba boards use category/id format (e.g., computer/43680)
• FTS5 search works with CJK text (Japanese, Korean, Chinese). Queries of 3+ characters use the trigram index; shorter queries fall back to LIKE
• Images are ephemeral on live boards — always use -D when exporting threads that will feed into presentations or visual work