joeseesun-markdown-proxy

qiaomu-markdown-proxy

Convert any URL to clean Markdown, with built-in support for login-required pages (X/Twitter, WeChat, Feishu/Lark docs, etc.)

将任意 URL 转为干净的 Markdown，支持需要登录的页面（X/Twitter、微信公众号、飞书文档等）。

English | 中文

---

English

Features

Send any URL to Claude, and it automatically fetches the full content as Markdown. Five content types have dedicated extraction:

URL Type	Method	Why
WeChat Articles (mp.weixin.qq.com)	Built-in Playwright script	Anti-scraping protection requires headless browser
Feishu/Lark Docs (feishu.cn, larksuite.com)	Built-in Feishu API script	Requires API authentication, auto-converts to Markdown
YouTube	Dedicated YouTube skill	Video content has its own toolchain
PDF (remote or local)	Built-in PDF extraction (extract_pdf.sh)	Three-method cascade: marker-pdf → pdftotext → pypdf
All other URLs	Proxy cascade via fetch.sh: r.jina.ai → defuddle.md → agent-fetch	Free, no API key, content validation built-in

Prerequisites

• Claude Code installed
• curl (built-in on macOS/Linux)
• (Optional - WeChat scraping) Python 3.8+ with playwright

  pip install playwright beautifulsoup4 lxml
  playwright install chromium
  

• (Optional - PDF extraction) One of:
  • marker-pdf (best quality): pip install marker-pdf
  • pdftotext (fast): brew install poppler
  • pypdf (fallback): pip install pypdf
• (Optional - Proxy fallback) agent-fetch

  npx agent-fetch --help  # No pre-install needed, npx auto-downloads
  

• (Optional - Feishu docs) Environment variables FEISHU_APP_ID and FEISHU_APP_SECRET

  echo $FEISHU_APP_ID  # Verify configured
  

Installation

npx skills add joeseesun/qiaomu-markdown-proxy

Verify:

ls ~/.claude/skills/qiaomu-markdown-proxy/SKILL.md

Usage

Just send Claude a URL:

• "Read this article: https://example.com/post"
• "Fetch this tweet: https://x.com/user/status/123456"
• "Read this WeChat article: https://mp.weixin.qq.com/s/abc123"
• "Convert this Feishu doc to Markdown: https://xxx.feishu.cn/docx/xxxxxxxx"

Proxy Priority

1. r.jina.ai — Most complete content, preserves image links
2. defuddle.md — Cleaner output with YAML frontmatter
3. agent-fetch — Local tool, no network proxy needed
4. defuddle CLI — Local CLI, good for standard web pages

Feishu/Lark Document Support

Built-in fetch_feishu.py script fetches documents via Feishu Open API and auto-converts to Markdown:

• Supports new docs (docx), legacy docs (doc), and wiki pages
• Auto-parses document blocks into Markdown format
• Supports headings, lists, code blocks, quotes, todos, equations, images, etc.
• Requires FEISHU_APP_ID and FEISHU_APP_SECRET environment variables
• App needs docx:document:readonly permission

Troubleshooting

Issue	Solution
WeChat scraping fails	Run playwright install chromium to install browser
Feishu returns permission error	Check FEISHU_APP_ID and FEISHU_APP_SECRET env vars, confirm app has document read permission
Feishu wiki page fails	Confirm app has wiki:wiki:readonly permission
r.jina.ai returns empty	Auto-falls back to defuddle.md (no action needed)
All proxies fail	URL may have strict auth restrictions, try npx agent-fetch

Credits

• r.jina.ai — Free URL-to-Markdown proxy by Jina AI
• defuddle.md — Clean article extraction service
• agent-fetch — Local URL content extraction tool
• Playwright — Browser automation for WeChat scraping
• Feishu Open Platform — Feishu Document API

---

中文

功能

给 Claude 发一个 URL，自动抓取完整内容并转为 Markdown。支持五种内容类型的专用抓取：

URL 类型	抓取方式	原因
微信公众号 (mp.weixin.qq.com)	内置 Playwright 脚本	公众号有反爬，需无头浏览器
飞书文档 (feishu.cn/docx/, /wiki/, /docs/)	内置飞书 API 脚本	需要 API 认证，自动转 Markdown
YouTube	专用 YouTube skill	视频内容有专用工具链
PDF（远程 URL 或本地文件）	内置 PDF 提取（extract_pdf.sh）	三级 fallback：marker-pdf → pdftotext → pypdf
其他所有 URL	代理级联 fetch.sh：r.jina.ai → defuddle.md → agent-fetch	免费、无需 API key、内置内容验证

前置条件