recommend-tools

Recommend Tools

A curated catalog. Not every corpus problem should be solved inside Claude Code — sometimes the right move is to point at an established library and set the user up to use it.

Classical NLP toolkits

• spaCy — production-grade NLP: tokenization, POS, NER, dependency parsing, lemmatization. Use as the default for anything ner-extraction, parametric-analysis, or tokenization-heavy.
• TextBlob — friendly wrapper over NLTK/pattern. Good for quick sentiment, noun-phrase extraction, translation, simple classification on small corpora. Lower quality than spaCy/transformers but very approachable.
• NLTK — classic toolkit, strong for linguistic/academic work. Slower than spaCy for production.
• TextAnalysis.jl — the main NLP library for the Julia ecosystem. Reach for this when the user is already in Julia (scientific computing, notebooks) — covers document representations, TF-IDF, LDA, summarization, sentiment.

Topic modeling & clustering

• BERTopic — default recommendation for topic-analysis. Embeddings + UMAP + HDBSCAN + c-TF-IDF. Dynamic topics over time built-in.
• gensim — LDA, LSI, word2vec. Use when BERTopic is overkill or when LDA is explicitly wanted.
• Top2Vec — similar space to BERTopic.

Aspect-based sentiment / fine-grained opinion

• PyABSA — aspect-based sentiment analysis framework. Use when the user wants per-aspect sentiment ("the battery is good, but the screen is bad" → {battery: +, screen: -}), not just document-level polarity. Strong for review corpora.

Corpus indexing & search

• BlackLab — Lucene-backed corpus retrieval engine for linguistically annotated corpora. Use when the user has a large annotated corpus and wants structured queries ("find every sentence where a PERSON is the subject of a verb of speech") rather than plain text search. Heavier setup; justifies itself on research-grade corpora.
• Whoosh / Tantivy — lightweight full-text indexes for simpler cases.

Entity extraction (beyond spaCy)

• GLiNER — zero-shot NER with user-specified entity types. Excellent when your entity types are domain-specific and don't match spaCy's defaults.
• flair — strong NER/POS taggers, multilingual.

Embeddings / similarity

• sentence-transformers — de-facto default for local embeddings. all-MiniLM-L6-v2 is fast + good enough for most clustering.
• nomic-embed-text — via Ollama, solid local embedding model.

Keyword / keyphrase extraction

• KeyBERT — BERT-based keyphrase extraction. Good for tag derivation.
• YAKE — unsupervised, language-agnostic, statistical.

Readability & text stats

• textstat — Flesch, Flesch-Kincaid, Gunning Fog, etc. Used by parametric-analysis.

When to pick what

• "I want to explore — quick and friendly" → TextBlob or spaCy.
• "I need topic clusters and evolution over time" → BERTopic.
• "I have reviews and want per-feature sentiment" → PyABSA.
• "I'm doing corpus linguistics research" → BlackLab (+ spaCy for annotation).
• "I'm in Julia already" → TextAnalysis.jl.
• "I need zero-shot domain-specific entities" → GLiNER.

Integration

When recommending a tool, show the user a minimal install + a 10-line example that works on their corpus. Don't just drop a link.