Enterprise Search

---

name: enterprise-search description: Design cross-tool knowledge retrieval strategies, architect enterprise search systems, and tune relevance models. Use when building internal search experiences, consolidating knowledge across tools, or improving search result quality. tags: [search, knowledge-management, information-retrieval]

Enterprise Search

Business-oriented framework for designing cross-tool knowledge retrieval, architecting enterprise search systems, and tuning relevance models. Focused on strategy and requirements — for technical implementation, see hybrid-search-implementation and similarity-search-patterns in the ai-ml domain.

Use this skill when

• Designing an enterprise search strategy across multiple internal tools (Confluence, Slack, Drive, SharePoint, GitHub)
• Choosing between federated, centralized, or hybrid search architectures
• Defining relevance tuning requirements and quality metrics
• Building a knowledge taxonomy or metadata schema for searchable content
• Creating search UX requirements for internal portals
• Evaluating search quality and measuring improvement

Do not use this skill when

• Implementing vector search or embeddings at code level (use hybrid-search-implementation)
• Building similarity search with specific vector databases (use similarity-search-patterns)
• Optimizing web SEO for external search engines (use seo-audit)
• Building RAG pipelines for LLM applications (use RAG skills in ai-ml domain)

Instructions

1. Audit current state — inventory all content sources, volumes, and access patterns.
2. Choose architecture — federated, centralized, or hybrid based on your constraints.
3. Design taxonomy — define metadata schema, facets, and tagging standards.
4. Define relevance model — scoring factors, boosting rules, and personalization signals.
5. Set quality metrics — establish baselines and targets for search quality.
6. Design search UX — autocomplete, facets, snippets, and result presentation.

---

Search Architecture Patterns

Architecture Comparison

Pattern	How It Works	Pros	Cons	Best For
Federated	Query multiple sources in real-time, merge results	No data duplication, real-time freshness	Slower, limited cross-source ranking	Small orgs (<500 people), few sources
Centralized	Ingest all content into single search index	Best relevance, fastest queries, unified ranking	Data duplication, sync complexity, stale content risk	Large orgs, search-critical workflows
Hybrid	Centralized index for primary sources + federated for long-tail	Balanced cost vs. quality	Most complex to maintain	Mid-to-large orgs with diverse source landscape

Connector Architecture

Source	Connector Type	Sync Method	Typical Latency
Confluence / Wiki	REST API	Incremental (webhook + poll)	Near real-time
Slack / Teams	Events API	Streaming	Real-time
Google Drive	Drive API + Changes API	Incremental	5-15 min
SharePoint	Graph API	Delta query	5-15 min
GitHub	Webhooks + REST API	Event-driven	Near real-time
Jira / Linear	REST API + Webhooks	Incremental	Near real-time
Email	Graph API / Gmail API	Incremental	15-30 min
Database	CDC (Change Data Capture)	Streaming	Near real-time

Access Control

Critical requirement: Search results must respect source-level permissions.

Approach	How It Works	Trade-off
Early binding	Filter at index time (only index what user can access)	Secure but requires per-user indices or ACL tagging
Late binding	Filter at query time (check permissions on each result)	Simpler indexing but slower queries at scale
Hybrid	Group-based ACL at index + user-level check at query	Best balance for most orgs

---

Knowledge Taxonomy Design

Metadata Schema

Every indexed document should carry these metadata fields:

Field	Type	Purpose	Example
title	string	Primary display and search field	"Q4 Revenue Report"
source	enum	Origin system	confluence, slack, drive, github
content_type	enum	Document classification	document, conversation, code, ticket
team	string	Owning team or department	"Engineering", "Sales"
created_at	datetime	For freshness scoring	2026-01-15T10:30:00Z
updated_at	datetime	For freshness and deduplication	2026-02-28T14:00:00Z
author	string	For personalization and credibility	"jane.doe@company.com"
access_groups	list[string]	For permission filtering	["engineering", "all-staff"]
tags	list[string]	For faceted navigation	["architecture", "adr", "database"]
status	enum	Content lifecycle	draft, published, archived

Tagging Standards

Rule	Rationale
Use controlled vocabulary (not free-text tags)	Prevents tag proliferation and inconsistency
Max 5 tags per document	Forces specificity over over-tagging
Tags use kebab-case	Consistency with URLs and search queries
Review tag taxonomy quarterly	Remove unused tags, merge synonyms
Auto-tag where possible	Use classification models to suggest tags on creation

Content Freshness Policies

Content Type	Freshness Target	Stale Threshold	Action When Stale
Documentation	Updated quarterly	>6 months	Flag for review
Meeting notes	Permanent	N/A	Reduce ranking weight over time
Code / PRs	Always current (live sync)	N/A	N/A
Tickets / Issues	Live sync	N/A	Archive closed items after 12 months
Policies / Runbooks	Updated semi-annually	>12 months	Alert content owner

---

Relevance Tuning Framework

Scoring Factors

Factor	Weight	Description
Text relevance (BM25)	40%	Keyword match quality — title, body, tags
Freshness	20%	More recent content ranked higher (decay function)
Popularity	15%	View count, link count, citation count
Personalization	15%	User's team, recent searches, frequently accessed sources
Source authority	10%	Official docs > Slack messages > personal notes

Field Boosting

Field	Boost Factor	Rationale
Title	3.0x	Titles are the strongest relevance signal
Tags	2.0x	Curated metadata is high-signal
Headings (H1-H3)	1.5x	Section headers indicate topic boundaries
Body text	1.0x	Baseline — full content match
Comments	0.5x	Noisy, often tangential

Query Understanding

Technique	Purpose	Example
Synonym expansion	Match equivalent terms	"deploy" → "deploy, release, ship"
Spell correction	Handle typos	"kuberntes" → "kubernetes"
Intent classification	Route to specialized search	"how do I deploy" → tutorial filter
Entity recognition	Boost specific entities	"John's PR for auth" → person + code filter

---

Search Quality Metrics

Core Metrics

Metric	Formula	Target	How to Measure
MRR (Mean Reciprocal Rank)	Average of 1/rank of first relevant result	>0.6	Relevance judgments on sample queries
NDCG@10	Normalized discounted cumulative gain at position 10	>0.7	Graded relevance judgments
Precision@5	% of top 5 results that are relevant	>60%	Binary relevance judgments
Zero-Result Rate	% of queries returning no results	<5%	Log analysis
Click-Through Rate	% of searches that result in a click	>40%	Click tracking
Query Reformulation Rate	% of searches followed by a refined query	<20%	Session analysis
Time to Result	p50 and p95 query latency	p50 <200ms, p95 <1s	Infrastructure monitoring

Quality Improvement Loop

1. Sample 100 queries weekly from search logs
2. Have 2+ raters judge relevance of top 10 results (0-3 scale)
3. Calculate MRR, NDCG@10, Precision@5
4. Identify failure patterns (categories of bad results)
5. Adjust relevance model (boosting, synonyms, freshness weights)
6. A/B test changes against baseline
7. Repeat monthly

---

Search UX Patterns

Pattern	Purpose	Implementation Notes
Autocomplete	Reduce typing, guide to known content	Suggest from titles, tags, and popular queries
Faceted navigation	Filter by source, type, team, date	Show counts per facet; update dynamically
Snippets / Highlights	Show matching content in context	Highlight query terms in 2-3 sentence excerpts
Related queries	Help users refine or explore	"People also searched for..." based on co-occurrence
Source badges	Indicate content origin	Confluence icon, Slack icon, etc.
Freshness indicator	Show content age	"Updated 2 days ago" vs. "Updated 2 years ago"
"Did you mean?"	Handle typos gracefully	Only suggest when confidence >80%

---

Output Template: Enterprise Search Requirements Document

# Enterprise Search Requirements — [Project Name]

## Current State
- **Content sources:** [list with estimated volumes]
- **Current search tools:** [what people use today]
- **Top pain points:** [from user interviews]

## Architecture Decision
- **Pattern:** [Federated / Centralized / Hybrid]
- **Rationale:** [why this pattern]
- **Search platform:** [Elasticsearch, Typesense, Algolia, Vespa, etc.]

## Scope (Phase 1)
- **Sources to index:** [list with priority]
- **Content types:** [documents, conversations, code, tickets]
- **Users:** [target audience and access model]

## Relevance Model
- **Scoring factors:** [weights per factor]
- **Field boosting:** [title, tags, headings, body]
- **Freshness decay:** [function and parameters]

## Quality Targets
| Metric | Baseline | Target |
|--------|----------|--------|
| MRR | [current] | [goal] |
| Zero-result rate | [current] | <5% |
| p95 latency | [current] | <1s |

## Roadmap
- Phase 1: [Core sources, basic search] — [timeline]
- Phase 2: [Additional sources, relevance tuning] — [timeline]
- Phase 3: [Personalization, AI-powered features] — [timeline]

---

Common Mistakes

• Indexing everything without curation — more content does not mean better search; noisy sources dilute quality
• Ignoring access control — leaking confidential documents through search is a security incident
• No freshness weighting — returning 3-year-old docs before this week's update frustrates users
• Not measuring search quality — if you don't measure MRR/NDCG, you can't improve
• Building search without user research — understand what people actually search for before designing the system
• Treating search as a one-time project — relevance tuning is ongoing; plan for continuous improvement

---

Additional Resources

• Related skills: hybrid-search-implementation (ai-ml — technical implementation), similarity-search-patterns (ai-ml — vector search)
• Elasticsearch / OpenSearch — open-source search engines
• Algolia — managed search platform
• Vespa — open-source search and recommendation engine