HarnessML: Domain Research

HarnessML: Domain Research

Use when generating feature hypotheses from domain knowledge. This is not a one-time pre-work step — return here whenever results surprise you, progress stalls, or a new data source becomes available.

---

The Principle

The biggest ML gains come from features that capture real phenomena, not from hyperparameter tuning or model architecture changes. A feature that encodes domain knowledge — even imperfectly — gives the model information it cannot learn from the raw data alone.

Domain research is how you generate those features.

---

Generating Hypotheses

Start with the Domain, Not the Data

Before looking at correlations, ask: what does domain expertise say should matter?

• Use web search, Wikipedia, academic literature, practitioner knowledge
• Identify known predictive factors and the mechanisms behind them
• Look for phenomena that are well-understood in the domain but not yet captured in your features

For Each Hypothesis, Document

• What it is: The phenomenon in plain language
• Why it should be predictive: The causal or correlational mechanism — not "it might help" but "it should help because..."
• What data would capture it: Raw columns, derived signals, or external sources
• Expected signal strength: Strong / Medium / Weak, with reasoning

Types of Domain Signals

Direct predictors — Features that directly measure the outcome driver.

• Hospital readmission: count_of_comorbidities
• House price: square_footage

Proxy signals — Indirect indicators when direct measurement is unavailable.

• Financial distress: days_payable_outstanding (when cash flow data is missing)
• Health risk: pharmacy_visit_frequency (when medical records are incomplete)

Interaction effects — Two features weak alone, strong together.

• high_leverage * rising_rates — leverage is fine until rates move
• is_diabetic * high_bmi — captures a specific high-risk population

Conditional effects — A feature that only matters in certain contexts.

• marketing_spend only predicts sales for products with existing brand awareness
• rainfall only affects crop yield during the growing season

Regime indicators — Signals that relationships change under different conditions.

• vix_above_30 — volatility regime where correlations break down
• product_lifecycle_stage — growth vs maturity dynamics differ

Contrarian signals — Counter-intuitive predictive direction. Often the most valuable.

• Higher satisfaction surveys sometimes predict turnover (dissatisfied employees don't respond)
• More safety incidents can predict fewer fatalities (reporting culture catches problems early)

---

Mapping Hypotheses to Features

For each hypothesis:

1. Raw columns needed — what must exist or be sourced
2. Transformation — the formula or logic
3. Feature type: instance, grouped, interaction, ratio, indicator, regime
4. Novelty check — does an existing feature already capture this signal?

features(action="discover")

If an existing feature correlates >0.8 with your proposed feature, yours is likely redundant. Either skip it or refine the hypothesis to capture what the existing feature misses.

---

The Research Log

Maintain a running log. This is the connective tissue between domain knowledge and experiment results.

### Hypothesis: [Name]
- **Domain reasoning**: Why this should be predictive (the mechanism)
- **Source**: Where you found evidence
- **Feature(s)**: Name and formula
- **Expected signal**: Strong / Medium / Weak
- **Result**: What happened when tested
- **Learning**: What this tells us about the domain
- **Follow-up**: Next hypothesis generated by this result

The Follow-up field is the most important. Every tested hypothesis should generate at least one new question.

---

When to Return Here

• Project start: Generate initial hypotheses before modeling
• After surprising results: A model that fails where you expected it to succeed means your domain understanding is incomplete
• When progress stalls: If experiments aren't teaching you anything new, you need new hypotheses from a different angle
• New data source: Map new data through domain reasoning before feature engineering
• After diagnosis reveals error patterns: If the model consistently fails on a subgroup, ask what domain knowledge explains that subgroup's behavior

---

Anti-Patterns

• Blind feature generation: Adding features without a hypothesis for why they should work
• Over-relying on auto-search: features(action="auto_search") finds statistical artifacts; domain reasoning finds real signals. Use auto-search as a supplement, not a replacement.
• Stopping after initial research: Domain research is continuous. Your best hypotheses often come after seeing what the model gets wrong.
• Ignoring negative results: A feature that doesn't work is information. Why didn't it work? Was the hypothesis wrong, or is the signal already captured?