designing-experiments

Designing Experiments

Design 1-3 experiment paths to verify or falsify a refined hypothesis. Prefer experiments executable with available tools — code to run, math to prove, data to gather, evidence to find — over those requiring unavailable external resources.

---

Arguments

• Hypothesis file -- path to hypothesis-NN.md

---

Workflow

Step 0: Check state

Read the hypothesis file. If it already contains a ## Experiments section with content, skip -- already designed. The file is the checkpoint.

If ## Status is already confirmed or refuted (set by literature review), skip experiment design. Note that the hypothesis was resolved by prior work and no experiments are needed.

Read problem.md in the parent directory for constraints and success criteria.

Step 1: Assess what is testable

From the refined hypothesis statement (or original if not yet refined), determine:

• What type of evidence would prove or disprove the claim
• What tools are available: Bash (run code), WebSearch/WebFetch (gather evidence), mathematical reasoning, logical deduction, data analysis
• What the minimum viable test looks like -- the simplest thing that would produce a decisive signal

Experiment types:

• code -- write and run code to test the claim
• math-proof -- work through a formal or informal proof or derivation
• evidence-gathering -- search for empirical evidence, studies, or data online
• data-analysis -- fetch a dataset and analyze it
• logical-deduction -- reason from stated premises to a conclusion

Step 2: Design 1–3 experiments

Order experiments from simplest and most decisive to most involved:

• 1 experiment: one clean test is sufficient
• 2 experiments: two independent approaches cross-validate
• 3 experiments: the hypothesis is complex or no single test is decisive

Each experiment must specify:

• Title: concise name (5–8 words)
• Type: one of the types listed above
• Approach: precise narrative of what to do, described with enough detail that an agent can execute without making behavioral decisions
• Confirms if: specific result that would support the hypothesis
• Refutes if: specific result that would contradict the hypothesis
• Confidence: how decisive a positive/negative result would be (high / medium / low) -- be honest about limitations
• Publishability potential: whether this experiment, if successful, could yield a novel contribution relative to prior art in the ## Literature section (high / medium / low, with one sentence of rationale)

When multiple experiment types are equally valid, lean toward the type with higher publishability potential — but never sacrifice scientific validity for publishability.

Step 3: Write experiment designs into hypothesis file

Note: This skill only writes to the hypothesis file. It does not create files in experiments/ — artifact creation is exclusively handled by /running-experiments.

Append to the hypothesis file after the existing content:

## Experiments

### Experiment 1: <Title>

- **Type:** <type>
- **Approach:** <precise narrative of what to do>
- **Confirms if:** <expected result if the hypothesis is true>
- **Refutes if:** <expected result if the hypothesis is false>
- **Confidence:** <high | medium | low>
- **Publishability potential:** <high | medium | low — one sentence assessing whether a successful result would be novel relative to prior art in the Literature section>

#### Results
<!-- To be filled by running-experiments -->

---

### Experiment 2: <Title>
...

Use Edit to append -- never overwrite existing content.