paper-to-skill

Paper-to-Skill Pipeline

Transform research papers into production-grade skill packages. The pipeline extracts the actionable methodology from a paper, structures it as a skill specification, and feeds it through co-evolutionary refinement to produce a validated package.

This closes the loop between research and practice: a paper published today can become an executable skill tomorrow, without manual authoring.

Reference Files

File	Contents	Load When
references/extraction-patterns.md	Patterns for extracting methodology from papers	Always

Prerequisites

• The to-markdown skill (for PDF/document conversion)
• The research-critique skill (for paper analysis)
• The test-engineer agent (for co-evolutionary skill generation)

Workflow

Phase 1: Paper Intake

Accept the paper in any supported format:

Input Format	Action
arXiv ID (e.g., 2604.01687)	Fetch via https://arxiv.org/abs/<id>, convert PDF
arXiv URL	Extract ID, fetch and convert
PDF file path	Convert using to-markdown skill
URL to paper	Fetch via WebFetch, convert if PDF
Pasted text	Use directly

For PDF conversion, invoke the to-markdown skill:

Convert this PDF to clean markdown, preserving section structure, tables, equations, and algorithm pseudocode. Drop references section but keep inline citations.

Phase 2: Critical Analysis

Invoke the research-critique skill on the converted paper:

Analyze this paper focusing on:

1. Core contribution: what is the novel methodology?
2. Algorithm description: extract the step-by-step procedure
3. Input/output specification: what goes in, what comes out?
4. Key parameters and their valid ranges
5. Claimed results and the evidence supporting them
6. Failure modes and limitations acknowledged by the authors
7. Prerequisites and dependencies (tools, data, compute)

The critique output becomes the foundation for the skill specification.

Phase 3: Skill Specification Extraction

From the critique output, build a structured skill specification:

specification:
  name: <kebab-case derived from paper's methodology name>
  domain: <paper's application domain>
  source_paper:
    title: <paper title>
    arxiv_id: <if available>
    url: <paper URL>
    authors: <first author et al.>
    date: <publication date>
  
  capabilities:
    - <capability 1 derived from the methodology>
    - <capability 2>
    - <capability 3>
  
  input_format: <what the skill accepts>
  output_format: <what the skill produces>
  
  algorithm_steps:
    - step: 1
      description: <from paper's algorithm>
      parameters: [<key params with ranges>]
    - step: 2
      description: <next step>
  
  failure_modes:
    - <from paper's limitations section>
  
  example_tasks:
    - <task 1 the methodology would solve>
    - <task 2>
    - <task 3>

Extraction rules:

• Prefer the paper's own algorithm pseudocode over prose descriptions
• Include parameter ranges from the paper's experiments (e.g., "learning rate: 0.001-0.01")
• Map the paper's terminology to armory conventions (e.g., "module" → "skill", "pipeline" → "workflow")
• If the paper describes multiple variants, extract the best-performing one

See references/extraction-patterns.md for patterns specific to common paper types.

Phase 4: Skill Generation

Hand off the specification to the test-engineer agent for co-evolutionary generation:

Evolve a skill for: [specification.domain]

Capabilities: [specification.capabilities] Algorithm: [specification.algorithm_steps] Input: [specification.input_format] Output: [specification.output_format] Failure modes: [specification.failure_modes] Example tasks: [specification.example_tasks]

Source: [specification.source_paper.title] ([specification.source_paper.url])

The test-engineer runs its full co-evolutionary loop (generate → verify → oracle → refine) using the specification as the task description.

Phase 5: Attribution and Finalization

Ensure the generated skill properly attributes the source paper:

1. Frontmatter: Add source: <paper_url> to the metadata
2. Body: Include an attribution section at the end of SKILL.md:

   ## Attribution
   
   This skill implements the methodology from:
   > <paper title>
   > <authors>
   > <venue/arxiv, date>
   > <URL>
   

3. References: If the paper has supplementary materials (code, datasets), create a source materials reference file in the generated skill's references/ directory linking to them
4. Verify the skill name does not conflict with existing packages in manifest.yaml

Output

The complete skill package at skills/<name>/:

• SKILL.md with attribution and paper-derived workflow
• evals/cases.yaml with assertions generated by the co-evolutionary loop
• references/ with extraction patterns and source materials
• evals/evolution-log.yaml from the test-engineer's refinement process

Error Handling

Error	Resolution
Paper has no clear algorithm	Extract the methodology from the experiments section
Paper is purely theoretical	Report: no actionable methodology; suggest literature-review instead
PDF conversion fails	Try alternative: fetch HTML version or request user paste text
Paper methodology requires data/compute	Note in skill's prerequisites; skill may be a workflow template only
test-engineer budget exhausted	Return best-scoring iteration with manual review warning

Limitations

• Cannot extract visual methodologies (circuit diagrams, neural architecture figures) — works on textual algorithm descriptions only
• Papers with multiple interdependent contributions may produce overly complex skills — consider splitting into multiple skills
• Non-English papers require translation before processing
• The generated skill's quality depends on the paper's clarity of methodology description