auto-research

AIRAS automated research (backend-LLM mode)

You drive the research; AIRAS provides retrieval, curated generation steps (run on its backend LLM), and execution infrastructure as MCP tools (server name: airas).

Prerequisites

• Credentials live in ~/.airas/credentials.json, re-read on every tool call. The easiest editor is the dashboard: run open_dashboard and open its API Keys page.
• This mode requires an LLM provider key (e.g. OPENAI_API_KEY) for the generation tools, and GH_PERSONAL_ACCESS_TOKEN for repository/experiment tools.

Flow

1. Discover: generate_research_queries → search_papers → retrieve_papers (structured study data).
2. Hypothesize & design: generate_hypothesis → generate_experimental_design (pass compute_environment so the design fits the hardware; retrieve_models / retrieve_datasets list curated candidates).
3. Set up the experiment repository: prepare_repository, then clone it locally with git.
4. Write the experiment code yourself in the clone. Read its AGENTS.md first — it defines the contract (allowed files, CLI shape, sanity / pilot / full modes, validation verdict lines, W&B conventions). For library-specific guidance (fine-tuning frameworks, distributed training, inference), get_library_docs returns each library's official docs and llms.txt endpoints — fetch those for current API usage instead of relying on memory. (The AI-Research-SKILLs library, which the template's code-generation workflows install on their runners, can also be installed locally: npx @orchestra-research/ai-research-skills.) Run mode=sanity locally until it prints SANITY_VALIDATION: PASS, then commit and push.
5. Run experiments: dispatch_experiment (async; backend="github_actions" or "aixs" with a compute_type). Poll get_workflow_runs (GitHub Actions) or get_experiment_run_status (either backend; returns stdout/stderr tails for debugging). Fix code locally and re-dispatch as needed.
6. Analyze: fetch_experiment_results → analyze_experiment (pass the experiment code from your clone as {"files": {"<path>": "<content>"}}).
7. Figures (see conventions below).
8. Write the paper: generate_bibfile → generate_paper → generate_latex. Write the returned LaTeX to .research/latex/{template}/main.tex in the clone and push with git.
9. Publish (two independent exits, use either or both): compile_latex builds the PDF on GitHub Actions; open_in_overleaf returns a link that creates an editable Overleaf project (pass local_path to export the local working tree without pushing).
10. Persist: upload_research_history saves the state; download_research_history restores it in a later session.

Figure conventions

• Result charts: build a Vega-Lite spec (data inline under data.values) and render_chart it to .research/results/chart/<name>.pdf in the clone. Rendering is fully local; no API keys.
• Method diagrams: write text notation (mermaid / graphviz / d2 / …) and render_diagram it to .research/results/diagram/<name>.pdf. Uses https://kroki.io by default; KROKI_BASE_URL switches to self-hosted.
• Keep filenames unique, commit and push. compile_latex and open_in_overleaf collect every PDF under .research/results/ into the paper's images/ (structure preserved) — reference them in LaTeX as images/<path>, e.g. images/chart/loss.pdf.

Notes

• Long-running tools return immediately; never block waiting. Poll between other work.
• All repository writes go through your local clone and git; there are no file-upload tools.