infra-hydra-config

Infra · Hydra config

curryTrain's configuration layer is Hydra + OmegaConf. The choice is opinionated — Hydra is mature, the Optuna sweeper is first-class, and the defaults-list pattern composes well across stages.

Recommended layout

project/
├── configs/
│   ├── config.yaml             # entry point; only `defaults:` and globals
│   ├── model/
│   │   ├── default.yaml
│   │   ├── llama_lite.yaml
│   │   └── csla_snn.yaml
│   ├── data/
│   │   ├── default.yaml
│   │   └── wmt14.yaml
│   ├── training/
│   │   ├── default.yaml        # lr, schedule, accumulation, weight_decay
│   │   └── long_run.yaml
│   ├── parallelism/
│   │   ├── single_gpu.yaml
│   │   ├── single_node_8gpu.yaml
│   │   └── multi_node.yaml
│   ├── logging/
│   │   ├── tb_only.yaml
│   │   └── tb_plus_wandb.yaml
│   └── search/
│       └── lr_wd.yaml          # Optuna sweep configs (Stage 4)

config.yaml entry point

defaults:
  - _self_
  - model: llama_lite
  - data: wmt14
  - training: default
  - parallelism: single_node_8gpu
  - logging: tb_only

experiment:
  name: ???                     # required; user must override on CLI
  seed: 42

paths:
  runs: ${oc.env:CURRYTRAIN_RUNS_DIR, runs}
  checkpoints: ${paths.runs}/${experiment.name}/checkpoints
  journal: ${paths.runs}/${experiment.name}/journal

hydra:
  run:
    dir: ${paths.runs}/${experiment.name}/${now:%Y-%m-%d_%H-%M-%S}
  sweep:
    dir: ${paths.runs}/${experiment.name}/sweep
    subdir: ${hydra.job.num}

CLI override patterns

# Pick a different model
python train.py model=csla_snn experiment.name=snn-v1

# Override a single field
python train.py training.lr=3e-4

# Multi-run sweep (built-in)
python train.py -m training.lr=1e-4,3e-4,1e-3

# Hydra-Optuna sweep
python train.py -m hydra/sweeper=optuna experiment.name=optuna-lr search=lr_wd

Structured configs (validation)

For type safety, use Hydra's structured configs:

from dataclasses import dataclass, field
from hydra.core.config_store import ConfigStore

@dataclass
class TrainingConfig:
    lr: float = 3e-4
    weight_decay: float = 0.1
    warmup_steps: int = 1000
    total_steps: int = 100_000
    gradient_accumulation_steps: int = 1

@dataclass
class CurryTrainConfig:
    experiment: ExperimentConfig
    model: ModelConfig
    data: DataConfig
    training: TrainingConfig = field(default_factory=TrainingConfig)
    parallelism: ParallelismConfig = field(default_factory=ParallelismConfig)
    logging: LoggingConfig = field(default_factory=LoggingConfig)
    paths: PathsConfig = field(default_factory=PathsConfig)

cs = ConfigStore.instance()
cs.store(name="curry_train", node=CurryTrainConfig)

The structured config catches typos at config-load time, not at training-loop time.

Procedure when assisting a user

1. If the user has no configs/ directory, set up the layout above. Don't paste 400 lines of defaults; start minimal and grow.

2. Use defaults: with _self_ first to allow config.yaml to override sub-config fields. Hydra evaluates in order; _self_ first means the entry-point overrides win.

3. For sweeps, point them at infra-optuna-sweep for the search-config skill.

4. For tracking, the logging group selects the backend; see infra-tracking-backend.

5. Avoid:

   • Putting the entire training-loop code path in _target_ instantiate. Heavy use of _target_ is a "lightning-hydra-template" anti-pattern that makes debugging hell.
   • Deep config inheritance (4+ layers). Hard to trace; refactor flat.

Boundaries

• Hydra has a learning curve. For users who don't already use it, suggest sticking with the curryTrain-provided template rather than trying to teach Hydra from first principles.
• For multi-config sweeps, Hydra's job-array layout is runs/<exp>/sweep/0/, 1/, etc. — point users at this so they don't try to override the dir naming.

Common mistakes

• Using ${...} interpolation that references missing keys → cryptic errors at runtime. Add structured configs to catch these earlier.
• Forgetting _self_ in defaults → entry-point overrides don't take effect.
• Mixing instantiate() for everything → see anti-pattern in subagent B's findings.

Related

• skills/infra-optuna-sweep — Hydra's Optuna sweeper integration.
• skills/infra-tracking-backend — selects logging backend via logging config group.
• skills/infra-fabric-launch — Lightning Fabric in the launch script.
• Hydra docs: hydra.cc.