linux-debugging

Inspect boot health on a systemd Linux system — failed units, slow services, kernel boot messages, and boot-to-boot comparison. Use when the user reports slow boot, services not starting, or wants to check post-reboot health. Targets Ubuntu/Debian + Wayland desktops but works on any systemd host.

Find past out-of-memory kill events in the journal — both kernel OOM killer invocations and userspace systemd-oomd / earlyoom kills — and reconstruct what was running at the time. Use when the user reports an unexplained freeze, lost work, or a process disappearing.

Collaboratively debug a misbehaving Linux desktop app or service in real time with the user. Two modes — live (tail logs while the user reproduces) and retrospective (something just happened, capture evidence before it gets buried). Produces a diagnosis and optionally a written-up bug report with attached log excerpts. Use when the user says "something won't launch", "let's debug this together", "X just crashed", "a bug just happened", or describes an issue they want to reproduce.

Open a forensic investigation into the most recent Linux crash, freeze, or unexpected reboot. Collects kernel dumps, previous-boot journal, userspace core dumps, and sar metrics, correlates them around the crash timestamp, and writes a findings report. Use when the user says "investigate the crash", "why did my system reboot", "open a crash investigation", or similar.

Run targeted journalctl queries to surface only the relevant log lines for a debugging task — by unit, time window, priority, boot, or kernel/user scope. Emits structured output (JSON or short-form) suited for AI analysis. Use when the user reports a problem and you need to inspect logs without dumping everything. Assumes systemd + journald (Ubuntu, Debian, Fedora, Arch, etc.).

Inspect non-journal log sources on a Linux desktop — Xorg/Wayland session logs, application logs in ~/.cache and ~/.local/state, /var/log/* leftovers, and dmesg — using AI-friendly tools (ripgrep, jq, structured queries) that produce focused, parseable output instead of raw log dumps. Use when journald doesn't have what you need or when an app writes its own log files.

Inspect /sys/fs/pstore for kernel oops and panic messages captured in firmware-backed storage (EFI vars, ACPI ERST, or ramoops) that survive a reboot. Use when the user says "check pstore", "any firmware-saved crash logs", or after a hard freeze on a system where kdump may not have worked.

Configure netconsole to stream kernel oops and panic messages over UDP to another host on the LAN, for capturing crashes on systems that hang before writing to disk. Use when the user says "set up netconsole", "stream kernel logs over the network", or is dealing with crashes that leave no local log trail.

Install and configure userspace OOM protection (systemd-oomd or earlyoom) so a desktop running out of memory kills the offending process before the kernel hard-locks the machine. Use when the user reports freezes under memory pressure, or proactively when setting up a new desktop. Ubuntu 22.04+ ships systemd-oomd by default; this skill verifies its config and falls back to earlyoom on older systems.

Verify that systemd-journald is configured for persistent storage so logs survive reboot — essential for diagnosing hard crashes and unexpected reboots. Configures it if not. Use proactively on any new desktop, and as a precondition before any post-crash investigation skill.