Convert a recipe video (TikTok, Instagram, YouTube, etc.) to Cooklang format
Extracts recipes from video URLs (TikTok, Instagram, YouTube) and converts them to Cooklang format with ingredient detection, transcription analysis, and automatic meal category sorting.
/plugin marketplace add keiththompson/video-to-cooklang/plugin install keiththompson-video-to-cooklang@keiththompson/video-to-cooklang<video-url>Convert the recipe video at: $ARGUMENTS
Supports TikTok, Instagram, YouTube, and any site supported by yt-dlp.
DO NOT web search for the recipe. If extraction fails, tell the user what went wrong and stop. Never search the web or guess a recipe from another source - the whole point is to capture THIS specific creator's version.
If any step fails:
brew install yt-dlp ffmpeg openai-whisper)Always start by clearing tmp files to avoid cross-contamination from previous recipes:
rm -rf /tmp/recipe_video.mp4 /tmp/recipe_audio.mp3 /tmp/recipe_audio.txt /tmp/recipe_frames
Use WebFetch to get basic metadata (title, creator). Note: Social media descriptions rarely contain full recipes - this is mainly for the recipe name and attribution.
Recipe videos contain instructions in BOTH visuals AND audio narration. Extract both:
# Download video (install with: brew install yt-dlp)
yt-dlp -o "/tmp/recipe_video.mp4" "$ARGUMENTS"
# Extract frames every 3 seconds (install with: brew install ffmpeg)
mkdir -p /tmp/recipe_frames
ffmpeg -i /tmp/recipe_video.mp4 -vf "fps=1/3" /tmp/recipe_frames/frame_%03d.jpg
# Extract and transcribe audio (install with: brew install openai-whisper)
ffmpeg -i /tmp/recipe_video.mp4 -vn -acodec mp3 /tmp/recipe_audio.mp3
whisper /tmp/recipe_audio.mp3 --model small --output_format txt --output_dir /tmp
From the transcript (/tmp/recipe_audio.txt):
From the frames:
The transcript is the primary source for accuracy - visuals confirm and supplement.
If audio transcription fails or is unclear, estimate from visuals:
If video download fails, ask the user to:
Convert the extracted recipe to Cooklang syntax:
---
source: [Video URL]
servings: [number if known]
---
@Chicken{} not @chicken{}large #Skillet{} not #large skillet{}Use @Ingredient{quantity%unit} format with optional prep instructions in parentheses:
@Chicken Breast{500%g}@Olive Oil{2%tbsp}@Garlic{3%cloves}(minced) - prep in parentheses@Onion{1}(diced)@Salt{} - no quantity = to taste-, ?, or & prefixes - they're not supported by Cooklang appsUse #Equipment{} format with adjectives BEFORE the keyword:
Use ~{time%unit} format with single values only (NO ranges):
~{5%minutes} - correct~{10-15%minutes} - WRONG, will breakUse == Section Name == format:
== Marinade ==== Assembly ==Based on the recipe content, classify into one of:
First, read the config file to get the Cooklang directory:
cat config.json # in the plugin directory, or ~/.claude/config.json
The cooklang_directory value is the base path. Save to:
[cooklang_directory]/[MealType]/[Recipe Name].cook
Filename rules:
Select the most appealing frame from the extracted frames (usually the final plated dish or a key cooking moment) and copy it with the same name as the recipe:
cp /tmp/recipe_frames/frame_XXX.jpg "[cooklang_directory]/[MealType]/[Recipe Name].jpg"
The image filename must match the recipe filename exactly (just with .jpg instead of .cook).
Remove temporary files to prevent cross-contamination with future runs:
rm -rf /tmp/recipe_video.mp4 /tmp/recipe_audio.mp3 /tmp/recipe_audio.txt /tmp/recipe_frames
After saving, tell the user:
For a garlic shrimp recipe saved to Dinner:
Created: Garlic Butter Shrimp.cook
Location: Dinner/Garlic Butter Shrimp.cook
Image: Dinner/Garlic Butter Shrimp.jpg
Category: Dinner (main protein dish)
The recipe includes 5 ingredients and takes about 15 minutes to prepare.
Source: [Video URL]