Help us improve
Share bugs, ideas, or general feedback.
From twelvelabs
Asynchronously analyzes up to 2-hour videos by URL, file upload, or asset ID using Pegasus 1.5. Supports time-based metadata extraction, multimodal reference images and structured JSON output.
npx claudepluginhub twelvelabs-io/twelve-labs-claude-code-plugin --plugin twelvelabsHow this skill is triggered — by the user, by Claude, or both
Slash command
/twelvelabs:async-analyzeThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Asynchronously analyse a video using Pegasus 1.5 (default). Accepts a direct http(s) URL, a local file path (auto-uploaded as an asset), a previously uploaded asset ID, or base64 data — no prior indexing required. Handles videos up to 2 hours. Returns a task ID immediately; poll with `get-analyse-task` for the result.
Analyzes a video synchronously using TwelveLabs AI to return a summary or answer questions about its content. Accepts video URLs, file paths, asset IDs, or indexed video IDs.
Analyzes video files or YouTube URLs: extracts frames/audio, detects scenes/motion/silence/transitions via ffmpeg tools with structured workflow.
Imports, searches, and analyzes videos from YouTube, TikTok, Instagram using Memories.ai LVMM for persistent intelligence, summarization, knowledge bases, and social trends research.
Share bugs, ideas, or general feedback.
Asynchronously analyse a video using Pegasus 1.5 (default). Accepts a direct http(s) URL, a local file path (auto-uploaded as an asset), a previously uploaded asset ID, or base64 data — no prior indexing required. Handles videos up to 2 hours. Returns a task ID immediately; poll with get-analyse-task for the result.
Use this skill when the user:
promptV2 + mediaSources)jsonSchema)For short, prompt-only analysis of an already-indexed video, use sync-analyze instead — it returns inline.
pegasus1.5 for everything. Only set pegasus1.2 when the user explicitly asks for legacy mode.
http(s)://... URL → videoUrlasset_* → assetIdcreate-asset:
Tool: mcp__twelvelabs-mcp__create-asset
Parameters:
file: "<absolute-path>"
Surface the returned assetId to the user — they can reuse or delete it via /twelvelabs:assets. Then continue as assetId.If a URL is given that's a YouTube/Drive/Dropbox share link, refuse — the MCP only accepts direct http(s) links to raw media.
Use the user's question/instruction as prompt. Default:
"Provide a comprehensive summary of this video including key topics, main points, and important moments."
Exactly one source (videoUrl / assetId / base64Video), modelName: "pegasus1.5", plus prompt:
Tool: mcp__twelvelabs-mcp__async-analyse-video
Parameters:
<videoUrl | assetId | base64Video>: "..."
modelName: "pegasus1.5"
prompt: "..."
Add these when the user's intent matches.
Time-based metadata extraction — the user wants structured per-segment output:
analysisMode: "time_based_metadata"
segmentDefinitions:
- id: "highlight"
description: "A notable moment in the video"
fields:
- { name: "summary", type: "string", description: "What happened" }
- { name: "importance", type: "integer", description: "1-5" }
# Optional: mediaSources (up to 4 reference images per definition)
# Optional: timeRanges (restrict extraction to specific [start,end] windows)
# Optional: minSegmentDuration / maxSegmentDuration (auto-segmenting bounds)
Don't set prompt or promptV2 in this mode — the per-segment description + fields drive the output. Pegasus 1.5 only.
Structured JSON output (general mode) — the user wants JSON, not per-segment:
jsonSchema: { ...Draft 2020-12 schema... }
Mutually exclusive with segmentDefinitions.
Multimodal prompting with reference images — the user wants the model to compare the video against images:
promptV2:
inputText: "Find every shot that looks like <@product> with the lighting of <@light-ref>."
mediaSources:
- { name: "product", mediaType: "image", url: "https://..." } # or assetId / base64String
- { name: "light-ref", mediaType: "image", assetId: "asset_..." }
Mutually exclusive with prompt. Requires Pegasus 1.5.
Clip windowing — restrict analysis to a sub-range (Pegasus 1.5 only):
startTime: 60
endTime: 180
end - start ≥ 4. Mutually exclusive with segmentDefinition.timeRanges.
Other knobs:
temperature (0-1, default 0.2)maxTokens (Pegasus 1.5: 2048-32768, default 32768)customId (1-64 chars, alphanumeric+-_; round-trips through get-analyse-task / list-analyse-tasks — useful for caller-side correlation)Async analysis started!
Task ID: <task_id>
Model: pegasus1.5
<customId surfaced if set>
Polling with: /twelvelabs:async-analyze status <task_id>
(typical wall time: 30s-several minutes; 2-hour videos can take ~10 min)
User asks "is my analysis done?" / "check status of X":
Tool: mcp__twelvelabs-mcp__get-analyse-task
Parameters:
taskId: "<task-id>"
The MCP returns plain text "Task is READY." with the result below when done, "Task is FAILED." with error info on failure, or "Task is PROCESSING/QUEUED/PENDING." with a "poll again in 5-10s" hint while running. Watch for the substring "is READY"/"is FAILED" to decide; don't try to JSON-parse the response.
If the task is ready and the result.data is JSON (from jsonSchema) or a per-segment object map (from time_based_metadata), the response carries it as a string — parse if needed for display.
If finishReason=length is present, warn the user that the output may be truncated and that any JSON output might fail to parse.
The MCP list-analyse-tasks tool supports:
page (default 1)pageLimit (max 50, default 10)status — queued | pending | processing | ready | failedvideoUrl — exact match on the source URLassetId — exact match on the source asset IDanalysisMode — general | time_based_metadataExamples:
status: "failed"videoUrl: "X", analysisMode: "time_based_metadata"pageLimit: 50Default (no filter): newest first, 10 per page.
Tool: mcp__twelvelabs-mcp__delete-analyse-task
Parameters:
taskId: "<task-id>"
The MCP returns a 409 if the task is still processing — surface that as "still processing, wait for it to finish before deleting".
"Analyze https://nasa.gov/video.mp4 and tell me what's shown"
→ async-analyse-video with videoUrl + default prompt → poll status → report result.
"Analyze /Users/me/clip.mp4"
→ create-asset with file: ... → async-analyse-video with the new assetId + Pegasus 1.5 → poll → report. Surface the asset ID for reuse.
"For this 90-minute keynote, extract every product mention with timestamp"
→ async-analyse-video with analysisMode: "time_based_metadata", segmentDefinitions: [{ id: "product-mention", description: "...", fields: [{name: "product_name", type: "string"}, {name: "context", type: "string"}] }].
"Find every shot of <@anchor1> giving a presentation"
→ Resolve the entity ID via the entities skill (or assume the user already has it) → async-analyse-video with promptV2: { inputText: "<@anchor1> ...", mediaSources: [reference images] }.
"List my failed async analyses"
→ list-analyse-tasks with status: "failed".
get-analyse-task.