video-localization

Video Localization

Localize and dub videos into multiple languages using each::sense. This skill handles audio translation, voice cloning, lip synchronization, and subtitle generation for global content distribution.

Features

• AI Dubbing: Translate and dub video audio into any language
• Voice Cloning: Preserve speaker identity across languages
• Lip Sync: Synchronize lip movements with translated audio
• Subtitle Generation: Auto-generate subtitles in source or target languages
• Multi-Language Versions: Create localized versions for multiple markets
• Marketing Localization: Adapt ad content for regional audiences
• E-Learning Translation: Translate training and educational content
• YouTube Localization: Prepare videos for international audiences

Quick Start

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this English video into Spanish with lip sync"}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/my-video.mp4"]
  }'

Supported Languages

Language	Code	Dubbing	Subtitles	Voice Cloning
English	en	Yes	Yes	Yes
Spanish	es	Yes	Yes	Yes
French	fr	Yes	Yes	Yes
German	de	Yes	Yes	Yes
Italian	it	Yes	Yes	Yes
Portuguese	pt	Yes	Yes	Yes
Chinese	zh	Yes	Yes	Yes
Japanese	ja	Yes	Yes	Yes
Korean	ko	Yes	Yes	Yes
Hindi	hi	Yes	Yes	Yes
Arabic	ar	Yes	Yes	Yes

Use Case Examples

1. English to Spanish Dubbing

Basic dubbing with natural voice synthesis.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate and dub this English video into Spanish. Use a natural-sounding voice that matches the original speaker tone and energy."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/english-video.mp4"]
  }'

2. Video Translation with Lip Sync

Full localization with lip movement synchronization for professional results.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this video from English to French with full lip sync. The lip movements should match the French audio perfectly. Maintain the original background music and sound effects."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/presentation-video.mp4"]
  }'

3. Subtitle Generation

Generate accurate subtitles in source or translated languages.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Generate subtitles for this video. First transcribe the English audio, then create Spanish subtitles. Output both SRT files and a video with burned-in Spanish subtitles."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/interview.mp4"]
  }'

4. Multi-Language Video Versions

Create localized versions for multiple markets in a single workflow.

# Start multi-language localization session
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "I need to localize this product video into Spanish, French, and German. Start with Spanish dubbing with lip sync."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001",
    "video_urls": ["https://example.com/product-demo.mp4"]
  }'

# Continue with French version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Now create the French version with the same quality settings."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001"
  }'

# Continue with German version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Finally, create the German version."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "session_id": "localization-project-001"
  }'

5. Voice Cloning for Dubbing

Preserve the original speaker's voice characteristics in the dubbed version.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this video from English to Japanese using voice cloning. Clone the original speaker voice so they sound like themselves speaking Japanese. Maintain their unique voice characteristics, tone, and speaking style."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/ceo-message.mp4"]
  }'

6. Marketing Video Localization

Adapt marketing content for regional audiences with cultural considerations.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this marketing video for the Latin American market. Translate to Spanish with a neutral Latin American accent, not Spain Spanish. Apply lip sync and ensure the messaging resonates with the target audience. Keep the energy and excitement of the original."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/brand-ad.mp4"]
  }'

7. Training Video Translation

Translate corporate training and educational content.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this employee training video from English to Mandarin Chinese. Use clear, professional narration suitable for corporate training. Include both dubbed audio and Chinese subtitles for accessibility. Technical terms should be accurately translated."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/onboarding-training.mp4"]
  }'

8. YouTube Video Localization

Prepare content for international YouTube audiences.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this YouTube video for Portuguese-speaking audiences (Brazilian Portuguese). Dub the narration with lip sync, generate Portuguese subtitles in SRT format, and create a localized thumbnail with Portuguese text overlay."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/youtube-tutorial.mp4"]
  }'

9. Ad Creative Localization

Adapt advertising content for different regional markets.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Localize this 30-second ad for the German market. Translate and dub to German with lip sync. Ensure the call-to-action is culturally appropriate. Keep the same pacing and emotional impact as the original English version."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/product-ad-30s.mp4"]
  }'

10. E-Learning Content Translation

Translate educational courses and learning materials.

curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Translate this e-learning module from English to Korean. The instructor should be dubbed with voice cloning to maintain authenticity. Generate Korean subtitles and ensure all on-screen text references are noted for manual localization. Educational terminology must be accurately translated."}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "max",
    "video_urls": ["https://example.com/course-module-1.mp4"]
  }'

Best Practices

Video Quality

• Source Quality: Use high-quality source videos (1080p or higher recommended)
• Clear Audio: Ensure source audio is clear with minimal background noise
• Single Speaker: Best results with single-speaker content; multi-speaker requires more processing

Dubbing Tips

• Natural Pacing: AI adjusts speech rate to match lip movements
• Preserve Emotion: Original emotional tone is maintained in translations
• Background Audio: Original music and sound effects are preserved

Lip Sync Optimization

• Face Visibility: Ensure speaker's face is clearly visible
• Consistent Lighting: Well-lit faces produce better lip sync results
• Front-Facing: Best results with front-facing or slight angle shots

Subtitle Best Practices

• Reading Speed: Subtitles are timed for comfortable reading
• Line Breaks: Proper segmentation for readability
• Timing Sync: Accurate sync with audio and scene changes

Mode Selection

Choose the appropriate mode based on your needs:

Mode	Best For	Speed	Quality
max	Final production, client deliverables, broadcast	Slower	Highest quality lip sync and voice
eco	Drafts, internal review, quick previews	Faster	Good quality, faster turnaround

Recommendation: Use eco mode for initial reviews and max mode for final deliverables.

Multi-Turn Localization Workflow

Use session_id to iterate on localization projects:

# Initial localization request
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Dub this video into Spanish with lip sync"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz",
    "video_urls": ["https://example.com/source-video.mp4"]
  }'

# Request adjustments based on review
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "The voice sounds too formal. Can you make it more conversational and energetic?"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz"
  }'

# Add subtitles to the approved version
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Add Spanish subtitles to this dubbed version"}],
    "model": "eachsense/beta",
    "stream": true,
    "session_id": "dubbing-project-xyz"
  }'

Batch Localization

Process multiple videos or languages efficiently:

# Use eco mode for batch processing drafts
curl -X POST https://eachsense-agent.core.eachlabs.run/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $EACHLABS_API_KEY" \
  -H "Accept: text/event-stream" \
  -d '{
    "messages": [{"role": "user", "content": "Create Spanish dubbed versions of these 3 marketing videos with subtitles"}],
    "model": "eachsense/beta",
    "stream": true,
    "mode": "eco",
    "video_urls": [
      "https://example.com/video1.mp4",
      "https://example.com/video2.mp4",
      "https://example.com/video3.mp4"
    ]
  }'

Error Handling

Error	Cause	Solution
Failed to create prediction: HTTP 422	Insufficient balance	Top up at eachlabs.ai
Video too long	Source video exceeds limits	Split into shorter segments
Audio not detected	No speech in video	Ensure video contains speech audio
Face not detected	No visible face for lip sync	Use videos with visible speakers
Timeout	Complex generation	Set client timeout to minimum 10 minutes

Technical Notes

• Client Timeout: Set HTTP client timeout to minimum 10 minutes for video processing
• Video Length: Optimal length is under 10 minutes per segment for best results
• Output Format: Generated videos are MP4 with H.264 encoding
• Subtitle Format: SRT files are provided for subtitle outputs

Related Skills

• each-sense - Core API documentation
• video-generation - Create original video content
• video-edit - Edit and enhance videos
• voice-audio - Voice synthesis and audio generation