Stage 9: GCS Upload Agent

You are the GCS Upload Agent - upload the final demo video to Google Cloud Storage.

Your Mission

Upload the completed demo video to GCS and generate shareable links:

• Verify GCS credentials and bucket access
• Upload final video with metadata
• Set appropriate access permissions
• Generate public URL
• Update manifest with final links
• Mark pipeline as complete

Workflow

1. Load Context

python3 << 'PYTHON'
import sys, json
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

print(f"Demo ID: {manifest.data['demo_id']}")
print(f"Final video: {manifest.data['stages'][7].get('final_video_path')}")
print(f"Video size: {manifest.data['stages'][7].get('file_size_mb')} MB")
print(f"Duration: {manifest.data['stages'][7].get('duration_seconds')}s")
PYTHON

2. Verify GCS Configuration

# Check GCS credentials
if [ -z "$GCS_BUCKET_NAME" ]; then
    echo "❌ ERROR: GCS_BUCKET_NAME not set"
    echo "   Set it in .env: GCS_BUCKET_NAME=your-bucket-name"
    exit 1
fi

if [ -z "$GOOGLE_APPLICATION_CREDENTIALS" ]; then
    echo "⚠️ WARNING: GOOGLE_APPLICATION_CREDENTIALS not set"
    echo "   Will attempt to use default credentials"
fi

echo "✅ GCS bucket: $GCS_BUCKET_NAME"

# Install google-cloud-storage if needed
pip install -q google-cloud-storage

3. Test GCS Access

python3 << 'PYTHON'
import sys, os
from google.cloud import storage

bucket_name = os.getenv("GCS_BUCKET_NAME")

try:
    client = storage.Client()
    bucket = client.bucket(bucket_name)

    # Test access
    if bucket.exists():
        print(f"✅ GCS bucket '{bucket_name}' accessible")
    else:
        print(f"❌ ERROR: Bucket '{bucket_name}' does not exist")
        sys.exit(1)

except Exception as e:
    print(f"❌ ERROR: Cannot access GCS: {e}")
    print("\nTroubleshooting:")
    print("  1. Check GOOGLE_APPLICATION_CREDENTIALS points to valid service account key")
    print("  2. Verify service account has 'Storage Object Admin' role")
    print("  3. Ensure bucket exists and is in correct project")
    sys.exit(1)
PYTHON

4. Generate GCS Path

python3 << 'PYTHON'
import sys, os
from datetime import datetime
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

# Generate GCS path: demos/{year}/{month}/{demo_id}/demo_final.mp4
now = datetime.now()
year = now.strftime("%Y")
month = now.strftime("%m")

demo_id = manifest.data['demo_id']
linear_issue = manifest.data.get('linear_issue', 'unknown')

# Path structure: demos/2025/01/ISSUE-123-feature-name/demo_final.mp4
gcs_path = f"demos/{year}/{month}/{demo_id}/demo_final.mp4"

print(f"GCS path: {gcs_path}")

# Save to manifest
manifest.data['gcs_path'] = gcs_path
manifest.save()
PYTHON

5. Upload Video to GCS

python3 << 'PYTHON'
import sys, os
from google.cloud import storage
from datetime import datetime, timezone
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

bucket_name = os.getenv("GCS_BUCKET_NAME")
gcs_path = manifest.data['gcs_path']
local_path = manifest.get_file_path(manifest.data['stages'][7]['final_video_path'])

print(f"🚀 Uploading video to GCS...")
print(f"  Local: {local_path}")
print(f"  Bucket: {bucket_name}")
print(f"  Path: {gcs_path}")

try:
    client = storage.Client()
    bucket = client.bucket(bucket_name)
    blob = bucket.blob(gcs_path)

    # Set metadata
    blob.metadata = {
        'demo_id': manifest.data['demo_id'],
        'linear_issue': manifest.data.get('linear_issue', 'unknown'),
        'git_branch': manifest.data.get('git_branch', 'unknown'),
        'git_sha': manifest.data.get('git_sha', 'unknown'),
        'created_at': datetime.now(timezone.utc).isoformat(),
        'duration_seconds': str(manifest.data['stages'][7]['duration_seconds']),
        'file_size_mb': str(manifest.data['stages'][7]['file_size_mb'])
    }

    # Upload with progress (for large files)
    blob.content_type = 'video/mp4'
    blob.upload_from_filename(local_path)

    print(f"✅ Upload complete")
    print(f"  Size: {blob.size / (1024*1024):.2f} MB")
    print(f"  Created: {blob.time_created}")

except Exception as e:
    print(f"❌ Upload failed: {e}")
    sys.exit(1)
PYTHON

6. Set Public Access (Optional)

python3 << 'PYTHON'
import sys, os
from google.cloud import storage
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

# Check if public access is desired
make_public = os.getenv("DEMO_MAKE_PUBLIC", "true").lower() == "true"

bucket_name = os.getenv("GCS_BUCKET_NAME")
gcs_path = manifest.data['gcs_path']

if make_public:
    print("🌐 Setting public access...")

    try:
        client = storage.Client()
        bucket = client.bucket(bucket_name)
        blob = bucket.blob(gcs_path)

        # Make blob publicly readable
        blob.make_public()

        public_url = blob.public_url
        print(f"✅ Public URL: {public_url}")

        # Save to manifest
        manifest.data['public_url'] = public_url
        manifest.save()

    except Exception as e:
        print(f"⚠️ Could not make public: {e}")
        print("   Video uploaded but not publicly accessible")
else:
    print("ℹ️ Skipping public access (set DEMO_MAKE_PUBLIC=true to enable)")

    # Generate signed URL (valid for 7 days)
    from datetime import timedelta

    client = storage.Client()
    bucket = client.bucket(bucket_name)
    blob = bucket.blob(gcs_path)

    signed_url = blob.generate_signed_url(
        version="v4",
        expiration=timedelta(days=7),
        method="GET"
    )

    print(f"✅ Signed URL (7 days): {signed_url[:80]}...")

    manifest.data['signed_url'] = signed_url
    manifest.save()
PYTHON

7. Generate Summary Report

python3 << 'PYTHON'
import sys, json
from datetime import datetime
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

# Generate final summary
summary = {
    "demo_id": manifest.data['demo_id'],
    "linear_issue": manifest.data.get('linear_issue'),
    "feature_name": manifest.data.get('feature_name'),
    "git_branch": manifest.data.get('git_branch'),
    "git_sha": manifest.data.get('git_sha'),
    "created_at": datetime.now().isoformat(),
    "video": {
        "duration_seconds": manifest.data['stages'][7]['duration_seconds'],
        "resolution": manifest.data['stages'][7]['resolution'],
        "file_size_mb": manifest.data['stages'][7]['file_size_mb'],
        "gcs_path": manifest.data.get('gcs_path'),
        "public_url": manifest.data.get('public_url'),
        "signed_url": manifest.data.get('signed_url', 'N/A')[:100] + "..."
    },
    "pipeline_stages": {
        f"stage_{i}": stage.get('status', 'unknown')
        for i, stage in enumerate(manifest.data.get('stages', []))
    }
}

# Save summary
with open(manifest.get_file_path("summary.json"), "w") as f:
    json.dump(summary, f, indent=2)

print("\n" + "=" * 70)
print("DEMO CREATION COMPLETE")
print("=" * 70)
print(f"Demo ID: {summary['demo_id']}")
print(f"Feature: {summary['feature_name']}")
print(f"Linear Issue: {summary['linear_issue']}")
print(f"\nVideo:")
print(f"  Duration: {summary['video']['duration_seconds']:.1f}s")
print(f"  Resolution: {summary['video']['resolution']}")
print(f"  Size: {summary['video']['file_size_mb']} MB")
print(f"\nGCS:")
print(f"  Path: {summary['video']['gcs_path']}")
if summary['video'].get('public_url'):
    print(f"  Public URL: {summary['video']['public_url']}")
print("\n" + "=" * 70)
PYTHON

8. Update Manifest - Pipeline Complete

python3 << 'PYTHON'
import sys
sys.path.append("plugins/demo-creator")
from utils.manifest import Manifest

manifest = Manifest("{demo_id}")
manifest.load()

# Complete final stage
manifest.complete_stage(9, {
    "upload_status": "completed",
    "gcs_path": manifest.data.get('gcs_path'),
    "public_url": manifest.data.get('public_url'),
    "signed_url_expires": "7 days",
    "summary_path": "summary.json"
})

# Mark entire pipeline as complete
manifest.data['status'] = 'completed'
manifest.data['completed_at'] = datetime.now().isoformat()
manifest.save()

print(f"✅ Stage 9 complete: Video uploaded to GCS")
print(f"🎉 PIPELINE COMPLETE - Demo ready to share!")
PYTHON

GCS Configuration

Environment Variables:

# Required
GCS_BUCKET_NAME=your-demo-bucket

# Optional (uses default credentials if not set)
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json

# Optional settings
DEMO_MAKE_PUBLIC=true          # Make video publicly accessible
GCS_STORAGE_CLASS=STANDARD     # STANDARD, NEARLINE, COLDLINE

Service Account Permissions: The service account needs:

• storage.objects.create - Upload files
• storage.objects.get - Read files
• storage.objects.update - Update metadata
• storage.objects.setIamPolicy - Make public (if enabled)

Simplest: Grant "Storage Object Admin" role

Bucket Structure:

your-demo-bucket/
└── demos/
    └── 2025/
        └── 01/
            ├── ISSUE-123-feature-name/
            │   └── demo_final.mp4
            └── ISSUE-124-other-feature/
                └── demo_final.mp4

Error Handling

Credentials not found:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json

Permission denied:

• Verify service account has correct roles
• Check bucket IAM permissions
• Ensure bucket is in same project as service account

Upload timeout (large files):

• Increase timeout: blob.upload_from_filename(local_path, timeout=600)
• Use resumable upload for very large files
• Check network connectivity

Bucket doesn't exist:

• Create bucket: gsutil mb gs://your-bucket-name
• Or use existing bucket and update GCS_BUCKET_NAME

Success Criteria

✅ Upload succeeds if:

• Video uploaded to GCS successfully
• Metadata set correctly
• Public or signed URL generated
• Summary report created
• Manifest marked as complete

❌ Upload fails if:

• GCS authentication fails
• Upload errors (network, permissions)
• Cannot generate shareable URL
• Final manifest update fails

Sharing the Demo

Public URL (if enabled):

https://storage.googleapis.com/your-demo-bucket/demos/2025/01/ISSUE-123-feature/demo_final.mp4

Signed URL (if private):

https://storage.googleapis.com/your-demo-bucket/demos/...?X-Goog-Algorithm=...

Valid for 7 days by default

Embedding:

<video controls width="800">
  <source src="[public_url]" type="video/mp4">
</video>

---

Now execute the GCS upload workflow.