Groq Deploy Integration

Overview

Deploy applications powered by Groq's ultra-fast LLM inference API (api.groq.com). Groq's sub-second latency makes it ideal for real-time applications.

Prerequisites

• Groq API key stored in GROQ_API_KEY environment variable
• Application using groq-sdk package
• Platform CLI installed (vercel, docker, or gcloud)

Instructions

Step 1: Configure Secrets

# Vercel (Edge-compatible)
vercel env add GROQ_API_KEY production

# Cloud Run
echo -n "your-key" | gcloud secrets create groq-api-key --data-file=-

Step 2: Vercel Edge Deployment

// api/chat.ts - Ultra-low latency with Groq + Vercel Edge
import Groq from "groq-sdk";

export const config = { runtime: "edge" };

export default async function handler(req: Request) {
  const groq = new Groq({ apiKey: process.env.GROQ_API_KEY! });
  const { messages, stream } = await req.json();

  if (stream) {
    const completion = await groq.chat.completions.create({
      model: "llama-3.3-70b-versatile",
      messages,
      stream: true,
    });

    const encoder = new TextEncoder();
    const readable = new ReadableStream({
      async start(controller) {
        for await (const chunk of completion) {
          const content = chunk.choices[0]?.delta?.content;
          if (content) {
            controller.enqueue(encoder.encode(`data: ${JSON.stringify({ content })}\n\n`));
          }
        }
        controller.enqueue(encoder.encode("data: [DONE]\n\n"));
        controller.close();
      },
    });

    return new Response(readable, {
      headers: { "Content-Type": "text/event-stream" },
    });
  }

  const completion = await groq.chat.completions.create({
    model: "llama-3.3-70b-versatile",
    messages,
  });

  return Response.json(completion);
}

Step 3: Docker Deployment

FROM node:20-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
EXPOSE 3000  # 3000: 3 seconds in ms
CMD ["node", "dist/index.js"]

Step 4: Cloud Run with Streaming

gcloud run deploy groq-api \
  --source . \
  --region us-central1 \
  --set-secrets=GROQ_API_KEY=groq-api-key:latest \
  --set-env-vars=GROQ_MODEL=llama-3.3-70b-versatile \
  --min-instances=1 \
  --cpu=1 --memory=512Mi

Step 5: Health Check

export async function GET() {
  try {
    const groq = new Groq({ apiKey: process.env.GROQ_API_KEY! });
    await groq.chat.completions.create({
      model: "llama-3.1-8b-instant",
      messages: [{ role: "user", content: "ping" }],
      max_tokens: 1,
    });
    return Response.json({ status: "healthy" });
  } catch {
    return Response.json({ status: "unhealthy" }, { status: 503 });  # HTTP 503 Service Unavailable
  }
}

Error Handling

Issue	Cause	Solution
Rate limited (429)	Too many requests	Implement request queuing
Model unavailable	Capacity constraint	Fall back to smaller model
Edge timeout	Long completion	Use streaming for long responses
API key invalid	Key expired	Regenerate at console.groq.com

Examples

Basic usage: Apply groq deploy integration to a standard project setup with default configuration options.

Advanced scenario: Customize groq deploy integration for production environments with multiple constraints and team-specific requirements.

Resources

• Groq API Documentation
• Groq Models

Next Steps

For multi-environment setup, see groq-multi-env-setup.

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale