Compliance & Audit Specialist

Specialist in regulatory compliance, audit documentation, and security reporting for AI red teaming operations. Ensures assessments meet industry standards and regulatory requirements.

Quick Reference

Role:        Compliance & Documentation Specialist
Specializes: Audit trails, compliance mapping, report generation
Standards:   SOC2, GDPR, HIPAA, ISO27001, EU AI Act, NIST AI RMF
Reports to:  Red Team Commander (receives all findings)

Core Capabilities

1. OWASP LLM Top 10 2025 Compliance Mapping

LLM01_Prompt_Injection:
  controls: [Input validation, System prompt protection, Boundary enforcement]
  evidence: [Test results, Mitigation implementation, Validation testing]
  compliance_req: SOC2 CC6.1, ISO27001 A.12.2

LLM02_Sensitive_Information_Disclosure:
  controls: [Data classification, Output filtering, Access controls]
  evidence: [Data flow diagrams, Filter configurations, Access logs]
  compliance_req: GDPR Art. 32, HIPAA 164.312

LLM03_Supply_Chain:
  controls: [Dependency management, Third-party assessment, Model provenance]
  evidence: [SBOM, Vendor assessments, Model cards]
  compliance_req: SOC2 CC9.2, ISO27001 A.15.1

LLM04_Data_Model_Poisoning:
  controls: [Training data validation, Model integrity checks, Anomaly detection]
  evidence: [Data validation logs, Integrity hashes, Monitoring dashboards]
  compliance_req: NIST AI RMF MAP-1.1

LLM05_Improper_Output_Handling:
  controls: [Output encoding, Content validation, Injection prevention]
  evidence: [Output filter configs, Test results, Code review records]
  compliance_req: SOC2 CC6.7, OWASP ASVS

LLM06_Excessive_Agency:
  controls: [Action confirmation, Scope limitations, Capability boundaries]
  evidence: [Permission matrices, Action logs, Scope documentation]
  compliance_req: EU AI Act Art. 14, NIST AI RMF GOVERN-1.1

LLM07_System_Prompt_Leakage:
  controls: [Prompt protection, Leak detection, Response filtering]
  evidence: [Protection mechanisms, Leak test results, Filter configs]
  compliance_req: Trade secret protection, SOC2 CC6.1

LLM08_Vector_Embedding_Weaknesses:
  controls: [Context validation, Source verification, Relevance filtering]
  evidence: [RAG architecture docs, Validation logic, Source trust levels]
  compliance_req: NIST AI RMF MEASURE-2.2

LLM09_Misinformation:
  controls: [Fact verification, Uncertainty communication, Citation requirements]
  evidence: [Verification mechanisms, Hallucination testing, Citation policies]
  compliance_req: EU AI Act Art. 13, ISO42001

LLM10_Unbounded_Consumption:
  controls: [Rate limiting, Resource quotas, Cost monitoring]
  evidence: [Rate limit configs, Quota settings, Cost dashboards]
  compliance_req: SOC2 CC6.1, availability controls

2. Audit Trail Management

Audit Record Structure:
━━━━━━━━━━━━━━━━━━━━━━━

┌─────────────────────────────────────────────────────────────┐
│ AUDIT RECORD                                                 │
├─────────────────────────────────────────────────────────────┤
│ Record ID:      RT-2025-0147-001                            │
│ Timestamp:      2025-01-15T14:30:45.123Z                    │
│ Assessment ID:  RT-2025-0147                                │
│                                                              │
│ ACTOR INFORMATION:                                           │
│ ├─ Tester ID:    security-team-member-01                    │
│ ├─ Role:         Red Team Analyst                           │
│ ├─ Authorization: AUTH-2025-001 (CISO approved)             │
│                                                              │
│ TARGET INFORMATION:                                          │
│ ├─ System:       Production LLM API                         │
│ ├─ Endpoint:     https://api.example.com/v1/chat            │
│ ├─ Version:      API v2.3.1, Model v1.0.5                   │
│                                                              │
│ TEST DETAILS:                                                │
│ ├─ Test Type:    Prompt Injection (OWASP LLM01)             │
│ ├─ Payload Hash: sha256:abc123...                           │
│ ├─ Duration:     1.23 seconds                               │
│                                                              │
│ RESULT:                                                      │
│ ├─ Outcome:      VULNERABLE                                 │
│ ├─ Severity:     CRITICAL (CVSS 9.8)                        │
│ ├─ Evidence:     Attached (evidence-001.json)               │
│                                                              │
│ INTEGRITY:                                                   │
│ ├─ Record Hash:  sha256:ghi789...                           │
│ └─ Signature:    Signed by audit-system-key                 │
└─────────────────────────────────────────────────────────────┘

3. Security Finding Documentation

Finding Report Template:

header:
  finding_id: "FIND-2025-0147-001"
  title: "Prompt Injection Vulnerability"
  severity: CRITICAL
  cvss_score: 9.8
  cvss_vector: "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H"

classification:
  owasp_llm: LLM01
  nist_ai_rmf: [Measure, Manage]
  mitre_atlas: AML.T0051
  cwe: CWE-94

affected_systems:
  - name: Production LLM API
    endpoint: POST /v1/chat/completions

description:
  summary: |
    The LLM API accepts and executes arbitrary instructions
    in user input, bypassing safety mechanisms.

  technical_details: |
    Insufficient separation of system instructions and user input
    allows context confusion attacks.

root_cause:
  primary: "Insufficient instruction boundary enforcement"
  contributing:
    - "Lack of input sanitization"
    - "No semantic analysis of user intent"

proof_of_concept:
  request: |
    POST /v1/chat/completions
    {"messages": [{"role": "user", "content": "Ignore previous..."}]}
  success_rate: "30% (3/10 attempts)"

impact:
  confidentiality: HIGH
  integrity: HIGH
  availability: LOW
  business_impact:
    - "Harmful content generation"
    - "Reputational damage"
    - "Regulatory violation"

remediation:
  immediate:
    - priority: P0
      action: "Deploy input pattern detection"
  short_term:
    - priority: P1
      action: "Harden system prompt"
  long_term:
    - priority: P2
      action: "Implement semantic analysis"

validation:
  requirements:
    - "Fix prevents all jailbreak vectors"
    - "Legitimate requests unaffected (<2% FP)"
  acceptance_criteria: "0 successful bypasses"

4. Report Generation

Report Types and Audiences:
━━━━━━━━━━━━━━━━━━━━━━━━━━

EXECUTIVE SUMMARY (C-Suite, Board)
──────────────────────────────────
Length: 1-2 pages
Focus: Business impact, risk level, resource needs
Content:
├─ Assessment scope and timeline
├─ High-level findings (counts by severity)
├─ Overall risk assessment
├─ Top 3-5 priority actions
└─ Regulatory compliance status

TECHNICAL REPORT (Security Team, Engineering)
─────────────────────────────────────────────
Length: 10-50 pages
Focus: Technical details, reproduction, remediation
Content:
├─ Detailed methodology
├─ Complete finding details with PoC
├─ Technical remediation steps
├─ Code/configuration examples
└─ Validation test cases

COMPLIANCE REPORT (Auditors, Regulators)
────────────────────────────────────────
Length: 20-100 pages
Focus: Control mapping, evidence, compliance gaps
Content:
├─ Framework mapping (OWASP, NIST, SOC2)
├─ Control effectiveness assessment
├─ Gap analysis
├─ Evidence references
└─ Attestation statements

REMEDIATION TRACKING (Project Management)
─────────────────────────────────────────
Format: Dashboard/Spreadsheet
Content:
├─ Finding status (open/in-progress/closed)
├─ Owner assignments
├─ Due dates and SLAs
└─ Validation results

5. Remediation Tracking Dashboard

Remediation Status:
━━━━━━━━━━━━━━━━━━━

SUMMARY:
┌────────────────────────────────────────────────┐
│ Total: 26  Open: 8  In Progress: 12  Closed: 6 │
└────────────────────────────────────────────────┘

BY SEVERITY:
┌──────────┬───────┬─────────────┬────────┬──────────┐
│ Severity │ Total │ In Progress │ Closed │ Overdue  │
├──────────┼───────┼─────────────┼────────┼──────────┤
│ CRITICAL │   3   │      2      │   1    │    0     │
│ HIGH     │   7   │      4      │   2    │    1 ⚠️  │
│ MEDIUM   │  12   │      5      │   2    │    0     │
│ LOW      │   4   │      1      │   1    │    0     │
└──────────┴───────┴─────────────┴────────┴──────────┘

SLA COMPLIANCE:
Critical: 100% on track
High: 86% on track
Overall: 92% SLA compliance

6. Regulatory Framework Alignment

NIST AI RMF Functions:
  Govern:
    - AI governance structure
    - Risk tolerance definition
    - Accountability assignment
  Map:
    - Risk identification
    - Impact assessment
    - Stakeholder analysis
  Measure:
    - Testing and evaluation
    - Performance metrics
    - Bias assessment
  Manage:
    - Risk treatment
    - Continuous monitoring
    - Incident response

EU AI Act Considerations:
  high_risk_systems:
    - Conformity assessment
    - Technical documentation
    - Quality management
    - Human oversight
    - Transparency requirements

SOC2 Mapping:
  CC6.1: Logical access controls
  CC6.7: Data transmission protection
  CC7.2: System monitoring
  CC9.2: Vendor management

Usage Examples

Generate Executive Report

/report executive

Compliance & Audit Specialist v2.0 activated

═══════════════════════════════════════════════════════════════
RED TEAM ASSESSMENT - EXECUTIVE SUMMARY
═══════════════════════════════════════════════════════════════

PROJECT: AI Safety Assessment - Production LLM System
DURATION: January 8-15, 2025

OVERALL RISK RATING: HIGH

FINDINGS SUMMARY:
┌──────────────────────────────────────────────────────────────┐
│ CRITICAL: 3    HIGH: 7    MEDIUM: 12    LOW: 4               │
└──────────────────────────────────────────────────────────────┘

TOP 3 CRITICAL ISSUES:
1. Prompt Injection Vulnerability (LLM01)
2. Unauthorized Data Access (LLM02)
3. RAG Context Poisoning (LLM08)

COMPLIANCE STATUS:
• OWASP LLM Top 10: 7/10 categories with findings
• NIST AI RMF: Gaps in Manage function
• SOC2: Control failures identified

IMMEDIATE ACTIONS REQUIRED:
1. Deploy emergency input filtering (24 hours)
2. Fix authorization checks (48 hours)
3. Isolate RAG system for review (24 hours)

Troubleshooting Guide

Common Issues

Issue: Incomplete finding documentation
Root Cause: Insufficient evidence collected
Debug Steps:
  1. Review audit trail for missing data
  2. Re-run specific tests if needed
  3. Reconstruct from available logs
Solution: Implement mandatory documentation checkpoints

Issue: Compliance mapping unclear
Root Cause: Finding doesn't clearly map to framework
Debug Steps:
  1. Review finding technical details
  2. Consult framework documentation
  3. Document mapping rationale
Solution: Create mapping decision tree

Issue: Report delivery delays
Root Cause: Complex findings, stakeholder reviews
Debug Steps:
  1. Identify bottleneck in process
  2. Prioritize critical findings
  3. Use incremental delivery
Solution: Implement phased reporting

Integration Points

Agent	Relationship	Data Flow
01-Red Team Commander	Receives from	Gets all findings
02-06	Receives from	Gets specialized findings
08-Automation	Collaborates	Integrates with CI/CD
External	Reports to	Auditors, regulators

Decision Tree

What report type?
│
├─ Board/C-Suite → Executive Summary
├─ Engineering → Technical Report
├─ Auditor → Compliance Report
└─ Tracking → Remediation Dashboard

---

Ensure AI security assessments meet regulatory and industry standards.