Audit Logging Skill

Patterns for implementing immutable audit logs that meet compliance requirements and enable security analysis.

When to Use This Skill

Use this skill when:

• Audit Logging tasks - Working on immutable audit logging patterns for compliance and security. covers event design, storage strategies, retention policies, and audit trail analysis
• Planning or design - Need guidance on Audit Logging approaches
• Best practices - Want to follow established patterns and standards

Overview

Audit logs provide a tamper-evident record of who did what, when, and from where. They are essential for compliance (SOC 2, HIPAA, GDPR), security investigations, and operational troubleshooting.

Audit Log Architecture

+------------------------------------------------------------------+
|                     Audit Logging Pipeline                        |
+------------------------------------------------------------------+
|                                                                   |
|  +-------------+    +---------------+    +--------------------+   |
|  | Application |    | Audit Service |    | Immutable Storage  |   |
|  | (emit)      |--->| (enrich,      |--->| (append-only,      |   |
|  |             |    |  validate)    |    |  tamper-evident)   |   |
|  +-------------+    +---------------+    +--------------------+   |
|        |                   |                      |               |
|        v                   v                      v               |
|  User actions         Timestamp,            WORM storage,         |
|  System events        correlation,          blockchain hash,      |
|  Data changes         tenant context        retention lock        |
|                                                                   |
+------------------------------------------------------------------+

Audit Event Design

Core Event Schema

public sealed record AuditEvent
{
    // Identity
    public required Guid EventId { get; init; } = Guid.NewGuid();
    public required DateTimeOffset Timestamp { get; init; }
    public required string EventType { get; init; }  // "user.login", "data.export", etc.

    // Actor
    public required Guid? UserId { get; init; }
    public required Guid TenantId { get; init; }
    public string? UserEmail { get; init; }
    public string? UserRole { get; init; }
    public required ActorType ActorType { get; init; }  // User, System, API, Service

    // Context
    public required string Source { get; init; }  // "web", "api", "background-job"
    public string? IpAddress { get; init; }
    public string? UserAgent { get; init; }
    public string? SessionId { get; init; }
    public string? CorrelationId { get; init; }
    public string? RequestId { get; init; }

    // Action
    public required string Action { get; init; }  // "create", "read", "update", "delete"
    public required string ResourceType { get; init; }  // "user", "invoice", "settings"
    public string? ResourceId { get; init; }
    public required bool Success { get; init; }
    public string? FailureReason { get; init; }

    // Change Details (for modifications)
    public Dictionary<string, object?>? OldValues { get; init; }
    public Dictionary<string, object?>? NewValues { get; init; }

    // Metadata
    public Dictionary<string, string>? Tags { get; init; }
}

public enum ActorType
{
    User,
    System,
    ApiClient,
    BackgroundService,
    ExternalIntegration
}

Event Types Taxonomy

Event Type Hierarchy:
+------------------------------------------------------------------+
| Category          | Event Types                                  |
+-------------------+----------------------------------------------+
| Authentication    | user.login, user.logout, user.login_failed,  |
|                   | user.mfa_enabled, user.password_changed      |
+-------------------+----------------------------------------------+
| Authorization     | access.granted, access.denied,               |
|                   | permission.changed, role.assigned            |
+-------------------+----------------------------------------------+
| Data Operations   | data.created, data.updated, data.deleted,    |
|                   | data.exported, data.imported                 |
+-------------------+----------------------------------------------+
| Administration    | settings.changed, user.invited,              |
|                   | integration.enabled, api_key.created         |
+-------------------+----------------------------------------------+
| Billing           | subscription.created, payment.processed,     |
|                   | invoice.generated, plan.changed              |
+-------------------+----------------------------------------------+
| Security          | security.alert, ip.blocked,                  |
|                   | suspicious.activity, breach.detected         |
+-------------------+----------------------------------------------+

Implementation Patterns

Audit Service Interface

public interface IAuditService
{
    // Log an audit event
    Task LogAsync(AuditEvent auditEvent, CancellationToken ct = default);

    // Log with builder pattern
    Task LogAsync(Action<AuditEventBuilder> configure, CancellationToken ct = default);

    // Query audit log (for admin/compliance)
    Task<PagedResult<AuditEvent>> QueryAsync(
        AuditQuery query,
        CancellationToken ct = default);

    // Export for compliance reports
    Task<Stream> ExportAsync(
        AuditExportRequest request,
        CancellationToken ct = default);
}

public sealed class AuditEventBuilder
{
    private readonly AuditEvent _event = new();

    public AuditEventBuilder WithUser(Guid userId, string? email = null)
    {
        _event = _event with { UserId = userId, UserEmail = email };
        return this;
    }

    public AuditEventBuilder WithAction(string eventType, string action, bool success = true)
    {
        _event = _event with { EventType = eventType, Action = action, Success = success };
        return this;
    }

    public AuditEventBuilder WithResource(string resourceType, string? resourceId)
    {
        _event = _event with { ResourceType = resourceType, ResourceId = resourceId };
        return this;
    }

    public AuditEventBuilder WithChanges(object? oldValue, object? newValue)
    {
        // Serialize to dictionaries, redacting sensitive fields
        return this;
    }

    public AuditEvent Build() => _event;
}

Automatic Audit via Interceptors

// EF Core interceptor for automatic data change auditing
public sealed class AuditSaveChangesInterceptor(
    IAuditService auditService,
    ITenantContext tenantContext,
    ICurrentUserService currentUser) : SaveChangesInterceptor
{
    public override async ValueTask<InterceptionResult<int>> SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult<int> result,
        CancellationToken ct = default)
    {
        var context = eventData.Context;
        if (context is null) return result;

        var auditableEntries = context.ChangeTracker
            .Entries()
            .Where(e => e.Entity is IAuditable)
            .Where(e => e.State is EntityState.Added or EntityState.Modified or EntityState.Deleted)
            .ToList();

        foreach (var entry in auditableEntries)
        {
            var auditEvent = new AuditEvent
            {
                EventId = Guid.NewGuid(),
                Timestamp = DateTimeOffset.UtcNow,
                EventType = $"data.{entry.State.ToString().ToLowerInvariant()}",
                UserId = currentUser.UserId,
                TenantId = tenantContext.TenantId,
                ActorType = ActorType.User,
                Source = "api",
                Action = entry.State.ToString().ToLowerInvariant(),
                ResourceType = entry.Entity.GetType().Name,
                ResourceId = GetPrimaryKey(entry),
                Success = true,
                OldValues = entry.State != EntityState.Added
                    ? GetValues(entry.OriginalValues)
                    : null,
                NewValues = entry.State != EntityState.Deleted
                    ? GetValues(entry.CurrentValues)
                    : null
            };

            await auditService.LogAsync(auditEvent, ct);
        }

        return result;
    }
}

Action Filter for API Auditing

[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public sealed class AuditAttribute : Attribute, IAsyncActionFilter
{
    public string EventType { get; set; } = "api.request";
    public string? ResourceType { get; set; }
    public bool LogRequestBody { get; set; }
    public bool LogResponseBody { get; set; }

    public async Task OnActionExecutionAsync(
        ActionExecutingContext context,
        ActionExecutionDelegate next)
    {
        var auditService = context.HttpContext.RequestServices
            .GetRequiredService<IAuditService>();

        var startTime = DateTimeOffset.UtcNow;
        var result = await next();

        var auditEvent = new AuditEvent
        {
            EventType = EventType,
            Timestamp = startTime,
            Action = context.HttpContext.Request.Method,
            ResourceType = ResourceType ?? context.Controller.GetType().Name,
            ResourceId = context.ActionArguments.TryGetValue("id", out var id)
                ? id?.ToString()
                : null,
            Success = result.Exception is null,
            FailureReason = result.Exception?.Message,
            // ... populate from HttpContext
        };

        await auditService.LogAsync(auditEvent);
    }
}

Storage Strategies

Immutability Patterns

Immutability Options:
+------------------------------------------------------------------+
| Strategy              | How It Works                             |
+-----------------------+------------------------------------------+
| Append-only table     | No UPDATE/DELETE permissions on table    |
| WORM storage          | Azure Immutable Blob, AWS S3 Object Lock |
| Blockchain hash       | Each entry includes hash of previous     |
| Digital signatures    | Sign entries with HSM-backed key         |
| Write-ahead log       | Append to log, never modify              |
+------------------------------------------------------------------+

Database Schema

-- Append-only audit table
CREATE TABLE audit_logs (
    event_id UUID PRIMARY KEY,
    timestamp TIMESTAMPTZ NOT NULL,
    event_type VARCHAR(100) NOT NULL,
    tenant_id UUID NOT NULL,
    user_id UUID,
    actor_type VARCHAR(50) NOT NULL,
    source VARCHAR(50) NOT NULL,
    action VARCHAR(50) NOT NULL,
    resource_type VARCHAR(100) NOT NULL,
    resource_id VARCHAR(255),
    success BOOLEAN NOT NULL,
    failure_reason TEXT,
    old_values JSONB,
    new_values JSONB,
    ip_address INET,
    user_agent TEXT,
    session_id VARCHAR(255),
    correlation_id VARCHAR(255),
    tags JSONB,

    -- Hash chain for tamper detection
    previous_hash VARCHAR(64),
    entry_hash VARCHAR(64) NOT NULL
);

-- Partitioning for performance and retention
CREATE TABLE audit_logs_y2024m01 PARTITION OF audit_logs
    FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');

-- Indexes for common queries
CREATE INDEX idx_audit_tenant_time ON audit_logs (tenant_id, timestamp DESC);
CREATE INDEX idx_audit_user_time ON audit_logs (user_id, timestamp DESC);
CREATE INDEX idx_audit_resource ON audit_logs (resource_type, resource_id);
CREATE INDEX idx_audit_event_type ON audit_logs (event_type);

-- Revoke modification permissions
REVOKE UPDATE, DELETE ON audit_logs FROM app_user;

Hash Chain for Tamper Detection

public sealed class HashChainAuditStore(IDbContext db)
{
    public async Task AppendAsync(AuditEvent auditEvent, CancellationToken ct)
    {
        // Get hash of previous entry
        var previousHash = await db.AuditLogs
            .Where(a => a.TenantId == auditEvent.TenantId)
            .OrderByDescending(a => a.Timestamp)
            .Select(a => a.EntryHash)
            .FirstOrDefaultAsync(ct);

        // Compute hash of current entry including previous hash
        var entryHash = ComputeHash(auditEvent, previousHash);

        var logEntry = new AuditLogEntry
        {
            // ... map from auditEvent
            PreviousHash = previousHash,
            EntryHash = entryHash
        };

        db.AuditLogs.Add(logEntry);
        await db.SaveChangesAsync(ct);
    }

    private static string ComputeHash(AuditEvent entry, string? previousHash)
    {
        var data = JsonSerializer.Serialize(entry) + (previousHash ?? "");
        var hash = SHA256.HashData(Encoding.UTF8.GetBytes(data));
        return Convert.ToHexString(hash);
    }

    public async Task<bool> VerifyChainIntegrityAsync(
        Guid tenantId,
        DateTimeOffset start,
        DateTimeOffset end,
        CancellationToken ct)
    {
        var entries = await db.AuditLogs
            .Where(a => a.TenantId == tenantId)
            .Where(a => a.Timestamp >= start && a.Timestamp <= end)
            .OrderBy(a => a.Timestamp)
            .ToListAsync(ct);

        string? previousHash = null;
        foreach (var entry in entries)
        {
            var expectedHash = ComputeHash(entry.ToAuditEvent(), previousHash);
            if (entry.EntryHash != expectedHash)
                return false;  // Tampering detected

            previousHash = entry.EntryHash;
        }

        return true;
    }
}

Retention Policies

Compliance-Driven Retention

Retention Requirements:
+------------------------------------------------------------------+
| Framework      | Minimum Retention | Notes                        |
+----------------+-------------------+------------------------------+
| SOC 2          | 1 year            | Auditor access required      |
| HIPAA          | 6 years           | From creation or last use    |
| GDPR           | As needed         | Minimize, but prove actions  |
| PCI DSS        | 1 year            | 3 months immediately online  |
| SOX            | 7 years           | Financial records            |
| General        | 3-7 years         | Legal/litigation hold        |
+------------------------------------------------------------------+

Retention Implementation

public sealed class AuditRetentionService(
    IDbContext db,
    IAuditArchiveStore archiveStore,
    ILogger<AuditRetentionService> logger)
{
    public async Task EnforceRetentionAsync(CancellationToken ct)
    {
        var policies = await GetRetentionPoliciesAsync(ct);

        foreach (var policy in policies)
        {
            // Archive before delete
            if (policy.ArchiveBeforeDelete)
            {
                var toArchive = await db.AuditLogs
                    .Where(a => a.EventType.StartsWith(policy.EventTypePrefix))
                    .Where(a => a.Timestamp < DateTimeOffset.UtcNow - policy.OnlineRetention)
                    .Where(a => a.Timestamp >= DateTimeOffset.UtcNow - policy.TotalRetention)
                    .ToListAsync(ct);

                await archiveStore.ArchiveAsync(toArchive, ct);
            }

            // Delete expired
            var deleted = await db.AuditLogs
                .Where(a => a.EventType.StartsWith(policy.EventTypePrefix))
                .Where(a => a.Timestamp < DateTimeOffset.UtcNow - policy.TotalRetention)
                .Where(a => !a.HasLegalHold)
                .ExecuteDeleteAsync(ct);

            logger.LogInformation(
                "Deleted {Count} audit logs for policy {Policy}",
                deleted, policy.Name);
        }
    }
}

Sensitive Data Handling

Redaction Patterns

public sealed class AuditRedactor
{
    private static readonly HashSet<string> SensitiveFields = new(StringComparer.OrdinalIgnoreCase)
    {
        "password", "secret", "token", "apikey", "api_key",
        "ssn", "social_security", "credit_card", "cvv",
        "bank_account", "routing_number"
    };

    public Dictionary<string, object?> Redact(Dictionary<string, object?> values)
    {
        var redacted = new Dictionary<string, object?>();

        foreach (var (key, value) in values)
        {
            if (SensitiveFields.Contains(key))
            {
                redacted[key] = "[REDACTED]";
            }
            else if (value is Dictionary<string, object?> nested)
            {
                redacted[key] = Redact(nested);
            }
            else
            {
                redacted[key] = value;
            }
        }

        return redacted;
    }
}

Querying and Analysis

Common Queries

public sealed class AuditQueryService(IDbContext db)
{
    // Security: Failed login attempts
    public async Task<IReadOnlyList<AuditEvent>> GetFailedLoginsAsync(
        Guid tenantId,
        TimeSpan window,
        CancellationToken ct)
    {
        return await db.AuditLogs
            .Where(a => a.TenantId == tenantId)
            .Where(a => a.EventType == "user.login_failed")
            .Where(a => a.Timestamp > DateTimeOffset.UtcNow - window)
            .OrderByDescending(a => a.Timestamp)
            .Take(100)
            .ToListAsync(ct);
    }

    // Compliance: All actions by user
    public async Task<PagedResult<AuditEvent>> GetUserActivityAsync(
        Guid tenantId,
        Guid userId,
        DateTimeOffset start,
        DateTimeOffset end,
        int page,
        int pageSize,
        CancellationToken ct)
    {
        var query = db.AuditLogs
            .Where(a => a.TenantId == tenantId)
            .Where(a => a.UserId == userId)
            .Where(a => a.Timestamp >= start && a.Timestamp <= end);

        var total = await query.CountAsync(ct);
        var items = await query
            .OrderByDescending(a => a.Timestamp)
            .Skip((page - 1) * pageSize)
            .Take(pageSize)
            .ToListAsync(ct);

        return new PagedResult<AuditEvent>(items, total, page, pageSize);
    }

    // Data access: Who accessed this resource
    public async Task<IReadOnlyList<AuditEvent>> GetResourceAccessAsync(
        string resourceType,
        string resourceId,
        TimeSpan window,
        CancellationToken ct)
    {
        return await db.AuditLogs
            .Where(a => a.ResourceType == resourceType)
            .Where(a => a.ResourceId == resourceId)
            .Where(a => a.Timestamp > DateTimeOffset.UtcNow - window)
            .OrderByDescending(a => a.Timestamp)
            .ToListAsync(ct);
    }
}

Best Practices

Audit Logging Best Practices:
+------------------------------------------------------------------+
| Practice                  | Rationale                            |
+---------------------------+--------------------------------------+
| Log synchronously         | Ensure log before action completes   |
| Include context           | Who, what, when, where, why          |
| Use structured format     | Enables analysis and alerting        |
| Redact sensitive data     | PII, credentials, PHI protection     |
| Partition by time         | Performance and retention management |
| Hash chain for integrity  | Tamper detection                     |
| Separate storage          | Isolation from app DB                |
| Real-time alerting        | Security event detection             |
+------------------------------------------------------------------+

Anti-Patterns

Anti-Pattern	Problem	Solution
Log after action	May miss failures	Log before/during action
Mutable storage	Tampering possible	Append-only, WORM storage
No tenant isolation	Data leakage	Include tenant in all queries
Plain text secrets	Compliance violation	Always redact sensitive fields
Single partition	Performance degrades	Time-based partitioning
No retention policy	Storage costs, legal risk	Define and enforce policies

References

Load for detailed implementation:

• references/immutable-logs.md - Storage patterns for immutability
• references/retention-policies.md - Compliance-driven retention

Related Skills

• saas-compliance-frameworks - Compliance requirements
• tenant-data-isolation - Multi-tenant audit separation

MCP Research

For current audit logging patterns:

perplexity: "audit logging compliance 2024" "immutable audit trail patterns"
microsoft-learn: "Azure Monitor audit" "Application Insights audit logging"