analyzing-network-traffic-of-malware

Analyzing Network Traffic of Malware

When to Use

• Sandbox execution has captured a PCAP file and the network behavior needs detailed analysis
• Identifying the C2 protocol structure for writing network detection signatures
• Determining what data the malware exfiltrates and to which external infrastructure
• Analyzing DNS tunneling, domain generation algorithms (DGA), or fast-flux behavior
• Creating Suricata/Snort signatures based on observed malware network patterns

Do not use for host-based analysis of malware behavior; use Cuckoo sandbox reports or Volatility memory analysis for process-level activity.

Prerequisites

• Wireshark 4.x installed for interactive PCAP analysis
• tshark (Wireshark CLI) for scripted packet extraction
• Zeek installed for automated metadata generation from PCAPs
• Suricata with ET Open/ET Pro rulesets for signature matching
• NetworkMiner for file extraction and credential detection from PCAPs
• Python 3.8+ with scapy and dpkt for programmatic packet analysis

Workflow

Step 1: Initial PCAP Overview

Get a high-level understanding of the network traffic:

# Capture statistics
capinfos malware.pcap

# Protocol hierarchy
tshark -r malware.pcap -q -z io,phs

# Endpoint statistics (top talkers)
tshark -r malware.pcap -q -z endpoints,ip

# Conversation statistics
tshark -r malware.pcap -q -z conv,tcp

# DNS query summary
tshark -r malware.pcap -q -z dns,tree

Step 2: Analyze DNS Activity

Examine DNS queries for DGA, tunneling, or C2 domain resolution:

# Extract all DNS queries
tshark -r malware.pcap -T fields -e frame.time -e dns.qry.name -e dns.a \
  -Y "dns.flags.response == 1" | sort

# Detect DGA patterns (high entropy domain names)
python3 << 'PYEOF'
import math
from collections import Counter

def entropy(s):
    p = [n/len(s) for n in Counter(s).values()]
    return -sum(pi * math.log2(pi) for pi in p if pi > 0)

# Parse DNS queries from tshark output
import subprocess
result = subprocess.run(
    ["tshark", "-r", "malware.pcap", "-T", "fields", "-e", "dns.qry.name",
     "-Y", "dns.flags.response == 0"],
    capture_output=True, text=True
)

domains = set(result.stdout.strip().split('\n'))
print("Suspicious DNS queries (high entropy):")
for domain in domains:
    if domain:
        subdomain = domain.split('.')[0]
        ent = entropy(subdomain)
        if ent > 3.5 and len(subdomain) > 10:
            print(f"  {domain} (entropy: {ent:.2f})")
PYEOF

# Detect DNS tunneling (large TXT responses)
tshark -r malware.pcap -T fields -e dns.qry.name -e dns.txt \
  -Y "dns.resp.type == 16 and dns.resp.len > 100"

Step 3: Analyze HTTP/HTTPS C2 Communication

Examine web-based command-and-control traffic:

# Extract HTTP requests
tshark -r malware.pcap -T fields \
  -e frame.time -e ip.src -e ip.dst -e http.host \
  -e http.request.method -e http.request.uri -e http.user_agent \
  -Y "http.request"

# Extract HTTP response bodies (potential payload downloads)
tshark -r malware.pcap -T fields \
  -e http.host -e http.request.uri -e http.content_type -e tcp.len \
  -Y "http.response and tcp.len > 1000"

# Extract POST data (potential exfiltration)
tshark -r malware.pcap -T fields \
  -e http.host -e http.request.uri -e http.file_data \
  -Y "http.request.method == POST"

# TLS analysis (SNI, JA3 fingerprints)
tshark -r malware.pcap -T fields \
  -e tls.handshake.extensions_server_name \
  -e tls.handshake.ja3 \
  -Y "tls.handshake.type == 1"

# Extract TLS certificate details
tshark -r malware.pcap -T fields \
  -e x509ce.dNSName -e x509af.serialNumber \
  -e x509sat.utf8String \
  -Y "tls.handshake.type == 11"

# Export HTTP objects (downloaded files)
tshark -r malware.pcap --export-objects http,exported_files/

Step 4: Detect Beaconing Patterns

Identify regular periodic communication indicating C2 beaconing:

# Beacon detection from PCAP
from scapy.all import rdpcap, IP, TCP
from collections import defaultdict
import statistics

packets = rdpcap("malware.pcap")

# Group connections by destination IP:port
connections = defaultdict(list)
for pkt in packets:
    if IP in pkt and TCP in pkt:
        if pkt[TCP].flags & 0x02:  # SYN flag
            dst = f"{pkt[IP].dst}:{pkt[TCP].dport}"
            connections[dst].append(float(pkt.time))

# Analyze timing intervals for beaconing
print("Beacon Analysis:")
for dst, times in connections.items():
    if len(times) >= 5:
        intervals = [times[i+1] - times[i] for i in range(len(times)-1)]
        avg = statistics.mean(intervals)
        stdev = statistics.stdev(intervals) if len(intervals) > 1 else 0
        jitter = (stdev / avg * 100) if avg > 0 else 0

        if 10 < avg < 3600 and jitter < 30:  # Regular interval with < 30% jitter
            print(f"  [!] {dst}: {len(times)} connections")
            print(f"      Interval: {avg:.1f}s ± {stdev:.1f}s (jitter: {jitter:.1f}%)")
            print(f"      Pattern: LIKELY BEACONING")

Step 5: Generate Network Detection Signatures

Create Suricata/Snort rules from observed traffic patterns:

# Run Suricata against the PCAP for existing signature matches
suricata -r malware.pcap -l suricata_output/ -c /etc/suricata/suricata.yaml

# Review alerts
cat suricata_output/fast.log

# Create custom Suricata rule from observed patterns
cat << 'EOF' > custom_malware.rules
# C2 beacon detection based on observed URI pattern
alert http $HOME_NET any -> $EXTERNAL_NET any (
    msg:"MALWARE MalwareX C2 Beacon";
    flow:established,to_server;
    http.method; content:"POST";
    http.uri; content:"/gate.php?id=";
    http.user_agent; content:"Mozilla/5.0 (compatible; MSIE 10.0)";
    sid:9000001; rev:1;
)

# DNS query for known C2 domain
alert dns $HOME_NET any -> any any (
    msg:"MALWARE MalwareX C2 DNS Query";
    dns.query; content:"update.malicious.com";
    sid:9000002; rev:1;
)

# JA3 hash match for malware TLS client
alert tls $HOME_NET any -> $EXTERNAL_NET any (
    msg:"MALWARE MalwareX JA3 Match";
    ja3.hash; content:"a0e9f5d64349fb13191bc781f81f42e1";
    sid:9000003; rev:1;
)
EOF

Step 6: Extract Files and Artifacts from Traffic

Recover transferred files and embedded data:

# Extract files using Zeek
zeek -r malware.pcap /opt/zeek/share/zeek/policy/frameworks/files/extract-all-files.zeek
ls extract_files/

# Extract files using NetworkMiner (GUI)
# Or use tshark for specific protocol exports
tshark -r malware.pcap --export-objects http,http_objects/
tshark -r malware.pcap --export-objects smb,smb_objects/
tshark -r malware.pcap --export-objects tftp,tftp_objects/

# Hash all extracted files
sha256sum http_objects/* smb_objects/* 2>/dev/null

# Generate Zeek logs for comprehensive metadata
zeek -r malware.pcap
# Output: conn.log, dns.log, http.log, ssl.log, files.log, etc.

Key Concepts

Term	Definition
Beaconing	Regular periodic connections from malware to C2 server, identifiable by consistent time intervals and packet sizes
JA3/JA3S	TLS fingerprinting method creating a hash from ClientHello/ServerHello parameters to uniquely identify malware TLS implementations
DGA (Domain Generation Algorithm)	Algorithm generating pseudo-random domain names that malware queries to locate C2 servers, evading static domain blocklists
DNS Tunneling	Encoding data in DNS queries and responses to establish a C2 channel or exfiltrate data through DNS infrastructure
Fast Flux	DNS technique rapidly rotating IP addresses for a domain to avoid takedown and distribute C2 across many compromised hosts
SNI (Server Name Indication)	TLS extension revealing the hostname the client is connecting to; visible even in encrypted HTTPS connections
Network Signature	Suricata/Snort rule matching specific patterns in network traffic (headers, payloads, timing) to detect malicious communications

Tools & Systems

• Wireshark: Open-source packet analyzer for deep interactive inspection of network traffic at the protocol level
• Zeek: Network analysis framework generating structured metadata logs (conn, dns, http, ssl) from live or captured traffic
• Suricata: High-performance network IDS/IPS for signature-based detection with Lua scripting for custom detection logic
• NetworkMiner: Network forensic analysis tool for extracting files, images, and credentials from PCAP files
• Scapy: Python packet manipulation library for programmatic packet analysis, beacon detection, and protocol decoding

Common Scenarios

Scenario: Decoding a Custom Binary C2 Protocol

Context: Malware communicates with its C2 server using a custom binary protocol over TCP port 8443. Standard HTTP analysis yields no results. The protocol structure needs to be reverse engineered from the PCAP.

Approach:

1. Filter the PCAP for TCP port 8443 conversations and follow the TCP stream
2. Identify the message framing (length prefix, delimiter, fixed-size headers)
3. Compare multiple messages to identify static header fields vs variable data fields
4. Cross-reference with reverse engineering findings from Ghidra (if the binary was analyzed)
5. Write a Wireshark dissector or Scapy parser for the custom protocol
6. Create Suricata rules matching the static header bytes for network detection
7. Document the full protocol specification for threat intelligence sharing

Pitfalls:

• Analyzing only the first few packets; some C2 protocols change behavior after initial handshake
• Not decrypting TLS traffic when the sandbox has MITM capabilities
• Confusing legitimate CDN or cloud traffic with C2 (validate destination IPs)
• Missing C2 traffic that uses DNS or ICMP instead of TCP/UDP

Output Format

MALWARE NETWORK TRAFFIC ANALYSIS
===================================
PCAP File:        malware_sandbox.pcap
Duration:         300 seconds
Total Packets:    12,847
Total Bytes:      4.2 MB

DNS ACTIVITY
Total Queries:    47
DGA Detected:     Yes (23 high-entropy queries to .com TLD)
Tunneling:        No
Resolved C2:      update.malicious[.]com -> 185.220.101[.]42

C2 COMMUNICATION
Protocol:         HTTPS (TLS 1.2)
Server:           185.220.101[.]42:443
SNI:              update.malicious[.]com
JA3 Hash:         a0e9f5d64349fb13191bc781f81f42e1
Beacon Interval:  60.2s ± 6.8s (11.3% jitter)
Total Sessions:   237
Data Sent:        147 MB
Data Received:    2.3 MB
Certificate:      CN=update.malicious[.]com (self-signed, expired)

PAYLOAD DOWNLOADS
GET /payload.dll from compromised-site[.]com
  Size: 98,304 bytes
  SHA-256: abc123def456...
  Content-Type: application/octet-stream

EXFILTRATION
Method:           HTTPS POST to /gate.php
Content-Type:     application/octet-stream
Average Size:     15,432 bytes per request
Total Volume:     147 MB over 4 hours

SURICATA ALERTS
[1:2028401] ET MALWARE Generic C2 Beacon Pattern
[1:2028500] ET POLICY Self-Signed Certificate

GENERATED SIGNATURES
SID 9000001: MalwareX HTTP beacon pattern
SID 9000002: MalwareX DNS C2 domain
SID 9000003: MalwareX JA3 TLS fingerprint