data-serialization

Data Serialization Skill

You are a data transformation specialist. Your job is to help convert between data formats and query structured data efficiently, with a focus on token efficiency for LLM contexts.

Supported Formats

Format	Extension	Best For	Tools
JSON	.json	Data interchange, APIs, tooling	jq
YAML	.yaml, .yml	Human editing, config files	yq
TOON	.toon	LLM prompts (30-60% fewer tokens)	Python toon_format
XML	.xml	Legacy systems, SOAP, external requirements	xmllint
HTML	.html	Web content conversion	Python xmltodict

When to Use Each Format

JSON

Pros:

• Universal support across all languages
• Strict syntax reduces ambiguity
• Best tooling ecosystem (jq)
• Required by most APIs

Cons:

• No comments allowed
• Verbose for human editing
• No multi-line strings (must escape)
• ~40% of tokens are formatting (braces, quotes, commas)

Use when: API responses, data interchange, tool input/output

YAML

Pros:

• Human-readable and editable
• Supports comments
• Multi-line strings with | or >
• Anchors and aliases for DRY configs

Cons:

• Significant whitespace (indentation matters)
• Security concerns with arbitrary YAML (code execution possible)
• Multiple ways to represent same data

Use when: Config files that humans edit, CI/CD pipelines, Kubernetes

Security Note: Never load untrusted YAML with yaml.load() - use yaml.safe_load()

TOON (Token-Oriented Object Notation)

Pros:

• 30-60% token reduction vs JSON
• Lossless conversion to/from JSON
• YAML-like indentation (human-readable)
• CSV-style tabular arrays (compact)
• Explicit [N] lengths and {fields} headers help LLMs parse reliably
• Improves LLM accuracy (73.9% vs JSON's 69.7% in benchmarks)

Cons:

• Less tooling than JSON/YAML (newer format)
• Less efficient for deeply nested, non-uniform data
• Requires Python library for conversion

Use when:

• Sending structured data to LLMs
• Reducing API costs (fewer input tokens)
• Maximizing context window usage
• Tabular data with uniform structure

Best for: Uniform arrays of objects (same fields across items)

XML/HTML

Pros:

• Schema validation (XSD)
• Namespaces for complex documents
• XPath for powerful querying
• Required by many enterprise systems

Cons:

• Verbose syntax
• Complex to parse and generate
• Falling out of favor for new projects

Use when: SOAP APIs, enterprise integrations, document formats (Office, SVG)

TOON Format Guide

Basic Syntax

Objects use key-value pairs with colon-space separation:

name: Alice
age: 30
active: true

Nested objects use indentation:

user:
  id: 123
  profile:
    role: admin

Primitive arrays (inline):

tags[3]: admin,ops,dev

Tabular arrays (uniform objects - TOON's sweet spot):

users[2]{id,name,role}:
 1,Alice,admin
 2,Bob,user

Expanded lists (mixed types):

tasks[2]:
 - Complete report
 - Review code

TOON Token Savings Example

JSON (257 tokens):

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" },
    { "id": 3, "name": "Carol", "role": "guest" }
  ]
}

TOON (166 tokens - 35% reduction):

users[3]{id,name,role}:
 1,Alice,admin
 2,Bob,user
 3,Carol,guest

When NOT to Use TOON

• Deeply nested hierarchies: JSON may be more compact
• Non-uniform data: Mixed object shapes reduce efficiency
• Flat tabular data: CSV is more compact (no TOON metadata)

Conversion Guide

Using the convert.sh Script

The plugin provides a convert.sh script for format conversion:

# Basic usage
/path/to/plugins/data-serialization/scripts/convert.sh <input-file> <output-format>

# Examples
convert.sh data.json yaml      # JSON to YAML
convert.sh config.yaml toon    # YAML to TOON
convert.sh data.toon json      # TOON to JSON
convert.sh data.xml json       # XML to JSON
convert.sh page.html yaml      # HTML to YAML

# With explicit source format (if auto-detect fails)
convert.sh data.txt json --from yaml

# Playwright accessibility snapshot conversion
convert.sh playwright-snapshot.md json --playwright
convert.sh playwright-snapshot.md toon --playwright

Manual Conversion Commands

JSON to YAML:

yq -P '.' input.json > output.yaml

YAML to JSON:

yq -o=json '.' input.yaml > output.json

JSON to TOON:

python3 -c "
from toon_format import encode
import json
with open('input.json') as f:
    data = json.load(f)
print(encode(data))
" > output.toon

TOON to JSON:

python3 -c "
from toon_format import decode
import json
with open('input.toon') as f:
    data = decode(f.read())
print(json.dumps(data, indent=2))
" > output.json

XML to JSON:

python3 -c "
import json, xmltodict
with open('input.xml') as f:
    data = xmltodict.parse(f.read())
print(json.dumps(data, indent=2))
" > output.json

HTML to JSON:

python3 -c "
import json, xmltodict
from html.parser import HTMLParser
with open('input.html') as f:
    # Parse HTML as XML-like structure
    data = xmltodict.parse(f.read())
print(json.dumps(data, indent=2))
" > output.json

Querying Data

jq for JSON

Basic selection:

# Get a field
jq '.fieldName' data.json

# Get nested field
jq '.parent.child' data.json

# Get array element
jq '.[0]' data.json
jq '.items[0]' data.json

Filtering:

# Select objects matching condition
jq '.[] | select(.status == "active")' data.json

# Multiple conditions
jq '.[] | select(.age > 18 and .country == "US")' data.json

# Null-safe filtering
jq '.[] | select(.optional // empty)' data.json

Transformation:

# Extract specific fields
jq '.[] | {name, email}' data.json

# Rename fields
jq '.[] | {userName: .name, userEmail: .email}' data.json

# Create arrays
jq '[.[] | .name]' data.json

Aggregation:

# Count items
jq 'length' data.json
jq '[.[] | select(.active)] | length' data.json

# Sum values
jq '[.[] | .price] | add' data.json

# Group by field
jq 'group_by(.category)' data.json

yq for YAML

yq uses the same syntax as jq:

# Basic queries work the same
yq '.fieldName' data.yaml
yq '.[] | select(.status == "active")' data.yaml

# Output as JSON
yq -o=json '.' data.yaml

# Edit in place
yq -i '.version = "2.0"' data.yaml

XPath for XML

Using xmllint:

# Get element text
xmllint --xpath '//element/text()' data.xml

# Get attribute
xmllint --xpath 'string(//element/@attr)' data.xml

# Count elements
xmllint --xpath 'count(//item)' data.xml

# Get multiple elements
xmllint --xpath '//item/name/text()' data.xml

Querying TOON

For TOON, convert to JSON first, then use jq:

python3 -c "
from toon_format import decode
import json
with open('data.toon') as f:
    print(json.dumps(decode(f.read())))
" | jq '.users[] | select(.role == "admin")'

Playwright Accessibility Snapshots

Playwright MCP returns accessibility snapshots in a YAML-like format. These can be converted for querying.

Snapshot Format

Playwright snapshots look like:

- button "Submit" [ref=s1e2]
- link "Home" [ref=s1e3]
  - text "Home"
- textbox "Email" [ref=s1e4]

Converting Snapshots

Use the convert script with --playwright flag:

convert.sh snapshot.md json --playwright
convert.sh snapshot.md toon --playwright  # For token efficiency

This produces queryable JSON:

[
  { "role": "button", "name": "Submit", "ref": "s1e2" },
  {
    "role": "link",
    "name": "Home",
    "ref": "s1e3",
    "children": [{ "role": "text", "name": "Home" }]
  },
  { "role": "textbox", "name": "Email", "ref": "s1e4" }
]

Querying Playwright Data

Find all buttons:

jq '[.. | objects | select(.role == "button")]' snapshot.json

Find element by name:

jq '.. | objects | select(.name == "Submit")' snapshot.json

Get all refs:

jq '[.. | objects | .ref // empty]' snapshot.json

Find clickable elements:

jq '[.. | objects | select(.role == "button" or .role == "link")]' snapshot.json

Best Practices

1. Use TOON for LLM input - 30-60% token savings on structured data
2. Save API/tool output to files first before querying
3. Use JSON as intermediate format when converting between formats
4. Validate before converting - malformed input causes cryptic errors
5. Preserve original files during conversion experiments
6. For deeply nested data, stick with JSON (TOON overhead increases)

Tool Installation

Ensure these tools are available:

# macOS
brew install jq yq

# Python dependencies (for TOON and XML)
pip install git+https://github.com/toon-format/toon-python.git xmltodict dicttoxml

# Verify
jq --version
yq --version
python3 -c "from toon_format import encode; print('toon_format OK')"

References

• TOON Specification - Official format spec
• TOON Python Library - Python implementation
• jq Manual - JSON query language
• yq Documentation - YAML processor
• XPath Reference - XML query language
• Playwright Accessibility - Accessibility snapshots

Plugin Location

This skill is part of the data-serialization plugin.

Sources:

• GitHub: https://github.com/nsheaps/ai-mktpl
• Local Path: ~/src/nsheaps/ai/plugins/data-serialization