Query and Transform JSON Data with jq

jq Skill covers practical, expert-level jq usage - filtering deeply nested JSON, transforming structures, aggregating values, and composing jq into shell pipelines - with every example copy-paste ready for real workflows. jq applies a filter expression to JSON input, composing filters with pipes and handling arrays, objects, strings, numbers, booleans, and null natively.

It covers basic selection (field access, nested access, array indexing/slicing, iterating all elements with .[]), filtering with select (matching conditions, numeric comparisons, non-null checks, combined boolean conditions), mapping and transformation (extracting a field across elements, map() as shorthand for [.[] | f], building new objects per element, adding computed fields, renaming keys), and aggregation (add to sum, length to count, max_by/min_by, reduce for custom accumulation, group_by plus a group-and-count pattern). String interpolation and formatting cover \(...) interpolation, @csv/@tsv output formatting, @uri URL-encoding, and @base64 encoding. Working with keys and paths covers keys, has(), del() for single and nested-across-array deletion, recursive descent (..) to find a key anywhere in the tree, and paths(scalars) for all leaf paths. Conditionals and error handling cover if-then-elif-else-end, the // alternative operator for null/false fallbacks, try...catch to skip errors instead of halting, and // empty to suppress unwanted null output.

Practical shell integration covers reading from a file, compact (-c) output for further piping into a while-read loop, safely injecting shell variables via --arg (strings) and --argjson (numbers/JSON values) rather than string interpolation, slurping multiple JSON lines or files into one array (-s), and worked pipeline examples against kubectl get pods -o json, gh pr list --json, aws ec2 describe-instances, and docker inspect. Advanced patterns cover transposing an object of arrays into an array of objects, flatten(1), unique_by(.field), sort-and-dedupe chains, walk() for recursive whole-tree transformation, and reading environment variables inside a filter via env.

Best practices: always use -r (raw output) when passing results to shell variables or other commands, to strip JSON string quotes; use --arg/--argjson to inject shell variables safely rather than interpolating them directly into the filter string; prefer map(f) over [.[] | f] for readability; use -c for newline-delimited JSON pipelines and omit it for human-readable debugging; test filters interactively with jq -n before embedding them in scripts; and use empty to drop unwanted elements rather than filtering to null. Security notes: jq is read-only by design (it cannot write files or execute commands), and untrusted JSON field values should never be embedded directly into shell commands - always quote or use --arg.

Common pitfalls and fixes: null output usually means a key-name typo (inspect actual field names with keys, remembering JSON is case-sensitive); numbers coming out quoted as strings means --arg was used where --argjson was needed; a filter that works interactively but fails in a script usually means double quotes were used in the shell instead of single quotes (allowing unwanted variable expansion); add returning null on an empty array needs a // 0 or // "" fallback; and slow processing on very large files calls for jq --stream or a tool like jstream/gron instead.

Use this skill when parsing JSON output from APIs, CLI tools (AWS, GitHub, kubectl, docker), or log files, transforming JSON structure (renaming keys, flattening arrays, grouping records), using jq inside a bash script or one-liner, or explaining what a complex jq expression does. Do not treat the output as a substitute for environment-specific validation, testing, or expert review, and stop to ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.

Developers and DevOps engineers who regularly parse, filter, or reshape JSON from APIs and CLI tools in shell pipelines and want correct, safe jq patterns rather than trial-and-error filter syntax.