Reverse-engineer custom bytecode VMs in obfuscated JS

DSL VM Reverse is a workflow for reverse-engineering custom JavaScript bytecode-VM interpreters used by risk-control and CAPTCHA engines (the examples throughout reference Alibaba's AWSC risk-control module specifically). It defines five identification characteristics that mark a file as a DSL VM rather than regular JS or WASM: an IIFE opening with many single-letter variables mapping numeric constants, an interpreter main loop (DG()-style) with a switch-case opcode dispatch, files over 500KB with a zero-byte ratio under 1%, C[number] constant-table references, and single-line minified code - with explicit exclusion rules routing standard WASM binaries, embedded WASM, and regular Webpack-bundled JS to other skills instead. The six-phase workflow: Phase 1 classifies the file type in about 5 minutes with a script that checks for the WASM magic bytes, the zero-byte ratio, and the var U=void 0 single-letter-variable pattern:

python3 << 'EOF'
with open('target.js', 'rb') as f:
    head = f.read(100)

# 1. Check for WASM magic bytes
if head[:4] == b'\x00asm':
    print("Standard WASM binary")
    exit()

# 2. Check zero-byte ratio
data = open('target.js', 'rb').read()
zero_pct = data.count(b'\x00') / len(data) * 100
print(f"Zero-byte ratio: {zero_pct:.1f}%")

if zero_pct > 20:
    print("WASM binary")
elif head[:2] == b'!f':
    # Check for single-letter variable pattern
    if b'var U=void 0' in head or b'U=void 0,y=parseInt' in head:
        print("→ DSL VM!")
    else:
        print("Regular JS IIFE")
EOF

Phase 2 extracts the var X=number constant-mapping table with a regex pass over the first 2000 characters; Phase 3 extracts and classifies every case N: opcode by inspecting each case body for signature patterns (a d[7]= assignment means BRANCH, W(C[ means CALL, new means ALLOC, try/catch means EXCEPTION); Phase 4 analyzes every C[9][index] constant-table reference and its surrounding context; Phase 5 traces exported functions from a module-registration call (for example AWSCInner.register()) through to the interpreter call that actually executes them; and Phase 6 falls back to runtime injection - either a minimal fake AWSCInner environment executed alongside the real DSL VM code, or a real browser (Selenium with CDP-level native mouse events and anti-detection overrides, or Playwright with network interception) - when static analysis alone can't recover the semantics, noting that pure-protocol replay is "extremely low success rate" because generated tokens bind to the browser's TLS fingerprint, IP, cookies, and headers. It ships a reference opcode-classification table (26 example opcodes mapped to BRANCH/CALL/ARITH/STORE/RETURN/ALLOC/STRING/TABLE/EXCEPTION/DOM types with their characteristic code patterns) and a table of common risk-control status codes (0 = verification passed, 300 = blocked, 8778/8776 = retry needed).

Use it when a protected web asset runs a custom bytecode VM - typically a risk-control or CAPTCHA engine - and the goal is recovering opcode semantics from a JS interpreter loop. Do NOT use it for a standard WASM binary (route to reverse-engineering/languages.md or extract the .wasm and use IDA/Ghidra instead) or for regular Webpack-bundled JavaScript with no custom VM (route to js-reverse/) - the skill's own exclusion table draws this line on the WASM magic bytes, zero-byte ratio, and bundler signature. Runtime capture requires actually controlling the execution environment (a real or automated browser), and the methodology is explicitly noted to transfer between different custom VMs while the specific opcode meanings do not - each target still needs its own Phase 2-4 extraction.

Input is a suspected DSL-VM JavaScript file. Output is a classification (DSL VM / WASM / regular JS), an extracted variable-mapping table, an opcode list with per-opcode type classification, an analyzed range of constant-table references, a traced path from an exported function name to its interpreter call, and - when static analysis alone is insufficient - a working runtime-capture script (fake environment injection, or a Selenium/Playwright automation sequence) that exercises the VM to observe real behavior.

Uses only Python's standard library and re for static analysis; for runtime capture it integrates with Selenium (via Chrome DevTools Protocol commands) or Playwright for headless-browser automation. Cross-references two adjacent skills in its routing table: browser-automation/ for capture strategy detail and js-reverse/ for general JS reverse-engineering (observe/capture/rebuild).

Security researchers and reverse engineers analyzing risk-control or CAPTCHA systems built on custom JavaScript bytecode VMs who need a repeatable identify-extract-trace-capture workflow instead of manually stepping through obfuscated interpreter code.