Calibrate First

Calibrate First uses assay-lite to measure whether daily, cross-sectional returns can reveal a signal before interpreting a backtest. The single-file Python tool runs offline. Its floor operation injects synthetic signals of known strength into the user's returns to establish the smallest detectable effect for the chosen prices and horizon.

The skill also tracks repeated testing. Every test is charged to an append-only registry before computation, and the threshold is 0.05 divided by every trial ever charged. The bar for the last charged trial differs from the bar the next test will face. Rerunning a signal, changing a parameter, or choosing another horizon counts as another trial; deleting registry lines or selecting a fresh home directory must not reset that history.

A result below the calibrated floor is UNDECIDABLE: the effect is too small for the data to see. PASS means the signal cleared the multiple-testing bar, while FAIL means the observed effect is large but unreliable. Neither verdict establishes profitability or gives permission to trade.

Use it when evaluating whether a trading signal, factor, or ranking has an edge, when interpreting an information coefficient or a null cross-sectional result, or before trying several variants and selecting the best. Measure floor before interpreting any result, and use the registry's applicable threshold rather than a conventional unadjusted 0.05.

It does not cover single-asset timing, intraday signals, live trading, position sizing, or portfolio simulation. Tests use Spearman IC on daily cross-sectional data and exclude transaction costs. Bonferroni correction can be conservative for correlated trials. Scaling the independent-noise floor by daily-IC dispersion accounts for factor structure on average rather than exactly.

Prices use date,asset,close columns. Signals use date,asset,signal and must be known at that date's close. The procedure checks those inputs, runs breadth, and reports effective independent bets beside the asset count. It then measures floor for the exact prices and horizon, records a one-sentence expectation with its rationale, and runs test once.

python /absolute/reviewed/assay_lite.py breadth  --prices prices.csv

The report quotes the verdict line verbatim and includes IC, p, the threshold faced, the dispersion ratio, and the scaled floor. The registry preserves the full trial count.

Execution requires a previously reviewed local assay_lite.py at an explicit absolute path, verified against commit 78104a4356541affe2a469dfdf69cb1e418eb0c5, plus a fully locked and reviewed numpy/pandas/scipy environment. The skill neither downloads code nor resolves dependencies. If these prerequisites are missing, it returns the input schema and analysis plan without executing, and it must not substitute a same-named project file.

A separate installation task must stage and inspect every file and dependency with installation hooks disabled, verify immutable bytes, and obtain approval before execution or registry writes. uv caching provides no isolation. Once approved, assay-lite makes no network calls, needs no API keys, reads only the specified CSVs, and writes within its own .assay-lite/ folder. Its append-only registry must not be edited or deleted.