Pre-register technical decisions with expert AI personas

ask-the-board

A board of expert personas whose every decision is a pre-registered,
time-anchored, reality-graded bet. Not a chatbot that agrees with you -- a board
that keeps score, before the fact.

Landing page & docs: https://danilushin.github.io/asktheboard/

Mechanism on sample data - the 60-second, no-key walkthrough below reproduces it exactly.

pip install asktheboard

Why this exists

Anyone can clone a "panel of AI personas" in a weekend, and a dozen have. The
debate mechanic is a commodity. What it leaves out is the thing that makes advice
worth trusting: a record of having been right before the outcome was knowable.
That record is hard to fake -- you can buy model outputs, but you can't
back-date a timestamp. It only accrues the slow way: by calling decisions in
advance and letting reality grade them, one resolution date at a time.

So ask-the-board records, for every decision:

1. your stated prior (what you believed going in),
2. the per-seat dissent vector -- each seat's stance + its own probability,
3. a dated, falsifiable prediction, anchored before the outcome is knowable,
4. on the resolution date, reality's realized outcome, auto-reconciled into a
   Brier/calibration score per seat.

The board-minute is a git-committable ADR. Your git history is the external
attestation of the anchor timestamp. The accumulating, reality-graded record is
the durable asset.

See it keep score (60s, no API key)

create -> resolve -> score is pure data -- no LLM, no key, no network. This is a
worked example on sample data: you supply the outcome with resolve, and the
engine computes each seat's Brier score (lower is better). It shows the mechanism,
not a track record -- the integrity comes from the anchor timestamp your git history
attests, which no demo can fabricate. The committed artifacts live in
examples/.

# pip-installed (no repo)? paste the sample spec below. Cloned the repo?
# skip the heredoc and use --spec tests/sample_minute.json instead.
cat > sample_minute.json <<'JSON'
{
  "id": "2026-01-postgres-vs-vectordb",
  "question": "Adopt Postgres + pgvector, or a dedicated vector DB?",
  "prior": "Leaning toward a dedicated vector DB for the embeddings workload.",
  "decision": "Stay on Postgres + pgvector for now.",
  "prediction": {
    "statement": "We will NOT migrate off Postgres for vectors within 3 months.",
    "resolution_date": "2026-04-01",
    "board_probability": 0.75
  },
  "seats": [
    {"seat": "pragmatist", "stance": "affirm", "probability": 0.8, "rationale": "Boring tech; pgvector is enough at this scale."},
    {"seat": "skeptic", "stance": "dissent", "probability": 0.35, "rationale": "Recall/latency will bite once the corpus 10x's."}
  ],
  "created_at": "2026-01-05T10:30:00"
}
JSON

asktheboard create  --spec sample_minute.json
asktheboard resolve --id 2026-01-postgres-vs-vectordb --outcome true
asktheboard score

seat               n  mean_brier  wins  losses
----------------------------------------------
pragmatist         1       0.040     0       0
skeptic            1       0.423     0       1

Full walkthrough + committed artifacts: examples/README.md.

And a real one, still open: this repo pre-registered a board-minute about its own
launch -- examples/open-minute.md, anchored in git on
2026-06-26, resolving 2026-09-24. No score yet; that's the point. The board may turn
out wrong, and the anchor means it can't pretend otherwise.

Live bet #1 (resolves in days): the board's call on the June 2026 US jobs report
-- examples/2026-06-jobs-report.md,
anchored 2026-06-27, resolving 2026-07-02 against the BLS Employment Situation
release. The board says +150k or more at 56%; the skeptic dissents at 40%. Bet #1 of
a public, recurring cadence -- come back on the date and watch it grade against a source
nobody controls.

BYOK (bring your own API key)

The engine ships no provider and makes no calls of its own. You supply your own
LLM key; you pay your own inference. The open-source core therefore costs nothing
to run at any scale -- the cost lives with the user, not a host. (A managed,
capped hosted tier -- for people who would rather not manage keys -- is the
separate, paid product.)

Hosted tier -- join the waitlist

The OSS engine is free forever and runs on your own key. If you'd rather not
manage keys -- or you want the aged, reality-graded public scoreboard hosted
for you -- a managed, capped paid tier is coming.

Want early access? Email support@chu6a.dev with the subject waitlist
(a one-liner on what you'd decide with it helps, but isn't required). No spam --
one note when it opens.

Integrity guarantees (enforced in code)

• A prediction cannot be pre-registered to resolve in the past (no backfilling
  an "old" call onto a known outcome).
• A minute cannot be graded before its resolution date -- the outcome must not
  be knowable yet. That is what makes it foresight.
• The anchor timestamp and the prediction are frozen once created; grading
  never moves them.

See tests/test_model.py -- these are the load-bearing tests.

Quick start

pip install asktheboard

# pre-register a decision (board-minute spec is JSON -- see "See it keep score" above)
asktheboard create  --spec sample_minute.json

# ... months later, on/after the resolution date, grade it against reality
asktheboard resolve --id 2026-01-postgres-vs-vectordb --outcome false

# per-seat calibration scoreboard, best-calibrated first
asktheboard score

create writes both <id>.json (the record) and <id>.md (the committable ADR)
into board-minutes/.

Convene a board (BYOK)

create pre-registers a minute you wrote by hand. convene runs the live LLM
fan-out: every seat answers through your key, and the board's consensus
probability is the mean of the seats' calls. It ships no provider -- bring an
OpenAI-compatible endpoint (HTTPLLMClient is stdlib-only, zero dependencies).

from asktheboard import convene, Seat, HTTPLLMClient

minute = convene(
    id="pgvector-scale",
    question="Will pgvector hold our scale, or do we need a dedicated vector DB?",
    prior="leaning postgres to avoid a new service",
    decision="stay on postgres + pgvector",
    statement="pgvector serves p95<150ms at 50M embeddings without a dedicated DB",
    seats=[Seat("pragmatist", "ML researcher"), Seat("skeptic", "find the failure mode")],
    client=HTTPLLMClient(model="gpt-4o-mini"),   # reads OPENAI_API_KEY
    decision_type="library",                     # -> 90-day resolution horizon
)

Or from the CLI (key in OPENAI_API_KEY):

asktheboard convene --spec convene.json --model gpt-4o-mini

Any OpenAI-compatible API works -- point --base-url (or HTTPLLMClient(base_url=...))
at OpenRouter, Together, or a local server. The engine still makes no calls of its
own; it only ever speaks through the client you pass.

Bundled roster -- seat a board by name

You can always hand-write Seat(name, persona). But a sensible default board ships
in the box: a curated set of role archetypes (the architect, the skeptic, the
operator -- functions, not impersonations of real people) and a few named panels,
so seating one is a single lookup.

from asktheboard import convene, panel, seats, HTTPLLMClient

minute = convene(
    id="pgvector-scale",
    question="Will pgvector hold our scale, or do we need a dedicated vector DB?",
    prior="leaning postgres",
    decision="stay on postgres + pgvector",
    statement="pgvector serves p95<150ms at 50M embeddings without a dedicated DB",
    seats=panel("tech"),                  # architect + skeptic + pragmatist
    # seats=seats(["architect", "operator", "skeptic"]),   # or pick your own
    client=HTTPLLMClient(model="gpt-4o-mini"),
    decision_type="library",
)

From the CLI, pass --panel or --seats instead of putting seats in the spec:

asktheboard roster                                   # list seats + panels
asktheboard convene --spec d.json --model gpt-4o-mini --panel tech
asktheboard convene --spec d.json --model gpt-4o-mini --seats architect,skeptic

Panels: tech (architect/skeptic/pragmatist), decision (strategist/skeptic/researcher),
ops (operator/architect/skeptic), default (architect/skeptic/pragmatist/strategist).
skeptic sits on every panel by design -- a board with no dissent keeps no honest score.

Decision types -> default horizons

A minute is only foresight if it has a date by which reality can grade it.
decision_type picks a sensible default horizon so the common case is one lookup
(and a 5-year horizon on a library swap stands out as dishonest):

Short-latency first on purpose: a fresh board earns a track record on fast library
calls before anyone trusts its slow architecture bets. Pass an explicit
resolution_date= to override.

A contrarian win

When a seat dissents from the board and turns out more right than the
consensus, that is a contrarian win -- the gold the public scoreboard is built
from. The board changed (or should have changed) its mind, and reality later
stamped the dissenter vindicated.

Stability

What's shipped: the foresight engine (data model + grading + committable
ADR) and the BYOK LLM fan-out that produces a board-minute
(asktheboard.convene, behind the asktheboard.llm Protocol). No provider is
bundled -- you plug in your own key.

The public API is 0.x / unstable. The LLMClient / HTTPLLMClient surface
and the board-minute JSON schema may change before 1.0 -- pin a version if you
depend on them.

Built with

Built by Dan Ilushin with Claude (Anthropic) in
the loop. Contributions welcome -- see CONTRIBUTING.md
(DCO sign-off) and SECURITY.md.