Deliberate on queries using multiple open-weight LLMs

LLM Council implements Andrej Karpathy's LLM Council concept, powered entirely by Fireworks AI's open-weight models, across three phases: Phase 1 runs all selected models on the query independently and in parallel; Phase 2 has each model rank the others' anonymized responses (labeled A, B, C...) from best to worst with brief reasoning; Phase 3 hands a Chairman model every response (names now revealed) plus every ranking to produce a final synthesis that incorporates the strongest elements, resolves contradictions, and corrects errors the cross-ranking surfaced. Fireworks' speed and pricing is called out as what makes this kind of multi-model deliberation practical rather than slow or expensive.

Use it for its documented three-phase deliberation workflow whenever multiple independent open-weight AI perspectives on a query would be more valuable than a single model's answer. Model and Chairman selection always go through AskUserQuestion (multiselect for the council, single-select for Chairman) rather than being assumed - since the tool only supports 4 options per question but 7 models are available, the skill shows 4 at a time and surfaces the rest via "Other," rotating which four are shown for variety.

Seven models are available via Fireworks model IDs: GLM 5, DeepSeek V3.1, DeepSeek V3.2, MiniMax M2.1, Kimi K2.5, Qwen3 235B, and Llama 4 Maverick. Each run creates a session directory at /tmp/llm-council/<timestamp>/ where every phase's raw output is saved unmodified as JSON (phase1_responses.json with per-model latency and token counts, phase2_rankings.json with the label-to-model mapping, phase3_synthesis.txt) and never summarized or truncated - the skill's hard rule is full transparency, displaying every individual response, every ranking, and the final synthesis, reading from the saved files rather than regenerating output from memory. A final final_output.md compiles the entire deliberation - query, council, Chairman, all Phase 1 responses with latency/token data, all Phase 2 rankings, and the Phase 3 synthesis - for display.

if [ -z "$FIREWORKS_API_KEY" ]; then
  echo "ERROR: FIREWORKS_API_KEY is not set."
  exit 1
fi

Runs entirely against the Fireworks AI inference API (api.fireworks.ai/inference/v1/chat/completions) using a FIREWORKS_API_KEY exported in the shell profile; Python calls use ThreadPoolExecutor to fan out the parallel Phase 1 and Phase 2 requests across however many models are selected. Fireworks bills per token, so more models or longer queries cost more (current pricing at fireworks.ai/pricing), and each call's latency is tracked so Fireworks' inference speed is visible in the output.

Anyone who wants a second (or fifth) opinion from multiple open-weight LLMs before trusting an answer - comparing how DeepSeek, GLM, Kimi, MiniMax, Qwen, and Llama models independently handle the same query, seeing how they rank each other's work, and getting a synthesized answer that reconciles the disagreements rather than picking one model's response at random.