Tune visual parameters with vision-model feedback loops

Automates visual tuning for parameters where the success criterion is "does this look or feel right" rather than a number to minimize, animation easing and timing, camera or zoom feel, color grade, layout and spacing, design parameters, render or encoder settings, or prompt parameters. It's the automated counterpart to a human lookdev session, for when there's no person available to sit the loop, and its worked reference point is the screenstudio-alternative skill's iteration.py, which tuned zoom-animation feel via fal-ai/video-understanding.

The loop runs in four steps. First, render several labeled variants across a small parameter spread into ONE artifact, burning each variant's parameters directly onto it (for example "A - 2.2Hz - zeta 0.5"): images become a labeled grid or contact sheet, while video or motion becomes a labeled sequence with a label card or burned-in overlay before or over each clip so the model can compare temporally. Second, make one model call with structured output: send the single artifact with an explicit rubric defining what "good" means and what "too much" or "too little" look like, and ask for per-variant ratings plus concrete suggested new values as JSON, in the shape {"ratings":{"A":n,...},"best_so_far":"X","suggest":[[p1,p2],...]}. Third, go coarse to fine: round one uses a wide spread to locate the region, round two renders the model's suggestions plus the current best into one artifact and asks it to pick the single best, and this usually converges in two rounds. Fourth, stop when sufficient, when the best rates high and the suggestions cluster, then apply the winner.

A set of token, quality and step reductions make the loop cheap: one artifact per round rather than one call per variant, since a 6-variant round becomes one upload and one inference instead of six; burning parameters onto the artifact itself so the model doesn't need separate "variant A used X" context; structured JSON output parsed by regexing the first {...} rather than free-text wrangling; tuning on a short representative sample (a 3-5 second clip, one frame, or one component) rather than the whole asset, then applying the found parameters to the full render once; capping variants at roughly 5-6, since more doesn't improve the model's discrimination and multiplies render and token cost; including one deliberately-bad and one safe-default variant as fixed calibration anchors each round, which gives the model a reference scale and exposes when its "best" is worse than the safe default; stating an independent rubric up front rather than asking "which do you like," since that framing lets the model echo the asker's bias; reusing the round-one winner's render in round two instead of re-rendering it; early-exiting round two if round one's top variant scores 9/10 or higher and the suggestions cluster tightly; and using the cheapest judge that can actually see the failure, an image model for spatial things like layout, color or crop, and a true video model only when the judgment is temporal, like easing, timing or motion smoothness, since those are invisible in still frames.

Learned caveats: the model's pick is an opinion, not ground truth, so the winner should be anchored against and sanity-checked against the safe default before committing, and vision/video models perceive gross differences well but fine ones poorly, so variant spacing should stay perceptible since near-identical variants get noise-rated.

Use it whenever "looks or feels right" is the success criterion and there is no cheap numeric metric to optimize directly, and specifically when there's no human available to sit the loop themselves. Skip it when a real numeric metric exists and correlates with quality, in which case optimize that directly instead of paying a model per step; when the judgment is subjective to a specific user's own taste or brand, since a model's "best" isn't necessarily their best and they should be shown the variants instead; and for only one or two variants, where a direct look is enough.

Input is a small set of rendered variants across a parameter spread, burned with their own labels, plus an explicit rubric defining "good." Output is a structured JSON rating and suggestion per variant, converging over one or two rounds to a single winning parameter set.

Works with any rendering or generation pipeline that can produce labeled image or video variants, judged by a vision model for spatial qualities or a video-understanding model (such as fal-ai/video-understanding) for temporal ones; its worked reference is the screenstudio-alternative skill's iteration.py.

Anyone tuning a visual or motion parameter with no cheap numeric proxy for quality, animation, camera feel, color grade, layout, render settings, who wants a model to run the look-and-compare loop instead of a human.