Track line-level authorship in AI-edited code

Us vs. Them

Line-level provenance for text under agentic editing - who wrote this line, us or them? - derived from a text's version history. Use it as library or as CLI tool.

The problem

With agentic coding and editing, provenance becomes a pertinent question. Text a human wrote or edited should be considered close to sacred: an agent should be hesitant and have a very good reason to touch it. Slop another agent has produced, on the other hand, is completely up for grabs.

A use case for this: Take a mostly vibecoded app in which you want to establish some corners in the code where you want to assert your ideas and ownership. You surely don't want another agent bulldoze over this piece of code in the next session.

Another use case: the README.md, originally generated, where you rewrite the opening paragraphs. The agent should feel free to redo or append parts further downwards but should really think twice changing anything in the opener.

How it works

The main constraint under which this should work is that this should not require for text to be marked up specifically for that. Omnipresent plain text (markdown) should be supported as is.

The only thing to leverage then, is that each new version of a text is created under identifable authorship - of either a human or an agent.

The output of an evaluation over a given text is a set of ranges - "islands" of human-authored lines inside a "sea" of machine generated text. Technically based on simple diffing, this is the guiding metaphor for development of the algorithm. We don't want to track authorship of individual lines only, but of meaningfully coherent pieces of text. So joining, splitting apart, and dilution of authorship are behaviours to be factored in, also in such a manner that we don't converge in full sea or full island.

Usage

Using us-vs-them as CLI tool requires bbin for a local install.

make install

A git repository is already a history of versions each carrying a provenance marker - every revision of the file, in order, with the author of the change that made it.

To use it anywhere inside a git repository:

us-vs-them --ours dan@eighttrigrams.net README.md

This yields a listing like

1-3          0.00
4            1.00
5-7          0.00
8-20         0.46
21-164       0.00

where 1.0 means fully human authored range.
0.46 means originally human authored range, modified by agents to a certain degree.
0.00 means fully agent authored.

Parameters are:

--ours: these are the humans, everone else is considered an agent

--theirs: these are agents, everyone else is considered human

Name whichever side is the shorter list.
Passing both arguments at the same time will be rejected.

Development

make test
make test-cljs   # the same suite on node, from the cookbook checkout beside this one

caution and core are .cljc. Cookbook compiles them into its browser bundle - its Recipes are encrypted, so its server cannot read the history the split is drawn from and the client has to - and a library whose suite only ever runs on one of two hosts is one that will drift on the other. There is no npm here on purpose, so make test-cljs points at the consumer that has one. That runs cookbook's whole node suite, most of which is cookbook's rather than this library's, so it can go red for a reason that is not yours - read the namespace in the failure. See the Makefile, and hosts_test.cljc for the two places the hosts do not agree: one number spelled differently, and one degenerate input refused on one host and invented over on the other.

Behaviour

The best way to understand the behaviour is to have a look at caution_test.cljc.