diff

thinkthen diff A B prints only the saved answers that changed between two runs. It sends no request.

Two runs, or two cuts

With two files, diff pairs each answer in A with the answer in B for the same record and answer name. With one file and --compare-threshold, it reads the same run under two rules. A reads under --threshold, and B under --compare-threshold. It warns when two paired answers came from different questions.

thinkthen diff memory.jsonl \
  --threshold 0.5 \
  --compare-threshold 0.2:0.8 \
  --key key.jsonl |
jq -c 'if .summary
  then .summary | {changed, right_a, right_b, gained, lost}
  else [.id, .from, .to, .effect]
  end'

["Blackbird","no","unsure","withdrawn"]
["Glass Onion","yes","unsure","withdrawn"]
["In My Life","yes","unsure","withdrawn"]
["Something","yes","unsure","withdrawn"]
["Taxman","yes","unsure","withdrawn"]
{"changed":5,"right_a":5,"right_b":3,"gained":0,"lost":0}

What it prints

One line for each changed answer, in the order of A. For decide and choose, a line holds id, name, from, to, the two probabilities, key, and effect. For recognize and relate, diff prints one line for each changed record. That line holds lost, gained, and changed_kind. A summary line comes last. It counts the records, the changes, each move, and the right answers on each side.

With --key, each change gets an effect:

A not-sure answer and a tied answer count the same way. Each one becomes resolved when it turns right or wrong. Without a key, every effect is null. The Beatles diff page compares a run from the title alone with a run that had context.

audit · The diff specification