# `diff`

`thinkthen diff A B` prints only the saved answers that changed between two runs. It sends no request.

## Two runs, or two cuts

With two files, `diff` pairs each answer in `A` with the answer in `B` for the same record and answer name. With one file and `--compare-threshold`, it reads the same run under two rules. `A` reads under `--threshold`, and `B` under `--compare-threshold`. It warns when two paired answers came from different questions.

*One saved run, read at 0.5 and then at the band 0.2:0.8. Five answers become not sure, and each is withdrawn. No request is sent.*

```
thinkthen diff memory.jsonl \
  --threshold 0.5 \
  --compare-threshold 0.2:0.8 \
  --key key.jsonl |
jq -c 'if .summary
  then .summary | {changed, right_a, right_b, gained, lost}
  else [.id, .from, .to, .effect]
  end'
```

*Output*

```
["Blackbird","no","unsure","withdrawn"]
["Glass Onion","yes","unsure","withdrawn"]
["In My Life","yes","unsure","withdrawn"]
["Something","yes","unsure","withdrawn"]
["Taxman","yes","unsure","withdrawn"]
{"changed":5,"right_a":5,"right_b":3,"gained":0,"lost":0}
```

*exit 0*

## What it prints

One line for each changed answer, in the order of `A`. For `decide` and `choose`, a line holds `id`, `name`, `from`, `to`, the two probabilities, `key`, and `effect`. For `recognize` and `relate`, diff prints one line for each changed record. That line holds `lost`, `gained`, and `changed_kind`. A summary line comes last. It counts the records, the changes, each move, and the right answers on each side.

With `--key`, each change gets an effect:

| Effect | From | To |
| --- | --- | --- |
| `gained` | wrong | right |
| `lost` | right | wrong |
| `resolved` | not sure or tied | right or wrong |
| `withdrawn` | right or wrong | not sure or tied |
| `changed` | any other change |  |

A not-sure answer and a tied answer count the same way. Each one becomes `resolved` when it turns right or wrong. Without a key, every effect is `null`. The [Beatles diff page](/learn/beatles-bench/diff/) compares a run from the title alone with a run that had context.

[audit](/reference/audit/) · [The diff specification](https://github.com/botassembly/thinkthen/blob/main/specification/diff.md)

On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen)