# `audit` grades a run.

You already know the right answer for some of your records. `audit` grades saved answers against those answers at any bar. It sends no request.

Here Jev was asked whether each of 10 songs is on Abbey Road, from the title alone. The slide's no-context column shows these answers. The slide reads each answer at the band 0.2:0.8. A red cross marks a wrong answer, and an amber ? marks a not-sure answer. The first example below runs `audit` on the same 10 songs. The [diff page](/learn/beatles-bench/diff/) asks about them again with context.

## Run it

*At the band 0.2:0.8, 3 are right, 2 are wrong, and 5 are not sure.*

```
thinkthen audit shown.jsonl shown-key.jsonl \
  --threshold 0.2:0.8 |
jq '{
  songs: .rows,
  right,
  wrong,
  not_sure: .unsure
}'
```

*Output*

```
{
  "songs": 10,
  "right": 3,
  "wrong": 2,
  "not_sure": 5
}
```

*exit 0*

## Try the default bar

*At the default bar of 0.5, 5 are right. Jev says yes wrongly 5 times.*

```
thinkthen audit shown.jsonl shown-key.jsonl |
jq '{
  songs: .rows,
  right,
  wrong_yes: .false_yes,
  missed_yes: .false_no
}'
```

*Output*

```
{
  "songs": 10,
  "right": 5,
  "wrong_yes": 5,
  "missed_yes": 0
}
```

*exit 0*

## Change the bar

*At 0.78, 8 are right. audit suggests 0.78 for the whole recorded run.*

```
thinkthen audit shown.jsonl shown-key.jsonl \
  --threshold 0.78 |
jq '{
  songs: .rows,
  right,
  wrong_yes: .false_yes,
  missed_yes: .false_no
}'
```

*Output*

```
{
  "songs": 10,
  "right": 8,
  "wrong_yes": 2,
  "missed_yes": 0
}
```

*exit 0*

## The lesson

From the title alone, half the answers fall inside the band. At 0.5, Jev says yes to 5 songs from other albums. At 0.78, only A Day in the Life and The Long and Winding Road remain wrong, and no Abbey Road song is lost.

The answers you already know grade any bar, at no cost.

On GitHub: [github.com/botassembly/beatles-bench/tree/main/examples/audit](https://github.com/botassembly/beatles-bench/tree/main/examples/audit)