---
title: Fail a CI job on a yes-or-no question
description: Make a CI step pass when a pull request description says how the change was tested, fail when it doesn't, and ask a reviewer when ThinkThen is not sure.
goal: A developer leaves with a CI script that passes on yes, fails on no, sends not sure to a reviewer, and never passes when the check itself fails.
reader: A developer who wants a CI job to check what a pull request description, changelog or commit message says
kind: how-to
verified: 2026-11-03
---

# Fail a CI job on a yes-or-no question

At the end, your CI job passes when a pull request description says how the change was tested, fails when it doesn't, and asks a reviewer when ThinkThen is not sure.

## 1. Save the description in a file

Your CI job writes the pull request description to `pr.md`. This one doesn't say how the change was tested:

```markdown title="pr.md"
Fix the rounding of invoice totals.

Totals now round half up to two decimals, as the tax rules require.
```

## 2. Ask the question with `--check`

`--check` makes the answer the exit code: 0 for yes, 1 for no and 3 for not sure. The low and high bars send an unclear description to not sure.

```sh
thinkthen decide --check --low 0.3 --high 0.7 \
  'Does the description say how the change was tested?' \
  pr.md
status=$?
echo "exit code $status"
case "$status" in
  0|1|3) ;;
  *) exit "$status" ;;
esac
```

Output

```text
false
exit code 1
```


No: the description says what changed, not how it was tested, so the exit code is 1.

## 3. Turn each exit code into the job's result

`check-pr.sh` reads the exit code and prints a note for the author or the reviewer. Any other code means the check itself didn't run, so the job fails with 2 rather than passing.

```sh title="check-pr.sh"
#!/bin/sh
# Pass on yes. Fail on no, not sure or an error.
thinkthen decide --check --low 0.3 --high 0.7 \
  'Does the description say how the change was tested?' \
  "$1" > /dev/null
case $? in
  0) echo 'passed: the description has a test plan' ;;
  1) echo 'failed: say how you tested the change'
     exit 1 ;;
  3) echo 'not sure: a reviewer must read the test plan'
     exit 1 ;;
  *) echo 'error: the check did not run'
     exit 2 ;;
esac
```

```sh
sh check-pr.sh pr.md
```

Output

```text
failed: say how you tested the change
```


The job fails, and the note tells the author what to add.

## 4. Add the script to your CI job

Run `sh check-pr.sh pr.md` as a step after the step that writes `pr.md`. The job needs `thinkthen` installed and your key in a CI secret. [Installation](/install/) and [Configuration](/configuration/) cover both.

## Check the result

Run the script on a description that says how the change was tested:

```markdown title="pr-tested.md"
Fix the rounding of invoice totals.

Totals now round half up to two decimals, as the tax rules require.

Tested: I added unit tests for totals that end in 5, and ran
the full test suite, which passes.
```

```sh
sh check-pr.sh pr-tested.md
```

Output

```text
passed: the description has a test plan
```


Yes: the description names the tests, so the job passes.

A writer who knows the question can word a description to pass it. Treat the check as a reminder, and keep a reviewer on the not-sure branch. [Exit codes](/command-line/exit-codes/) lists every code the script can see.

## Next

[Measure answers against your own labels](/how-tos/measure-answers-against-your-own-labels/) checks your bars against answers you wrote yourself.