Beatles Bench · Run it

Run the Beatles Bench.

The bench is open source. The repository holds the songs, the questions and their right answers, and a saved recording of every answer. Each function has its own folder with a script to run it. The data is CC BY-SA 4.0.

Get the bench.

git clone https://github.com/botassembly/beatles-bench
cd beatles-bench

Replay the Jev run. It is free and needs no key.

./run.sh

Rerun it all fresh, on Jev or on your own backend.

export THINKTHEN_BASE_URL=https://your-server/v1
export THINKTHEN_API_KEY=...
./run.sh my-rerun

Replay every answer for free. Rerun it on the model you want to test.

On GitHub: github.com/botassembly/beatles-bench