✎ Edit this page

Conventions

← Handbook

Tasks and languages

Skills

Benchmark data format

One standard for every task benchmark, so each loads with Hugging Face datasets:

tasks/<task>/data/<lang-code>/v<major>/
  test.jsonl     one item per line: {"id": ..., "input": ..., "target": ..., "metadata": {...}}
  README.md      Hugging Face dataset card: YAML front matter (license, language, configs → test.jsonl),
                 then version, purpose, item count, source and license of the texts,
                 who made the answers, and a worked example

BuddhistBench uses its own item schema (tracks/<track>/data/<lang>/<split>.jsonl, docs/schema.md) because it is a Q&A set with traditions, reviewers and permissions. Its test.jsonl is private and never committed.

Versions

Everything that is scored has a semantic version (MAJOR.MINOR.PATCH).

Thing Where the version is Patch Minor Major
Benchmark data set Dataset card README.md + task CHANGELOG.md; tag <task>/<lang>@X.Y.Z (made by CI) Fixed references New items Scorer or format change: new v<major>/ folder
Scorer SCORER_VERSION in score.py and TASK.md — — Any change that moves scores; bump every data set’s major too
Skill or variant metadata.version in SKILL.md Wording fix New behaviour Changed output
BuddhistBench VERSION + each track.json + CHANGELOG.md; tag vX.Y.Z Gold fixes New items, tracks, languages Format, scorer or judge change
Workflow page version in its front matter (source_version is the owner’s own version) Small fix New step or task Restructure
Design system version in tokens.json + CHANGELOG.md Changed value New token or part Renamed or removed class

Licenses and imports

Design system

One look for every site. Source: BuddhistAI/design, style guide at https://buddhistai.github.io/design/.

Writing

Credit

✎ Edit this page · History · Discuss