✎ Edit this page

Status, decisions and roadmap

← Handbook

Status as of 2026-11-17. Update this page when something here changes.

What is real today

Part State
Homepage https://buddhistai.github.io/ Live. One static page. All scores on it are real results, copied by hand from the leaderboard. A banner says the arena and automatic scoring are planned.
Leaderboard https://buddhistai.github.io/buddhist-benchmarks/ Live. Generated from result.json files by CI. One task benchmark with results (Tibetan TOC), plus BuddhistBench and an external card (BDRC Tibetan OCR).
Knowledge base https://buddhistai.github.io/knowledge-base/ Live. 16 task pages, 2 workflow pages (both drafts, not yet reviewed by their owners), 2 experiments, 2 community papers, a Directory of 87 projects and 73 datasets, Languages pages for 21 Buddhist languages with a country overview, and this handbook.
Design system https://buddhistai.github.io/design/ Live, version 1.0.0. Used by the leaderboard and the knowledge base.
buddhist-skills Only released skills: one (toc-extraction-tibetan 1.0.0) and the router ask-buddhistai. Plugin marketplace works.
buddhist-prompts Shared, unmeasured files and skills, as a catalog grouped by kind. Two entries: my-dharma.md (Shoukei Matsumoto, CC0) and a link card to Drupchen’s method note. Holds the Share your files form.
buddhist-benchmarks 15 tasks in tasks/index.json; 1 active (toc-extraction), with data set toc-extraction/bo 2.0.0 (85 items). Published on Hugging Face.
buddhist-bench (BuddhistBench) Version 0.4.0, a scaffold. Format examples only, plus the imported Thai Dhamma exam set (4,075 questions). No live model runs yet (no OpenRouter key).
GitHub → Hugging Face sync Live since 2026-11-17.
Boards and forms All boards are public: Skills, Benchmarks and Prompt library (fed by issue forms), and the Development board for team work.

Current results: Tibetan TOC extraction

Benchmark toc-extraction/bo 2.0.0 (85 passages from two commentaries, one book held out). Headline metric: structure F1. Full table: leaderboard; report: REPORT.md; experiment: 2026-11-toc-tibetan-skills.

Planned or mock (labelled on the sites)

Waiting on NT

Waiting on a reader

Readers volunteer with the language reader form.

Team call, 2026-11-17

Decisions

Action items from the call

Who What State
Kalden Leaderboard web app MVP (FastAPI + React) by 2026-11-18, then the arena. Then add Sitar’s skills and invite other organizations; reply to the Italian group. In progress. See Building the leaderboard app.
Sitar Segmentation, annotation detection and Tibetan spell-check skills; gather data sets that can serve as benchmarks for each. Sitar’s team already has sets for Tibetan annotation detection and an author-name detection skill. Open. Task pages exist here; no benchmarks yet.
Evan Help document Buddhist data workflows (a place where organizations share theirs; structure to be worked out). Try Google’s “elephantgoldfish” for shared project memory; talk to Monlam about feedback. Open. Evan is the natural owner of the knowledge base Workflows section, not confirmed.
Owner not named A skill in the Rails template repo that syncs updated skills from the template into repos built from it. Open.

Open questions

Roadmap

In rough order. Owners in brackets where known.

  1. Leaderboard app MVP reading the existing JSON files and using the design tokens [Kalden]. Then the arena [Kalden].
  2. Domain: point buddhistai.org (Cloudflare) at the sites; decide where the app runs; update links everywhere [NT, Kalden].
  3. Sponsor key and automatic scoring: add OPENROUTER_API_KEY (and BRAVE_API_KEY) as secrets; a GitHub Action that scores a submitted repo; maintainer approval [NT, Kalden].
  4. First model runs: run commercial and open-weight models through OpenRouter on the Thai exam set and the TOC benchmark, so the model table fills in.
  5. Fix the TOC answer key with a Tibetan reader → benchmark 2.0.1 → re-run the top two with real token logging (replaces the estimated costs).
  6. New benchmarks from Sitar’s team: segmentation, annotation detection, Tibetan spell check. Then promote their skills through the leaderboard [Sitar].
  7. Split benchmarks into one repo each with a maturity level, and make buddhist-benchmarks the list. Move the TOC data first and keep tags and Hugging Face names stable.
  8. buddhist-skills holds both picks (Top performance and Best value) per task and language; decide the folder naming.
  9. Monastery exam papers as benchmarks, with permission records. Plan: buddhist-benchmarks#1 with sub-issues #2 to #11.
  10. Thai reader for the imported set; workflow owners review their pages; more workflows from teaching, art and practice.
  11. Community: Discussions is set up (2026-11-17). Still to do: a contributors list, a help-wanted list, decide who posts on LinkedIn and X; maybe a chat channel later.
  12. Homepage: read scores from the leaderboard data instead of a hand-copied array (or replace the page with the app).
  13. Access for partner organizations: before the first partner team gets access, give the Maintainers team (created 2026-11-17) access to the core repos, check member base permission is Read, and add branch rules (pull request + code-owner review) on the core repos (Governance) [NT].

Changes since the call

✎ Edit this page · History · Discuss