Experiments
This section of the BuddhistAI knowledge base is the record of BuddhistAI experiments: what we tried, why, and what we found. Each experiment has its own folder with a plan written before it ran, a dated log, and a report ready to share in Discussions or any forum.
Code, data and scores live in the other repos. This section holds the reasoning and links to them. (It used to be the separate research repo, which is now archived. Its full history is kept here.)
| Repo | Holds |
|---|---|
| buddhist-skills | The promoted skills users install |
| buddhist-benchmarks | Test sets, scorers and leaderboards for skills |
| buddhist-bench | BuddhistBench: Q&A benchmark ranking models on Buddhist knowledge |
| Workshop repos, e.g. toc-extraction-tibetan | Skill variants, runs and ranking reports for one skill |
Experiments
| Experiment | Status | Question |
|---|---|---|
| 2026-11-toc-tibetan-skills | Concluded, re-run pending | Which skill finds the sa-bcad outline of a Tibetan commentary best? |
| 2026-11-ai-intervention-study | Planned | Does publishing reliable information about a text change what AI models answer about it, and how fast? |
| 2026-11-tibetan-french-translation | Planned | Do skills built from Drupchen’s method translate Tibetan colophons into French better than plain Claude, and does writing from meaning units help? |
Status is one of: Planned (plan written, not started), Running, Concluded, Abandoned (with a note on why). Keep this table current when an experiment changes status.
How an experiment works
- Open an issue with the “Idea” form. Discuss there until it is worth running.
- Make the folder
experiments/<yyyy-mm>-<slug>/, named by the month it starts. Copy the three files from templates/experiment/. - Write
PLAN.mdbefore running anything. State the question, the hypothesis, the method, the metric and what result would count against the hypothesis. Commit it. Once the experiment starts, don’t edit the plan: record changes inLOG.mdas deviations, with the reason. Git history is the proof that the goalposts didn’t move. This matters most for experiments that try to change the world and then measure it, like the intervention study. - Optionally post the plan in Discussions (or your own forum) to invite feedback before starting.
- Keep
LOG.mdas you go: dated entries, newest last. Note runs, interventions, surprises, and anything you changed. - Write
REPORT.mdwhen it concludes. It should stand alone so it can be pasted into Discussions or a forum: result first, then what was measured, the numbers, the known problems and next steps. Link to data and results in the other repos at exact commits or versions. - Post the report in Discussions (and any forum you use) and add its link to the report’s header.
- Open issues for follow-ups (“To-do” or “Re-run” templates), each linking back to the folder.
Issues and the board
Issues track work; folders hold knowledge. Every issue names its experiment folder, so the reasoning is always one click away. Labels: idea, todo, re-run, plus one exp:<slug> label per experiment.
The BuddhistAI research board shows the pipeline: Ideas, Planned, Running, Writing up, Done.
Rules
- Link, don’t copy. Data, code and long designs stay in their own repos; refer to them by commit or version so a link still shows what we used.
- Numbers in a report come from a results file you can link to.
- A re-run after a benchmark version change is a new LOG entry and, if it changes the conclusion, a new report section. Don’t overwrite old numbers.
- Write for a forum post: plain language, result first, short sentences.
License
MIT, like the rest of the knowledge base. Written by the BuddhistAI contributors; anyone can propose an experiment with the “Idea” issue form.

BuddhistAI Knowledge base