✎ Edit this page

Experiments

This section of the BuddhistAI knowledge base is the record of BuddhistAI experiments: what we tried, why, and what we found. Each experiment has its own folder with a plan written before it ran, a dated log, and a report ready to share in Discussions or any forum.

Code, data and scores live in the other repos. This section holds the reasoning and links to them. (It used to be the separate research repo, which is now archived. Its full history is kept here.)

Repo Holds
buddhist-skills The promoted skills users install
buddhist-benchmarks Test sets, scorers and leaderboards for skills
buddhist-bench BuddhistBench: Q&A benchmark ranking models on Buddhist knowledge
Workshop repos, e.g. toc-extraction-tibetan Skill variants, runs and ranking reports for one skill

Experiments

Experiment Status Question
2026-11-toc-tibetan-skills Concluded, re-run pending Which skill finds the sa-bcad outline of a Tibetan commentary best?
2026-11-ai-intervention-study Planned Does publishing reliable information about a text change what AI models answer about it, and how fast?
2026-11-tibetan-french-translation Planned Do skills built from Drupchen’s method translate Tibetan colophons into French better than plain Claude, and does writing from meaning units help?

Status is one of: Planned (plan written, not started), Running, Concluded, Abandoned (with a note on why). Keep this table current when an experiment changes status.

How an experiment works

  1. Open an issue with the “Idea” form. Discuss there until it is worth running.
  2. Make the folder experiments/<yyyy-mm>-<slug>/, named by the month it starts. Copy the three files from templates/experiment/.
  3. Write PLAN.md before running anything. State the question, the hypothesis, the method, the metric and what result would count against the hypothesis. Commit it. Once the experiment starts, don’t edit the plan: record changes in LOG.md as deviations, with the reason. Git history is the proof that the goalposts didn’t move. This matters most for experiments that try to change the world and then measure it, like the intervention study.
  4. Optionally post the plan in Discussions (or your own forum) to invite feedback before starting.
  5. Keep LOG.md as you go: dated entries, newest last. Note runs, interventions, surprises, and anything you changed.
  6. Write REPORT.md when it concludes. It should stand alone so it can be pasted into Discussions or a forum: result first, then what was measured, the numbers, the known problems and next steps. Link to data and results in the other repos at exact commits or versions.
  7. Post the report in Discussions (and any forum you use) and add its link to the report’s header.
  8. Open issues for follow-ups (“To-do” or “Re-run” templates), each linking back to the folder.

Issues and the board

Issues track work; folders hold knowledge. Every issue names its experiment folder, so the reasoning is always one click away. Labels: idea, todo, re-run, plus one exp:<slug> label per experiment.

The BuddhistAI research board shows the pipeline: Ideas, Planned, Running, Writing up, Done.

Rules

License

MIT, like the rest of the knowledge base. Written by the BuddhistAI contributors; anyone can propose an experiment with the “Idea” issue form.

✎ Edit this page · History · Discuss