Update community notebooks to rapidfireai 0.16.1 (run-validated)

[kamran-rapidfireAI]

Changes

• Updates all 8 community notebooks from rapidfireai==0.14.0 to rapidfireai==0.16.1 and fixes issues found while executing the notebooks end-to-end.
• Keeps the change notebook-only: no library/runtime code is touched.
• Adds outcome-aware SFT notebook waits after Colab background run_fit() execution so failed or incomplete training runs raise before downstream metric cells continue.

Changelog Content

Additions

• community_notebooks: add a UCSB course-catalog dataset download cell because those files are not shipped by rapidfireai init.
• community_notebooks: add a RapidFire services startup cell for rag-fiqa-context-optimization, required by the MLflow AI Gateway generator path.
• community_notebooks: install FlagEmbedding in the QASPER setup cell because phase 2b imports it inside Ray actors.

Changes

• community_notebooks: pin all 8 notebooks to rapidfireai==0.16.1.
• community_notebooks: migrate the RAG notebooks to the 0.16 RFLangChainRagSpec cfg-dict API and switch FiQA generation to RFAPIModelConfig.
• community_notebooks: pin transformers>=4.55.2,<5.0.0 in SFT installs to avoid Colab's transformers 5.x incompatibility with trl==0.21.0.
• community_notebooks: wait for Colab background run_fit() completion in SFT notebooks, then validate experiment.get_runs_info() so only all-Completed runs are treated as successful.
• community_notebooks: update TensorBoard log paths to tensorboard_logs/<experiment>/<run>.
• community_notebooks: flatten nested 0.16 *_cfg knobs in RAG result analysis.

Fixes

• community_notebooks: deduplicate chunk-level hits before NDCG in FiQA so duplicate chunks cannot inflate DCG above IDCG.
• community_notebooks: remove FiQA's manual ray.init(num_cpus=2) restart, which can starve the 0.16 evals engine.
• community_notebooks: use 2 shards for FiQA's 6-query demo evaluation.
• community_notebooks: use a sqlite MLflow store for FiQA demo metrics instead of the retired filesystem backend path.
• community_notebooks: fix PII "Run all" plot ordering and normalize its batch polling status names.
• community_notebooks: make QASPER phase 2b summary, plot, and heatmap cells tolerate metrics that the phase does not persist.
• community_notebooks: strip stale execution counts from NFCorpus.

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have tested this change manually
• I have tested this change in the following environments:
  • Local development
  • Docker environment
  • Other: headless end-to-end execution of all 8 notebooks on GCP GPU VMs, plus lightweight local PR-review validation

Test methodology

Every notebook in this PR was executed top to bottom with papermill in a fresh Python 3.12 venv on GCP L4 GPU VMs configured to mirror Colab. The notebooks executed their own %pip install cells, /content and RapidFire state were wiped between runs, and failures were fixed forward until each notebook passed.

Validation for the latest review fix:

• Notebook JSON/output state checked for all edited SFT notebooks.
• Modified training cells parsed successfully.
• Synthetic completed, failed, empty, and unfinished run-status cases passed.
• git diff --check passed.
• Cursor Bugbot passed on commit bfa6482c9cd69233e294ecc696f71973383b7564.
• Actual Colab Enterprise T4 validation passed through the previously failing Experiment(...) initialization path on commit aabd819 (execution 5828237635323166720).

Per-notebook results:

Notebook	Result	Wall time	Evidence
sft-ecommerce-retail-chatbot	pass	~7 min	8 runs, eval_loss 2.0-2.6, metrics table, plots, artifact zip
sft-cybersecurity-qa	pass	~25 min	4 runs plus baseline, 40 eval checkpoints, checkpoint eval table
sft-child-facing-chatbot	pass	~2h20m	4 runs, BERTScore per checkpoint, 36 chunk-tasks
sft-pii-masking-redaction	pass	~1h20m	8 gpt2 configs, plots, TensorBoard scalar summary
rag-nfcorpus-biomedical-qa	pass	~30 min	4 configs, Precision 0.50-0.55, NDCG@5 0.29-0.31, MRR 0.55-0.65
rag-ucsb-course-catalog	pass	~30 min	4 configs including new dataset download cell
rag-fiqa-context-optimization	pass	~25 min	3 configs via gpt-4o-mini through MLflow AI Gateway, Precision up to 0.75, NDCG@5 0.68
rag-qasper-scientific-paper-qa	pass	~5.5h cumulative	12 chunking, 5 embedding, and 3 hybrid configs validated via notebook CSV resume logic

Validation used an uncommitted google.colab shim for Colab-only affordances. OPENAI_API_KEY came from the environment and is not committed.

Screenshots (if applicable)

N/A - notebook-only changes; executed outputs are intentionally stripped per repo convention.

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published

Performance Impact

None. This PR is notebook-only. The training-completion assertion only runs after notebook training waits finish and does not affect RapidFire library runtime behavior.

Related Issues

Related to #285, #286, #208

---

Note

Low Risk
Changes are confined to tutorial notebooks and documentation-style cells; they do not alter production library behavior or authentication/data paths in the repo.

Overview
Notebook-only refresh of all eight community_notebooks from rapidfireai 0.14.0 ΓåÆ 0.16.1, aligned with end-to-end Colab/GPU runs. No library or runtime code changes.

RAG notebooks move RFLangChainRagSpec to the 0.16 *_cfg dict API (embedding_cfg, vector_store_cfg, search_cfg, reranker_cfg). FiQA switches generation from RFOpenAIAPIModelConfig to RFAPIModelConfig with MLflow AI Gateway endpoint_config, adds MLflow/RapidFire service env vars, a services startup cell (ports 8851ΓÇô8853), dedupes document IDs in NDCG@k, drops manual ray.init(num_cpus=2), uses 2 shards for the small query set, and logs demo metrics to sqlite instead of filesystem mlruns. QASPER loads QASPER via parquet (datasets 3.x), adds FlagEmbedding, and makes phase 2b plots/heatmaps tolerate partial metric columns. NFCorpus/UCSB flatten nested *_cfg for plots/tables; UCSB adds dataset download, stricter install/init, optional Drive, Ray stop before evals, and Colab Enterprise iframe/save guards.

SFT notebooks pin transformers>=4.55.2,<5.0.0, wait on Colab background run_fit(), and _assert_training_completed via get_runs_info() so failed or incomplete runs error before metrics. TensorBoard paths move to tensorboard_logs/<experiment>/ (ecommerce artifact bundling updated). PII masking fixes batch polling statuses and best-config selection before eval-loss plots.

^{Reviewed by Cursor Bugbot for commit aabd819. Bugbot is set up for automated code reviews on this repo. Configure here.}

[david-rfai]

First notebook I tested rag-ucsb-course-catalog.ipynb failed on actual Colab. Some of these need to be actually tested on Colab as there is no way to directly replicate on a VM. Also, why were L4s used when Colab uses T4s?

Second notebook I tested also failed, sft-ecommerce-retail-chatbot.ipynb as it has the install process collapsed.

All notebooks should be using the pinned release that does not use transformers 5.0 so should not be needing: transformers>=4.55.2,<5.0.0

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit aabd819. Configure here.}

[cursor]

Stale rapidfireai skips upgrade

Medium Severity

Several notebooks only install rapidfireai==0.16.1 when the package is missing. If an older release (for example 0.14.0) is already importable, the install cell is skipped while later cells use the 0.16 *_cfg APIs and training-completion checks, which can fail at runtime on reused Colab or local environments.

Additional Locations (2)

• community_notebooks/sft-cybersecurity-qa.ipynb#L86-L93
• community_notebooks/sft-ecommerce-retail-chatbot.ipynb#L106-L112

 

^{Reviewed by Cursor Bugbot for commit aabd819. Configure here.}

[cursor]

Transformers pin not enforced

Medium Severity

The transformers>=4.55.2,<5.0.0 constraint is bundled with the rapidfireai install path only. When rapidfireai is already present (typical on Colab), transformers 5.x can remain installed and break trl==0.21.0 during training even though the PR intends to avoid that combination.

Additional Locations (2)

• community_notebooks/sft-child-facing-chatbot.ipynb#L106-L109
• community_notebooks/sft-pii-masking-redaction.ipynb#L88-L91

 

^{Reviewed by Cursor Bugbot for commit aabd819. Configure here.}