From b286be4def72616c8fa984708fd757d38b8d2793 Mon Sep 17 00:00:00 2001 From: Aidan Feldman Date: Sun, 30 Aug 2026 16:00:13 -0400 Subject: [PATCH 1/5] explain the site build process --- extras/scripts/school_ci.sh | 1 - meta/instructor_guide.md | 15 ++++++++++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/extras/scripts/school_ci.sh b/extras/scripts/school_ci.sh index 122e212b..43d927a2 100755 --- a/extras/scripts/school_ci.sh +++ b/extras/scripts/school_ci.sh @@ -7,7 +7,6 @@ set -x SCHOOL=$1 # remove irrelevant files - git rm -r \ .vscode/ \ nbdime_config.json \ diff --git a/meta/instructor_guide.md b/meta/instructor_guide.md index bca571f1..b4d1b770 100644 --- a/meta/instructor_guide.md +++ b/meta/instructor_guide.md @@ -25,7 +25,20 @@ make slides lec=N The site is generated using [JupyterBook](https://jupyterbook.org/) and deployed to [ReadTheDocs](https://readthedocs.org/). Markdown (`.md`) files and the files and folders that start with an underscore (`_`) are related to JupyterBook. -The HTML can be downloaded as an [artifact](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow) from [the latest `Publish` Action](https://github.com/afeld/python-public-policy/actions/workflows/publish.yml). +### Publishing + +This repository produces separate versions of the course site for Columbia and NYU. The source of truth is the `main` branch; the `columbia` and `nyu` branches contain generated, school-specific source files and their rendered sites. + +For each school, [the workflow](../.github/actions/setup/action.yml): + +1. Checks out the repository and installs the Python dependencies. +1. Switches to the corresponding school branch and merges the pushed branch using Git's `ours` strategy. It then restores the pushed branch's working tree. This records that the school branch includes the source revision without using the school branch's generated files as merge inputs. +1. Commits that preliminary source tree. +1. Runs [`school_ci.sh`](../extras/scripts/school_ci.sh) with the school ID. The script removes files that do not belong in a published school version, including development tools and tests. +1. Calls [`school.sh`](../extras/scripts/school.sh), which applies school-specific templates to Markdown and notebooks. It removes cells tagged for the other school or tagged `remove`. +1. Runs `make site`, which builds the Jupyter Book HTML into `_build/html`. + +When the push is to `main`, the workflow pushes the amended commit to the relevant school branch. Pushes to other branches still render both versions and upload their HTML, but do not update either published branch. The HTML can be downloaded as an [artifact](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow) from [the latest `Publish` Action](https://github.com/afeld/python-public-policy/actions/workflows/publish.yml). ### Building locally From 88c22e28d3d704b898a5f12831b79021663887ab Mon Sep 17 00:00:00 2001 From: Aidan Feldman Date: Sun, 30 Aug 2026 16:07:35 -0400 Subject: [PATCH 2/5] elaborate on how templating works --- meta/instructor_guide.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/instructor_guide.md b/meta/instructor_guide.md index b4d1b770..9dbd7d41 100644 --- a/meta/instructor_guide.md +++ b/meta/instructor_guide.md @@ -35,7 +35,10 @@ For each school, [the workflow](../.github/actions/setup/action.yml): 1. Switches to the corresponding school branch and merges the pushed branch using Git's `ours` strategy. It then restores the pushed branch's working tree. This records that the school branch includes the source revision without using the school branch's generated files as merge inputs. 1. Commits that preliminary source tree. 1. Runs [`school_ci.sh`](../extras/scripts/school_ci.sh) with the school ID. The script removes files that do not belong in a published school version, including development tools and tests. -1. Calls [`school.sh`](../extras/scripts/school.sh), which applies school-specific templates to Markdown and notebooks. It removes cells tagged for the other school or tagged `remove`. +1. Calls [`school.sh`](../extras/scripts/school.sh) to render school-specific templates. + - It renders every eligible file (Markdown, notebooks, etc.) with Jinja, replacing variables such as `{{school_name}}`, `{{lms_url}}`, and `{{assistant_name}}` from the [configuration](../extras/lib/school.py) for the selected school. + - It uses `{% if id == "columbia" %}` and `{% if id == "nyu" %}` conditionals for school-only content; the generated files must contain neither Jinja tags nor identifiers for the other school. + - For notebooks, `school.sh` first removes cells tagged for the other school (`columbia-only` or `nyu-only`) and cells tagged `remove`. It then renders each remaining cell source with the same Jinja variables, while resetting notebook kernel metadata to the default Python kernel for Colab. 1. Runs `make site`, which builds the Jupyter Book HTML into `_build/html`. When the push is to `main`, the workflow pushes the amended commit to the relevant school branch. Pushes to other branches still render both versions and upload their HTML, but do not update either published branch. The HTML can be downloaded as an [artifact](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow) from [the latest `Publish` Action](https://github.com/afeld/python-public-policy/actions/workflows/publish.yml). From 10f2135fed3cc799d1c7ee1f315e1ee47d576fb5 Mon Sep 17 00:00:00 2001 From: Aidan Feldman Date: Sun, 30 Aug 2026 16:10:22 -0400 Subject: [PATCH 3/5] explain the ReadTheDocs part of publishing --- meta/instructor_guide.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/meta/instructor_guide.md b/meta/instructor_guide.md index 9dbd7d41..08ce4901 100644 --- a/meta/instructor_guide.md +++ b/meta/instructor_guide.md @@ -40,8 +40,10 @@ For each school, [the workflow](../.github/actions/setup/action.yml): - It uses `{% if id == "columbia" %}` and `{% if id == "nyu" %}` conditionals for school-only content; the generated files must contain neither Jinja tags nor identifiers for the other school. - For notebooks, `school.sh` first removes cells tagged for the other school (`columbia-only` or `nyu-only`) and cells tagged `remove`. It then renders each remaining cell source with the same Jinja variables, while resetting notebook kernel metadata to the default Python kernel for Colab. 1. Runs `make site`, which builds the Jupyter Book HTML into `_build/html`. +1. When the push is to `main`, the workflow pushes the amended commit to the relevant school branch. Pushes to other branches still render both versions and upload their HTML, but do not update either published branch. +1. ReadTheDocs is notified of updates to the school-specific branches, which are then built as separate [versions](https://docs.readthedocs.com/platform/stable/versions.html). -When the push is to `main`, the workflow pushes the amended commit to the relevant school branch. Pushes to other branches still render both versions and upload their HTML, but do not update either published branch. The HTML can be downloaded as an [artifact](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow) from [the latest `Publish` Action](https://github.com/afeld/python-public-policy/actions/workflows/publish.yml). +The HTML can be downloaded as an [artifact](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow) from [the latest `Publish` Action](https://github.com/afeld/python-public-policy/actions/workflows/publish.yml). ### Building locally From adaee1e2a900ad2f2565a7b0a3ce14fa074c3f88 Mon Sep 17 00:00:00 2001 From: Aidan Feldman Date: Sun, 30 Aug 2026 16:22:17 -0400 Subject: [PATCH 4/5] escape Jinja tags that are meant to be shown --- extras/lib/school.py | 3 +++ meta/instructor_guide.md | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/extras/lib/school.py b/extras/lib/school.py index 1800478c..84a2c3c8 100644 --- a/extras/lib/school.py +++ b/extras/lib/school.py @@ -144,6 +144,9 @@ class SchoolText: # text that contains / is adjacent to prefixes above, which are allowed for both schools EXEMPT = [ + '{% if id == "columbia" %}` and `{% if id == "nyu" %}', + "`columbia-only` or `nyu-only`", + "`columbia` and `nyu` branches", ".zoom.us/rec", "autograder", # matches "grader" "baseurl: ", # _config.yml diff --git a/meta/instructor_guide.md b/meta/instructor_guide.md index 08ce4901..b72ef255 100644 --- a/meta/instructor_guide.md +++ b/meta/instructor_guide.md @@ -37,7 +37,7 @@ For each school, [the workflow](../.github/actions/setup/action.yml): 1. Runs [`school_ci.sh`](../extras/scripts/school_ci.sh) with the school ID. The script removes files that do not belong in a published school version, including development tools and tests. 1. Calls [`school.sh`](../extras/scripts/school.sh) to render school-specific templates. - It renders every eligible file (Markdown, notebooks, etc.) with Jinja, replacing variables such as `{{school_name}}`, `{{lms_url}}`, and `{{assistant_name}}` from the [configuration](../extras/lib/school.py) for the selected school. - - It uses `{% if id == "columbia" %}` and `{% if id == "nyu" %}` conditionals for school-only content; the generated files must contain neither Jinja tags nor identifiers for the other school. + - It uses {% raw %}`{% if id == "columbia" %}` and `{% if id == "nyu" %}`{% endraw %} conditionals for school-only content; the generated files must contain neither Jinja tags nor identifiers for the other school. - For notebooks, `school.sh` first removes cells tagged for the other school (`columbia-only` or `nyu-only`) and cells tagged `remove`. It then renders each remaining cell source with the same Jinja variables, while resetting notebook kernel metadata to the default Python kernel for Colab. 1. Runs `make site`, which builds the Jupyter Book HTML into `_build/html`. 1. When the push is to `main`, the workflow pushes the amended commit to the relevant school branch. Pushes to other branches still render both versions and upload their HTML, but do not update either published branch. From 0ceed2568027554715e832500a1fd42d1f6a3081 Mon Sep 17 00:00:00 2001 From: Aidan Feldman Date: Sun, 30 Aug 2026 16:55:23 -0400 Subject: [PATCH 5/5] elaborate on the templating --- meta/instructor_guide.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/instructor_guide.md b/meta/instructor_guide.md index b72ef255..e9af36f6 100644 --- a/meta/instructor_guide.md +++ b/meta/instructor_guide.md @@ -36,7 +36,8 @@ For each school, [the workflow](../.github/actions/setup/action.yml): 1. Commits that preliminary source tree. 1. Runs [`school_ci.sh`](../extras/scripts/school_ci.sh) with the school ID. The script removes files that do not belong in a published school version, including development tools and tests. 1. Calls [`school.sh`](../extras/scripts/school.sh) to render school-specific templates. - - It renders every eligible file (Markdown, notebooks, etc.) with Jinja, replacing variables such as `{{school_name}}`, `{{lms_url}}`, and `{{assistant_name}}` from the [configuration](../extras/lib/school.py) for the selected school. + - Jinja templating is used throughout the source files (Markdown, notebooks, etc.), rendered using [nbconvert](https://nbconvert.readthedocs.io/) with [a custom preprocessor](../extras/lib/school_template.py). + - Variables (such as `{{school_name}}`, `{{lms_url}}`, and `{{assistant_name}}`) are replaced with the values from the [configuration file](../extras/lib/school.py) for the selected school. - It uses {% raw %}`{% if id == "columbia" %}` and `{% if id == "nyu" %}`{% endraw %} conditionals for school-only content; the generated files must contain neither Jinja tags nor identifiers for the other school. - For notebooks, `school.sh` first removes cells tagged for the other school (`columbia-only` or `nyu-only`) and cells tagged `remove`. It then renders each remaining cell source with the same Jinja variables, while resetting notebook kernel metadata to the default Python kernel for Colab. 1. Runs `make site`, which builds the Jupyter Book HTML into `_build/html`.