From a2afc7f11a1713f2d5e36633d984b2c7dec25dcc Mon Sep 17 00:00:00 2001 From: Adrien Cotte Date: Fri, 15 May 2026 17:19:17 +0200 Subject: [PATCH 1/5] doc: new section for lesser known features Signed-off-by: Adrien Cotte --- .hunspell.en.dic | 10 + doc/source/index.rst | 1 + doc/source/latex_index.rst | 1 + doc/source/modulefile.rst | 2 + doc/source/user_guide.rst | 1024 ++++++++++++++++++++++++++++++++++++ 5 files changed, 1038 insertions(+) create mode 100644 doc/source/user_guide.rst diff --git a/.hunspell.en.dic b/.hunspell.en.dic index cd0ef5637..3ffdd5896 100644 --- a/.hunspell.en.dic +++ b/.hunspell.en.dic @@ -1605,3 +1605,13 @@ TOOL2 sphinxext opengraph PNG +HPSFCon +pu +xargs +jq +PyPI +gui +spack +mpivars +cuda +lindex diff --git a/doc/source/index.rst b/doc/source/index.rst index fd404cfe4..affd42309 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -184,6 +184,7 @@ or (at your option) any later version (`GPL-2.0-or-later`). INSTALL-win MIGRATING NEWS + user_guide FAQ changes other-implementations diff --git a/doc/source/latex_index.rst b/doc/source/latex_index.rst index 91f241e37..55912974c 100644 --- a/doc/source/latex_index.rst +++ b/doc/source/latex_index.rst @@ -18,4 +18,5 @@ Introduction .. include:: envml.rst .. include:: MIGRATING.rst .. include:: NEWS.rst +.. include:: user_guide.rst .. include:: changes.rst diff --git a/doc/source/modulefile.rst b/doc/source/modulefile.rst index 815b712f8..49194f469 100644 --- a/doc/source/modulefile.rst +++ b/doc/source/modulefile.rst @@ -2100,6 +2100,8 @@ or terse reporting of these sub-commands: :mconfig:`avail_output`, :mconfig:`list_terse_output`, :mconfig:`spider_output`, :mconfig:`spider_terse_output`. +.. _advanced_module_version_specifiers: + Advanced module version specifiers ---------------------------------- diff --git a/doc/source/user_guide.rst b/doc/source/user_guide.rst new file mode 100644 index 000000000..1b0f9e8f4 --- /dev/null +++ b/doc/source/user_guide.rst @@ -0,0 +1,1024 @@ +.. _user_guide: + +Lesser know features +==================== + +Environment Modules provides many capabilities beyond the commonly used +:command:`module` :subcmd:`load` command. Some features are lesser known, despite being very +useful for users, modulefile developers, and site administrators. + +This document gathers and explains a selection of these features through +practical examples and common use cases. + +It originates from the presentation *Beyond `module load`: Exploring the +Capabilities of Environment Modules* presented at `HPSFCon 2026 +`_. Original slides and video +demonstrations can be found on the `event session page +`_. + +Short command names +------------------- + +Most Modules sub-commands can be abbreviated. The :command:`ml` command also +provides a shorter alternative interface for common operations. + +.. list-table:: + :header-rows: 1 + :widths: 50 50 + + * - Standard command + - Short form + + * - ``module `` + - ``ml `` + + * - ``module avail`` + - ``module av`` + + * - ``module purge`` + - ``module pu`` + + * - ``module list`` + - ``ml`` + + * - ``module load `` + - ``ml `` + + * - ``module unload `` + - ``ml -`` + +Display/Parsing +--------------- + +--output=LIST +^^^^^^^^^^^^^ + +The :option:`--output`, or :option:`-o`, option customizes the information +displayed by Modules commands. + +It is supported by :command:`module` :subcmd:`avail`, :subcmd:`list`, and +:subcmd:`spider` sub-commands. + +By default, these commands display additional formatting elements such as +modulepaths, tags, default markers, or variant information. With +:option:`--output`, it is possible to restrict the displayed information to +only the needed fields. + +This option is especially useful when parsing command output in shell scripts, +Python programs, or monitoring tools. + +.. code-block:: console + + $ module avail -o '' + +Only module names are displayed. + +The output format is defined as a colon-separated list of elements. Supported +elements are documented in the corresponding sub-command manual pages. + +This feature helps avoid fragile text parsing pipelines. + +Instead of parsing formatted output: + +.. code-block:: bash + + modules_list=$(module avail | grep -v '^---' \ + | sed 's/(default)//g' | head -n -2 | xargs) + +A simpler and more robust approach is: + +.. code-block:: bash + + modules_list=$(module avail -o '') + +spider +^^^^^^ + +The :command:`module` :subcmd:`spider` sub-command lists all available +modulefiles found in enabled modulepaths, including modulepaths recursively +added by modulefiles. + +Unlike :subcmd:`avail`, :subcmd:`spider` explores the full dependency tree +created by :mfcmd:`module use`, :mfcmd:`append-path`, or +:mfcmd:`prepend-path` instructions. + +This command is especially useful with hierarchical module layouts, where some +modulefiles only become visible after loading compiler or MPI dependencies. + +The :subcmd:`spider` sub-command accepts the same display and formatting +options as :subcmd:`avail`, including :option:`--output`, +:option:`--latest`, :option:`--default`, :option:`--json`, +:option:`--indepth`, and :option:`--no-indepth`. + +For example, in a hierarchical environment: + +.. code-block:: console + + $ module avail + ------------------- /usr/share/modulefiles/common ------------------ + gcc/14.0 intel/25.0 + +Only compiler modules are initially visible. + +The :subcmd:`spider` sub-command reveals additional modules together with +their dependency path: + +.. code-block:: console + + $ module spider + ------------------- /usr/share/modulefiles/common ------------------ + gcc/14.0 intel/25.0 + + + ---------- /usr/share//modulefiles/intel (via intel/25.0) ---------- + intelmpi/25.0 + + ------------ /usr/share//modulefiles/gcc (via gcc/14.0) ------------ + openmpi/5.0.8 + + +This output indicates which modules must be loaded before these MPI stacks +become available. + +Modules caches are automatically used by :subcmd:`spider` when configured, +which significantly improves search performance on large installations. + +--latest and --default +^^^^^^^^^^^^^^^^^^^^^^ + +The :option:`--latest`, or :option:`-L`, and :option:`--default`, or +:option:`-d`, options restrict the output of the :command:`module` +:subcmd:`avail` and :subcmd:`spider` sub-commands. + +The :option:`--latest` option displays only the highest numerically sorted +version of each module name. + +.. code-block:: console + + $ module avail --latest + +The :option:`--default` option displays only the default version of each +module name. + +.. code-block:: console + + $ module avail --default + +These options may also be combined with other display options such as +:option:`--output` or :option:`--json`. + +--indepth and --no-indepth +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The :option:`--indepth` and :option:`--no-indepth` options control how the +:command:`module` :subcmd:`avail` and :subcmd:`spider` sub-commands search +for matching modulefiles. + +By default, Modules searches recursively and returns all matching +modulefiles. This behavior is equivalent to :option:`--indepth` and +could be configured with :command:`module` :subcmd:`config` :mconfig:`avail_indepth` +and :mconfig:`spider_indepth`. + +.. code-block:: console + + $ module avail + ------------------- /usr/share/modulefiles ------------------ + + intel/24.0 intel/25.0 gcc/8.3 gcc/11.1 gcc/14.0 + +The :option:`--no-indepth` option limits results to the depth level +expressed by the search query. Modulefiles contained in matching directories +are not displayed. + +.. code-block:: console + + $ module avail --no-indepth + ------------------- /usr/share/modulefiles ------------------ + + intel/ gcc/ + +--json +^^^^^^ + +The :option:`--json`, or :option:`-j`, option displays command results in +JSON format. + +It is supported by the :command:`module` :subcmd:`avail`, :subcmd:`list`, +:subcmd:`savelist`, :subcmd:`search`, :subcmd:`spider`, +:subcmd:`stashlist`, and :subcmd:`whatis` sub-commands. + +JSON output is intended for machine consumption and can be processed directly +by tools such as Python or `jq`. + +.. code-block:: console + + $ module avail -j | jq . + { + "/usr/share/modulefiles": { + "gcc/14.0": { + "name": "gcc/14.0", + "type": "modulefile", + "symbols": [], + "tags": [], + "pathname": "/usr/share/modulefiles/gcc/14.0", + "via": "" + }, + ... + } + +The JSON format is also useful for monitoring, reporting, and integration +with external tools. + +Manipulating environment for users +---------------------------------- + +Collections +^^^^^^^^^^^ + +Collections save and restore sets of loaded modules. + +They provide a convenient way to switch between software environments without +having to manually load and unload each module. + +Collections are managed through several :command:`module` sub-commands: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Sub-command + - Description + + * - :subcmd:`save` [collection] + - Save the current module environment to a collection. + + * - :subcmd:`restore` [collection] + - Restore a collection or collection file. + + * - :subcmd:`savelist` + - List saved collections. + + * - :subcmd:`saverm` [collection] + - Remove a saved collection. + + * - :subcmd:`saveshow` [collection] + - Display information about a collection. + + * - :subcmd:`is-saved` [collection] + - Test whether one or more collections exist. + +Collections are stored under the user collection directory, usually +:file:`$HOME/.module`. + +The :envvar:`MODULES_COLLECTION_TARGET` environment variable may be defined +to append a suffix to collection names. This feature is particularly useful +on systems sharing a common home directory across multiple clusters. + +.. code-block:: console + + $ module save + $ module purge + $ module restore + +mogui +^^^^^ + +``mogui`` provides a graphical interface to browse available modules and +manage collections. + +It offers an alternative to the command line and can be useful for new users +discovering a software stack or for users who prefer an interactive +interface. + +Any environment change made from the GUI, such as loading modules or +restoring collections, is applied back to the shell session that launched +the application. + +The project is available on GitHub: +`cea-hpc/mogui `_. It can be installed +from PyPI with :command:`pip` or from Spack. + +.. code-block:: console + + $ pip install modules-gui + +or: + +.. code-block:: console + + $ spack install py_modules_gui + +Then start the graphical interface: + +.. code-block:: console + + $ mogui + +.. image:: https://raw.githubusercontent.com/cea-hpc/mogui/main/doc/sneak_peek.gif + :alt: mogui graphical interface + :align: center + +stash commands +^^^^^^^^^^^^^^ + +Stash collections temporarily save the current module environment and make it +easy to switch to another environment. + +This feature is similar to the :command:`git stash` workflow: save the current +state, perform other work, then restore the original environment later. + +Stash collections are managed through several :command:`module` +sub-commands: + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Sub-command + - Description + + * - :subcmd:`stash` + - Save the current environment and reset it. + + * - :subcmd:`stashpop` + - Restore and remove a stash collection. + + * - :subcmd:`stashrm` + - Remove a stash collection. + + * - :subcmd:`stashshow` + - Display information about a stash collection. + + * - :subcmd:`stashclear` + - Remove all stash collections. + + * - :subcmd:`stashlist` + - List stash collections. + +Save the current environment: + +.. code-block:: console + + $ module stash + +Load a different software stack: + +.. code-block:: console + + $ module purge + $ module load ... + +Restore the original environment: + +.. code-block:: console + + $ module stashpop + +Unlike regular collections, stash collections are intended to be short-lived +and are automatically removed when restored with +:subcmd:`stashpop`. + +Envml +^^^^^ + +The :ref:`envml(1)` command executes a command in a specific module +environment. + +It applies the requested module actions, executes the command, then restores +the original environment. As a result, the current shell session remains +unchanged. + +This feature is particularly useful in scripts, tests, and automation +workflows where commands must be executed in a controlled software +environment. + +Supported module actions include: + +.. list-table:: + :header-rows: 1 + :widths: 35 65 + + * - Action + - Description + + * - ``purge`` + - Purge all loaded modules. + + * - ``restore=`` + - Restore a saved collection. + + * - ``unload=`` + - Unload one or more modules. + + * - ``switch=&`` + - Switch from one module to another. + + * - ``load=`` + - Load one or more modules. + + * - ```` + - Shorthand form of ``load=``. + +Multiple actions may be chained with the ``:`` separator. When specifying +multiple modules for a single action, use the ``&`` separator. In some +shells, such as Bash, the ``&`` character must be escaped or quoted. + +For example, execute a command in a clean environment: + +.. code-block:: console + + $ envml purge -- gcc hello.c + +The :ref:`envml(1)` command can simplify scripts by replacing sequences of +module operations with a single command. + +Instead of: + +.. code-block:: bash + + #!/bin/bash + module purge + module load gcc + gcc hello.c + +One can write: + +.. code-block:: bash + + #!/bin/bash + envml purge:gcc -- gcc hello.c + +Protected environment variables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The :mconfig:`protected_envvars` configuration option prevents Modules from +modifying selected environment variables. Multiple variables may be specified using the ``:`` separator. + +This feature is useful to protect critical environment settings from +unintended changes. + +When a modulefile attempts to modify a protected variable, Modules ignores +the modification and emits a warning. + +.. code-block:: console + + $ module config protected_envvars LD_PRELOAD + $ module load intel + Loading intel/25.0 + WARNING: Modification of protected environment variable LD_PRELOAD ignored + +Developing and configuring module files +--------------------------------------- + +module edit +^^^^^^^^^^^ + +The :command:`module` :subcmd:`edit` sub-command opens a modulefile in the +configured text editor (cf. :subcmd:`config` :mconfig:`editor`). + +Instead of manually locating a modulefile: + +.. code-block:: console + + $ vi $(module path appA) + +One can directly edit it with: + +.. code-block:: console + + $ module edit appA + +module lint +^^^^^^^^^^^ + +The :command:`module` :subcmd:`lint` sub-command analyzes modulefiles and +reports potential issues. + +It relies on the Nagelfar Tcl syntax checker to detect syntax errors and +common mistakes. + +Analyze a specific modulefile: + +.. code-block:: console + + $ module lint appA + +Reported issues may include: + +* Unknown variables +* Unknown commands +* Invalid command arguments +* Tcl syntax errors + +source-sh +^^^^^^^^^ + +The :mfcmd:`source-sh` modulefile command evaluates a shell script and tracks +the environment changes it performs. + +The detected changes are translated into modulefile actions, making it +possible to load and unload shell-based environments through Modules. + +It is particularly useful to integrate software distributions that provide +environment setup scripts such as ``setup-env.sh``, ``mpivars.sh``, or +similar rc-provided files. + +For example: + +.. code-block:: tcl + + #%Module + + source-sh bash /opt/spack/share/spack/setup-env.sh + +sh-to-mod +^^^^^^^^^ + +The :command:`module` :subcmd:`sh-to-mod` sub-command evaluates a shell script +and reports the resulting environment changes as modulefile commands. + +It can be used to convert existing shell initialization scripts into +modulefiles. + +For example: + +.. code-block:: console + + $ module sh-to-mod bash setup-env.sh > setup-env.mod + +.. tip:: + + Use :subcmd:`sh-to-mod` when you want to generate and maintain a regular + modulefile from an existing shell script. + +mod-to-sh +^^^^^^^^^ + +The :command:`module` :subcmd:`mod-to-sh` sub-command evaluates one or more +modulefiles and reports the resulting environment changes as shell code. + +For example: + +.. code-block:: console + + $ module mod-to-sh bash appA > setup-env.sh + +.. tip:: + + The :subcmd:`mod-to-sh` sub-command can be seen as the reverse operation of + :subcmd:`sh-to-mod`. + +prereq +^^^^^^ + +The :mfcmd:`prereq` modulefile command declares one or more modules that must +already be loaded before the current modulefile can be loaded. + +Instead of explicitly loading dependencies: + +.. code-block:: tcl + + if { ! [ is-loaded appA ] } { + module load appA + } + +Declare them with: + +.. code-block:: tcl + + prereq appA + +When the prerequisite is not satisfied, Modules reports an error and refuses +to load the modulefile. + +Multiple module names passed to a single :mfcmd:`prereq` command act as a +logical OR. Multiple :mfcmd:`prereq` commands act as a logical AND. + +conflict +^^^^^^^^ + +The :mfcmd:`conflict` modulefile command declares one or more modules that +cannot be loaded together with the current modulefile. + +Instead of manually checking for incompatible modules: + +.. code-block:: tcl + + if { [ is-loaded appB ] } { + puts stderr "ERROR: appB conflicts with appA!" + exit 1 + } + +Declare the conflict with: + +.. code-block:: tcl + + conflict appB + +When a conflicting module is loaded, Modules reports an error and refuses to +load the modulefile. + +The conflict check may be bypassed with the :option:`--force` option: + +.. code-block:: console + + $ module load --force appA + +.. tip:: + + To ensure that only one version of a module can be loaded at a time, + consider enabling the :mconfig:`unique_name_loaded` configuration option + instead of declaring self-conflicts in every modulefile. + +variant +^^^^^^^ + +The :mfcmd:`variant` modulefile command declares variants and their accepted +values. + +This command is only available inside modulefiles. + +Variants allow users to customize the behavior of a modulefile at load time. +They follow the same syntax as Spack variants. + +Boolean variants are enabled with ``+`` and disabled with ``~``: + +.. code-block:: tcl + + #%Module + variant --boolean cuda + +.. code-block:: console + + $ module load openmpi +cuda + +Variants with multiple values are declared by listing the accepted values: + +.. code-block:: tcl + + #%Module + variant compiler gcc intel + +And selected with the ``=`` syntax: + +.. code-block:: console + + $ module load openmpi compiler=intel + +The :mfcmd:`getvariant` modulefile command retrieves the value of a declared +variant, allowing the modulefile to adapt its behavior. + +For example: + +.. code-block:: tcl + + switch -- [getvariant compiler] { + gcc { + prereq gcc + } + intel { + prereq intel + } + } + +Configuration option :mconfig:`variant_shortcut` defines shortcut characters +for variants. + +For example: + +.. code-block:: console + + $ module config variant_shortcut compiler=% + +Allows users to write: + +.. code-block:: console + + $ module load openmpi %gcc + +Advanced manipulations +---------------------- + +Advanced specifications specifiers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:ref:`advanced_module_version_specifiers` provide a concise way to select modulefiles based on +their version or declared variants. They can be used with several +:command:`module` sub-commands, including :subcmd:`avail`, +:subcmd:`spider`, :subcmd:`load`, and :subcmd:`switch`. + +Version specifications use the ``@`` character: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Specification + - Description + + * - ``module avail cuda@default`` + - Select the default version. + + * - ``module avail cuda@latest`` + - Select the latest version. + + * - ``module avail cuda@12:`` + - Select versions greater than or equal to ``12``. + + * - ``module avail cuda@:12`` + - Select versions lower than or equal to ``12``. + + * - ``module avail cuda@12:13`` + - Select versions between ``12`` and ``13``. + +Variant specifications may also be used: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Specification + - Description + + * - ``module avail +mpi`` + - Select modules declaring the ``mpi`` variant. + + * - ``module avail cuda=12`` + - Select modules where the ``cuda`` variant is set to ``12``. + + * - ``module avail +mpi +cuda`` + - Select modules matching both variant requirements. + +--timer and --debug +^^^^^^^^^^^^^^^^^^^ + +The :option:`--timer` and :option:`--debug`, or :option:`-D`, options help +analyze and troubleshoot Modules commands. + +They are supported by most :command:`module` sub-commands. + +The :option:`--timer` option reports the total execution time of a command. + +For example: + +.. code-block:: console + + $ module avail --timer + +The :option:`--debug`, or :option:`-D`, option displays debugging messages +describing the internal execution of the command. + +For example: + +.. code-block:: console + + $ module avail -D + +The :option:`--timer` and :option:`--debug` options may be combined. In this +case, regular debug messages are replaced by execution time reports for each +internal procedure call. + +.. code-block:: console + + $ module avail --timer -D + +For even more detailed debugging information, use :option:`-DD`. + +.modulerc files +^^^^^^^^^^^^^^^ + +A :file:`.modulerc` file contains Tcl code automatically evaluated by +Modules when encountered during a command execution. + +Rc files are used to define module aliases, virtual +modules, tags, hidden modules, forbidden modules, ... + +Several :file:`.modulerc` files may be evaluated during a single +:command:`module` command. + +Depending on the executed sub-command, Modules may evaluate +:file:`.modulerc` files found in different locations, including: + +* :envvar:`MODULERCFILE` +* :file:`$HOME/.modulerc` +* Modulepath and modules directories + +For example, the following directory hierarchy contains several +:file:`.modulerc` files: + +.. code-block:: text + + /home/user/.modulerc + modulefiles/ + ├── .modulerc + ├── app/ + │ ├── .modulerc + │ └── 1.0 + └── mpi/ + ├── .modulerc + └── openmpi/5.0 + +Depending on the requested operation, one or more of these files are +evaluated automatically. + +See :ref:`Modulecmd startup` for details on the startup sequence, +:mconfig:`rcfile` and :mconfig:`ignore_user_rc` to configure the evaluation +of user rc files, and :sitevar:`modulerc_extra_vars` and +:sitevar:`modulerc_extra_cmds` to extend :file:`.modulerc` files. + +module-tag +^^^^^^^^^^ + +The :mfcmd:`module-tag` modulerc command associates one or more tags with +modulefiles. + +This command is only available inside :file:`.modulerc` files. + +Tags may be used to provide additional information about modulefiles or to +modify their behavior. + +For example: + +.. code-block:: tcl + + module-tag experimental app/2.0 + +Several predefined tags affect the behavior of Modules: + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Tag + - Description + + * - ``keep-loaded`` + - Prevent automatic unloading when unloading dependent modules. + + * - ``sticky`` + - Prevent unloading unless :option:`--force` is specified. + + * - ``super-sticky`` + - Prevent unloading under any circumstances. + +For example: + +.. code-block:: tcl + + module-tag sticky core + +Sticky modules can help ensure that essential software stacks remain loaded. +The behavior of :subcmd:`purge` with sticky modules can be configured through +:mconfig:`sticky_purge`. + +module-hide +^^^^^^^^^^^ + +The :mfcmd:`module-hide` modulerc command hides modulefiles from module +searches and selection. + +This command is only available inside :file:`.modulerc` files. + +A hidden module is excluded from regular searches but may still be selected +when explicitly referred to by its exact name. + +For example: + +.. code-block:: tcl + + module-hide app/1.0 + +Visibility may also be restricted to specific users or groups with options. + +Time-based restrictions can be defined with ``--after`` and ``--before``. + +For example: + +.. code-block:: tcl + + module-hide --after 2027-01-01 app/1.0 + +See :mfcmd:`module-hide` for additional visibility options, including **soft** +and **hard** hiding, as well as **users** and **groups** restrictions. + +module-forbid +^^^^^^^^^^^^^ + +The :mfcmd:`module-forbid` modulerc command prevents designated modulefiles +from being loaded. + +This command is only available inside :file:`.modulerc` files. + +Unlike a hidden module, a forbidden module remains visible in module searches +but cannot be loaded. + +For example: + +.. code-block:: tcl + + module-forbid app/1.0 + +A custom message may be displayed when a user attempts to load a forbidden +module with ``--message`` option. + +For example: + +.. code-block:: tcl + + module-forbid --message {Please use app/2.0 instead} app/1.0 + +See :mfcmd:`module-forbid` for additional restriction options, including +custom messages, as well as **after**/**before**, **users** and **groups** restrictions. + +module-virtual +^^^^^^^^^^^^^^ + +The :mfcmd:`module-virtual` modulerc command associates a virtual module name +with an existing modulefile. + +This command is only available inside :file:`.modulerc` files. + +Several virtual modules may refer to the same modulefile, reducing the number +of files required to define similar modules. + +For example: + +.. code-block:: text + + modulefiles/app/ + ├── .common + └── .modulerc + +The :file:`.modulerc` file may define several virtual versions: + +.. code-block:: tcl + + #%Module + module-virtual 1.0 .common + module-virtual 1.2 .common + module-virtual 1.5 .common + module-virtual 2.0 .common + +.. code-block:: console + + $ module avail + ------------------- /usr/share/modulefiles ------------------ + app/1.0 app/1.2 app/1.5 app/2.0 + +All these virtual modules are evaluated using the same :file:`.common` +modulefile. +Its behavior can be adapted according to the virtual module +name being evaluated. + +.. code-block:: tcl + + #%Module + + set version [lindex [split [module-info name] /] end] + + setenv APP_VERSION $version + +Module logger +^^^^^^^^^^^^^ + +Modules can log module activity through the :mconfig:`logger` and +:mconfig:`logged_events` configuration options. + +The :mconfig:`logger` option defines the command used to record log messages. + +For example: + +.. code-block:: console + + $ module config logger "/usr/bin/logger -t modules" + +The :mconfig:`logged_events` option defines which module events are recorded. + +.. list-table:: + :header-rows: 1 + :widths: 30 70 + + * - Event + - Description + + * - ``requested_cmd`` + - Record module commands requested by users. + + * - ``requested_eval`` + - Record modulefile evaluations requested by users. + + * - ``auto_eval`` + - Record modulefile evaluations automatically triggered by Modules. + +Multiple events may be specified using the ``:`` separator. + +For example: + +.. code-block:: console + + $ module config logged_events requested_cmd:requested_eval + +By default, no events are logged. + +The logger command can be customized to integrate Modules activity with an +existing logging or monitoring infrastructure. From 17120276aeb768375f5768f60848d3bcf24ad9f7 Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Sun, 30 Aug 2026 20:09:34 +0200 Subject: [PATCH 2/5] doc: polish lesser known features section Use literal markup on command, option and file names in section titles and move the mogui section next to the other collection-related features. Fix typos and grammar: "Lesser know" document title, garbled "Advanced specifications specifiers" section title, doubled slash in spider example paths, nested inline markup in the presentation title, missing articles and various wording issues. Also fix the module use cross-reference so it resolves, correct the Spack package name to py-modules-gui, and harmonize inline markup, line wrapping and example output spacing. Assisted-by: Claude:claude-fable-5 Signed-off-by: Xavier Delaruelle --- doc/source/index.rst | 2 +- doc/source/latex_index.rst | 2 +- doc/source/{user_guide.rst => user-guide.rst} | 207 +++++++++--------- 3 files changed, 105 insertions(+), 106 deletions(-) rename doc/source/{user_guide.rst => user-guide.rst} (93%) diff --git a/doc/source/index.rst b/doc/source/index.rst index affd42309..15ef84fc5 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -184,7 +184,7 @@ or (at your option) any later version (`GPL-2.0-or-later`). INSTALL-win MIGRATING NEWS - user_guide + user-guide FAQ changes other-implementations diff --git a/doc/source/latex_index.rst b/doc/source/latex_index.rst index 55912974c..54208eb2c 100644 --- a/doc/source/latex_index.rst +++ b/doc/source/latex_index.rst @@ -18,5 +18,5 @@ Introduction .. include:: envml.rst .. include:: MIGRATING.rst .. include:: NEWS.rst -.. include:: user_guide.rst +.. include:: user-guide.rst .. include:: changes.rst diff --git a/doc/source/user_guide.rst b/doc/source/user-guide.rst similarity index 93% rename from doc/source/user_guide.rst rename to doc/source/user-guide.rst index 1b0f9e8f4..bbee3cd72 100644 --- a/doc/source/user_guide.rst +++ b/doc/source/user-guide.rst @@ -1,7 +1,7 @@ -.. _user_guide: +.. _user-guide: -Lesser know features -==================== +Lesser known features +===================== Environment Modules provides many capabilities beyond the commonly used :command:`module` :subcmd:`load` command. Some features are lesser known, despite being very @@ -10,8 +10,8 @@ useful for users, modulefile developers, and site administrators. This document gathers and explains a selection of these features through practical examples and common use cases. -It originates from the presentation *Beyond `module load`: Exploring the -Capabilities of Environment Modules* presented at `HPSFCon 2026 +It originates from the presentation *Beyond "module load": Exploring the +Capabilities of Environment Modules* given at `HPSFCon 2026 `_. Original slides and video demonstrations can be found on the `event session page `_. @@ -50,8 +50,8 @@ provides a shorter alternative interface for common operations. Display/Parsing --------------- ---output=LIST -^^^^^^^^^^^^^ +``--output=LIST`` +^^^^^^^^^^^^^^^^^ The :option:`--output`, or :option:`-o`, option customizes the information displayed by Modules commands. @@ -91,15 +91,15 @@ A simpler and more robust approach is: modules_list=$(module avail -o '') -spider -^^^^^^ +``spider`` +^^^^^^^^^^ The :command:`module` :subcmd:`spider` sub-command lists all available modulefiles found in enabled modulepaths, including modulepaths recursively added by modulefiles. Unlike :subcmd:`avail`, :subcmd:`spider` explores the full dependency tree -created by :mfcmd:`module use`, :mfcmd:`append-path`, or +created by :mfcmd:`module use`, :mfcmd:`append-path`, or :mfcmd:`prepend-path` instructions. This command is especially useful with hierarchical module layouts, where some @@ -130,10 +130,10 @@ their dependency path: gcc/14.0 intel/25.0 - ---------- /usr/share//modulefiles/intel (via intel/25.0) ---------- + ---------- /usr/share/modulefiles/intel (via intel/25.0) ----------- intelmpi/25.0 - ------------ /usr/share//modulefiles/gcc (via gcc/14.0) ------------ + ------------ /usr/share/modulefiles/gcc (via gcc/14.0) ------------- openmpi/5.0.8 @@ -143,8 +143,8 @@ become available. Modules caches are automatically used by :subcmd:`spider` when configured, which significantly improves search performance on large installations. ---latest and --default -^^^^^^^^^^^^^^^^^^^^^^ +``--latest`` and ``--default`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :option:`--latest`, or :option:`-L`, and :option:`--default`, or :option:`-d`, options restrict the output of the :command:`module` @@ -167,8 +167,8 @@ module name. These options may also be combined with other display options such as :option:`--output` or :option:`--json`. ---indepth and --no-indepth -^^^^^^^^^^^^^^^^^^^^^^^^^^ +``--indepth`` and ``--no-indepth`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :option:`--indepth` and :option:`--no-indepth` options control how the :command:`module` :subcmd:`avail` and :subcmd:`spider` sub-commands search @@ -176,15 +176,14 @@ for matching modulefiles. By default, Modules searches recursively and returns all matching modulefiles. This behavior is equivalent to :option:`--indepth` and -could be configured with :command:`module` :subcmd:`config` :mconfig:`avail_indepth` +can be configured with :command:`module` :subcmd:`config` :mconfig:`avail_indepth` and :mconfig:`spider_indepth`. .. code-block:: console $ module avail ------------------- /usr/share/modulefiles ------------------ - - intel/24.0 intel/25.0 gcc/8.3 gcc/11.1 gcc/14.0 + intel/24.0 intel/25.0 gcc/8.3 gcc/11.1 gcc/14.0 The :option:`--no-indepth` option limits results to the depth level expressed by the search query. Modulefiles contained in matching directories @@ -194,11 +193,10 @@ are not displayed. $ module avail --no-indepth ------------------- /usr/share/modulefiles ------------------ + intel/ gcc/ - intel/ gcc/ - ---json -^^^^^^ +``--json`` +^^^^^^^^^^ The :option:`--json`, or :option:`-j`, option displays command results in JSON format. @@ -208,7 +206,7 @@ It is supported by the :command:`module` :subcmd:`avail`, :subcmd:`list`, :subcmd:`stashlist`, and :subcmd:`whatis` sub-commands. JSON output is intended for machine consumption and can be processed directly -by tools such as Python or `jq`. +by tools such as ``jq`` or Python scripts. .. code-block:: console @@ -229,8 +227,8 @@ by tools such as Python or `jq`. The JSON format is also useful for monitoring, reporting, and integration with external tools. -Manipulating environment for users ----------------------------------- +Manipulating the environment for users +-------------------------------------- Collections ^^^^^^^^^^^ @@ -280,46 +278,8 @@ on systems sharing a common home directory across multiple clusters. $ module purge $ module restore -mogui -^^^^^ - -``mogui`` provides a graphical interface to browse available modules and -manage collections. - -It offers an alternative to the command line and can be useful for new users -discovering a software stack or for users who prefer an interactive -interface. - -Any environment change made from the GUI, such as loading modules or -restoring collections, is applied back to the shell session that launched -the application. - -The project is available on GitHub: -`cea-hpc/mogui `_. It can be installed -from PyPI with :command:`pip` or from Spack. - -.. code-block:: console - - $ pip install modules-gui - -or: - -.. code-block:: console - - $ spack install py_modules_gui - -Then start the graphical interface: - -.. code-block:: console - - $ mogui - -.. image:: https://raw.githubusercontent.com/cea-hpc/mogui/main/doc/sneak_peek.gif - :alt: mogui graphical interface - :align: center - -stash commands -^^^^^^^^^^^^^^ +``stash`` commands +^^^^^^^^^^^^^^^^^^ Stash collections temporarily save the current module environment and make it easy to switch to another environment. @@ -378,8 +338,46 @@ Unlike regular collections, stash collections are intended to be short-lived and are automatically removed when restored with :subcmd:`stashpop`. -Envml -^^^^^ +``mogui`` +^^^^^^^^^ + +``mogui`` provides a graphical interface to browse available modules and +manage collections. + +It offers an alternative to the command line and can be useful for new users +discovering a software stack or for users who prefer an interactive +interface. + +Any environment change made from the GUI, such as loading modules or +restoring collections, is applied back to the shell session that launched +the application. + +The project is available on GitHub: +`cea-hpc/mogui `_. It can be installed +from PyPI with :command:`pip` or from Spack. + +.. code-block:: console + + $ pip install modules-gui + +or: + +.. code-block:: console + + $ spack install py-modules-gui + +Then start the graphical interface: + +.. code-block:: console + + $ mogui + +.. image:: https://raw.githubusercontent.com/cea-hpc/mogui/main/doc/sneak_peek.gif + :alt: mogui graphical interface + :align: center + +``envml`` +^^^^^^^^^ The :ref:`envml(1)` command executes a command in a specific module environment. @@ -452,7 +450,8 @@ Protected environment variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :mconfig:`protected_envvars` configuration option prevents Modules from -modifying selected environment variables. Multiple variables may be specified using the ``:`` separator. +modifying selected environment variables. Multiple variables may be specified +using the ``:`` separator. This feature is useful to protect critical environment settings from unintended changes. @@ -467,11 +466,11 @@ the modification and emits a warning. Loading intel/25.0 WARNING: Modification of protected environment variable LD_PRELOAD ignored -Developing and configuring module files ---------------------------------------- +Developing and configuring modulefiles +-------------------------------------- -module edit -^^^^^^^^^^^ +``module edit`` +^^^^^^^^^^^^^^^ The :command:`module` :subcmd:`edit` sub-command opens a modulefile in the configured text editor (cf. :subcmd:`config` :mconfig:`editor`). @@ -488,8 +487,8 @@ One can directly edit it with: $ module edit appA -module lint -^^^^^^^^^^^ +``module lint`` +^^^^^^^^^^^^^^^ The :command:`module` :subcmd:`lint` sub-command analyzes modulefiles and reports potential issues. @@ -510,8 +509,8 @@ Reported issues may include: * Invalid command arguments * Tcl syntax errors -source-sh -^^^^^^^^^ +``source-sh`` +^^^^^^^^^^^^^ The :mfcmd:`source-sh` modulefile command evaluates a shell script and tracks the environment changes it performs. @@ -531,8 +530,8 @@ For example: source-sh bash /opt/spack/share/spack/setup-env.sh -sh-to-mod -^^^^^^^^^ +``sh-to-mod`` +^^^^^^^^^^^^^ The :command:`module` :subcmd:`sh-to-mod` sub-command evaluates a shell script and reports the resulting environment changes as modulefile commands. @@ -551,8 +550,8 @@ For example: Use :subcmd:`sh-to-mod` when you want to generate and maintain a regular modulefile from an existing shell script. -mod-to-sh -^^^^^^^^^ +``mod-to-sh`` +^^^^^^^^^^^^^ The :command:`module` :subcmd:`mod-to-sh` sub-command evaluates one or more modulefiles and reports the resulting environment changes as shell code. @@ -568,8 +567,8 @@ For example: The :subcmd:`mod-to-sh` sub-command can be seen as the reverse operation of :subcmd:`sh-to-mod`. -prereq -^^^^^^ +``prereq`` +^^^^^^^^^^ The :mfcmd:`prereq` modulefile command declares one or more modules that must already be loaded before the current modulefile can be loaded. @@ -594,8 +593,8 @@ to load the modulefile. Multiple module names passed to a single :mfcmd:`prereq` command act as a logical OR. Multiple :mfcmd:`prereq` commands act as a logical AND. -conflict -^^^^^^^^ +``conflict`` +^^^^^^^^^^^^ The :mfcmd:`conflict` modulefile command declares one or more modules that cannot be loaded together with the current modulefile. @@ -630,8 +629,8 @@ The conflict check may be bypassed with the :option:`--force` option: consider enabling the :mconfig:`unique_name_loaded` configuration option instead of declaring self-conflicts in every modulefile. -variant -^^^^^^^ +``variant`` +^^^^^^^^^^^ The :mfcmd:`variant` modulefile command declares variants and their accepted values. @@ -690,7 +689,7 @@ For example: $ module config variant_shortcut compiler=% -Allows users to write: +This allows users to write: .. code-block:: console @@ -699,7 +698,7 @@ Allows users to write: Advanced manipulations ---------------------- -Advanced specifications specifiers +Advanced module version specifiers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :ref:`advanced_module_version_specifiers` provide a concise way to select modulefiles based on @@ -749,8 +748,8 @@ Variant specifications may also be used: * - ``module avail +mpi +cuda`` - Select modules matching both variant requirements. ---timer and --debug -^^^^^^^^^^^^^^^^^^^ +``--timer`` and ``--debug`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :option:`--timer` and :option:`--debug`, or :option:`-D`, options help analyze and troubleshoot Modules commands. @@ -784,8 +783,8 @@ internal procedure call. For even more detailed debugging information, use :option:`-DD`. -.modulerc files -^^^^^^^^^^^^^^^ +``.modulerc`` files +^^^^^^^^^^^^^^^^^^^ A :file:`.modulerc` file contains Tcl code automatically evaluated by Modules when encountered during a command execution. @@ -826,8 +825,8 @@ See :ref:`Modulecmd startup` for details on the startup sequence, of user rc files, and :sitevar:`modulerc_extra_vars` and :sitevar:`modulerc_extra_cmds` to extend :file:`.modulerc` files. -module-tag -^^^^^^^^^^ +``module-tag`` +^^^^^^^^^^^^^^ The :mfcmd:`module-tag` modulerc command associates one or more tags with modulefiles. @@ -871,8 +870,8 @@ Sticky modules can help ensure that essential software stacks remain loaded. The behavior of :subcmd:`purge` with sticky modules can be configured through :mconfig:`sticky_purge`. -module-hide -^^^^^^^^^^^ +``module-hide`` +^^^^^^^^^^^^^^^ The :mfcmd:`module-hide` modulerc command hides modulefiles from module searches and selection. @@ -888,7 +887,7 @@ For example: module-hide app/1.0 -Visibility may also be restricted to specific users or groups with options. +Visibility may also be restricted to specific users or groups. Time-based restrictions can be defined with ``--after`` and ``--before``. @@ -901,8 +900,8 @@ For example: See :mfcmd:`module-hide` for additional visibility options, including **soft** and **hard** hiding, as well as **users** and **groups** restrictions. -module-forbid -^^^^^^^^^^^^^ +``module-forbid`` +^^^^^^^^^^^^^^^^^ The :mfcmd:`module-forbid` modulerc command prevents designated modulefiles from being loaded. @@ -919,7 +918,7 @@ For example: module-forbid app/1.0 A custom message may be displayed when a user attempts to load a forbidden -module with ``--message`` option. +module with the ``--message`` option. For example: @@ -930,8 +929,8 @@ For example: See :mfcmd:`module-forbid` for additional restriction options, including custom messages, as well as **after**/**before**, **users** and **groups** restrictions. -module-virtual -^^^^^^^^^^^^^^ +``module-virtual`` +^^^^^^^^^^^^^^^^^^ The :mfcmd:`module-virtual` modulerc command associates a virtual module name with an existing modulefile. @@ -963,7 +962,7 @@ The :file:`.modulerc` file may define several virtual versions: $ module avail ------------------- /usr/share/modulefiles ------------------ - app/1.0 app/1.2 app/1.5 app/2.0 + app/1.0 app/1.2 app/1.5 app/2.0 All these virtual modules are evaluated using the same :file:`.common` modulefile. From 3d399f9bfe6931a53e1ae3bf27c86fa1f45b6494 Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Sun, 30 Aug 2026 20:15:52 +0200 Subject: [PATCH 3/5] doc: mention lesser known features guide in NEWS Assisted-by: Claude:claude-fable-5 Signed-off-by: Xavier Delaruelle --- NEWS.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index 24c149529..b30b07019 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -248,6 +248,9 @@ Modules 5.7.0 (not yet released) :ref:`CONTRIBUTING`: AI agents must never add their own ``Signed-off-by:`` trailer and their involvement must be disclosed with an ``Assisted-by:`` trailer. +* Doc: add the :ref:`user-guide` document that explains a selection of + useful but lesser known features through practical examples and common + use cases. .. _5.6 release notes: From 497e78e647d657bdc762c13eccda1e9732ef390b Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Mon, 31 Aug 2026 07:14:30 +0200 Subject: [PATCH 4/5] doc: improve sections of lesser known features guide Demonstrate what the described features bring by showing the output of the example commands, compared when relevant against a plain module avail run listing all existing modulefiles: - --output: empty LIST reporting only module names, plus the "+" and "-" LIST prefixes that adjust the currently configured element list - --latest and --default: filtered version lists, explaining that the default version is either explicitly set or implicitly the highest - --timer and --debug: timing report, debug messages and per-procedure execution time reports when both options are combined - module-tag: applied tag reported along the module name and a purge attempt on a sticky module showing its unload is skipped Rework the prereq section: present this command as a way to declare requirements, that are automatically loaded when the auto_handling mode is enabled, rather than a check that dependencies are loaded. Warn that requirements must not be declared within a condition block, as no requirement rule gets defined when the required module is already loaded. Mention that the module load modulefile command equally defines a requirement rule, loading missing modules even when auto_handling is disabled, and acting as a logical AND over the specified modules whereas prereq acts as a logical OR. Rework the conflict section: conflicting modules are automatically unloaded when the conflict_unload and auto_handling modes are enabled, so demonstrate this mechanism rather than advertising the --force option to bypass the conflict error. Describe the reflexive conflict pattern that ensures only one version of a module can be loaded at a time. Precise in the tip about unique_name_loaded that this option applies to all existing modules, which may not fit when application configurations are also handled through modulefiles. Assisted-by: Claude:claude-fable-5 Signed-off-by: Xavier Delaruelle --- .hunspell.en.dic | 2 + doc/source/user-guide.rst | 148 +++++++++++++++++++++++++++++++++----- 2 files changed, 132 insertions(+), 18 deletions(-) diff --git a/.hunspell.en.dic b/.hunspell.en.dic index 3ffdd5896..7482eb4aa 100644 --- a/.hunspell.en.dic +++ b/.hunspell.en.dic @@ -1615,3 +1615,5 @@ spack mpivars cuda lindex +defineModStartNbProc +isIcase diff --git a/doc/source/user-guide.rst b/doc/source/user-guide.rst index bbee3cd72..5fac3d0f0 100644 --- a/doc/source/user-guide.rst +++ b/doc/source/user-guide.rst @@ -67,15 +67,34 @@ only the needed fields. This option is especially useful when parsing command output in shell scripts, Python programs, or monitoring tools. +For example, with the following available modulefiles: + .. code-block:: console - $ module avail -o '' + $ module avail + ------------------- /usr/share/modulefiles ------------------ + gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 -Only module names are displayed. +When *LIST* is set to an empty value, only module names are displayed: + +.. code-block:: console + + $ module avail -o '' + gcc/8.3 gcc/11.1 gcc/14.0 openmpi/5.0.8 The output format is defined as a colon-separated list of elements. Supported elements are documented in the corresponding sub-command manual pages. +*LIST* may also be prefixed by ``+`` or ``-`` to respectively append elements +to or subtract them from the currently configured value. For example, report +the variants declared by each modulefile in addition to the default output: + +.. code-block:: console + + $ module avail -o +variant + ------------------- /usr/share/modulefiles ------------------ + gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8{cuda=on,off} + This feature helps avoid fragile text parsing pipelines. Instead of parsing formatted output: @@ -150,19 +169,32 @@ The :option:`--latest`, or :option:`-L`, and :option:`--default`, or :option:`-d`, options restrict the output of the :command:`module` :subcmd:`avail` and :subcmd:`spider` sub-commands. +For example, with the following available modulefiles: + +.. code-block:: console + + $ module avail + ------------------- /usr/share/modulefiles ------------------ + gcc/8.3 gcc/11.1(default) gcc/14.0 intel/24.0 intel/25.0 + The :option:`--latest` option displays only the highest numerically sorted version of each module name. .. code-block:: console $ module avail --latest + ------------------- /usr/share/modulefiles ------------------ + gcc/14.0 intel/25.0 The :option:`--default` option displays only the default version of each -module name. +module name. This version is either explicitly defined (as here for ``gcc``) +or implicitly the highest one (as for ``intel``). .. code-block:: console $ module avail --default + ------------------- /usr/share/modulefiles ------------------ + gcc/11.1(default) intel/25.0 These options may also be combined with other display options such as :option:`--output` or :option:`--json`. @@ -570,10 +602,13 @@ For example: ``prereq`` ^^^^^^^^^^ -The :mfcmd:`prereq` modulefile command declares one or more modules that must -already be loaded before the current modulefile can be loaded. +The :mfcmd:`prereq` modulefile command declares one or more modules as a +requirement of the current modulefile. When the automated module handling +mode is enabled (see :mconfig:`auto_handling`), which is the default, a +requirement is automatically loaded if not yet present in the user +environment. -Instead of explicitly loading dependencies: +Requirements are sometimes expressed with a condition block: .. code-block:: tcl @@ -581,17 +616,34 @@ Instead of explicitly loading dependencies: module load appA } -Declare them with: +This must be avoided: when ``appA`` is already loaded, the condition is +false and no requirement rule is defined, so Modules is not aware that the +current module depends on ``appA``. + +Declare the requirement unconditionally instead: .. code-block:: tcl prereq appA -When the prerequisite is not satisfied, Modules reports an error and refuses -to load the modulefile. +Loading a module declaring this requirement (here ``appB``) automatically +loads the required module when missing: -Multiple module names passed to a single :mfcmd:`prereq` command act as a -logical OR. Multiple :mfcmd:`prereq` commands act as a logical AND. +.. code-block:: console + + $ module load appB + Loading appB + Loading requirement: appA + +When automated module handling is disabled, a missing requirement is not +automatically loaded: an error is reported and the modulefile load fails. + +The :mfcmd:`module load` modulefile command equally declares a +requirement rule, and it loads the missing required module even when +automated module handling is disabled. The two commands also differ when +several modules are specified: :mfcmd:`module load` loads them all +(logical AND) whereas :mfcmd:`prereq` requires only one of them (logical +OR). Multiple :mfcmd:`prereq` commands act as a logical AND. ``conflict`` ^^^^^^^^^^^^ @@ -614,20 +666,43 @@ Declare the conflict with: conflict appB -When a conflicting module is loaded, Modules reports an error and refuses to -load the modulefile. +When the :mconfig:`conflict_unload` configuration option is enabled in +addition to the automated module handling mode (see :mconfig:`auto_handling`), +loading a module automatically unloads the conflicting modules found in the +user environment: + +.. code-block:: console + + $ module load appA + Loading appA + Unloading conflict: appB -The conflict check may be bypassed with the :option:`--force` option: +When these options are disabled, an error is reported instead and the +modulefile load fails. + +A modulefile declaring a conflict on its own module name defines a +*reflexive conflict*: only one version of this module can then be loaded at +a time. + +.. code-block:: tcl + + conflict appA .. code-block:: console - $ module load --force appA + $ module load appA/2.0 + Loading appA/2.0 + Unloading conflict: appA/1.0 .. tip:: - To ensure that only one version of a module can be loaded at a time, - consider enabling the :mconfig:`unique_name_loaded` configuration option - instead of declaring self-conflicts in every modulefile. + When only one module should be loaded at a time for any given module + name, enable the :mconfig:`unique_name_loaded` configuration option + instead of declaring a reflexive conflict in every modulefile. As this + option applies to all existing modules, every module name must follow + this rule. It is generally the case when only *application* modules are + provided, but may not be if application configurations are also handled + through modulefiles. ``variant`` ^^^^^^^^^^^ @@ -763,6 +838,10 @@ For example: .. code-block:: console $ module avail --timer + ------------------- /usr/share/modulefiles ------------------ + gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 + + TIMER Total execution took 20.045 ms The :option:`--debug`, or :option:`-D`, option displays debugging messages describing the internal execution of the command. @@ -772,6 +851,13 @@ For example: .. code-block:: console $ module avail -D + DEBUG setState: cmdline set to 'modulecmd.tcl bash avail -D' + DEBUG setState: shell set to 'bash' + DEBUG setState: subcmd set to 'avail' + DEBUG setConf: verbosity set to 'debug' + ... + ------------------- /usr/share/modulefiles ------------------ + gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 The :option:`--timer` and :option:`--debug` options may be combined. In this case, regular debug messages are replaced by execution time reports for each @@ -780,6 +866,11 @@ internal procedure call. .. code-block:: console $ module avail --timer -D + TIMER parseModuleCommandName avail help (0.151 ms) + TIMER isIcase (0.088 ms) + TIMER defineModStartNbProc 1 (0.060 ms) + ... + TIMER Total execution took 31.722 ms For even more detailed debugging information, use :option:`-DD`. @@ -842,6 +933,14 @@ For example: module-tag experimental app/2.0 +The tag is then reported along the module name in search results: + +.. code-block:: console + + $ module avail + ------------------- /usr/share/modulefiles ------------------ + app/1.0 app/2.0 + Several predefined tags affect the behavior of Modules: .. list-table:: @@ -866,6 +965,19 @@ For example: module-tag sticky core +Once loaded, the ``core`` module cannot be unloaded, even with the +:subcmd:`purge` sub-command. The ``sticky`` tag is abbreviated ```` in +the output. + +.. code-block:: console + + $ module purge + Unloading core/1.0 + ERROR: Unload of sticky module skipped + $ module list + Currently Loaded Modulefiles: + 1) core/1.0 + Sticky modules can help ensure that essential software stacks remain loaded. The behavior of :subcmd:`purge` with sticky modules can be configured through :mconfig:`sticky_purge`. From b33dc893f9ec904630e136c51b0ece420f679301 Mon Sep 17 00:00:00 2001 From: Xavier Delaruelle Date: Mon, 31 Aug 2026 07:24:31 +0200 Subject: [PATCH 5/5] doc: colorize console examples of lesser known features guide Turn console code-blocks into parsed-literal blocks using the color roles rendering output like in a terminal, as done in MIGRATING.rst. Modulepaths, default versions, tags, variants, messages and debugging output get the colors and styles of the real command output, whose rendering slightly differs from the monochrome mode: the default version is underlined rather than reported with a "(default)" marker and the sticky tag is rendered with a background color rather than an "" abbreviation. Assisted-by: Claude:claude-fable-5 Signed-off-by: Xavier Delaruelle --- doc/source/user-guide.rst | 260 +++++++++++++++++++------------------- 1 file changed, 130 insertions(+), 130 deletions(-) diff --git a/doc/source/user-guide.rst b/doc/source/user-guide.rst index 5fac3d0f0..617294bad 100644 --- a/doc/source/user-guide.rst +++ b/doc/source/user-guide.rst @@ -69,17 +69,17 @@ Python programs, or monitoring tools. For example, with the following available modulefiles: -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles ------------------ - gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 When *LIST* is set to an empty value, only module names are displayed: -.. code-block:: console +.. parsed-literal:: - $ module avail -o '' + :ps:`$` module avail -o '' gcc/8.3 gcc/11.1 gcc/14.0 openmpi/5.0.8 The output format is defined as a colon-separated list of elements. Supported @@ -89,11 +89,11 @@ elements are documented in the corresponding sub-command manual pages. to or subtract them from the currently configured value. For example, report the variants declared by each modulefile in addition to the default output: -.. code-block:: console +.. parsed-literal:: - $ module avail -o +variant - ------------------- /usr/share/modulefiles ------------------ - gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8{cuda=on,off} + :ps:`$` module avail -o +variant + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8\ :sgrse:`{`:sgrva:`cuda=on`:sgrse:`,`\ :sgrva:`off`:sgrse:`}` This feature helps avoid fragile text parsing pipelines. @@ -131,10 +131,10 @@ options as :subcmd:`avail`, including :option:`--output`, For example, in a hierarchical environment: -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles/common ------------------ + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles/common` ------------------ gcc/14.0 intel/25.0 Only compiler modules are initially visible. @@ -142,17 +142,17 @@ Only compiler modules are initially visible. The :subcmd:`spider` sub-command reveals additional modules together with their dependency path: -.. code-block:: console +.. parsed-literal:: - $ module spider - ------------------- /usr/share/modulefiles/common ------------------ + :ps:`$` module spider + ------------------- :sgrmp:`/usr/share/modulefiles/common` ------------------ gcc/14.0 intel/25.0 - ---------- /usr/share/modulefiles/intel (via intel/25.0) ----------- + ---------- :sgrmp:`/usr/share/modulefiles/intel` (via intel/25.0) ----------- intelmpi/25.0 - ------------ /usr/share/modulefiles/gcc (via gcc/14.0) ------------- + ------------ :sgrmp:`/usr/share/modulefiles/gcc` (via gcc/14.0) ------------- openmpi/5.0.8 @@ -171,30 +171,30 @@ The :option:`--latest`, or :option:`-L`, and :option:`--default`, or For example, with the following available modulefiles: -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles ------------------ - gcc/8.3 gcc/11.1(default) gcc/14.0 intel/24.0 intel/25.0 + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 intel/24.0 intel/25.0 The :option:`--latest` option displays only the highest numerically sorted version of each module name. -.. code-block:: console +.. parsed-literal:: - $ module avail --latest - ------------------- /usr/share/modulefiles ------------------ + :ps:`$` module avail --latest + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ gcc/14.0 intel/25.0 The :option:`--default` option displays only the default version of each module name. This version is either explicitly defined (as here for ``gcc``) or implicitly the highest one (as for ``intel``). -.. code-block:: console +.. parsed-literal:: - $ module avail --default - ------------------- /usr/share/modulefiles ------------------ - gcc/11.1(default) intel/25.0 + :ps:`$` module avail --default + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + :sgrde:`gcc/11.1` intel/25.0 These options may also be combined with other display options such as :option:`--output` or :option:`--json`. @@ -211,21 +211,21 @@ modulefiles. This behavior is equivalent to :option:`--indepth` and can be configured with :command:`module` :subcmd:`config` :mconfig:`avail_indepth` and :mconfig:`spider_indepth`. -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles ------------------ + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ intel/24.0 intel/25.0 gcc/8.3 gcc/11.1 gcc/14.0 The :option:`--no-indepth` option limits results to the depth level expressed by the search query. Modulefiles contained in matching directories are not displayed. -.. code-block:: console +.. parsed-literal:: - $ module avail --no-indepth - ------------------- /usr/share/modulefiles ------------------ - intel/ gcc/ + :ps:`$` module avail --no-indepth + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + :sgrdi:`intel`/ :sgrdi:`gcc`/ ``--json`` ^^^^^^^^^^ @@ -240,9 +240,9 @@ It is supported by the :command:`module` :subcmd:`avail`, :subcmd:`list`, JSON output is intended for machine consumption and can be processed directly by tools such as ``jq`` or Python scripts. -.. code-block:: console +.. parsed-literal:: - $ module avail -j | jq . + :ps:`$` module avail -j | jq . { "/usr/share/modulefiles": { "gcc/14.0": { @@ -304,11 +304,11 @@ The :envvar:`MODULES_COLLECTION_TARGET` environment variable may be defined to append a suffix to collection names. This feature is particularly useful on systems sharing a common home directory across multiple clusters. -.. code-block:: console +.. parsed-literal:: - $ module save - $ module purge - $ module restore + :ps:`$` module save + :ps:`$` module purge + :ps:`$` module restore ``stash`` commands ^^^^^^^^^^^^^^^^^^ @@ -349,22 +349,22 @@ sub-commands: Save the current environment: -.. code-block:: console +.. parsed-literal:: - $ module stash + :ps:`$` module stash Load a different software stack: -.. code-block:: console +.. parsed-literal:: - $ module purge - $ module load ... + :ps:`$` module purge + :ps:`$` module load ... Restore the original environment: -.. code-block:: console +.. parsed-literal:: - $ module stashpop + :ps:`$` module stashpop Unlike regular collections, stash collections are intended to be short-lived and are automatically removed when restored with @@ -388,21 +388,21 @@ The project is available on GitHub: `cea-hpc/mogui `_. It can be installed from PyPI with :command:`pip` or from Spack. -.. code-block:: console +.. parsed-literal:: - $ pip install modules-gui + :ps:`$` pip install modules-gui or: -.. code-block:: console +.. parsed-literal:: - $ spack install py-modules-gui + :ps:`$` spack install py-modules-gui Then start the graphical interface: -.. code-block:: console +.. parsed-literal:: - $ mogui + :ps:`$` mogui .. image:: https://raw.githubusercontent.com/cea-hpc/mogui/main/doc/sneak_peek.gif :alt: mogui graphical interface @@ -455,9 +455,9 @@ shells, such as Bash, the ``&`` character must be escaped or quoted. For example, execute a command in a clean environment: -.. code-block:: console +.. parsed-literal:: - $ envml purge -- gcc hello.c + :ps:`$` envml purge -- gcc hello.c The :ref:`envml(1)` command can simplify scripts by replacing sequences of module operations with a single command. @@ -491,12 +491,12 @@ unintended changes. When a modulefile attempts to modify a protected variable, Modules ignores the modification and emits a warning. -.. code-block:: console +.. parsed-literal:: - $ module config protected_envvars LD_PRELOAD - $ module load intel - Loading intel/25.0 - WARNING: Modification of protected environment variable LD_PRELOAD ignored + :ps:`$` module config protected_envvars LD_PRELOAD + :ps:`$` module load intel + Loading :sgrhi:`intel/25.0` + :sgrwa:`WARNING`: Modification of protected environment variable LD_PRELOAD ignored Developing and configuring modulefiles -------------------------------------- @@ -509,15 +509,15 @@ configured text editor (cf. :subcmd:`config` :mconfig:`editor`). Instead of manually locating a modulefile: -.. code-block:: console +.. parsed-literal:: - $ vi $(module path appA) + :ps:`$` vi $(module path appA) One can directly edit it with: -.. code-block:: console +.. parsed-literal:: - $ module edit appA + :ps:`$` module edit appA ``module lint`` ^^^^^^^^^^^^^^^ @@ -530,9 +530,9 @@ common mistakes. Analyze a specific modulefile: -.. code-block:: console +.. parsed-literal:: - $ module lint appA + :ps:`$` module lint appA Reported issues may include: @@ -573,9 +573,9 @@ modulefiles. For example: -.. code-block:: console +.. parsed-literal:: - $ module sh-to-mod bash setup-env.sh > setup-env.mod + :ps:`$` module sh-to-mod bash setup-env.sh > setup-env.mod .. tip:: @@ -590,9 +590,9 @@ modulefiles and reports the resulting environment changes as shell code. For example: -.. code-block:: console +.. parsed-literal:: - $ module mod-to-sh bash appA > setup-env.sh + :ps:`$` module mod-to-sh bash appA > setup-env.sh .. tip:: @@ -629,11 +629,11 @@ Declare the requirement unconditionally instead: Loading a module declaring this requirement (here ``appB``) automatically loads the required module when missing: -.. code-block:: console +.. parsed-literal:: - $ module load appB - Loading appB - Loading requirement: appA + :ps:`$` module load appB + Loading :sgrhi:`appB` + :sgrin:`Loading requirement`: appA When automated module handling is disabled, a missing requirement is not automatically loaded: an error is reported and the modulefile load fails. @@ -671,11 +671,11 @@ addition to the automated module handling mode (see :mconfig:`auto_handling`), loading a module automatically unloads the conflicting modules found in the user environment: -.. code-block:: console +.. parsed-literal:: - $ module load appA - Loading appA - Unloading conflict: appB + :ps:`$` module load appA + Loading :sgrhi:`appA` + :sgrin:`Unloading conflict`: appB When these options are disabled, an error is reported instead and the modulefile load fails. @@ -688,11 +688,11 @@ a time. conflict appA -.. code-block:: console +.. parsed-literal:: - $ module load appA/2.0 - Loading appA/2.0 - Unloading conflict: appA/1.0 + :ps:`$` module load appA/2.0 + Loading :sgrhi:`appA/2.0` + :sgrin:`Unloading conflict`: appA/1.0 .. tip:: @@ -722,9 +722,9 @@ Boolean variants are enabled with ``+`` and disabled with ``~``: #%Module variant --boolean cuda -.. code-block:: console +.. parsed-literal:: - $ module load openmpi +cuda + :ps:`$` module load openmpi +cuda Variants with multiple values are declared by listing the accepted values: @@ -735,9 +735,9 @@ Variants with multiple values are declared by listing the accepted values: And selected with the ``=`` syntax: -.. code-block:: console +.. parsed-literal:: - $ module load openmpi compiler=intel + :ps:`$` module load openmpi compiler=intel The :mfcmd:`getvariant` modulefile command retrieves the value of a declared variant, allowing the modulefile to adapt its behavior. @@ -760,15 +760,15 @@ for variants. For example: -.. code-block:: console +.. parsed-literal:: - $ module config variant_shortcut compiler=% + :ps:`$` module config variant_shortcut compiler=% This allows users to write: -.. code-block:: console +.. parsed-literal:: - $ module load openmpi %gcc + :ps:`$` module load openmpi %gcc Advanced manipulations ---------------------- @@ -835,42 +835,42 @@ The :option:`--timer` option reports the total execution time of a command. For example: -.. code-block:: console +.. parsed-literal:: - $ module avail --timer - ------------------- /usr/share/modulefiles ------------------ - gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 + :ps:`$` module avail --timer + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 - TIMER Total execution took 20.045 ms + :sgrtr:`TIMER Total execution took 20.045 ms` The :option:`--debug`, or :option:`-D`, option displays debugging messages describing the internal execution of the command. For example: -.. code-block:: console +.. parsed-literal:: - $ module avail -D - DEBUG setState: cmdline set to 'modulecmd.tcl bash avail -D' - DEBUG setState: shell set to 'bash' - DEBUG setState: subcmd set to 'avail' - DEBUG setConf: verbosity set to 'debug' + :ps:`$` module avail -D + :sgrtr:`DEBUG setState: cmdline set to 'modulecmd.tcl bash avail -D'` + :sgrtr:`DEBUG setState: shell set to 'bash'` + :sgrtr:`DEBUG setState: subcmd set to 'avail'` + :sgrtr:`DEBUG setConf: verbosity set to 'debug'` ... - ------------------- /usr/share/modulefiles ------------------ - gcc/8.3 gcc/11.1(default) gcc/14.0 openmpi/5.0.8 + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + gcc/8.3 :sgrde:`gcc/11.1` gcc/14.0 openmpi/5.0.8 The :option:`--timer` and :option:`--debug` options may be combined. In this case, regular debug messages are replaced by execution time reports for each internal procedure call. -.. code-block:: console +.. parsed-literal:: - $ module avail --timer -D - TIMER parseModuleCommandName avail help (0.151 ms) - TIMER isIcase (0.088 ms) - TIMER defineModStartNbProc 1 (0.060 ms) + :ps:`$` module avail --timer -D + :sgrtr:`TIMER parseModuleCommandName avail help (0.151 ms)` + :sgrtr:`TIMER isIcase (0.088 ms)` + :sgrtr:`TIMER defineModStartNbProc 1 (0.060 ms)` ... - TIMER Total execution took 31.722 ms + :sgrtr:`TIMER Total execution took 31.722 ms` For even more detailed debugging information, use :option:`-DD`. @@ -935,11 +935,11 @@ For example: The tag is then reported along the module name in search results: -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles ------------------ - app/1.0 app/2.0 + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ + app/1.0 app/2.0 :sgrse:`<`\ experimental\ :sgrse:`>` Several predefined tags affect the behavior of Modules: @@ -966,17 +966,17 @@ For example: module-tag sticky core Once loaded, the ``core`` module cannot be unloaded, even with the -:subcmd:`purge` sub-command. The ``sticky`` tag is abbreviated ```` in -the output. +:subcmd:`purge` sub-command. The ``sticky`` tag is rendered with a specific +background color in the output. -.. code-block:: console +.. parsed-literal:: - $ module purge - Unloading core/1.0 - ERROR: Unload of sticky module skipped - $ module list + :ps:`$` module purge + Unloading :sgrshi:`core/1.0` + :sgrer:`ERROR`: Unload of sticky module skipped + :ps:`$` module list Currently Loaded Modulefiles: - 1) core/1.0 + 1) :sgrs:`core/1.0` Sticky modules can help ensure that essential software stacks remain loaded. The behavior of :subcmd:`purge` with sticky modules can be configured through @@ -1070,10 +1070,10 @@ The :file:`.modulerc` file may define several virtual versions: module-virtual 1.5 .common module-virtual 2.0 .common -.. code-block:: console +.. parsed-literal:: - $ module avail - ------------------- /usr/share/modulefiles ------------------ + :ps:`$` module avail + ------------------- :sgrmp:`/usr/share/modulefiles` ------------------ app/1.0 app/1.2 app/1.5 app/2.0 All these virtual modules are evaluated using the same :file:`.common` @@ -1099,9 +1099,9 @@ The :mconfig:`logger` option defines the command used to record log messages. For example: -.. code-block:: console +.. parsed-literal:: - $ module config logger "/usr/bin/logger -t modules" + :ps:`$` module config logger "/usr/bin/logger -t modules" The :mconfig:`logged_events` option defines which module events are recorded. @@ -1125,9 +1125,9 @@ Multiple events may be specified using the ``:`` separator. For example: -.. code-block:: console +.. parsed-literal:: - $ module config logged_events requested_cmd:requested_eval + :ps:`$` module config logged_events requested_cmd:requested_eval By default, no events are logged.