From d9b53a84863dc4bf5708fdbf65fc52bfc4c0504f Mon Sep 17 00:00:00 2001 From: RafsanNeloy Date: Sat, 14 Mar 2026 05:43:29 +0600 Subject: [PATCH 1/3] Add README dfor the data and formatter layers. Signed-off-by: RafsanNeloy --- cbrain_cli/data/readme.md | 34 +++++++++++++++++++++++++++++++++ cbrain_cli/formatter/readme.md | 35 ++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 cbrain_cli/data/readme.md create mode 100644 cbrain_cli/formatter/readme.md diff --git a/cbrain_cli/data/readme.md b/cbrain_cli/data/readme.md new file mode 100644 index 0000000..ea4ca41 --- /dev/null +++ b/cbrain_cli/data/readme.md @@ -0,0 +1,34 @@ +# Welcome to the CBRAIN CLI Data Layer! πŸ‘‹ + +This directory is the core engine connecting our command-line tools to **CBRAIN**β€”a powerful web-enabled platform built to manage large, distributed research datasets and mediate high-performance computing (HPC) tasks. + +Whenever the CLI needs to talk to CBRAIN's architecture (specifically the **BrainPortal** API), it happens right here. + +## How It Is Used πŸ› οΈ + +Think of these Python modules as your direct bridge to CBRAIN's resources. They handle all the API calls, data fetching, and object modeling. If a user command needs to interact with the CBRAIN database, these modules do the heavy lifting. + +### The Modules at a Glance (with Example Commands!): + +* **`background_activities.py`**: Tracks backend jobs in progress (`cbrain background`). + * *Example:* `cbrain background list` +* **`data_providers.py`**: Connects to the systems storing your research data (`cbrain dataprovider`). + * *Example:* `cbrain dataprovider show 15` +* **`files.py`**: Manages data uploads, caching, and transfers across CBRAIN (`cbrain file`). + * *Example:* `cbrain file move --file-id 2 --dp-id 15` +* **`projects.py`**: Keeps user research organized (`cbrain project`). + * *Example:* `cbrain project switch 10` +* **`remote_resources.py`**: Interfaces with external network capabilities (`cbrain remote-resource`). + * *Example:* `cbrain remote-resource list` +* **`tags.py`**: Organizes and categorizes files and tasks (`cbrain tag`). + * *Example:* `cbrain tag create --name NewTag1 --user-id 2 --group-id 3` +* **`tasks.py`**: Orchestrates intensive compute jobs on remote HPCs (`cbrain task`). + * *Example:* `cbrain task show 2` +* **`tools.py` & `tool_configs.py`**: Manages the scientific tools available on CBRAIN (`cbrain tool` / `cbrain tool-config`). + * *Example:* `cbrain tool list` + +## How It Fits Together 🧩 + +**Data here, Display there.** + +This folder is strictly for fetching raw objects from the CBRAIN server. Once we pull the data, we hand it off to the `formatter/` directory, which takes those raw responses and turns them into clean, human-readable outputs for your terminal! diff --git a/cbrain_cli/formatter/readme.md b/cbrain_cli/formatter/readme.md new file mode 100644 index 0000000..d591632 --- /dev/null +++ b/cbrain_cli/formatter/readme.md @@ -0,0 +1,35 @@ +# Welcome to the CBRAIN CLI Formatter Layer! 🎨 + +This directory is all about **presentation**. Once the CLI fetches raw data objects from the CBRAIN backend (using the `data/` modules), it hands them over to these formatter scripts to make them look great in your terminal. + +These formatters are responsible for creating the clean tables, organized summaries, and human-readable text you see when you run a command. + +## How It Fits Together 🧩 + +**Data there, Display here.** + +While the `data/` directory handles backend API interactions, the `formatter/` directory focuses exclusively on presenting that info. Every `_fmt.py` file here perfectly matches a corresponding data module. For example, `tasks.py` gets the data, and `tasks_fmt.py` decides how to draw it on your screen! + +## What the Formatter Actually Does πŸͺ„ + +Imagine you ask CBRAIN for a list of your files using `cbrain file list`. The raw data coming from the server is a dense machine-readable bundle filled with IDs, timestamps, and metadata. + +Instead of throwing that raw JSON at you, the formatter layer catches the data and turns it into a beautiful, easy-to-read table right in your terminal. Here is exactly what these scripts do: + +**The Raw Data (what `data/` gets):** +```json +{"id": 1024, "name": "my_mri_scan.nii.gz", "size": 47185920, "status": "synced"} +``` + +**The Formatted Output (what `formatter/` shows you):** +```text ++---------+----------------------+---------+-------------+ +| File ID | Name | Size | Status | ++---------+----------------------+---------+-------------+ +| 1024 | my_mri_scan.nii.gz | 45.0 MB | Synced | ++---------+----------------------+---------+-------------+ +``` + +*(Fun fact: We don't use heavy external libraries like `PrettyTable` or `Rich` for this! The CLI uses a custom-built, lightweight, dynamic table formatter powered completely by Python's built-in `textwrap` and `shutil` libraries. That means zero extra dependencies to slow you down!)* + +Every `_fmt.py` script in this folder is basically a tiny artist that knows exactly how to draw its specific type of data so it's perfectly readable for you! From e1538202eee216e463872054c4ce89b7b0b25fd5 Mon Sep 17 00:00:00 2001 From: rafsanneloy Date: Sat, 8 Aug 2026 02:35:19 +0600 Subject: [PATCH 2/3] Update README and formatter documentation & unit tests overview Signed-off-by: rafsanneloy --- cbrain_cli/README.md | 43 +++++++++++++++ cbrain_cli/data/readme.md | 60 ++++++++++++++++----- cbrain_cli/formatter/readme.md | 38 +++++++++---- tests/README.md | 97 ++++++++++++++++++++++++++++++++++ 4 files changed, 214 insertions(+), 24 deletions(-) create mode 100644 cbrain_cli/README.md create mode 100644 tests/README.md diff --git a/cbrain_cli/README.md b/cbrain_cli/README.md new file mode 100644 index 0000000..76f5987 --- /dev/null +++ b/cbrain_cli/README.md @@ -0,0 +1,43 @@ +# Welcome to the CBRAIN CLI Package! + +This directory is the heart of the `cbrain` command β€” parse input, check the session, talk to CBRAIN, and print a clean result. + +## How It Fits Together + +**Parse β†’ Auth β†’ Handle β†’ Fetch β†’ Format.** + +| Piece | Role | +| --- | --- | +| `main.py` | Build the parser, gate auth, dispatch | +| `sessions.py` / `config.py` | Login, logout, credential file | +| `handlers.py` | Call data, then formatter; set exit codes | +| [`data/`](data/README.md) | BrainPortal API calls (no printing) | +| [`formatter/`](formatter/README.md) | Tables, summaries, `--json`, `--jsonl` | +| `cli_utils.py` | `CbrainClient`, errors, pagination, output helpers | +| `users.py` | Current-user helpers for `whoami` | + +## A Quick Mental Model + +```mermaid +flowchart TD + CMD["cbrain file list"] --> MAIN["main.py"] + MAIN --> AUTH{"authenticated?"} + AUTH -->|no| STOP["ask login"] + AUTH -->|yes| H["handlers.py"] + + H -->|"1. ask for data"| D["data/"] + D -->|"2. HTTP via CbrainClient
token from config.py"| API["CBRAIN API"] + API -->|"3. raw JSON"| D + D -->|"4. Python objects"| H + + H -->|"5. present"| F["formatter/"] + F --> OUT["terminal"] +``` + +Use `CbrainClient` for new API calls. Credentials load at command time via `from_credentials()` β€” never at import. + +## Golden Rules ✨ + +* **Data never prints** β€” validate and return objects only. +* **Formatters own presentation** β€” including `--json` / `--jsonl`. +* **Handlers glue both** β€” exit codes and `--yes` for destructive actions. diff --git a/cbrain_cli/data/readme.md b/cbrain_cli/data/readme.md index ea4ca41..76bf8ac 100644 --- a/cbrain_cli/data/readme.md +++ b/cbrain_cli/data/readme.md @@ -6,29 +6,63 @@ Whenever the CLI needs to talk to CBRAIN's architecture (specifically the **Brai ## How It Is Used πŸ› οΈ -Think of these Python modules as your direct bridge to CBRAIN's resources. They handle all the API calls, data fetching, and object modeling. If a user command needs to interact with the CBRAIN database, these modules do the heavy lifting. +Think of these Python modules as your direct bridge to CBRAIN's resources. They handle BrainPortal API interactions β€” reads and writes β€” plus client-side validation. If a user command needs the CBRAIN API, these modules do the heavy lifting. ### The Modules at a Glance (with Example Commands!): * **`background_activities.py`**: Tracks backend jobs in progress (`cbrain background`). - * *Example:* `cbrain background list` -* **`data_providers.py`**: Connects to the systems storing your research data (`cbrain dataprovider`). - * *Example:* `cbrain dataprovider show 15` + * `cbrain background list` + * `cbrain background show 42` + +* **`data_providers.py`**: Connects to the systems storing your research data (`cbrain data-provider`). + * `cbrain data-provider list` + * `cbrain data-provider show 15` + * `cbrain data-provider is-alive 15` + * `cbrain data-provider delete-unregistered-files 15` + * **`files.py`**: Manages data uploads, caching, and transfers across CBRAIN (`cbrain file`). - * *Example:* `cbrain file move --file-id 2 --dp-id 15` + * `cbrain file list` + * `cbrain file show 99` + * `cbrain file upload /path/to/file.nii --data-provider-id 15 --group-id 3` + * `cbrain file copy --file-id 2 3 --data-provider-id 15` + * `cbrain file move --file-id 2 --data-provider-id 15` + * `cbrain file delete 99` + * **`projects.py`**: Keeps user research organized (`cbrain project`). - * *Example:* `cbrain project switch 10` -* **`remote_resources.py`**: Interfaces with external network capabilities (`cbrain remote-resource`). - * *Example:* `cbrain remote-resource list` + * `cbrain project list` + * `cbrain project show 10` + * `cbrain project switch 10` + * `cbrain project switch all` + * `cbrain project unswitch` + +* **`remote_resources.py`**: Interfaces with bourreaux / execution servers (`cbrain remote-resource`). + * `cbrain remote-resource list` + * `cbrain remote-resource show 5` + * **`tags.py`**: Organizes and categorizes files and tasks (`cbrain tag`). - * *Example:* `cbrain tag create --name NewTag1 --user-id 2 --group-id 3` + * `cbrain tag list` + * `cbrain tag show 7` + * `cbrain tag create --name NewTag1 --user-id 2 --group-id 3` + * `cbrain tag update 7 --name UpdatedTag --user-id 2 --group-id 3` + * `cbrain tag delete 7` + * **`tasks.py`**: Orchestrates intensive compute jobs on remote HPCs (`cbrain task`). - * *Example:* `cbrain task show 2` + * `cbrain task list` + * `cbrain task list bourreau-id 4` + * `cbrain task show 2` + * `cbrain task operation hold --task-id 2` + * `cbrain task operation terminate --task-id 1 2` + * `cbrain --debug task list` + * **`tools.py` & `tool_configs.py`**: Manages the scientific tools available on CBRAIN (`cbrain tool` / `cbrain tool-config`). - * *Example:* `cbrain tool list` + * `cbrain tool list` + * `cbrain tool show 8` + * `cbrain tool-config list` + * `cbrain tool-config show 12` + * `cbrain tool-config boutiques-descriptor 12` ## How It Fits Together 🧩 -**Data here, Display there.** +**API here, Display there.** -This folder is strictly for fetching raw objects from the CBRAIN server. Once we pull the data, we hand it off to the `formatter/` directory, which takes those raw responses and turns them into clean, human-readable outputs for your terminal! +**API interaction layer**: GET/list/show plus state-changing calls (upload, delete, move/copy, tag create/update/delete, project switch/unswitch, task operations, and similar). Modules talk to BrainPortal via `CbrainClient`, validate inputs, and return domain data (or raise typed errors). They do not print. Handlers pass results to `formatter/` for terminal output. diff --git a/cbrain_cli/formatter/readme.md b/cbrain_cli/formatter/readme.md index d591632..9b57264 100644 --- a/cbrain_cli/formatter/readme.md +++ b/cbrain_cli/formatter/readme.md @@ -1,6 +1,6 @@ -# Welcome to the CBRAIN CLI Formatter Layer! 🎨 +# Welcome to the CBRAIN CLI Formatter Layer! -This directory is all about **presentation**. Once the CLI fetches raw data objects from the CBRAIN backend (using the `data/` modules), it hands them over to these formatter scripts to make them look great in your terminal. +This directory is all about **presentation**. Once a handler has domain data from the `data/` API layer, it hands that data to these formatters for terminal output. These formatters are responsible for creating the clean tables, organized summaries, and human-readable text you see when you run a command. @@ -16,20 +16,36 @@ Imagine you ask CBRAIN for a list of your files using `cbrain file list`. The ra Instead of throwing that raw JSON at you, the formatter layer catches the data and turns it into a beautiful, easy-to-read table right in your terminal. Here is exactly what these scripts do: -**The Raw Data (what `data/` gets):** +**The Raw Data (what `data/` returns):** ```json -{"id": 1024, "name": "my_mri_scan.nii.gz", "size": 47185920, "status": "synced"} +{"id": 1024, "type": "SingleFile", "name": "my_mri_scan.nii.gz"} ``` -**The Formatted Output (what `formatter/` shows you):** +**The Formatted Output (what `formatter/` shows you for `cbrain file list`):** ```text -+---------+----------------------+---------+-------------+ -| File ID | Name | Size | Status | -+---------+----------------------+---------+-------------+ -| 1024 | my_mri_scan.nii.gz | 45.0 MB | Synced | -+---------+----------------------+---------+-------------+ +ID Type File Name +---- ---------- ------------------ +1024 SingleFile my_mri_scan.nii.gz ``` *(Fun fact: We don't use heavy external libraries like `PrettyTable` or `Rich` for this! The CLI uses a custom-built, lightweight, dynamic table formatter powered completely by Python's built-in `textwrap` and `shutil` libraries. That means zero extra dependencies to slow you down!)* -Every `_fmt.py` script in this folder is basically a tiny artist that knows exactly how to draw its specific type of data so it's perfectly readable for you! +### Raw JSON Output + +Every command also supports a `--json` flag that skips the formatter entirely and dumps the raw API response straight to your terminal. Useful for scripting or piping into other tools! + +## The Formatter Files πŸ—‚οΈ + +Each `_fmt.py` file knows exactly how to display its specific type of data: + +* **`background_activities_fmt.py`**: Renders background job lists and individual activity details. +* **`data_providers_fmt.py`**: Renders data provider lists and full provider detail views. +* **`files_fmt.py`**: Renders file lists, file detail views, upload results, move/copy results, and delete confirmations. +* **`projects_fmt.py`**: Renders project lists, the current active project, full project details, the "no project" state, and unswitch results. +* **`remote_resources_fmt.py`**: Renders remote resource (bourreau) lists and individual resource details. +* **`tags_fmt.py`**: Renders tag lists, tag details, and create/update/delete operation results. +* **`tasks_fmt.py`**: Renders task lists, detailed task views, and bulk operation results. +* **`tool_configs_fmt.py`**: Renders tool configuration lists, config details, and Boutiques descriptors. +* **`tools_fmt.py`**: Renders tool lists and individual tool detail views. + +Every formatter script is basically a tiny artist that knows exactly how to draw its specific type of data so it's perfectly readable for you! diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..3f154a4 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,97 @@ +# CBRAIN CLI Unit Tests + +This directory contains the unit test suite for the CBRAIN CLI. + +Every test mocks the HTTP layer (`urllib.request.urlopen`), so **no real CBRAIN +server is required**. For live-server end-to-end output checks, see +[capture_tests/README.md](../capture_tests/README.md). Those capture tests +expect a seeded CBRAIN test instance; server setup follows the same pattern as +CBRAIN's own [API testing frameworks](https://github.com/aces/cbrain/blob/master/BrainPortal/test_api/README.md). + +Alongside the test modules, __conftest.py__ provides shared helpers and pytest +fixtures used across the suite. + +## Running the tests + +From the repository root (with the `dev` extra installed): + +```bash +pytest +``` + +With a coverage report: + +```bash +pytest --cov=cbrain_cli +``` + +Selecting a subset by path or by test name substring: + +```bash +pytest tests/test_files.py +pytest tests/test_sessions.py -k login +pytest tests -k "handler and not ops" +``` + +More options are available via `pytest --help`. + +## Shared helpers and fixtures + +__conftest.py__ is the common entry point for test scaffolding. + +Helper functions: + +- `make_args(**kwargs)` β€” fake `argparse.Namespace` (defaults: `page=1`, `per_page=25`) +- `parse_json_output(capsys)` β€” parse captured stdout as JSON +- `run_main(monkeypatch, argv)` β€” run `main()` with a fake `sys.argv` +- `install_auth()` β€” write known credentials so `CbrainClient.from_credentials()` succeeds +- `write_auth_credentials(path, ...)` β€” write a credentials JSON file to a given path +- `sample_credentials(**overrides)` β€” credentials dict with sensible defaults + +Fixtures: + +- `_isolate_credentials` (autouse) β€” redirect the credentials file to a temp directory so the real `~/.config/cbrain` file is never touched +- `creds_file` / `sessions_creds_file` β€” path to the isolated credentials file +- `fake_credentials` β€” known credentials for `is_authenticated()` and `CbrainClient` +- `mock_urlopen` β€” patch `urlopen` with one configurable response: `mock_urlopen(response_json, status=200)` +- `capture_urlopen` β€” like `mock_urlopen`, but also records URL, headers, method, and body; returns `(configure, captured)` + +**Note:** credential isolation is automatic. Unit tests must not depend on a real login session. + +## Test modules + +### Data layer + +- `test_background_activities.py` β€” list, show, empty list, missing ID +- `test_data_providers.py` β€” list, show (ID fallback), `is-alive`, `delete-unregistered-files` +- `test_files.py` β€” list (filters), show, upload, copy, move, delete, missing-arg validation +- `test_projects.py` β€” list, show, switch, unswitch, stale group cleanup +- `test_remote_resources.py` β€” list, show, empty list, missing ID +- `test_tags.py` β€” list, show, create, update, delete, missing-arg validation +- `test_tasks.py` β€” list (`bourreau-id` filter), show, operation +- `test_tool_configs.py` β€” list, show, Boutiques descriptor, missing ID +- `test_tools.py` β€” list (pagination), show (multi-page search, not-found) + +### Formatter layer + +- `test_formatters.py` β€” all `_fmt.py` outputs: normal, empty, `--json`, `--jsonl` + +### Handlers + +- `test_handlers.py` β€” list/show/switch/unswitch/`whoami` handler contracts; task operation success/empty/validation/API +- `test_handlers_ops.py` β€” dataprovider ops, file upload/copy/delete (incl. abort without `--yes`), tags, Boutiques + +### Infrastructure + +- `test_cbrain_client.py` β€” URL normalization, timeouts, GET/POST/DELETE, query params, multipart, HTTP errors +- `test_cli_utils_output.py` β€” tables, JSONL, `version_info`, `confirm_destructive` +- `test_config.py` β€” credential load/save, permissions, corrupt JSON +- `test_exit_codes.py` β€” mapped exit codes for HTTP, URL, validation, interrupt, unexpected errors +- `test_sessions.py` β€” login/logout edge cases and file cleanup +- `test_users.py` β€” `user_details` / `whoami`, including login-then-whoami flow + +### Parsing and dispatch + +- `test_parser.py` β€” command registration, kebabβ†’snake flags, `--json` / `--jsonl`, pagination flags +- `test_main_dispatch.py` β€” auth bypass for logout/version, help paths, `task list bourreau-id` +- `test_pagination.py` β€” page / per-page bounds and query-param injection From 8596c0fb320e9d161036caeda65f48deb2b7bce7 Mon Sep 17 00:00:00 2001 From: Rafsanul Islam Neloy <89649374+RafsanNeloy@users.noreply.github.com> Date: Fri, 7 Aug 2026 20:49:31 +0000 Subject: [PATCH 3/3] Refactor README files Signed-off-by: Rafsanul Islam Neloy <89649374+RafsanNeloy@users.noreply.github.com> --- cbrain_cli/README.md | 6 +++--- cbrain_cli/data/{readme.md => README.md} | 0 cbrain_cli/formatter/{readme.md => README.md} | 0 3 files changed, 3 insertions(+), 3 deletions(-) rename cbrain_cli/data/{readme.md => README.md} (100%) rename cbrain_cli/formatter/{readme.md => README.md} (100%) diff --git a/cbrain_cli/README.md b/cbrain_cli/README.md index 76f5987..6b4ca89 100644 --- a/cbrain_cli/README.md +++ b/cbrain_cli/README.md @@ -1,8 +1,8 @@ -# Welcome to the CBRAIN CLI Package! +# Welcome to the CBRAIN CLI Package! This directory is the heart of the `cbrain` command β€” parse input, check the session, talk to CBRAIN, and print a clean result. -## How It Fits Together +## How It Fits Together **Parse β†’ Auth β†’ Handle β†’ Fetch β†’ Format.** @@ -16,7 +16,7 @@ This directory is the heart of the `cbrain` command β€” parse input, check the s | `cli_utils.py` | `CbrainClient`, errors, pagination, output helpers | | `users.py` | Current-user helpers for `whoami` | -## A Quick Mental Model +## A Quick Mental Model ```mermaid flowchart TD diff --git a/cbrain_cli/data/readme.md b/cbrain_cli/data/README.md similarity index 100% rename from cbrain_cli/data/readme.md rename to cbrain_cli/data/README.md diff --git a/cbrain_cli/formatter/readme.md b/cbrain_cli/formatter/README.md similarity index 100% rename from cbrain_cli/formatter/readme.md rename to cbrain_cli/formatter/README.md