Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ and this project adheres to [Semantic Versioning][].
[keep a changelog]: https://keepachangelog.com/en/1.0.0/
[semantic versioning]: https://semver.org/spec/v2.0.0.html

## 2.2.0

### Added
- Bundled a Claude Code [Agent Skill](https://docs.claude.com/en/docs/claude-code/skills) that teaches coding agents to use decoupler correctly (method selection, calling conventions, output locations, priors, datasets, and benchmarking)
- Added the `decoupler-install-skills` console script to install the skill into `~/.claude/skills/decoupler/` (use `--force` to refresh after upgrading, or `--print-path` to point `CLAUDE_SKILLS_PATH` at the bundled copy in place)

## 2.1.6

### Bugfixes
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,20 @@ mamba create -n=dcp conda-forge::decoupler-py
pip install git+https://github.com/scverse/decoupler.git@main
```

## Claude Code Skill

decoupler ships an [Agent Skill](https://docs.claude.com/en/docs/claude-code/skills) for [Claude Code](https://claude.com/claude-code) that teaches the agent to use the library correctly. Install it once into your personal skills directory:

```bash
decoupler-install-skills
```

This copies the skill to `~/.claude/skills/decoupler/`, making it available in **every** project (re-run with `--force` after upgrading). Once installed, just ask Claude Code to do decoupler tasks ("find the most active TFs in this cell cluster", "score hallmark gene sets with ora", "build a consensus across methods") and the skill is consulted automatically. To avoid copying into your home directory, point Claude Code at the bundled copy instead:

```bash
export CLAUDE_SKILLS_PATH="$(decoupler-install-skills --print-path)"
```

## Release notes

See the [changelog][].
Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ requires = [ "hatchling" ]

[project]
name = "decoupler"
version = "2.1.6"
version = "2.2.0"
description = "Python package to perform enrichment analysis from omics data."
readme = "README.md"
license = { file = "LICENSE" }
Expand Down Expand Up @@ -38,6 +38,7 @@ dependencies = [
urls.Documentation = "https://decoupler.readthedocs.io/"
urls.Homepage = "https://github.com/scverse/decoupler"
urls.Source = "https://github.com/scverse/decoupler"
scripts.decoupler-install-skills = "decoupler._skills.install:main"

[dependency-groups]
dev = [
Expand Down
6 changes: 6 additions & 0 deletions src/decoupler/_skills/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""Bundled Claude Code Agent Skill and its installer.

This subpackage keeps the decoupler Agent Skill (the ``data/`` directory) and the
command that installs it (``install.py``) together and out of the main package
namespace. See ``install.py`` for usage.
"""
85 changes: 85 additions & 0 deletions src/decoupler/_skills/data/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
name: decoupler
description: Use for any task involving the decoupler library — inferring biological activity/enrichment scores from omics data (bulk, single-cell, spatial). Triggers on estimating transcription factor (TF) activity, pathway activity, or gene-set enrichment from an AnnData/DataFrame; running ulm, mlm, ora, gsea, gsva, aucell, viper, zscore, waggr, mdt, udt; multi-method consensus; fetching prior knowledge (CollecTRI, DoRothEA, PROGENy, MSigDB hallmarks, OmniPath); pseudobulk and expression filtering; using bundled example datasets (pbmc3k, covid5k, erygast1k, hsctgfb, msvisium, knocktf, toy) and whether they are raw counts or normalized; ranking/plotting activity scores; or benchmarking methods against ground truth. This is a router skill — read the relevant file under references/ before writing decoupler code, because outputs land in different places and several defaults are non-obvious.
---

# decoupler

`decoupler` estimates biological activities (transcription factors, pathways, gene
sets) from omics data. You give it a **data matrix** (observations × features, e.g.
cells × genes) and a **prior-knowledge network** (which features belong to which
biological program), and it returns an **enrichment score per observation per
program** — all methods sharing one unified interface. Part of scverse; works
directly on `AnnData`, `pandas.DataFrame`, or raw matrices.

This skill is a **router**. Each topic below has a detailed reference file with exact
signatures and footguns. **Read the relevant reference before writing code** — do not
rely on memory of the API, because results are written to different places depending
on the input type and several methods have non-obvious defaults.

## Two cross-cutting concepts (read these first if unsure)

- **[The unified method call](references/calling-convention.md)** — every method is
called the same way: `dc.mt.<method>(data, net, tmin=5, ...)`. Covers the accepted
input types (AnnData / DataFrame / `[matrix, obs, var]` **list**), the shared
arguments (`tmin`, `raw`, `layer`, `empty`, `bsize`), and the big footguns: input
must be **normalized (e.g. log1p), not raw counts**, and `net` must be a
**long-format** `source`/`target`/`weight` table.

- **[Where the results go](references/io-and-outputs.md)** — the return type depends
on the input type. **AnnData → written in place** into `.obsm["score_<method>"]`
(and `.obsm["padj_<method>"]` if the method tests), returns `None`. **DataFrame /
list → returns an `(es, pv)` tuple** (`pv` is `None` for non-testing methods). Read
scores back out of an AnnData with `dc.pp.get_obsm`. **This is the #1 thing agents
get wrong.**

## Task → reference file

| If the task is… | Read |
|---|---|
| **starting from scratch** — set up an end-to-end run (load data → get a net → score → rank → plot) | [references/getting-started.md](references/getting-started.md) |
| how to **call a method**, what data/net formats are accepted, `tmin`/`raw`/`layer` semantics | [references/calling-convention.md](references/calling-convention.md) |
| **where scores land** and how to read them back (`.obsm` vs `(es, pv)` tuple, `get_obsm`) | [references/io-and-outputs.md](references/io-and-outputs.md) |
| **choosing and configuring a method** — ulm/mlm/ora/gsea/gsva/aucell/viper/zscore/waggr/mdt/udt + their kwargs | [references/methods.md](references/methods.md) |
| getting **prior knowledge** — TF regulons (CollecTRI, DoRothEA), pathways (PROGENy), gene sets (MSigDB hallmarks), OmniPath, `.gmt`, organism translation | [references/priors.md](references/priors.md) |
| **preprocessing** — pseudobulk from single cell, expression/sample filtering, layers, spatial kNN | [references/preprocessing.md](references/preprocessing.md) |
| running **many methods at once** and combining them into a **consensus** | [references/multi-method.md](references/multi-method.md) |
| **ranking** activity scores by group/ordering and **plotting** them (barplot, dotplot, volcano, network) or overlaying them on a **UMAP / spatial H&E** via scanpy | [references/ranking-and-plotting.md](references/ranking-and-plotting.md) |
| **benchmarking** methods/nets against a perturbation ground truth | [references/benchmarking.md](references/benchmarking.md) |
| using a **bundled example dataset** (`dc.ds.*`) — and whether its `.X` is raw counts or already normalized | [references/datasets.md](references/datasets.md) |

## Quick orientation (submodule map)

decoupler exposes everything through namespaced submodules — canonical import is
`import decoupler as dc`:

- `dc.mt` — **methods**: the enrichment/activity algorithms (`ulm`, `mlm`, `ora`,
`gsea`, `gsva`, `aucell`, `viper`, `zscore`, `waggr`, `mdt`, `udt`) plus
`dc.mt.decouple` (run several) and `dc.mt.consensus`. `dc.mt.show()` lists them.
- `dc.op` — **prior knowledge** (OmniPath): `collectri`, `dorothea`, `progeny`,
`hallmark`, `resource`, `translate`.
- `dc.pp` — **preprocessing**: `pseudobulk`, `filter_by_expr`, `filter_by_prop`,
`filter_samples`, `get_obsm`, `extract`, `swap_layer`, `knn`, `read_gmt`.
- `dc.ds` — **datasets**: `toy` (used in every example: `adata, net = dc.ds.toy()`),
`pbmc3k`, `covid5k`, and other real examples. **They differ in normalization state**
(some raw counts, some already log-normalized) — see
[datasets.md](references/datasets.md) before scoring one.
- `dc.tl` — **tools**: `rankby_group`, `rankby_obsm`, `rankby_order`.
- `dc.pl` — **plotting**: `barplot`, `dotplot`, `volcano`, `network`, `obsm`, etc.
- `dc.bm` — **benchmarking**: `benchmark`, `metric`, `pl`.

## Conventions used throughout

- **Data** = observations × features (cells/samples × genes), **normalized** (log1p),
not raw counts. **Net** = long-format DataFrame: `source` (program), `target`
(feature), optional signed `weight`.
- Output score matrices are observations × sources. AnnData results live in `.obsm`
under `score_<method>` / `padj_<method>`.
- `tmin` (default 5) silently drops sources with fewer than `tmin` targets present in
the data — lower it for small/toy nets (examples use `tmin=3`).
- `verbose=True` on any method prints the pruning/run log — use it to see how many
sources survived `tmin`.
- **Plotting**: use `dc.pl.*` for summary plots and scanpy `sc.pl.umap` / `sc.pl.spatial`
(on a `dc.pp.get_obsm` score AnnData) for embedding/tissue overlays. Never reimplement
a plot from matplotlib primitives — see
[ranking-and-plotting.md](references/ranking-and-plotting.md).
59 changes: 59 additions & 0 deletions src/decoupler/_skills/data/references/benchmarking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Benchmarking methods and nets

`dc.bm` evaluates how well a method + net recover a known ground truth — typically a
perturbation dataset where you know which program was up/down in each sample. Use it
to **choose a method** or **validate a net**, not for routine scoring.

## `dc.bm.benchmark`

```python
dc.bm.benchmark(
adata, # AnnData with a perturbation ground truth in .obs
net, # a net, or a dict of {name: net} to compare several
metrics=None, # e.g. ["auroc", "auprc"] (None = defaults)
groupby=None, # obs column(s) to stratify the evaluation
runby="expr", # evaluate per-observation ("expr") or per-source
sfilt=False, # filter sources to those with a ground-truth label
thr=0.1, emin=5,
kws_decouple=None, # kwargs forwarded to dc.mt.decouple (methods, args, tmin, ...)
verbose=False,
)
```

- Internally runs `dc.mt.decouple`, so pass method selection and per-method args via
`kws_decouple` (e.g. `kws_decouple={"methods": ["ulm", "mlm"], "tmin": 3}`).
- Pass a **dict of nets** to benchmark several priors head to head.
- The ground truth (which source is perturbed, and its sign) must be encoded in
`adata.obs` — see the decoupler benchmarking tutorial for the expected columns.

## Metrics and plots

- `dc.bm.metric` — the scoring functions: `auc`, `fscore`, `qrank`, `hmean`,
`dict_metric`.
- `dc.bm.pl` — plots of benchmark results: `auc`, `bar`, `fscore`, `qrank`, `summary`.

```python
res = dc.bm.benchmark(adata, {"collectri": net_a, "dorothea": net_b},
kws_decouple={"methods": ["ulm", "mlm"]})
dc.bm.pl.summary(res) # compare methods × nets
```

## When to use

- Deciding **which method** to trust for your data/organism.
- Comparing **candidate nets** (e.g. CollecTRI vs DoRothEA levels).
- Sanity-checking a **custom net** against a perturbation benchmark
(`dc.ds.knocktf` and similar provide ground-truth datasets).

## Footguns

- Benchmarking is only meaningful with a **real ground truth** — a toy net won't tell
you anything.
- It re-runs scoring internally; keep `kws_decouple` consistent with how you intend to
score for real, or the comparison won't transfer.

## Related references

[multi-method.md](multi-method.md) (`decouple`, which `benchmark` wraps),
[methods.md](methods.md) (candidate methods),
[priors.md](priors.md) (candidate nets).
91 changes: 91 additions & 0 deletions src/decoupler/_skills/data/references/calling-convention.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# The unified method call

Every enrichment method in `dc.mt` is a callable object with the **same signature**.
Learn it once and it applies to `ulm`, `mlm`, `ora`, `gsea`, `gsva`, `aucell`,
`viper`, `zscore`, `waggr`, `mdt`, `udt` (and `dc.mt.decouple`).

```python
import decoupler as dc

adata, net = dc.ds.toy() # AnnData (obs × var) + long-format net
dc.mt.ulm(adata, net, tmin=3) # writes adata.obsm["score_ulm"], ["padj_ulm"]
```

## Shared signature

```python
dc.mt.<method>(
data, # AnnData | pandas.DataFrame | [matrix, obs_names, var_names]
net, # long-format DataFrame: source, target, [weight]
tmin=5, # drop sources with < tmin targets present in the data
raw=False, # use adata.raw instead of adata.X
layer=None, # use adata.layers[layer] instead of adata.X (passed via kwargs)
empty=True, # drop all-zero observations/features before scoring
bsize=250_000, # batch size (rows) for sparse / backed matrices
verbose=False,
**kwargs, # method-specific args — see references/methods.md
)
```

`layer=` is accepted even though it is not in the explicit signature — it is forwarded
through `**kwargs` to the runner. Use `layer=` **or** `raw=`, not both.

## Accepted input types (`data`)

| Form | What it is | Returns |
|---|---|---|
| `AnnData` | scored on `.X` (or `.raw`/`layer`) | **`None`** — results written to `.obsm` in place |
| `pandas.DataFrame` | observations (rows) × features (cols) | `(es, pv)` tuple of DataFrames |
| `list` `[matrix, obs_names, var_names]` | numpy/sparse matrix + name arrays | `(es, pv)` tuple of DataFrames |

See [io-and-outputs.md](io-and-outputs.md) for the return details.

### Footgun — the matrix form must be a `list`, not a `tuple`

```python
# WRONG — a tuple raises "mat must be a list of [matrix, samples, features]"
es, pv = dc.mt.ulm((X, obs, var), net)

# RIGHT — wrap it in a list
es, pv = dc.mt.ulm([X, obs, var], net)
```

## The three footguns that silently give wrong results

1. **Input must be normalized, not raw counts.** decoupler expects
log-normalized expression — the same matrix you would use for differential
expression. **When normalizing, use `sc.pp.normalize_total(adata, target_sum=1e4)`**
followed by `sc.pp.log1p(adata)`. Passing raw counts runs without error but produces
meaningless scores. If your normalized data is in a layer, pass
`layer="lognorm"`; if in `.raw`, pass `raw=True`. **Check before normalizing:** if
`.X` is integer-valued with a large max it is raw counts; if it is small floats
(often with `adata.uns["log1p"]`) it is already normalized — do not normalize twice.
The bundled datasets differ; see [datasets.md](datasets.md) (e.g. `pbmc3k` is already
log-normalized, `covid5k` is raw counts).

2. **`net` must be long-format** with columns `source`, `target`, and (for weighted
methods) `weight` — one row per (program, feature) edge. A wide/matrix net or wrong
column names will fail or misbehave. See [priors.md](priors.md).

3. **`tmin` silently prunes.** Sources with fewer than `tmin` (default 5) targets
present in `data` are dropped *before* scoring — they simply won't appear in the
output. On small or toy nets this can drop everything; the examples use `tmin=3`.
Run with `verbose=True` to see `<method> - X sources with < tmin targets` and how
many survived.

## Listing and introspecting methods

```python
dc.mt.show() # DataFrame: name, desc, stype, weight, test, limits, reference
```

- `weight` — whether the method uses signed edge weights (`True` for ulm/mlm/viper/…).
- `test` — whether it produces p-values (`padj`). `False` for aucell/gsva/mdt/udt.
- `stype` / `limits` — value type and output range (e.g. aucell `(0, 1)`).

## Related references

[io-and-outputs.md](io-and-outputs.md) (return types, reading `.obsm` back),
[methods.md](methods.md) (per-method kwargs and how to choose one),
[priors.md](priors.md) (building/fetching `net`),
[preprocessing.md](preprocessing.md) (getting a normalized matrix / layers).
Loading
Loading