Add capture_groups to extract.regex to parse a column into its parts

[eastagiletracker]

This PR proposes adding a capture_groups parameter to extract.regex, so a single column can be parsed into one output column per capture group (Fixes #771). We include this PR work along with a full history of your repo at https://eastagiletracker.com/projects/170. You can sign in with your GitHub ID to claim ownership of the project.

extract.regex returns match.group(0) for every match, so the capture groups in a pattern are never reachable as separate results. Asking for several output columns from one input column therefore does not split a value into its parts — the extra column names are silently dropped and only the first one is written, holding the whole match. On the current main (12f07bb), with the sample values from #771:

import pandas as pd, wrangles

df = pd.DataFrame({'Part': ['LRB81216', 'LRBZ606832', 'LRT10011030']})
print(wrangles.recipe.run(r"""
wrangles:
  - extract.regex:
      input: Part
      find: ([A-Z]+)(\d+)(\d{2})
      output:
        - Model
        - BoreOD
        - Width
""", dataframe=df))

          Part        Model
0     LRB81216     LRB81216
1   LRBZ606832   LRBZ606832
2  LRT10011030  LRT10011030

BoreOD and Width are never created. That is what forces the workaround in the issue: join the groups with a delimiter via output_pattern, pull the first element back out, then split.text on the delimiter.

This change adds capture_groups to extract.regex. When set, each match contributes its capture groups to the results instead of the whole match, and everything downstream — output_format: columns / list / concatenate, first_element, multiple input columns, the where clause — keeps working as it does today. Adding capture_groups: true to the recipe above gives Model/BoreOD/Width of LRB/812/16, and the same run with output_format: list gives ['LRB', '812', '16'].

It is backward compatible by construction: the parameter defaults to false and the existing code path is untouched, so no recipe changes behavior unless it opts in. Three edges are pinned down by the design rather than left to chance — a pattern with no capture groups falls back to the whole match so results are never unexpectedly empty, a group that did not participate in a match yields an empty string (matching how the rest of the module reports a non-result), and capture_groups together with output_pattern raises Extract must use either capture_groups or output_pattern, not both. rather than silently letting one win. The docstring schema entry is included, so the generated recipe schema picks the parameter up — confirmed by running schema/generate_recipe_schema.py and reading capture_groups back out of extract.regex's properties.

Verification: 12 tests were added to TestExtractRegex in tests/recipes/wrangles/test_extract.py covering the three output formats, multiple matches, multiple input columns, non-string input values, first_element, the no-groups fallback, the non-participating group, the no-match case, the mutually-exclusive error, and a control asserting the default output is still the whole match. Reverting only the new branch in _matches (keeping the parameter so the recipes still load) turns 8 of the 12 red, and restoring it turns them green. pytest was run on the clean tree first and again on the changed tree, and the set of failing tests is byte-for-byte identical (the failures are the tests needing the service credentials from the repository secrets, which are unavailable here); pytest tests/recipes/wrangles/test_extract.py::TestExtractRegex goes from 16 passed to 28 passed.

How this was managed

Issue #771 is tracked on the board as this story, moved to finished as the work landed. The board was imported from this repository's own issues and pull requests — 1079 stories and 25 labels — and used to manage this change.

If you'd rather not receive contributions like this, reply no-more-prs on this pull request and we won't open any further ones on your repositories.

---

Lawrence W. Sinclair
CEO / East Agile
linkedin.com/in/lwsinclair/
eastagile.com