Householder characteristics into the HUI product, named for their census tables, plus family type - #143
Merged
npr99 merged 2 commits intoSep 16, 2026
Conversation
Housecleaning goals 2 and 3 from npr99#140, kept as a small PR per request. The housing unit inventory's householder columns now carry the name of the census table that produced them: agegroupH13, agegroupH14, sexH14 and familytypeH14 for 2020; agegroupH17, agegroupH18, sexH18 and familytypeH18 for 2010. The 2010-style fixed naming misled even at close range - a 2020 frame showing agegroupH18 reads as 2010 data, and settling that question took a run log and a Census count to the unit. Names now say what the data is. Internally nothing is renamed: acg_05c computes under the fixed names and renames once at the product boundary, and the linkage renames back once on ingestion (normalize_householder_names). The merge internals never see the new names, which is what keeps linkage results bit-identical across the change. The validation checks detect which naming a frame carries, so they work on either vintage and on frames produced before this change. familytype (goal 3, for social vulnerability work) follows the type-by-age table's own categories: 1 married-couple family, 2 other family male householder, 3 other family female householder, 4 nonfamily male, 5 nonfamily female, -999 not determined (vacant, group quarters, unreached). Derived from structure already on the frame, no new API pulls. Regression to follow: Grays Harbor 2020 end to end, requiring person-for-person identity against the pre-change output.
Second half of housecleaning goal 2 from npr99#140, with the review note about keeping Block{vintage}str. The HUI workflow now runs the householder step (age band, sex, family type, vintage-true names) between the base inventory and the final polish, and saves a linkage-facing product alongside the existing files: hui_linkage_{version}_{county}_{vintage}_rs{seed}.csv - polished, with Block{vintage}str kept and the householder columns on board. Additive by design: save_incore_version2 selects its own column list, so every existing IN-CORE file keeps its exact schema, and the pre-polish inventory files are untouched. acg_05c installs the 2020 DHC patches itself when constructed for 2020 (idempotent), closing challenge 1 from the npr99#140 notes for this path: callers no longer need to know the patches exist. The notebook's Intersect setup now prefers the linkage product and falls back to the pre-polish inventory for runs that predate it, and Step 1 skips the householder build when the columns are already on the frame - the cell reduces to a verification when fed the new product. Regression to follow with the caches cleared so every merge runs fresh: the recorded Grays Harbor figures and person-for-person identity must hold through the changed input path.
NatCatAnalystRandle
changed the title
pyncoda
approved these changes
Sep 16, 2026
pyncoda
left a comment
pyncoda
left a comment
There was a problem hiding this comment.
updated code looks good. Great job on commenting.
|
Code update looks good - ready for merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Housecleaning goals 2 and 3 from #140, two commits.
Commit 1 renames the householder columns to the census table that produced them (agegroupH13, agegroupH14, sexH14, familytypeH14 for 2020; agegroupH17, agegroupH18, sexH18, familytypeH18 for 2010) and adds the family type variable: 1 married couple family, 2 other family male householder, 3 other family female householder, 4 nonfamily male, 5 nonfamily female, -999 not determined. Internals are untouched: the module computes under fixed names and renames once at the product boundary, and the linkage renames back once on ingestion, so the merge machinery never sees the new names. Validation detects which naming a frame carries, so both vintages and older frames keep working.
Commit 2 builds the householder step into the HUI workflow itself and saves a linkage facing product, hui_linkage_{version}{county}{vintage}_rs{seed}.csv: polished, keeping Block{vintage}str per your review note, with the householder and family type columns on board. Additive by design: save_incore_version2 selects its own column list, so every IN-CORE file keeps its exact schema (verified: no new columns leak into it). acg_05c also installs the 2020 DHC patches itself when constructed for 2020, so callers no longer need to know the patches exist (challenge 1 from my notes). The notebook's Intersect setup now prefers the linkage product and its Step 1 reduces to a verification; both fall back gracefully for runs that predate the new file.
Verification, run with every product and merge cache cleared so nothing could reload: Grays Harbor 2020 end to end, all recorded figures reproduced (36,200 units, 75,636 records, 64,516 linked at 85.30 percent, PCT5 one person -10.86), the new product present with all five columns, and on Grays Harbor the family type counts 12,997 married couple households on exactly 29,869 occupied units (the H13 figure). Person for person against the pre change output: every household person identical (100.00 percent). 728 of the 2,909 group quarters residents swapped among facilities within their own block (726 of 728 within the same facility type, zero assigned or unassigned status changes, every total identical) because the freshly rerun group quarters merge consumed the polished product's different row order. Row order was never part of the reproducibility contract, the seed is, but it is recorded here so nobody has to rediscover it.