Commit 605f833
committed
fix(populate): make Dependencies picklable for multiprocessing
Parallel populate pickles the table (and its connection's Dependencies) to
worker processes, which then reload dependencies and reconnect. Dependencies
held an `itertools.count` (_node_alias_count) that is not picklable. This was
latent on Linux through Python 3.13 because multiprocessing defaulted to
`fork` (which inherits rather than pickles); Python 3.14 defaults to
`forkserver`, which pickles the initializer args and surfaced the failure
(`TypeError: cannot pickle 'itertools.count' object` in test_multi_processing).
Add __getstate__/__setstate__ to drop the counter on pickle and rebuild it on
load — workers reload dependencies anyway, so the value need not be carried.1 parent 0a0aeb2 commit 605f833
1 file changed
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
143 | 156 | | |
144 | 157 | | |
145 | 158 | | |
| |||
0 commit comments