Add Inventory Flow tab with quantity tracking and date filtering#5313
Add Inventory Flow tab with quantity tracking and date filtering#5313AlexeyKasianenko wants to merge 14 commits into
Conversation
|
notes for my functional testing (which won't be until at least tomorrow): beyond basic checks, check what happens if we try to go back before the first snapshot. Are we already constraining this / should we be and how are we communicating that otherwise. (this may require using prod data to test well) (Note: thinking about it - if we base this on eventables (which is probably the best bet ) we should constrain it to start no earlier than the first snapshot) |
|
@AlexeyKasianenko Thank you for this! I'm going to add things I notice here on-the-go while I test. I'm not looking at the code, just trying out the functionality. |
|
1/ First thing is that I'm afraid the performance is slow. I'm checking it with the production data for one of active bank. Using the production branch, the storage location view comes up in 4 seconds, on your current branch it takes 89 seconds. |
|
2/ You're showing items that aren't on the inventory in / out tabs (again, this is testing with production data). I'm about 90% sure that they don't show inactive items, if so you should do the same. |
|
3/ If I update the date range and click "filter", it's going back to the first tab. This is potentially confusion/annoying for our users. Let's have it stay on the same tab. |
|
4/ The current "all time" is not working properly (with all time, in plus out should equal inventory on the first page) with production data. However, I want to check with the senior team about whether we should just constrain the date range-- i suspect that's going to make sense. |
|
5/ When I did a "this year" in the date range, and filtered (with production data), when I returned to the page there was only one item. I know there are more. I'll try to recreate that with the seed data. (Edit: no immediate luck -- I'll need to poke further on this one) |
cielf
left a comment
cielf
left a comment
There was a problem hiding this comment.
Hey @AlexeyKasianenko -- I'm at the end of working on this for today. Could you look at points 1, 2, and 3 above, please. I'll confirm number 4 with the senior team on Sunday, and number 5 is going to take some digging to figure out -- I'll probably wait and try to recreate it with production data once you've addressed 1, 2, and 3.
awwaiid
left a comment
awwaiid
left a comment
There was a problem hiding this comment.
Hi! This was a good first run at this, but unfortunately we need a different approach. The idea here is to base this off of Events instead of Items. Events form our improved transactional log, and we'd be able to get the per-storage-location Events to accurately capture the quantity ins and outs for an item over a period of time.
You MIGHT be able to update the queries you built to be Event centric, still joining off of Items to group them. Then you pass in the time range filter and limit to increases or decreases in the quantity depending on which column you're populating.
That also means, as an aside, that we want to hard-wire in a start-date for the filter to 2024-09-01, by which time every organization was using Events to track inventory changes.
Let me know if this makes sense and if you're up for it!
|
@awwaiid I would counter that we want to hard-wire that the start date can be no earlier than 2024-09-01. I, for one, don't want to have to work more than a full year of data if I'm checking into something that happened 2 weeks ago. |
8e2ab73 to
0da6387
Compare
0da6387 to
d5c4aed
Compare
|
Hmmm. 
I then went to the Inventory Flow tab, and selected the date range "Today". |
There was a problem hiding this comment.
Hey @AlexeyKasianenko -- Thank you -- I'm going to hold off on any further functional testing until the above is addressed, as I expect its fix will impact anything else I would try.
I looked very briefly at the code, and didn't see any reference to Events. Did you make that change as well?
c761f96 to
5233723
Compare
|
Hey @cielf. Fixed that |
…curate item creation timestamps
cielf
left a comment
cielf
left a comment
There was a problem hiding this comment.
This is working better. However, I was able to enter dates from before the snapshot, and I think, based on a very quick look at the code, that it is still Items-based rather than Events-based. Forwarding to @awwaiid for further comment.
…d accuracy and clarity
|
Hey, @cielf , we discussed with stakeholders on Wednesday and they said that it makes sense to include adjustments, so that all totals are represented |
|
@AlexeyKasianenko ok ONE more change! Everything looks good, but we forgot to include the |
…e related specs for accurate quantity calculations
|
@awwaiid added Audits |
|
Automatically unassigned after 7 days of inactivity. |
|
@awwaiid check this one out. |
Validating against a production snapshot showed the flow tab's numbers diverging from inventory by millions of units. Three causes: 1. Editing a donation/purchase/distribution/transfer/adjustment republishes the full event for the same eventable, and inventory replay diffs against the previous version - but the query counted every version at full value. Dedupe to the latest version per eventable (destroy events publish zeroed items, so destroyed records net to nothing and drop out). 2. Kit allocation/deallocation events were not counted at all. On the heaviest production location that is 1.6M items consumed into kits that the tab silently omitted. They now flow like any other event (each row applies in full, matching InventoryAggregate). 3. Audits record the absolute counted quantity, not a delta, so the tab showed e.g. 'audited at 5000' as 5000 in. Audit contributions are now computed as deltas via a single replay pass through the event log. Also collapse the per-type to/from conditions (which had and/or precedence bugs) into generic to/from matching over an explicit list of flow event types, filter by event_time rather than created_at to match replay semantics, drop all-zero rows, and move totals to Ruby. With these fixes, snapshot + in - out reconciles with current inventory to within a few hundred units over multi-year windows moving 7M+ units on production data (residuals trace to UpdateExistingEvent group handling quirks), and the page renders in under a second on the largest production org. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNy2e1J81K8xbvXraCeN8e
The flow tab now shows, per item: Quantity at Start, Quantity In, Quantity Out, Quantity Adjustment/Audit, Quantity Change, and Quantity at End, so each row self-validates (start + in - out + adjustment = end) and users can see stock levels bracketing the flows. - Start/end quantities come from raw InventoryAggregate state at the window boundaries (View::Inventory prunes inactive items, so the raw aggregate is used instead) - Adjustments move out of in/out into the combined adjustment/audit column, netted with their signed quantities; audit deltas land there too. The audit replay pass now doubles as the window-end state computation, so the whole page needs only two replays - The effective window start is clamped to just after the organization's most recent usable snapshot: inventory state cannot be derived from events before it, and unclamped ranges silently double-counted pre-snapshot events against the snapshot baseline - Rows appear when they have flow activity or a start/end difference Validated against a production snapshot: row identity holds for all but 1-7 rows per organization (rare event-correction corner cases in aggregate diff ordering, off by tens of units against multi-million unit flows), totals reconcile start to end, and the page renders in under a second on the largest organizations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNy2e1J81K8xbvXraCeN8e
5 participants
Resolves #5249
Description
Quantity in has donations, purchases, adjustments that increase the inventory.
Quantity out has distributions and adjustments that decrease the inventory
Kit allocations result in quantity in for the kit, and quantity out for the components
See the current Inventory In and Inventory Out tabs for details
Change is Quantity In minus Quantity Out. Negatives are presented in red.
Include anything else we should know about. -->
Type of change
How Has This Been Tested?
go to /storage_locations page
Screenshots