Skip to content

return-dataframe-with-succeeded-rows-even-during-TTDAPIError - #62

Open
adithyasamavedhi-ttd wants to merge 1 commit into
mainfrom
return-already-processed-rows-on-TTDAPIError
Open

return-dataframe-with-succeeded-rows-even-during-TTDAPIError#62
adithyasamavedhi-ttd wants to merge 1 commit into
mainfrom
return-already-processed-rows-on-TTDAPIError

Conversation

@adithyasamavedhi-ttd

@adithyasamavedhi-ttd adithyasamavedhi-ttd commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

  • Catches and proccesses Exceptions in both ush_data and batch_process so that neither riase / propagate exceptions
  • We catch exceptions and mark the batxch failed. If its a transient error we continue processing remaining entries. If its a unrecoverable error we abort the remaining entries and set a ABORTED error code for all these rows.
  • ABORTED means these were never sent to TTD and are safe to retry
  • The logic is now that only 401/403 result in fail all batchs, rest should just fail current batch

Tests:

  1. push_data
  • Before this change, push_data used to raise exception and we lost all the data processed before point of exception
Screenshot 2026-08-18 at 12 50 33 AM
  • After this change, we respond with a DF which has processed data
Screenshot 2026-08-18 at 1 10 08 AM
  1. batch_process
  • Before this change, batch_process used to raise exception and we lost all the data processed before point of exception
Screenshot 2026-08-18 at 1 04 22 AM
  • After this change, we respond with a DF which has processed data
Screenshot 2026-08-18 at 1 11 08 AM

@adithyasamavedhi-ttd
adithyasamavedhi-ttd requested a review from a team August 17, 2026 17:07
@adithyasamavedhi-ttd
adithyasamavedhi-ttd force-pushed the return-already-processed-rows-on-TTDAPIError branch 9 times, most recently from 8e22d94 to 320d6f7 Compare August 18, 2026 21:07
@adithyasamavedhi-ttd
adithyasamavedhi-ttd force-pushed the return-already-processed-rows-on-TTDAPIError branch from 320d6f7 to af8ab2f Compare August 18, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants