Skip to content

Copy unresolved failed messages and their bodies to SQL - #6005

Open
warwickschroeder wants to merge 2 commits into
warwick/migration-engine-1from
warwick/migration-engine-4
Open

warwickschroeder wants to merge 2 commits into
warwick/migration-engine-1from
warwick/migration-engine-4

Conversation

@warwickschroeder

Copy link
Copy Markdown
Contributor

What this adds

The migration now copies unresolved and retry-issued failed messages, with their failure groups and bodies, from RavenDB to SQL Server or PostgreSQL while ServiceControl is closed. It does not copy archived or resolved messages yet, and a Release build still refuses to copy anything.

Files Lines
Code 31 +1,344 −198
Tests 31 +3,819 −34
Docs 3 +25 −17
  • Reader: FailedMessagesReader reads only unresolved messages through a RavenDB index, and reads the whole collection if the index can't be trusted.
  • Writer: FailedMessagesWriter writes one row per message, from its newest attempt, plus its failure groups.
  • Bodies: MigrationBodyLoader reads them from RavenDB, and they are stored inline or externally by the same rule ingestion uses.
  • New skip reasons: a bad id, no attempts, a value too long, a malformed value, a missing body.
  • SQL Server: the insert is faster.
  • Stalls: a stalled category no longer hangs startup.
  • Debug switch: SERVICECONTROL_MIGRATION_ALLOW_UNRELEASED lets a Debug build copy for real for testing. This will be removed once the migration engine is ready for release.
  • Docs: docs/migration/ is updated to match.

Tests

  • ServiceControl.UnitTests/Migration: body loading, stalls, refusal wording and the Debug switch.
  • ServiceControl.Persistence.Tests/EFCore/Migration: mapping, skips, groups and bodies.
  • ServiceControl.Persistence.Tests.RavenDB/DataMigration: reading through the index and the fallbacks, resuming, and body attachments.
  • ServiceControl.Persistence.Tests.SqlServer and .PostgreSql: value length checks and the SQL Server insert.
  • ServiceControl.Migration.AcceptanceTests: an end-to-end copy of failed messages and their bodies.

…andling for failure details without timestamps, and optimize SQL Server and PostgreSQL dialects for better performance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant