feat(profiling): Add pipeline workflow to perfetto profiling by markushi · Pull Request #5932 · getsentry/relay

markushi · 2026-05-05T06:38:39Z

As a follow up to:
https://github.com/getsentry/relay/pull/5659/changes/BASE..db555e68ad45debd66f46d28e84aa6952b7498b7#r3167376271, introduces a pipeline workflow instead of doing everything in one place.

Dav1dde

Thanks this addresses my biggest concern of the other PR.

I think there may still be some stricter typing we can do, and maybe de-duplicate some of the filtering logic, but this goes into the territory of maybe not worth it at this time. I tried some stuff locally and realized it'll need more changes quickly.

So ended up just leaving some nits.

For order of PRs, happy to merge them separately into master, as each one of the PRs is functional standalone.

Should also give other reviewers some time to take a look!

…add missing tests As ProfileChunkOutput::Expanded is only used in processing mode, there's no need to carry around the headers / platform fields. - added platform validation for perfetto profiles - added test for existing JSON-only profiles, ensuring no change in behavior - refactored validation / quantities handling to be more re-usable across profile formats

markushi · 2026-05-07T15:07:56Z

@Dav1dde @jjbayer thanks for the detailed PR feedback (and bearing with me 😅 ). I've applied all suggestions, so this is ready for review again.

Dav1dde · 2026-05-08T08:34:39Z


+#[derive(Debug)]
+#[cfg_attr(all(not(feature = "processing"), not(test)), expect(dead_code))]
+pub struct RawProfile {


Really small nit, would move these structs under SerializedProfileChunks, structure mirrored by other processors and kind-of a logical order: Processor -> Serialized -> Expanded.

Dav1dde · 2026-05-08T08:37:49Z

+#[expect(
+    clippy::large_enum_variant,
+    reason = "variants are sized by Managed<T> which wraps different pipeline stages"
+)]


Urgh, the old problem again where EnvelopeHeaders is 700+ bytes.

Something we need to figure out in general.

Dav1dde · 2026-05-08T08:41:49Z

+    item.platform()
+        .ok_or_else(|| err(relay_profiling::ProfileError::PlatformNotSupported.into()))?;


That's neat but I think I'd prefer:

Suggested change

item.platform()

.ok_or_else(|| err(relay_profiling::ProfileError::PlatformNotSupported.into()))?;

if item.platform.is_none() {

return Err(..);

}

Dav1dde · 2026-05-08T08:42:53Z

-        _ => return Err(relay_profiling::ProfileError::PlatformNotSupported.into()),
-    }
+    payload: Bytes,
+) -> Result<ExpandedProfileChunk, (Error, Quantities)> {


The quantity dance isn't necessary here, the function should return a normal error, then the caller can deal with the outcomes and quantities (see comment below how to).

Dav1dde · 2026-05-08T08:47:54Z

+    chunks: Managed<SerializedProfileChunks>,
+    ctx: Context<'_>,
+) -> Managed<ExpandedProfileChunks> {
+    chunks.map(|serialized, records| {


We have two options here and I think either are fine:

We consider the entire envelope invalid if it contains a single broken profile chunk. If that's the case we want to use try_map here and just ? the errors.

We only want to skip broken chunks but process other chunks in the same envelope, in this case we need to keep map here but reject items (with their outcomes) like:

match result { Ok(chunk) => expanded.push(chunk), Err(err) => drop(records.reject_err(err, &item)), }

Latter should not require that dance with quantities and track_outcomes.

…kushi/perfetto-profiling-support-pipeline

sentry · 2026-05-15T19:18:57Z

+impl Counted for ExpandedProfileChunks {
+    fn quantities(&self) -> Quantities {
+        let mut q = Quantities::new();
+        for chunk in &self.chunks {
+            q.extend(chunk.quantities());
+        }
+        q
+    }


Bug: The quantities() implementation for ExpandedProfileChunks uses extend() instead of aggregating counts, leading to duplicate entries for same-category chunks and incorrect quota enforcement.
_{Severity: HIGH}

Suggested Fix

Modify the quantities() method in ExpandedProfileChunks to aggregate counts for the same data category. Use a BTreeMap to sum quantities by category, similar to the implementation of Counted for [T]. This ensures each data category appears only once with its total count, allowing for correct quota consumption.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: relay-server/src/processing/profile_chunks/mod.rs#L128-L135 Potential issue: The `quantities()` method for `ExpandedProfileChunks` builds a list of data categories and their counts. It uses `extend()` to combine quantities from multiple profile chunks. If an envelope contains multiple chunks of the same type, this results in duplicate entries in the final list, for example `[(DataCategory::ProfileChunkUi, 1), (DataCategory::ProfileChunkUi, 1)]`. The quota enforcement mechanism iterates over these entries separately, which can lead to incorrect quota application. This bug is triggered when processing envelopes with multiple profile chunks of the same type, which is a realistic scenario.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1d8c95c. Configure here.}

cursor · 2026-05-15T19:22:43Z

-            if item
-                .profile_type()
-                .is_some_and(|pt| pt != pc.profile_type())
-            {


Error path loses outcome tracking for platformless chunks

Medium Severity

When a JSON profile chunk without a platform header fails during processing (e.g., filtering or expansion), track_quantities is called with &item.quantities() which returns an empty smallvec![] for ProfileChunk items without a profile_type. Then records.reject_err(err, &item) also uses the item's empty quantities. This means the rejection outcome is recorded with no data category. The old code called item.set_platform(...) and records.modify_by(...) before filtering/expansion could fail, so rejections always carried the correct data category. This regression silently drops outcome tracking for platformless chunks that error after parsing.

Additional Locations (1)

relay-server/src/processing/profile_chunks/process.rs#L133-L149

^{Reviewed by Cursor Bugbot for commit 1d8c95c. Configure here.}

Introduce more typing, apply pipeline pattern

069ad03

Dav1dde reviewed May 5, 2026

View reviewed changes

Address PR feedback

4108a6e

markushi marked this pull request as ready for review May 5, 2026 09:18

markushi requested a review from a team as a code owner May 5, 2026 09:18

markushi mentioned this pull request May 5, 2026

feat(profiling): Add Perfetto trace format support #5659

Open

cursor Bot reviewed May 5, 2026

View reviewed changes

Comment thread relay-server/src/processing/profile_chunks/mod.rs

Comment thread relay-server/src/processing/profile_chunks/mod.rs Outdated

Dav1dde added the skip-changelog label May 5, 2026

jjbayer reviewed May 5, 2026

View reviewed changes

markushi added 5 commits May 6, 2026 09:44

Flatten ProfileChunkKafkaMessage

67cd159

Ensure platform is correctly populated

8de85af

Validate platform for perfetto profiles

e43f1a3

Inline expand_item

51b5a83

cursor Bot reviewed May 7, 2026

View reviewed changes

Comment thread relay-server/src/processing/profile_chunks/process.rs

Extend tests, apply style feedback

2858a57

sentry Bot reviewed May 7, 2026

View reviewed changes

Comment thread relay-server/src/processing/profile_chunks/process.rs

cursor Bot reviewed May 7, 2026

View reviewed changes

Comment thread relay-server/src/services/store.rs

Dav1dde reviewed May 8, 2026

View reviewed changes

markushi added 3 commits May 13, 2026 17:37

move struct definition

5b19c27

Simplify expand error handling, move quantity tracking to caller

18e340e

Merge branch 'feat/markushi/perfetto-profiling-support' into feat/mar…

1d8c95c

…kushi/perfetto-profiling-support-pipeline

sentry Bot reviewed May 15, 2026

View reviewed changes

cursor Bot reviewed May 15, 2026

View reviewed changes

		item.platform()
		.ok_or_else(\|\| err(relay_profiling::ProfileError::PlatformNotSupported.into()))?;

Conversation

markushi commented May 5, 2026

Uh oh!

Dav1dde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markushi commented May 7, 2026

Uh oh!

Dav1dde May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Dav1dde May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Dav1dde May 8, 2026

Choose a reason for hiding this comment

Uh oh!

Dav1dde May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dav1dde May 8, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot May 15, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 15, 2026

Choose a reason for hiding this comment

Error path loses outcome tracking for platformless chunks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Dav1dde May 8, 2026 •

edited

Loading