Skip to content

⚗️ Collect WebSocket resource events#4718

Open
bdibon wants to merge 8 commits into
mainfrom
boris.dibon/ws-5-collection
Open

⚗️ Collect WebSocket resource events#4718
bdibon wants to merge 8 commits into
mainfrom
boris.dibon/ws-5-collection

Conversation

@bdibon

@bdibon bdibon commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Motivation

WebSocket connections are a blind spot in RUM resource monitoring. This PR adds WebSocket tracking behind the track_web_sockets experimental feature flag, collecting per-connection metrics (message counts, sizes, timing, handshake duration, close codes) and reporting them as resource events with resource.type: "websocket".

Changes

packages/browser-core

  • webSocketObservable: Instruments the WebSocket constructor and send method via instrumentConstructor/instrumentMethod to emit lifecycle events: connecting, open, message-in, message-out, closed. Tracks payload sizes and clock timestamps for each event.
  • Exports the observable from the package index.
  • Adds ResourceType.WEBSOCKET = 'websocket' to the resource type enum.
  • Adds ExperimentalFeature.TRACK_WEB_SOCKETS = 'track_web_sockets' experimental flag.

packages/browser-rum-core

  • webSocketCollection: Consumes webSocketObservable and maintains an in-memory registry of open connections. Aggregates metrics (message counts/sizes, first-message offsets, longest inbound silence, buffered amount peaks) and emits WEBSOCKET_COMPLETED lifecycle events on close or session expiry.
  • resourceCollection: Subscribes to WEBSOCKET_COMPLETED and assembles RumResourceEvent objects with resource.type: "websocket" and a resource.websocket sub-object containing the full connection metrics.
  • lifeCycle: Adds the WEBSOCKET_COMPLETED event type and its payload type.
  • rawRumEvent.types: Adds WebSocketResourceProperties interface and wires it into RawRumResourceEvent.
  • startRum: Starts webSocketCollection when the TRACK_WEB_SOCKETS experimental feature is enabled.

scripts/dev-server

  • Adds Access-Control-Allow-Origin: * response header to support cross-origin WebSocket testing from the sandbox.

Test instructions

Run unit tests:

yarn test:unit --spec packages/browser-core/src/browser/webSocketObservable.spec.ts
yarn test:unit --spec packages/browser-rum-core/src/domain/webSocketCollection.spec.ts
yarn test:unit --spec packages/browser-rum-core/src/domain/resource/resourceCollection.spec.ts

Manual end-to-end test:

# Terminal 1: Start dev server
yarn dev-server start

# Terminal 2: Spin up a local WebSocket echo server (uses the ws package already in the repo)
yarn node -e "
  const WebSocket = require('ws');
  const wss = new WebSocket.Server({ port: 8765 });
  wss.on('connection', ws => { ws.on('message', msg => ws.send(msg)); });
  console.log('WS echo server on :8765');
"

Open http://localhost:8080 and replace sandbox/index.html with:

<script>
  DD_RUM.init({
    applicationId: '...',
    clientToken: '...',
    trackResources: true,
    enableExperimentalFeatures: ['track_web_sockets'],
    proxy: '/proxy',
  })
</script>
<button onclick="
  let ws = new WebSocket('ws://localhost:8765');
  ws.onopen = () => { ws.send('hello'); ws.send('world'); };
  ws.onmessage = e => console.log('in:', e.data);
  setTimeout(() => ws.close(1000, 'done'), 2000);
">Test WebSocket</button>

Click the button, wait ~3 seconds, then open a new tab to flush events:

yarn dev-server intake rum-resources | jq 'select(.resource.type == "websocket")'

Expected output — a resource event with:

{
  "resource": {
    "type": "websocket",
    "url": "ws://localhost:8765",
    "websocket": {
      "handshake_succeeded": true,
      "messages_in": { "count": 2, "size": 22 },
      "messages_out": { "count": 2, "size": 22 },
      "close_code": 1000,
      "was_clean": true,
      "tracking_end_reason": "close_event"
    }
  }
}

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated documentation and/or relevant AGENTS.md file

bdibon commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jun 3, 2026

Copy link
Copy Markdown

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

❄️ 1 New flaky test detected

webSocketCollection records inboundIdleDurationBeforeClose from last message-in to close from Firefox 78.0 (Windows 10)   View in Datadog (Fix with Cursor)
Expected 31 to be 30.
&lt;Jasmine&gt;
./packages/browser-rum-core/src/domain/webSocketCollection.spec.ts/&lt;/&lt;@webpack:///packages/browser-rum-core/src/domain/webSocketCollection.spec.ts:208:57 &lt;- /tmp/_karma_webpack_538320/commons.js:110844:61
&lt;Jasmine&gt;

New test introduced in this PR is flaky.

View in Flaky Test Management

ℹ️ Info

No other issues found (see more)

🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 78.89%
Overall Coverage: 76.85% (+0.05%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4251e33 | Docs | Datadog PR Page | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jun 3, 2026

Copy link
Copy Markdown

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum N/A 177.42 KiB N/A N/A N/A
Rum Profiler N/A 7.88 KiB N/A N/A N/A
Rum Recorder N/A 21.09 KiB N/A N/A N/A
Logs N/A 54.68 KiB N/A N/A N/A
Rum Slim N/A 134.89 KiB N/A N/A N/A
Worker N/A 22.96 KiB N/A N/A N/A

@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from f9f774b to f67e33e Compare June 4, 2026 09:16
@bdibon bdibon force-pushed the boris.dibon/ws-4-track-config branch from ec073a8 to 98e75c8 Compare June 4, 2026 09:16
@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from f67e33e to adaf9f3 Compare June 4, 2026 12:30
@bdibon bdibon force-pushed the boris.dibon/ws-4-track-config branch from 98e75c8 to 1afcc3a Compare June 4, 2026 12:30
@bdibon bdibon changed the base branch from boris.dibon/ws-4-track-config to graphite-base/4718 June 8, 2026 14:48
@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from adaf9f3 to cb90533 Compare June 10, 2026 11:39
@bdibon bdibon force-pushed the graphite-base/4718 branch from 1afcc3a to 9b239da Compare June 12, 2026 14:54
@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from cb90533 to 7fc9595 Compare June 12, 2026 14:54
@bdibon bdibon changed the base branch from graphite-base/4718 to boris.dibon/ws-4-track-config June 12, 2026 14:54
@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from 7fc9595 to 770842b Compare June 12, 2026 15:10
@bdibon bdibon changed the title ✨ Collect WebSocket resource events ⚗️ Collect WebSocket resource events Jun 12, 2026
@bdibon bdibon changed the base branch from boris.dibon/ws-4-track-config to main June 12, 2026 15:12
@bdibon bdibon marked this pull request as ready for review June 12, 2026 15:23
@bdibon bdibon requested a review from a team as a code owner June 12, 2026 15:23

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 770842b8f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

type: RumEventType.RESOURCE,
resource: {
id: generateUUID(),
type: ResourceType.WEBSOCKET,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add schema support before emitting websocket resources

The generated RUM event schema still only allows the existing resource types (see packages/browser-rum-core/src/rumEvent.types.ts:665-676), and this PR also skips websocket resources in format validation (packages/browser-rum-core/test/formatValidation.ts:25-27). When track_web_sockets is enabled, events assembled here have resource.type: "websocket", so they do not match the SDK/intake schema and can be rejected or hidden by downstream validation. Please land/regenerate schema support before emitting this resource type.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a prototype and the data collected is subject to change, downstream validation has been handled.

Comment thread packages/browser-rum-core/src/domain/resource/resourceCollection.ts Outdated
Comment thread packages/browser-core/src/browser/webSocketObservable.ts Outdated
Comment thread packages/browser-rum-core/src/domain/resource/resourceCollection.ts
Comment thread packages/browser-rum-core/src/domain/resource/resourceCollection.ts Outdated
'send',
({ target: instance, parameters: [data], onPostCall }) => {
const size = computePayloadSize(data)
const bufferedAmountPreSend = instance.bufferedAmount

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Sample bufferedAmount after send

For a single large send, bufferedAmount is often 0 before the native send() call and only increases once that call queues the payload, so sampling only bufferedAmountPreSend makes buffered_amount_max miss the last/only write and under-report the real queue peak. Capture the value in the post-call path (or include both before and after) before updating the max.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sampling bufferedAmountPreSend allows us to spot back pressure issues, if we instrument the post-call path, we'll just see the bufferedAmount grow by the size of the message that has just been sent.


if (isExperimentalFeatureEnabled(ExperimentalFeature.TRACK_WEB_SOCKETS)) {
const webSocketCollection = startWebSocketCollection(lifeCycle, viewHistory, vitalCollection.addDurationVital)
cleanupTasks.push(webSocketCollection.stop)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Flush open websockets before stopping resources

When RUM is stopped while a socket is still open, cleanup tasks run in registration order: resourceCollection.stop() is registered earlier and stops its task queue before this later webSocketCollection.stop() emits WEBSOCKET_COMPLETED for open connections. That means the intended session_end resource is queued after the resource pipeline is already stopped and can be lost; flush websocket connections before stopping the resource collection/batch, or avoid emitting them during stop.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice, the stop() cleanup isn't ran.

Also, the only way resourceCollection.stop() prevents a websocket event from being collected is when there's a corresponding task in taskQueue which is an unlikely window.

@bdibon bdibon changed the base branch from main to graphite-base/4718 June 15, 2026 11:27
@bdibon bdibon force-pushed the boris.dibon/ws-5-collection branch from 770842b to 4251e33 Compare June 15, 2026 11:27
@bdibon bdibon changed the base branch from graphite-base/4718 to boris.dibon/ws-4-track-config June 15, 2026 11:27
@bdibon bdibon changed the base branch from boris.dibon/ws-4-track-config to main June 15, 2026 14:33
@bdibon

bdibon commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

/to-staging

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jun 16, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-06-16 08:11:45 UTC ℹ️ Start processing command /to-staging


2026-06-16 08:11:51 UTC ℹ️ Branch Integration: starting soon, merge expected in approximately 0s (p90)

Commit 4251e33ae1 will soon be integrated into staging-25.


2026-06-16 08:30:42 UTC ℹ️ Branch Integration: this commit was successfully integrated

Commit 4251e33ae1 has been merged into staging-25 in merge commit 07d8c51b44.

If you need to revert this integration, you can use the following command: /code revert-integration -b staging-25

gh-worker-dd-mergequeue-cf854d Bot added a commit that referenced this pull request Jun 16, 2026
Integrated commit sha: 4251e33

Co-authored-by: bdibon <boris.dibon@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant