From b506f0cee715bdfc70ae2ddb6436b590061fd859 Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Thu, 7 May 2026 12:18:01 +0200 Subject: [PATCH 1/7] feat(webhooks): add verify_and_decode_webhook for compressed payloads (CHA-3071) Adds two new helpers on `StreamChat::Client` so customers can ingest gzip-compressed webhook payloads (and base64-wrapped SQS / SNS firehose envelopes) without wiring `Zlib`, `Base64`, and `OpenSSL::HMAC` together themselves: * `decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil)` - the primitive decoder, no signature check * `verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil)` - decodes and timing-safely verifies the X-Signature HMAC against the uncompressed JSON, raising `StreamChat::WebhookSignatureError` on any failure (decode, decompress, or signature mismatch). `nil` / empty for either encoding is a no-op, so the same handler keeps working whether or not compression is enabled at the dashboard. The existing `verify_webhook(body, x_signature)` boolean helper is left untouched for backward compatibility. Helper logic lives in `lib/stream-chat/webhook.rb` so the decode primitives can be exercised independently. New tests in `spec/webhook_compression_spec.rb` cover plain / gzip / base64 / base64 + gzip round-trips, case-insensitive encoding values, every documented unsupported encoding, malformed gzip / base64 input, and signature mismatch scenarios (including signatures wrongly computed over the compressed or wrapped bytes). Docs: new "Compressed webhook bodies" section in the webhooks overview with a Rails example and an SQS / SNS subsection that passes `payload_encoding: 'base64'`. Co-authored-by: Cursor --- .../webhooks_overview/webhooks_overview.md | 89 ++++++++++ lib/stream-chat/client.rb | 63 +++++++ lib/stream-chat/errors.rb | 30 ++++ lib/stream-chat/webhook.rb | 102 +++++++++++ spec/webhook_compression_spec.rb | 167 ++++++++++++++++++ 5 files changed, 451 insertions(+) create mode 100644 lib/stream-chat/webhook.rb create mode 100644 spec/webhook_compression_spec.rb diff --git a/docs/webhooks/webhooks_overview/webhooks_overview.md b/docs/webhooks/webhooks_overview/webhooks_overview.md index ab5f0c8..7a2793d 100644 --- a/docs/webhooks/webhooks_overview/webhooks_overview.md +++ b/docs/webhooks/webhooks_overview/webhooks_overview.md @@ -96,6 +96,95 @@ All webhook requests contain these headers: | X-Api-Key | Your application’s API key. Should be used to validate request signature | a1b23cdefgh4 | | X-Signature | HMAC signature of the request body. See Signature section | ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb | +## Compressed webhook bodies + +GZIP compression can be enabled for hooks payloads from the Dashboard. Enabling compression reduces the payload size significantly (often 70–90% smaller) reducing your bandwidth usage on Stream. The computation overhead introduced by the decompression step is usually negligible and offset by the much smaller payload. + +When payload compression is enabled, webhook HTTP requests will include the `Content-Encoding: gzip` header and the request body will be compressed with GZIP. Some HTTP servers and middleware (Rails, Django, Laravel, Spring Boot, ASP.NET) handle this transparently and strip the header before your handler runs — in that case the body you see is already raw JSON. + +Before enabling compression, make sure that: + +* Your backend integration is using a recent version of our official SDKs with compression support +* If you don't use an official SDK, make sure that your code supports receiving compressed payloads +* The payload signature check is done on the **uncompressed** payload + +### Decoding a compressed webhook in Ruby (Rails) + +The Ruby SDK exposes two helpers on `StreamChat::Client` so you do not have to wire `Zlib` and `OpenSSL::HMAC` together yourself: + +* `decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil)` — primitive that just decodes the body. No signature check. +* `verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil)` — decodes **and** verifies the `X-Signature` HMAC against the uncompressed JSON. Raises `StreamChat::WebhookSignatureError` if anything is wrong. + +Both methods return the raw JSON bytes as a binary `String`; you can `.force_encoding('UTF-8')` it or pass it straight to `JSON.parse`. Passing `nil` (or an empty string) for either encoding is a no-op, so the same handler works whether or not compression is enabled. + +```ruby +require 'stream-chat' +require 'json' + +STREAM_CLIENT = StreamChat::Client.new('STREAM_KEY', 'STREAM_SECRET') + +class WebhooksController < ApplicationController + skip_before_action :verify_authenticity_token + + def stream + body = request.raw_post # binary safe; do NOT use params + + json_bytes = STREAM_CLIENT.verify_and_decode_webhook( + body, + request.headers['X-Signature'], + request.headers['Content-Encoding'] # 'gzip' when compressed, nil otherwise + ) + + event = JSON.parse(json_bytes) + Rails.logger.info("Stream webhook: #{event['type']}") + + head :ok + rescue StreamChat::WebhookSignatureError => e + Rails.logger.warn("Rejected Stream webhook: #{e.message}") + head :unauthorized + end +end +``` + +> [!NOTE] +> If you sit behind middleware that already inflates `Content-Encoding: gzip` requests for you, pass `nil` for `content_encoding` (or just don't forward the header) — `request.raw_post` will already be raw JSON. + +### Decoding a compressed SQS / SNS firehose message + +SQS and SNS message bodies must be valid UTF-8, so when GZIP compression is enabled the gzipped bytes are additionally **base64-wrapped** before being placed on the queue. Pass `payload_encoding: 'base64'` so the SDK unwraps the queue envelope before decompressing: + +```ruby +require 'aws-sdk-sqs' +require 'json' +require 'stream-chat' + +client = StreamChat::Client.new('STREAM_KEY', 'STREAM_SECRET') +sqs = Aws::SQS::Client.new + +resp = sqs.receive_message(queue_url: ENV.fetch('STREAM_SQS_URL'), max_number_of_messages: 10) + +resp.messages.each do |msg| + attrs = msg.message_attributes || {} + signature = attrs.dig('X-Signature', :string_value) + content_encoding = attrs.dig('Content-Encoding', :string_value) # 'gzip' when compressed + payload_encoding = attrs.dig('Payload-Encoding', :string_value) || 'base64' + + json_bytes = client.verify_and_decode_webhook( + msg.body, + signature, + content_encoding, + payload_encoding + ) + + event = JSON.parse(json_bytes) + # ...handle the event... + + sqs.delete_message(queue_url: ENV.fetch('STREAM_SQS_URL'), receipt_handle: msg.receipt_handle) +end +``` + +The exact attribute names that carry the signature and encoding metadata may vary — refer to the SQS / SNS pages in this section for the up-to-date list. The decoding rules themselves do not change: signature is always computed over the **uncompressed** JSON. + ## Webhook types In addition to the above there are 3 special webhooks. diff --git a/lib/stream-chat/client.rb b/lib/stream-chat/client.rb index e5503c4..119f9fa 100644 --- a/lib/stream-chat/client.rb +++ b/lib/stream-chat/client.rb @@ -1,7 +1,11 @@ # typed: strict # frozen_string_literal: true +require 'base64' require 'open-uri' +require 'openssl' +require 'stringio' +require 'zlib' require 'faraday' require 'faraday/multipart' require 'faraday/net_http_persistent' @@ -18,6 +22,7 @@ require 'stream-chat/types' require 'stream-chat/moderation' require 'stream-chat/channel_batch_updater' +require 'stream-chat/webhook' module StreamChat DEFAULT_BLOCKLIST = 'profanity' @@ -705,6 +710,64 @@ def verify_webhook(request_body, x_signature) signature == x_signature end + # Decodes a webhook body that may have been base64-wrapped (SQS / SNS) and + # / or gzip-compressed (Content-Encoding: gzip), without verifying the + # signature. Returns the raw JSON bytes as a binary `String`; callers can + # `.force_encoding('UTF-8')` or pass the result to `JSON.parse` directly. + # + # @param body [String, Array] The raw webhook payload. + # @param content_encoding [String, nil] Value of the `Content-Encoding` + # HTTP header (or the corresponding SQS / SNS message attribute). Pass + # `nil` or an empty string to skip decompression. + # @param payload_encoding [String, nil] How the queue wrapped the bytes, + # typically `"base64"` for SQS / SNS firehoses. Pass `nil` for plain + # HTTP webhooks. + # @return [String] Uncompressed JSON bytes (binary encoded). + sig do + params( + body: T.any(String, T::Array[Integer]), + content_encoding: T.nilable(String), + payload_encoding: T.nilable(String) + ).returns(String) + end + def decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil) + raw = StreamChat::Webhook.normalize_body(body) + raw = StreamChat::Webhook.apply_payload_encoding(raw, payload_encoding) + StreamChat::Webhook.apply_content_encoding(raw, content_encoding) + end + + # Decodes a (possibly base64-wrapped, possibly gzip-compressed) webhook + # body and verifies its `X-Signature` HMAC against `@api_secret`. The + # signature is always computed over the *uncompressed* JSON bytes — both + # for plain HTTP webhooks and for SQS / SNS messages — matching what the + # Stream backend signs. + # + # @param body [String, Array] The raw webhook payload. + # @param x_signature [String] Value of the `X-Signature` header (or the + # corresponding SQS / SNS message attribute). + # @param content_encoding [String, nil] Value of the `Content-Encoding` + # HTTP header. `nil` or empty means the body is already raw JSON. + # @param payload_encoding [String, nil] Set to `"base64"` for SQS / SNS + # firehose envelopes; `nil` for plain HTTP. + # @return [String] Uncompressed JSON bytes (binary encoded). + # @raise [WebhookSignatureError] if decoding fails or the signature does + # not match. + sig do + params( + body: T.any(String, T::Array[Integer]), + x_signature: String, + content_encoding: T.nilable(String), + payload_encoding: T.nilable(String) + ).returns(String) + end + def verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil) + decoded = decompress_webhook_body(body, content_encoding, payload_encoding) + expected = OpenSSL::HMAC.hexdigest('SHA256', @api_secret, decoded) + raise WebhookSignatureError, 'invalid webhook signature' unless StreamChat::Webhook.constant_time_equal?(expected, x_signature) + + decoded + end + # Allows you to send custom events to a connected user. sig { params(user_id: String, event: StringKeyHash).returns(StreamChat::StreamResponse) } def send_user_event(user_id, event) diff --git a/lib/stream-chat/errors.rb b/lib/stream-chat/errors.rb index df5cef0..c2a750b 100644 --- a/lib/stream-chat/errors.rb +++ b/lib/stream-chat/errors.rb @@ -47,4 +47,34 @@ def to_s end class StreamChannelException < StandardError; end + + # Raised when a webhook payload cannot be decoded, decompressed, or its + # signature does not match the expected HMAC. Carries a human readable + # reason instead of an HTTP response so it can be raised from the local + # webhook helpers (decompress / verify) without involving Faraday. + class WebhookSignatureError < StreamAPIException + extend T::Sig + + sig { returns(String) } + attr_reader :reason + + sig { params(reason: String).void } + def initialize(reason) # rubocop:disable Lint/MissingSuper + @reason = T.let(reason, String) + @json_response = T.let(false, T::Boolean) + @error_code = T.let(0, Integer) + @error_message = T.let(reason, String) + StandardError.instance_method(:initialize).bind_call(self, reason) + end + + sig { returns(String) } + def message + @reason + end + + sig { returns(String) } + def to_s + @reason + end + end end diff --git a/lib/stream-chat/webhook.rb b/lib/stream-chat/webhook.rb new file mode 100644 index 0000000..ec42625 --- /dev/null +++ b/lib/stream-chat/webhook.rb @@ -0,0 +1,102 @@ +# typed: strict +# frozen_string_literal: true + +require 'base64' +require 'openssl' +require 'sorbet-runtime' +require 'stringio' +require 'zlib' + +require 'stream-chat/errors' + +module StreamChat + # Stateless helpers used by the webhook decoding and verification methods on + # `StreamChat::Client`. Kept in a module so the decode/verify primitives can + # be exercised in isolation, and so `Client#verify_webhook` (the legacy + # boolean-returning helper) stays untouched for backward compatibility. + module Webhook + extend T::Sig + + SUPPORTED_CONTENT_ENCODINGS = T.let(%w[gzip].freeze, T::Array[String]) + SUPPORTED_PAYLOAD_ENCODINGS = T.let(%w[base64 b64].freeze, T::Array[String]) + + # Coerces the webhook body into a binary `String` regardless of whether + # the caller hands us a `String` (HTTP `request.raw_post`) or an array of + # bytes (which is what some Ruby SQS clients yield when the message body + # is binary safe). + sig { params(body: T.any(String, T::Array[Integer])).returns(String) } + def self.normalize_body(body) + raw = + if body.is_a?(Array) + body.pack('C*') + else + String.new(body) + end + raw.force_encoding(Encoding::ASCII_8BIT) + end + + # Decodes the outer `payload_encoding` wrapper if present. SQS / SNS + # base64-wrap the gzipped bytes so they remain valid UTF-8 over the queue; + # plain HTTP webhooks pass `nil` here and this is a no-op. + sig { params(body: String, payload_encoding: T.nilable(String)).returns(String) } + def self.apply_payload_encoding(body, payload_encoding) + normalized = payload_encoding.to_s.strip.downcase + return body if normalized.empty? + + case normalized + when 'base64', 'b64' + decoded = + begin + Base64.strict_decode64(body) + rescue ArgumentError => e + raise WebhookSignatureError, "failed to decode webhook body using payload_encoding=#{normalized}: #{e.message}" + end + String.new(decoded).force_encoding(Encoding::ASCII_8BIT) + else + raise WebhookSignatureError, "unsupported webhook payload_encoding: #{normalized}. This SDK only supports base64." + end + end + + # Decompresses the payload according to the HTTP `Content-Encoding` + # header reported by the dashboard / SQS message attribute. `nil` / + # empty means the body is already the raw JSON document. + sig { params(body: String, content_encoding: T.nilable(String)).returns(String) } + def self.apply_content_encoding(body, content_encoding) + normalized = content_encoding.to_s.strip.downcase + return body if normalized.empty? + + case normalized + when 'gzip' + begin + inflated = Zlib::GzipReader.new(StringIO.new(body)).read + rescue Zlib::Error => e + raise WebhookSignatureError, "failed to decompress webhook body: #{e.message}" + end + String.new(inflated).force_encoding(Encoding::ASCII_8BIT) + else + raise WebhookSignatureError, %(unsupported webhook Content-Encoding: #{normalized}. This SDK only supports gzip; set webhook_compression_algorithm to "gzip" on the app config.) + end + end + + # Timing-safe equality check used to compare the locally computed HMAC + # with the `X-Signature` header. Prefers the OpenSSL primitive when the + # Ruby build exposes it, otherwise falls back to a manual byte XOR loop + # that does not short-circuit on the first mismatch. + sig { params(left: String, right: String).returns(T::Boolean) } + def self.constant_time_equal?(left, right) + a = left.b + b = right.b + return false unless a.bytesize == b.bytesize + + if OpenSSL.respond_to?(:fixed_length_secure_compare) + OpenSSL.fixed_length_secure_compare(a, b) + else + a_bytes = a.bytes + b_bytes = b.bytes + diff = 0 + a_bytes.each_with_index { |byte, i| diff |= byte ^ b_bytes[i] } + diff.zero? + end + end + end +end diff --git a/spec/webhook_compression_spec.rb b/spec/webhook_compression_spec.rb new file mode 100644 index 0000000..2483299 --- /dev/null +++ b/spec/webhook_compression_spec.rb @@ -0,0 +1,167 @@ +# frozen_string_literal: true + +require 'base64' +require 'openssl' +require 'stringio' +require 'zlib' + +require 'stream-chat' + +describe 'StreamChat webhook compression' do + let(:json_body) { '{"type":"message.new","message":{"text":"the quick brown fox"}}' } + let(:api_key) { 'tkey' } + let(:api_secret) { 'tsec2' } + + def gzip(bytes) + io = StringIO.new + io.set_encoding(Encoding::ASCII_8BIT) + Zlib::GzipWriter.wrap(io) { |gz| gz.write(bytes) } + io.string + end + + def hmac_hex(secret, data) + OpenSSL::HMAC.hexdigest('SHA256', secret, data) + end + + let(:client) { StreamChat::Client.new(api_key, api_secret) } + + describe '#verify_webhook (existing helper, must remain backwards compatible)' do + it 'returns true when the signature matches the raw body' do + signature = hmac_hex(api_secret, json_body) + expect(client.verify_webhook(json_body, signature)).to be true + end + + it 'returns false when the signature does not match' do + expect(client.verify_webhook(json_body, 'not-a-real-signature')).to be false + end + end + + describe '#decompress_webhook_body' do + context 'when both encodings are nil or empty' do + it 'returns the body unchanged when encodings are nil' do + expect(client.decompress_webhook_body(json_body, nil, nil)).to eq(json_body) + end + + it 'returns the body unchanged when encodings are empty strings' do + expect(client.decompress_webhook_body(json_body, '', '')).to eq(json_body) + end + + it 'returns the body unchanged when encodings are whitespace' do + expect(client.decompress_webhook_body(json_body, ' ', ' ')).to eq(json_body) + end + + it 'accepts a body provided as an array of integers (byte array)' do + bytes = json_body.bytes + expect(client.decompress_webhook_body(bytes, nil, nil)).to eq(json_body) + end + end + + context 'gzip round-trip' do + it 'decompresses a gzip-compressed body' do + compressed = gzip(json_body) + expect(client.decompress_webhook_body(compressed, 'gzip', nil)).to eq(json_body) + end + + it 'is case-insensitive for the content_encoding value' do + compressed = gzip(json_body) + expect(client.decompress_webhook_body(compressed, 'GZIP', nil)).to eq(json_body) + expect(client.decompress_webhook_body(compressed, ' Gzip ', nil)).to eq(json_body) + end + end + + context 'base64 round-trip' do + it 'decodes a strict-base64 wrapped body' do + wrapped = Base64.strict_encode64(json_body) + expect(client.decompress_webhook_body(wrapped, nil, 'base64')).to eq(json_body) + end + + it 'accepts the b64 alias' do + wrapped = Base64.strict_encode64(json_body) + expect(client.decompress_webhook_body(wrapped, nil, 'b64')).to eq(json_body) + end + + it 'is case-insensitive for the payload_encoding value' do + wrapped = Base64.strict_encode64(json_body) + expect(client.decompress_webhook_body(wrapped, nil, 'BASE64')).to eq(json_body) + expect(client.decompress_webhook_body(wrapped, nil, ' Base64 ')).to eq(json_body) + end + end + + context 'base64 + gzip round-trip (SQS / SNS firehose shape)' do + it 'decodes the base64 wrapper and then decompresses the gzip body' do + wrapped = Base64.strict_encode64(gzip(json_body)) + expect(client.decompress_webhook_body(wrapped, 'gzip', 'base64')).to eq(json_body) + end + end + + context 'unsupported encodings' do + %w[br brotli zstd deflate compress lz4].each do |unsupported| + it "rejects content_encoding=#{unsupported.inspect}" do + expect { client.decompress_webhook_body(json_body, unsupported, nil) } + .to raise_error(StreamChat::WebhookSignatureError, /unsupported webhook Content-Encoding/i) + end + end + + %w[hex url binary].each do |unsupported| + it "rejects payload_encoding=#{unsupported.inspect}" do + expect { client.decompress_webhook_body(json_body, nil, unsupported) } + .to raise_error(StreamChat::WebhookSignatureError, /unsupported webhook payload_encoding/i) + end + end + end + + context 'malformed payloads' do + it 'raises when the gzip bytes are corrupt' do + expect { client.decompress_webhook_body('not-actually-gzip', 'gzip', nil) } + .to raise_error(StreamChat::WebhookSignatureError, /failed to decompress webhook body/i) + end + + it 'raises on invalid base64 input' do + expect { client.decompress_webhook_body("\x00\x01\xff not base64", nil, 'base64') } + .to raise_error(StreamChat::WebhookSignatureError, /payload_encoding=base64/i) + end + end + end + + describe '#verify_and_decode_webhook' do + context 'happy paths' do + it 'verifies and returns the body for a plain HTTP webhook' do + signature = hmac_hex(api_secret, json_body) + expect(client.verify_and_decode_webhook(json_body, signature, nil, nil)).to eq(json_body) + end + + it 'verifies and returns the body for a gzip-compressed webhook (signature over uncompressed bytes)' do + compressed = gzip(json_body) + signature = hmac_hex(api_secret, json_body) + expect(client.verify_and_decode_webhook(compressed, signature, 'gzip', nil)).to eq(json_body) + end + + it 'verifies and returns the body for a base64+gzip SQS / SNS webhook (signature over uncompressed bytes)' do + wrapped = Base64.strict_encode64(gzip(json_body)) + signature = hmac_hex(api_secret, json_body) + expect(client.verify_and_decode_webhook(wrapped, signature, 'gzip', 'base64')).to eq(json_body) + end + end + + context 'signature mismatches' do + it 'raises WebhookSignatureError when the signature is wrong' do + expect { client.verify_and_decode_webhook(json_body, 'definitely-wrong', nil, nil) } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end + + it 'rejects a gzip body when the signature was computed over the compressed bytes (not the JSON)' do + compressed = gzip(json_body) + wrong_signature_over_compressed = hmac_hex(api_secret, compressed) + expect { client.verify_and_decode_webhook(compressed, wrong_signature_over_compressed, 'gzip', nil) } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end + + it 'rejects a base64+gzip body when the signature was computed over the wrapped bytes (not the JSON)' do + wrapped = Base64.strict_encode64(gzip(json_body)) + wrong_signature_over_wrapped = hmac_hex(api_secret, wrapped) + expect { client.verify_and_decode_webhook(wrapped, wrong_signature_over_wrapped, 'gzip', 'base64') } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end + end + end +end From 1dc1bfae538559a15c6f47886cc4fe13e51653e0 Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Fri, 8 May 2026 15:58:53 +0200 Subject: [PATCH 2/7] refactor(webhooks): switch to verify_and_parse_* API (CHA-3071) Replaces verify_and_decode_webhook / decompress_webhook_body with the cross-SDK contract documented at https://getstream.io/chat/docs/node/webhooks_overview/. Module-level helpers in StreamChat::Webhook: Primitives: ungzip_payload - gzip magic-byte detection + inflate decode_sqs_payload - base64 then ungzip-if-magic decode_sns_payload - alias for decode_sqs_payload verify_signature - constant-time HMAC-SHA256 comparison (parameter order matches the cross-SDK spec) parse_event - JSON -> Hash (typed event lands later) Composite (return parsed event Hash): verify_and_parse_webhook verify_and_parse_sqs verify_and_parse_sns The composite functions auto-detect compression from body bytes, so the same handler stays correct whether or not Stream is currently compressing payloads, and behind middleware that auto-decompresses. Client#verify_and_parse_{webhook,sqs,sns} mirror the three composite helpers with the api_secret pulled from the client. The legacy Client#verify_webhook(body, signature) -> bool helper is kept for backward compatibility (now delegates to verify_signature). Co-authored-by: Cursor --- lib/stream-chat/client.rb | 96 ++++++----- lib/stream-chat/webhook.rb | 178 +++++++++++++++----- spec/webhook_compression_spec.rb | 277 +++++++++++++++++-------------- 3 files changed, 332 insertions(+), 219 deletions(-) diff --git a/lib/stream-chat/client.rb b/lib/stream-chat/client.rb index 119f9fa..519503b 100644 --- a/lib/stream-chat/client.rb +++ b/lib/stream-chat/client.rb @@ -703,69 +703,67 @@ def get_rate_limits(server_side: false, android: false, ios: false, web: false, get('rate_limits', params: params) end - # Verify the signature added to a webhook event. + # Verify the signature on a webhook event using the client's API secret. + # + # Backward-compatible boolean helper. New integrations should call + # {#verify_and_parse_webhook} (or the SQS / SNS variants), which also handle + # gzip payload compression and return the parsed event. sig { params(request_body: String, x_signature: String).returns(T::Boolean) } def verify_webhook(request_body, x_signature) - signature = OpenSSL::HMAC.hexdigest('SHA256', @api_secret, request_body) - signature == x_signature + StreamChat::Webhook.verify_signature(request_body, x_signature, @api_secret) end - # Decodes a webhook body that may have been base64-wrapped (SQS / SNS) and - # / or gzip-compressed (Content-Encoding: gzip), without verifying the - # signature. Returns the raw JSON bytes as a binary `String`; callers can - # `.force_encoding('UTF-8')` or pass the result to `JSON.parse` directly. + # Verify and parse an HTTP webhook event. + # + # Decompresses `body` when gzipped (detected from the body bytes), verifies + # the `X-Signature` header against the client's API secret, and returns the + # parsed event as a `Hash`. Typed event classes are planned for a future + # release. # - # @param body [String, Array] The raw webhook payload. - # @param content_encoding [String, nil] Value of the `Content-Encoding` - # HTTP header (or the corresponding SQS / SNS message attribute). Pass - # `nil` or an empty string to skip decompression. - # @param payload_encoding [String, nil] How the queue wrapped the bytes, - # typically `"base64"` for SQS / SNS firehoses. Pass `nil` for plain - # HTTP webhooks. - # @return [String] Uncompressed JSON bytes (binary encoded). + # @param body [String, Array] Raw HTTP request body bytes Stream + # signed. + # @param signature [String] Value of the `X-Signature` header. + # @return [Hash] The parsed event. + # @raise [WebhookSignatureError] When the signature does not match or the + # gzip envelope is malformed. sig do params( body: T.any(String, T::Array[Integer]), - content_encoding: T.nilable(String), - payload_encoding: T.nilable(String) - ).returns(String) + signature: String + ).returns(T::Hash[String, T.untyped]) end - def decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil) - raw = StreamChat::Webhook.normalize_body(body) - raw = StreamChat::Webhook.apply_payload_encoding(raw, payload_encoding) - StreamChat::Webhook.apply_content_encoding(raw, content_encoding) + def verify_and_parse_webhook(body, signature) + StreamChat::Webhook.verify_and_parse_webhook(body, signature, @api_secret) end - # Decodes a (possibly base64-wrapped, possibly gzip-compressed) webhook - # body and verifies its `X-Signature` HMAC against `@api_secret`. The - # signature is always computed over the *uncompressed* JSON bytes — both - # for plain HTTP webhooks and for SQS / SNS messages — matching what the - # Stream backend signs. + # Verify and parse an SQS firehose webhook event. # - # @param body [String, Array] The raw webhook payload. - # @param x_signature [String] Value of the `X-Signature` header (or the - # corresponding SQS / SNS message attribute). - # @param content_encoding [String, nil] Value of the `Content-Encoding` - # HTTP header. `nil` or empty means the body is already raw JSON. - # @param payload_encoding [String, nil] Set to `"base64"` for SQS / SNS - # firehose envelopes; `nil` for plain HTTP. - # @return [String] Uncompressed JSON bytes (binary encoded). - # @raise [WebhookSignatureError] if decoding fails or the signature does - # not match. - sig do - params( - body: T.any(String, T::Array[Integer]), - x_signature: String, - content_encoding: T.nilable(String), - payload_encoding: T.nilable(String) - ).returns(String) + # Reverses the base64 (+ optional gzip) wrapping on the SQS `Body`, + # verifies the `X-Signature` message attribute against the client's API + # secret, and returns the parsed event. + # + # @param message_body [String] SQS message `Body` string. + # @param signature [String] Value of the `X-Signature` message attribute. + # @return [Hash] The parsed event. + # @raise [WebhookSignatureError] + sig { params(message_body: String, signature: String).returns(T::Hash[String, T.untyped]) } + def verify_and_parse_sqs(message_body, signature) + StreamChat::Webhook.verify_and_parse_sqs(message_body, signature, @api_secret) end - def verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil) - decoded = decompress_webhook_body(body, content_encoding, payload_encoding) - expected = OpenSSL::HMAC.hexdigest('SHA256', @api_secret, decoded) - raise WebhookSignatureError, 'invalid webhook signature' unless StreamChat::Webhook.constant_time_equal?(expected, x_signature) - decoded + # Verify and parse an SNS firehose webhook event. + # + # Reverses the base64 (+ optional gzip) wrapping on the SNS notification + # `Message`, verifies the `X-Signature` message attribute against the + # client's API secret, and returns the parsed event. + # + # @param message [String] SNS notification `Message` field. + # @param signature [String] Value of the `X-Signature` message attribute. + # @return [Hash] The parsed event. + # @raise [WebhookSignatureError] + sig { params(message: String, signature: String).returns(T::Hash[String, T.untyped]) } + def verify_and_parse_sns(message, signature) + StreamChat::Webhook.verify_and_parse_sns(message, signature, @api_secret) end # Allows you to send custom events to a connected user. diff --git a/lib/stream-chat/webhook.rb b/lib/stream-chat/webhook.rb index ec42625..7248283 100644 --- a/lib/stream-chat/webhook.rb +++ b/lib/stream-chat/webhook.rb @@ -2,6 +2,7 @@ # frozen_string_literal: true require 'base64' +require 'json' require 'openssl' require 'sorbet-runtime' require 'stringio' @@ -10,15 +11,21 @@ require 'stream-chat/errors' module StreamChat - # Stateless helpers used by the webhook decoding and verification methods on - # `StreamChat::Client`. Kept in a module so the decode/verify primitives can - # be exercised in isolation, and so `Client#verify_webhook` (the legacy - # boolean-returning helper) stays untouched for backward compatibility. - module Webhook + # Stateless helpers implementing the cross-SDK webhook contract documented at + # https://getstream.io/chat/docs/node/webhooks_overview/. + # + # The composite functions (`verify_and_parse_webhook`, `verify_and_parse_sqs`, + # `verify_and_parse_sns`) are the recommended entry points. The primitives + # they compose (`ungzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, + # `verify_signature`, `parse_event`) are exposed so callers can build custom + # flows or run individual steps in isolation. + # + # The Ruby SDK currently returns the parsed JSON as a `Hash`; typed event + # classes will land in a future release. + module Webhook # rubocop:disable Metrics/ModuleLength extend T::Sig - SUPPORTED_CONTENT_ENCODINGS = T.let(%w[gzip].freeze, T::Array[String]) - SUPPORTED_PAYLOAD_ENCODINGS = T.let(%w[base64 b64].freeze, T::Array[String]) + GZIP_MAGIC = T.let("\x1f\x8b\x08".b.freeze, String) # Coerces the webhook body into a binary `String` regardless of whether # the caller hands us a `String` (HTTP `request.raw_post`) or an array of @@ -35,47 +42,132 @@ def self.normalize_body(body) raw.force_encoding(Encoding::ASCII_8BIT) end - # Decodes the outer `payload_encoding` wrapper if present. SQS / SNS - # base64-wrap the gzipped bytes so they remain valid UTF-8 over the queue; - # plain HTTP webhooks pass `nil` here and this is a no-op. - sig { params(body: String, payload_encoding: T.nilable(String)).returns(String) } - def self.apply_payload_encoding(body, payload_encoding) - normalized = payload_encoding.to_s.strip.downcase - return body if normalized.empty? - - case normalized - when 'base64', 'b64' - decoded = - begin - Base64.strict_decode64(body) - rescue ArgumentError => e - raise WebhookSignatureError, "failed to decode webhook body using payload_encoding=#{normalized}: #{e.message}" - end - String.new(decoded).force_encoding(Encoding::ASCII_8BIT) - else - raise WebhookSignatureError, "unsupported webhook payload_encoding: #{normalized}. This SDK only supports base64." + # Returns `body` unchanged unless it starts with the gzip magic + # (`1f 8b 08`), in which case the gzip stream is inflated and the + # decompressed bytes are returned. + # + # Magic-byte detection (rather than relying on a header) keeps the same + # handler correct when middleware - Rack, Rails - auto-decompresses the + # request before your code sees it. + sig { params(body: T.any(String, T::Array[Integer])).returns(String) } + def self.ungzip_payload(body) + raw = normalize_body(body) + return raw unless raw.start_with?(GZIP_MAGIC) + + begin + Zlib::GzipReader.new(StringIO.new(raw)).read.force_encoding(Encoding::ASCII_8BIT) + rescue Zlib::Error => e + raise WebhookSignatureError, "failed to decompress gzip payload: #{e.message}" end end - # Decompresses the payload according to the HTTP `Content-Encoding` - # header reported by the dashboard / SQS message attribute. `nil` / - # empty means the body is already the raw JSON document. - sig { params(body: String, content_encoding: T.nilable(String)).returns(String) } - def self.apply_content_encoding(body, content_encoding) - normalized = content_encoding.to_s.strip.downcase - return body if normalized.empty? - - case normalized - when 'gzip' + # Reverses the SQS firehose envelope: the message `Body` is base64-decoded + # and, when the result begins with the gzip magic, gzip-decompressed. The + # same call works whether or not Stream is currently compressing payloads. + sig { params(body: String).returns(String) } + def self.decode_sqs_payload(body) + decoded = begin - inflated = Zlib::GzipReader.new(StringIO.new(body)).read - rescue Zlib::Error => e - raise WebhookSignatureError, "failed to decompress webhook body: #{e.message}" + Base64.strict_decode64(body) + rescue ArgumentError => e + raise WebhookSignatureError, "failed to base64-decode payload: #{e.message}" end - String.new(inflated).force_encoding(Encoding::ASCII_8BIT) - else - raise WebhookSignatureError, %(unsupported webhook Content-Encoding: #{normalized}. This SDK only supports gzip; set webhook_compression_algorithm to "gzip" on the app config.) - end + ungzip_payload(decoded) + end + + # Identical to `decode_sqs_payload`; exposed under both names so call sites + # read intent. + sig { params(message: String).returns(String) } + def self.decode_sns_payload(message) + decode_sqs_payload(message) + end + + # Constant-time HMAC-SHA256 verification of `signature` against the digest + # of `body` keyed by `secret`. + # + # The signature is always computed over the **uncompressed** JSON bytes, + # so callers that decoded a gzipped or base64-wrapped payload must pass + # the inflated bytes here. + sig do + params( + body: T.any(String, T::Array[Integer]), + signature: String, + secret: String + ).returns(T::Boolean) + end + def self.verify_signature(body, signature, secret) + raw = normalize_body(body) + expected = OpenSSL::HMAC.hexdigest('SHA256', secret, raw) + constant_time_equal?(expected, signature) + end + + # Parse a JSON-encoded webhook event into a `Hash`. + # + # The Ruby SDK currently returns the parsed JSON as a `Hash`; typed event + # classes will land in a future release. The function name matches the + # documented primitive so callers can swap in a typed parser later without + # changing call sites. + sig { params(payload: String).returns(T::Hash[String, T.untyped]) } + def self.parse_event(payload) + result = JSON.parse(payload) + raise WebhookSignatureError, 'failed to parse webhook event: top-level value is not an object' unless result.is_a?(Hash) + + result + end + + sig do + params( + payload: String, + signature: String, + secret: String + ).returns(T::Hash[String, T.untyped]) + end + def self.verify_and_parse_internal(payload, signature, secret) + raise WebhookSignatureError, 'invalid webhook signature' unless verify_signature(payload, signature, secret) + + parse_event(payload) + end + private_class_method :verify_and_parse_internal + + # Decompress `body` when gzipped, verify the HMAC `signature`, and return + # the parsed event. + sig do + params( + body: T.any(String, T::Array[Integer]), + signature: String, + secret: String + ).returns(T::Hash[String, T.untyped]) + end + def self.verify_and_parse_webhook(body, signature, secret) + verify_and_parse_internal(ungzip_payload(body), signature, secret) + end + + # Decode the SQS `Body` (base64, then gzip-if-magic), verify the HMAC + # `signature` from the `X-Signature` message attribute, and return the + # parsed event. + sig do + params( + message_body: String, + signature: String, + secret: String + ).returns(T::Hash[String, T.untyped]) + end + def self.verify_and_parse_sqs(message_body, signature, secret) + verify_and_parse_internal(decode_sqs_payload(message_body), signature, secret) + end + + # Decode the SNS notification `Message` (identical to SQS handling), verify + # the HMAC `signature` from the `X-Signature` message attribute, and return + # the parsed event. + sig do + params( + message: String, + signature: String, + secret: String + ).returns(T::Hash[String, T.untyped]) + end + def self.verify_and_parse_sns(message, signature, secret) + verify_and_parse_internal(decode_sns_payload(message), signature, secret) end # Timing-safe equality check used to compare the locally computed HMAC diff --git a/spec/webhook_compression_spec.rb b/spec/webhook_compression_spec.rb index 2483299..9da2d8d 100644 --- a/spec/webhook_compression_spec.rb +++ b/spec/webhook_compression_spec.rb @@ -1,14 +1,16 @@ # frozen_string_literal: true require 'base64' +require 'json' require 'openssl' require 'stringio' require 'zlib' require 'stream-chat' -describe 'StreamChat webhook compression' do +describe 'StreamChat webhook verification + parsing' do let(:json_body) { '{"type":"message.new","message":{"text":"the quick brown fox"}}' } + let(:event_hash) { { 'type' => 'message.new', 'message' => { 'text' => 'the quick brown fox' } } } let(:api_key) { 'tkey' } let(:api_secret) { 'tsec2' } @@ -25,10 +27,94 @@ def hmac_hex(secret, data) let(:client) { StreamChat::Client.new(api_key, api_secret) } - describe '#verify_webhook (existing helper, must remain backwards compatible)' do + describe 'StreamChat::Webhook.ungzip_payload' do + it 'passes through plain bytes unchanged' do + expect(StreamChat::Webhook.ungzip_payload(json_body)).to eq(json_body) + end + + it 'inflates gzip-magic bytes' do + expect(StreamChat::Webhook.ungzip_payload(gzip(json_body))).to eq(json_body) + end + + it 'accepts a body provided as an array of integers' do + expect(StreamChat::Webhook.ungzip_payload(json_body.bytes)).to eq(json_body) + end + + it 'returns empty input unchanged' do + expect(StreamChat::Webhook.ungzip_payload('')).to eq('') + end + + it 'returns short input below magic length unchanged' do + expect(StreamChat::Webhook.ungzip_payload('ab')).to eq('ab') + end + + it 'raises on truncated gzip with magic' do + bad = "\x1f\x8b\x08\x00\x00\x00".b + expect { StreamChat::Webhook.ungzip_payload(bad) } + .to raise_error(StreamChat::WebhookSignatureError, /decompress gzip/i) + end + end + + describe 'StreamChat::Webhook.decode_sqs_payload' do + it 'decodes base64 only (no compression)' do + wrapped = Base64.strict_encode64(json_body) + expect(StreamChat::Webhook.decode_sqs_payload(wrapped)).to eq(json_body) + end + + it 'decodes base64 + gzip' do + wrapped = Base64.strict_encode64(gzip(json_body)) + expect(StreamChat::Webhook.decode_sqs_payload(wrapped)).to eq(json_body) + end + + it 'raises on invalid base64' do + expect { StreamChat::Webhook.decode_sqs_payload('!!!not-base64!!!') } + .to raise_error(StreamChat::WebhookSignatureError, /base64-decode/i) + end + end + + describe 'StreamChat::Webhook.decode_sns_payload' do + it 'aliases decode_sqs_payload' do + wrapped = Base64.strict_encode64(gzip(json_body)) + expect(StreamChat::Webhook.decode_sns_payload(wrapped)) + .to eq(StreamChat::Webhook.decode_sqs_payload(wrapped)) + end + end + + describe 'StreamChat::Webhook.verify_signature' do + it 'returns true for matching HMAC' do + sig = hmac_hex(api_secret, json_body) + expect(StreamChat::Webhook.verify_signature(json_body, sig, api_secret)).to be true + end + + it 'returns false for mismatched signature' do + expect(StreamChat::Webhook.verify_signature(json_body, '0' * 64, api_secret)).to be false + end + + it 'rejects signatures computed over compressed bytes' do + compressed = gzip(json_body) + sig_over_compressed = hmac_hex(api_secret, compressed) + expect(StreamChat::Webhook.verify_signature(json_body, sig_over_compressed, api_secret)).to be false + end + end + + describe 'StreamChat::Webhook.parse_event' do + it 'parses a known event type into a hash' do + expect(StreamChat::Webhook.parse_event(json_body)).to eq(event_hash) + end + + it 'still parses unknown event types' do + expect(StreamChat::Webhook.parse_event('{"type":"a.future.event","custom":42}')) + .to eq({ 'type' => 'a.future.event', 'custom' => 42 }) + end + + it 'raises on malformed JSON' do + expect { StreamChat::Webhook.parse_event('not json') }.to raise_error(JSON::ParserError) + end + end + + describe '#verify_webhook (legacy boolean helper, unchanged)' do it 'returns true when the signature matches the raw body' do - signature = hmac_hex(api_secret, json_body) - expect(client.verify_webhook(json_body, signature)).to be true + expect(client.verify_webhook(json_body, hmac_hex(api_secret, json_body))).to be true end it 'returns false when the signature does not match' do @@ -36,132 +122,69 @@ def hmac_hex(secret, data) end end - describe '#decompress_webhook_body' do - context 'when both encodings are nil or empty' do - it 'returns the body unchanged when encodings are nil' do - expect(client.decompress_webhook_body(json_body, nil, nil)).to eq(json_body) - end - - it 'returns the body unchanged when encodings are empty strings' do - expect(client.decompress_webhook_body(json_body, '', '')).to eq(json_body) - end - - it 'returns the body unchanged when encodings are whitespace' do - expect(client.decompress_webhook_body(json_body, ' ', ' ')).to eq(json_body) - end - - it 'accepts a body provided as an array of integers (byte array)' do - bytes = json_body.bytes - expect(client.decompress_webhook_body(bytes, nil, nil)).to eq(json_body) - end - end - - context 'gzip round-trip' do - it 'decompresses a gzip-compressed body' do - compressed = gzip(json_body) - expect(client.decompress_webhook_body(compressed, 'gzip', nil)).to eq(json_body) - end - - it 'is case-insensitive for the content_encoding value' do - compressed = gzip(json_body) - expect(client.decompress_webhook_body(compressed, 'GZIP', nil)).to eq(json_body) - expect(client.decompress_webhook_body(compressed, ' Gzip ', nil)).to eq(json_body) - end - end - - context 'base64 round-trip' do - it 'decodes a strict-base64 wrapped body' do - wrapped = Base64.strict_encode64(json_body) - expect(client.decompress_webhook_body(wrapped, nil, 'base64')).to eq(json_body) - end - - it 'accepts the b64 alias' do - wrapped = Base64.strict_encode64(json_body) - expect(client.decompress_webhook_body(wrapped, nil, 'b64')).to eq(json_body) - end - - it 'is case-insensitive for the payload_encoding value' do - wrapped = Base64.strict_encode64(json_body) - expect(client.decompress_webhook_body(wrapped, nil, 'BASE64')).to eq(json_body) - expect(client.decompress_webhook_body(wrapped, nil, ' Base64 ')).to eq(json_body) - end - end - - context 'base64 + gzip round-trip (SQS / SNS firehose shape)' do - it 'decodes the base64 wrapper and then decompresses the gzip body' do - wrapped = Base64.strict_encode64(gzip(json_body)) - expect(client.decompress_webhook_body(wrapped, 'gzip', 'base64')).to eq(json_body) - end - end - - context 'unsupported encodings' do - %w[br brotli zstd deflate compress lz4].each do |unsupported| - it "rejects content_encoding=#{unsupported.inspect}" do - expect { client.decompress_webhook_body(json_body, unsupported, nil) } - .to raise_error(StreamChat::WebhookSignatureError, /unsupported webhook Content-Encoding/i) - end - end - - %w[hex url binary].each do |unsupported| - it "rejects payload_encoding=#{unsupported.inspect}" do - expect { client.decompress_webhook_body(json_body, nil, unsupported) } - .to raise_error(StreamChat::WebhookSignatureError, /unsupported webhook payload_encoding/i) - end - end - end - - context 'malformed payloads' do - it 'raises when the gzip bytes are corrupt' do - expect { client.decompress_webhook_body('not-actually-gzip', 'gzip', nil) } - .to raise_error(StreamChat::WebhookSignatureError, /failed to decompress webhook body/i) - end - - it 'raises on invalid base64 input' do - expect { client.decompress_webhook_body("\x00\x01\xff not base64", nil, 'base64') } - .to raise_error(StreamChat::WebhookSignatureError, /payload_encoding=base64/i) - end + describe '#verify_and_parse_webhook' do + it 'parses a plain JSON body with a valid signature' do + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_webhook(json_body, sig)).to eq(event_hash) + end + + it 'parses a gzip-compressed body' do + compressed = gzip(json_body) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_webhook(compressed, sig)).to eq(event_hash) + end + + it 'accepts a body provided as an array of integers' do + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_webhook(json_body.bytes, sig)).to eq(event_hash) + end + + it 'raises WebhookSignatureError on signature mismatch' do + expect { client.verify_and_parse_webhook(json_body, 'definitely-wrong') } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end + + it 'rejects a gzip body when the signature was computed over compressed bytes' do + compressed = gzip(json_body) + sig_over_compressed = hmac_hex(api_secret, compressed) + expect { client.verify_and_parse_webhook(compressed, sig_over_compressed) } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') end end - describe '#verify_and_decode_webhook' do - context 'happy paths' do - it 'verifies and returns the body for a plain HTTP webhook' do - signature = hmac_hex(api_secret, json_body) - expect(client.verify_and_decode_webhook(json_body, signature, nil, nil)).to eq(json_body) - end - - it 'verifies and returns the body for a gzip-compressed webhook (signature over uncompressed bytes)' do - compressed = gzip(json_body) - signature = hmac_hex(api_secret, json_body) - expect(client.verify_and_decode_webhook(compressed, signature, 'gzip', nil)).to eq(json_body) - end - - it 'verifies and returns the body for a base64+gzip SQS / SNS webhook (signature over uncompressed bytes)' do - wrapped = Base64.strict_encode64(gzip(json_body)) - signature = hmac_hex(api_secret, json_body) - expect(client.verify_and_decode_webhook(wrapped, signature, 'gzip', 'base64')).to eq(json_body) - end - end - - context 'signature mismatches' do - it 'raises WebhookSignatureError when the signature is wrong' do - expect { client.verify_and_decode_webhook(json_body, 'definitely-wrong', nil, nil) } - .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') - end - - it 'rejects a gzip body when the signature was computed over the compressed bytes (not the JSON)' do - compressed = gzip(json_body) - wrong_signature_over_compressed = hmac_hex(api_secret, compressed) - expect { client.verify_and_decode_webhook(compressed, wrong_signature_over_compressed, 'gzip', nil) } - .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') - end - - it 'rejects a base64+gzip body when the signature was computed over the wrapped bytes (not the JSON)' do - wrapped = Base64.strict_encode64(gzip(json_body)) - wrong_signature_over_wrapped = hmac_hex(api_secret, wrapped) - expect { client.verify_and_decode_webhook(wrapped, wrong_signature_over_wrapped, 'gzip', 'base64') } - .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') - end + describe '#verify_and_parse_sqs' do + it 'parses a base64-only message body' do + wrapped = Base64.strict_encode64(json_body) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_sqs(wrapped, sig)).to eq(event_hash) + end + + it 'parses a base64 + gzip message body' do + wrapped = Base64.strict_encode64(gzip(json_body)) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_sqs(wrapped, sig)).to eq(event_hash) + end + + it 'rejects a wrapped body when the signature was computed over the wrapper' do + wrapped = Base64.strict_encode64(gzip(json_body)) + sig_over_wrapped = hmac_hex(api_secret, wrapped) + expect { client.verify_and_parse_sqs(wrapped, sig_over_wrapped) } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end + end + + describe '#verify_and_parse_sns' do + it 'parses a base64 + gzip notification' do + wrapped = Base64.strict_encode64(gzip(json_body)) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_sns(wrapped, sig)).to eq(event_hash) + end + + it 'returns the same event as verify_and_parse_sqs' do + wrapped = Base64.strict_encode64(gzip(json_body)) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_sns(wrapped, sig)) + .to eq(client.verify_and_parse_sqs(wrapped, sig)) end end end From 59dcac2510493d91134b39f4503118590c2ba9b5 Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Fri, 8 May 2026 16:53:25 +0200 Subject: [PATCH 3/7] refactor(webhooks): use 2-byte gzip magic per RFC 1952 (CHA-3071) RFC 1952 defines the gzip magic number as the two-byte sequence 1F 8B; the third byte (CM) is informational and not part of the identifier. Trim the magic check from three bytes to two to match the spec and stay consistent with the reference implementations in the public docs. Co-authored-by: Cursor --- lib/stream-chat/webhook.rb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/stream-chat/webhook.rb b/lib/stream-chat/webhook.rb index 7248283..1c9d439 100644 --- a/lib/stream-chat/webhook.rb +++ b/lib/stream-chat/webhook.rb @@ -25,7 +25,7 @@ module StreamChat module Webhook # rubocop:disable Metrics/ModuleLength extend T::Sig - GZIP_MAGIC = T.let("\x1f\x8b\x08".b.freeze, String) + GZIP_MAGIC = T.let("\x1f\x8b".b.freeze, String) # Coerces the webhook body into a binary `String` regardless of whether # the caller hands us a `String` (HTTP `request.raw_post`) or an array of @@ -43,8 +43,8 @@ def self.normalize_body(body) end # Returns `body` unchanged unless it starts with the gzip magic - # (`1f 8b 08`), in which case the gzip stream is inflated and the - # decompressed bytes are returned. + # (`1f 8b`, per RFC 1952), in which case the gzip stream is inflated and + # the decompressed bytes are returned. # # Magic-byte detection (rather than relying on a header) keeps the same # handler correct when middleware - Rack, Rails - auto-decompresses the From fdd5b4db648c38e330f0b2f46b9aadccbbc1125a Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Mon, 11 May 2026 11:12:34 +0200 Subject: [PATCH 4/7] fix(webhooks): inherit WebhookSignatureError from StandardError The previous parent (`StreamAPIException`) declares a non-nilable `Faraday::Response` and an HTTP-shaped `initialize`. A local webhook verification failure has no response, so the subclass was forced to skip `super`, fake `@error_code`/`@error_message`/`@json_response`, and call `StandardError#initialize` via `bind_call` to populate the message - while still exposing a `response` reader that violates its own type signature by returning nil. Inherit directly from `StandardError` instead. Drops the LSP violation, the `Lint/MissingSuper` disable, and three irrelevant accessors. No caller depended on the prior parent: the class was introduced in this PR and is only used by the webhook helpers. CHA-3071 Co-authored-by: Cursor --- lib/stream-chat/errors.rb | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) diff --git a/lib/stream-chat/errors.rb b/lib/stream-chat/errors.rb index c2a750b..a750487 100644 --- a/lib/stream-chat/errors.rb +++ b/lib/stream-chat/errors.rb @@ -49,32 +49,19 @@ def to_s class StreamChannelException < StandardError; end # Raised when a webhook payload cannot be decoded, decompressed, or its - # signature does not match the expected HMAC. Carries a human readable - # reason instead of an HTTP response so it can be raised from the local - # webhook helpers (decompress / verify) without involving Faraday. - class WebhookSignatureError < StreamAPIException + # signature does not match the expected HMAC. This is a local verification + # failure with no HTTP response attached, so it inherits directly from + # `StandardError` rather than `StreamAPIException`. + class WebhookSignatureError < StandardError extend T::Sig sig { returns(String) } attr_reader :reason sig { params(reason: String).void } - def initialize(reason) # rubocop:disable Lint/MissingSuper + def initialize(reason) @reason = T.let(reason, String) - @json_response = T.let(false, T::Boolean) - @error_code = T.let(0, Integer) - @error_message = T.let(reason, String) - StandardError.instance_method(:initialize).bind_call(self, reason) - end - - sig { returns(String) } - def message - @reason - end - - sig { returns(String) } - def to_s - @reason + super end end end From c89d8d84221f8055d75d68590f2cbeb41ad4c8a2 Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Mon, 11 May 2026 11:13:25 +0200 Subject: [PATCH 5/7] docs(webhooks): align Ruby snippets with verify_and_parse_* API The Ruby code samples still referenced an earlier draft of the API (`decompress_webhook_body`, `verify_and_decode_webhook`, with `content_encoding` / `payload_encoding` arguments returning raw JSON bytes). The shipped API renamed those to `verify_and_parse_*`, removed the encoding arguments (gzip is detected from the body bytes per RFC 1952), and returns a parsed `Hash` rather than bytes - so the previous snippets would have failed with NoMethodError, and the subsequent `JSON.parse(json_bytes)` would have raised TypeError on the returned Hash. Replace both snippets with the current API, add a pointer to the module-level primitives, and note that the legacy boolean `verify_webhook` remains for callers that don't need compression support. CHA-3071 Co-authored-by: Cursor --- .../webhooks_overview/webhooks_overview.md | 46 ++++++++----------- 1 file changed, 18 insertions(+), 28 deletions(-) diff --git a/docs/webhooks/webhooks_overview/webhooks_overview.md b/docs/webhooks/webhooks_overview/webhooks_overview.md index 7a2793d..e7016da 100644 --- a/docs/webhooks/webhooks_overview/webhooks_overview.md +++ b/docs/webhooks/webhooks_overview/webhooks_overview.md @@ -110,16 +110,16 @@ Before enabling compression, make sure that: ### Decoding a compressed webhook in Ruby (Rails) -The Ruby SDK exposes two helpers on `StreamChat::Client` so you do not have to wire `Zlib` and `OpenSSL::HMAC` together yourself: +The Ruby SDK exposes three composite helpers on `StreamChat::Client`, one per transport, so you do not have to wire `Zlib`, `Base64`, and `OpenSSL::HMAC` together yourself. Each method decodes the body, verifies the `X-Signature` HMAC against the **uncompressed** JSON, parses the JSON, and returns the event as a `Hash`. Any failure (bad signature, malformed gzip, malformed base64) raises `StreamChat::WebhookSignatureError`. -* `decompress_webhook_body(body, content_encoding = nil, payload_encoding = nil)` — primitive that just decodes the body. No signature check. -* `verify_and_decode_webhook(body, x_signature, content_encoding = nil, payload_encoding = nil)` — decodes **and** verifies the `X-Signature` HMAC against the uncompressed JSON. Raises `StreamChat::WebhookSignatureError` if anything is wrong. +* `verify_and_parse_webhook(body, x_signature)` — HTTP webhooks. +* `verify_and_parse_sqs(message_body, x_signature)` — SQS firehose messages (base64 + optional gzip). +* `verify_and_parse_sns(message, x_signature)` — SNS notification messages (base64 + optional gzip). -Both methods return the raw JSON bytes as a binary `String`; you can `.force_encoding('UTF-8')` it or pass it straight to `JSON.parse`. Passing `nil` (or an empty string) for either encoding is a no-op, so the same handler works whether or not compression is enabled. +GZIP is detected from the body bytes (the `1f 8b` magic prefix), not from the `Content-Encoding` header, so the same handler works whether or not your HTTP server transparently decompresses the request body before it reaches you. ```ruby require 'stream-chat' -require 'json' STREAM_CLIENT = StreamChat::Client.new('STREAM_KEY', 'STREAM_SECRET') @@ -129,15 +129,12 @@ class WebhooksController < ApplicationController def stream body = request.raw_post # binary safe; do NOT use params - json_bytes = STREAM_CLIENT.verify_and_decode_webhook( + event = STREAM_CLIENT.verify_and_parse_webhook( body, - request.headers['X-Signature'], - request.headers['Content-Encoding'] # 'gzip' when compressed, nil otherwise + request.headers['X-Signature'] ) - event = JSON.parse(json_bytes) Rails.logger.info("Stream webhook: #{event['type']}") - head :ok rescue StreamChat::WebhookSignatureError => e Rails.logger.warn("Rejected Stream webhook: #{e.message}") @@ -146,16 +143,14 @@ class WebhooksController < ApplicationController end ``` -> [!NOTE] -> If you sit behind middleware that already inflates `Content-Encoding: gzip` requests for you, pass `nil` for `content_encoding` (or just don't forward the header) — `request.raw_post` will already be raw JSON. +If you also want to support the legacy "I just want to check the signature myself" flow, `client.verify_webhook(body, x_signature)` is still available and returns a boolean. It does **not** handle compression — use `verify_and_parse_webhook` for new integrations. ### Decoding a compressed SQS / SNS firehose message -SQS and SNS message bodies must be valid UTF-8, so when GZIP compression is enabled the gzipped bytes are additionally **base64-wrapped** before being placed on the queue. Pass `payload_encoding: 'base64'` so the SDK unwraps the queue envelope before decompressing: +SQS and SNS message bodies must be valid UTF-8, so when GZIP compression is enabled the gzipped bytes are additionally **base64-wrapped** before being placed on the queue. `verify_and_parse_sqs` (and its SNS twin) unwrap the queue envelope before verifying the signature, so the same call works whether or not compression is enabled: ```ruby require 'aws-sdk-sqs' -require 'json' require 'stream-chat' client = StreamChat::Client.new('STREAM_KEY', 'STREAM_SECRET') @@ -164,26 +159,21 @@ sqs = Aws::SQS::Client.new resp = sqs.receive_message(queue_url: ENV.fetch('STREAM_SQS_URL'), max_number_of_messages: 10) resp.messages.each do |msg| - attrs = msg.message_attributes || {} - signature = attrs.dig('X-Signature', :string_value) - content_encoding = attrs.dig('Content-Encoding', :string_value) # 'gzip' when compressed - payload_encoding = attrs.dig('Payload-Encoding', :string_value) || 'base64' + signature = msg.message_attributes&.dig('X-Signature', :string_value) - json_bytes = client.verify_and_decode_webhook( - msg.body, - signature, - content_encoding, - payload_encoding - ) - - event = JSON.parse(json_bytes) - # ...handle the event... + event = client.verify_and_parse_sqs(msg.body, signature) + # ...handle event... sqs.delete_message(queue_url: ENV.fetch('STREAM_SQS_URL'), receipt_handle: msg.receipt_handle) +rescue StreamChat::WebhookSignatureError => e + # do NOT delete the message - leave it for the DLQ / retry policy + Rails.logger.warn("Rejected Stream SQS message: #{e.message}") end ``` -The exact attribute names that carry the signature and encoding metadata may vary — refer to the SQS / SNS pages in this section for the up-to-date list. The decoding rules themselves do not change: signature is always computed over the **uncompressed** JSON. +The exact attribute name that carries the signature may vary — refer to the SQS / SNS pages in this section for the up-to-date list. The decoding rules themselves do not change: the signature is always computed over the **uncompressed** JSON. + +If you need to build a custom pipeline, the module-level primitives are also exposed: `StreamChat::Webhook.ungzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, `verify_signature`, and `parse_event`. The composite `verify_and_parse_*` methods are thin wrappers on top of these. ## Webhook types From a29a1f0573d1a8a6ac2fc86d4e8ab6a58832ca0b Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Mon, 11 May 2026 13:08:46 +0200 Subject: [PATCH 6/7] fix(webhooks): unwrap SNS notification envelope in decode_sns_payload decode_sns_payload now JSON-parses the SNS HTTP notification envelope ({"Type":"Notification","Message":"..."}) and extracts the inner Message field before running the SQS pipeline. Falls through to the pre-extracted Message string when the input is not a JSON envelope so existing call sites keep working. Spec adds a realistic SNS HTTP notification body fixture and exercises both the new envelope path and the existing pre-extracted Message path. Co-authored-by: Cursor --- lib/stream-chat/webhook.rb | 32 ++++++++++++++++++---- spec/webhook_compression_spec.rb | 47 ++++++++++++++++++++++++++++++-- 2 files changed, 71 insertions(+), 8 deletions(-) diff --git a/lib/stream-chat/webhook.rb b/lib/stream-chat/webhook.rb index 1c9d439..04f4cc0 100644 --- a/lib/stream-chat/webhook.rb +++ b/lib/stream-chat/webhook.rb @@ -75,12 +75,34 @@ def self.decode_sqs_payload(body) ungzip_payload(decoded) end - # Identical to `decode_sqs_payload`; exposed under both names so call sites - # read intent. - sig { params(message: String).returns(String) } - def self.decode_sns_payload(message) - decode_sqs_payload(message) + # Reverses an SNS HTTP notification envelope. When `notification_body` is a + # JSON envelope (`{"Type":"Notification","Message":"..."}`), the inner + # `Message` field is extracted and run through the SQS pipeline + # (base64-decode, then gzip-if-magic). When the input is not a JSON + # envelope it is treated as the already-extracted `Message` string, so + # call sites that pre-unwrap continue to work. + sig { params(notification_body: String).returns(String) } + def self.decode_sns_payload(notification_body) + inner = extract_sns_message(notification_body) + decode_sqs_payload(inner.nil? ? notification_body : inner) + end + + sig { params(notification_body: String).returns(T.nilable(String)) } + def self.extract_sns_message(notification_body) + trimmed = notification_body.to_s.lstrip + return nil unless trimmed.start_with?('{') + + begin + parsed = JSON.parse(trimmed) + rescue JSON::ParserError + return nil + end + return nil unless parsed.is_a?(Hash) + + message = parsed['Message'] + message.is_a?(String) ? message : nil end + private_class_method :extract_sns_message # Constant-time HMAC-SHA256 verification of `signature` against the digest # of `body` keyed by `secret`. diff --git a/spec/webhook_compression_spec.rb b/spec/webhook_compression_spec.rb index 9da2d8d..937aa26 100644 --- a/spec/webhook_compression_spec.rb +++ b/spec/webhook_compression_spec.rb @@ -25,6 +25,20 @@ def hmac_hex(secret, data) OpenSSL::HMAC.hexdigest('SHA256', secret, data) end + def sns_envelope(inner_message) + JSON.generate({ + 'Type' => 'Notification', + 'MessageId' => '22b80b92-fdea-4c2c-8f9d-bdfb0c7bf324', + 'TopicArn' => 'arn:aws:sns:us-east-1:123456789012:stream-webhooks', + 'Message' => inner_message, + 'Timestamp' => '2026-05-11T10:00:00.000Z', + 'SignatureVersion' => '1', + 'MessageAttributes' => { + 'X-Signature' => { 'Type' => 'String', 'Value' => '' } + } + }) + end + let(:client) { StreamChat::Client.new(api_key, api_secret) } describe 'StreamChat::Webhook.ungzip_payload' do @@ -73,11 +87,23 @@ def hmac_hex(secret, data) end describe 'StreamChat::Webhook.decode_sns_payload' do - it 'aliases decode_sqs_payload' do + it 'treats a pre-extracted Message identically to decode_sqs_payload' do wrapped = Base64.strict_encode64(gzip(json_body)) expect(StreamChat::Webhook.decode_sns_payload(wrapped)) .to eq(StreamChat::Webhook.decode_sqs_payload(wrapped)) end + + it 'unwraps a full SNS HTTP notification envelope' do + wrapped = Base64.strict_encode64(gzip(json_body)) + envelope = sns_envelope(wrapped) + expect(StreamChat::Webhook.decode_sns_payload(envelope)).to eq(json_body) + end + + it 'handles whitespace before the envelope JSON' do + wrapped = Base64.strict_encode64(gzip(json_body)) + envelope = "\n #{sns_envelope(wrapped)}" + expect(StreamChat::Webhook.decode_sns_payload(envelope)).to eq(json_body) + end end describe 'StreamChat::Webhook.verify_signature' do @@ -174,17 +200,32 @@ def hmac_hex(secret, data) end describe '#verify_and_parse_sns' do - it 'parses a base64 + gzip notification' do + it 'parses a pre-extracted base64 + gzip notification' do wrapped = Base64.strict_encode64(gzip(json_body)) sig = hmac_hex(api_secret, json_body) expect(client.verify_and_parse_sns(wrapped, sig)).to eq(event_hash) end - it 'returns the same event as verify_and_parse_sqs' do + it 'returns the same event as verify_and_parse_sqs for pre-extracted Message' do wrapped = Base64.strict_encode64(gzip(json_body)) sig = hmac_hex(api_secret, json_body) expect(client.verify_and_parse_sns(wrapped, sig)) .to eq(client.verify_and_parse_sqs(wrapped, sig)) end + + it 'parses a full SNS HTTP notification envelope' do + wrapped = Base64.strict_encode64(gzip(json_body)) + envelope = sns_envelope(wrapped) + sig = hmac_hex(api_secret, json_body) + expect(client.verify_and_parse_sns(envelope, sig)).to eq(event_hash) + end + + it 'rejects signature computed over the envelope JSON, not the payload' do + wrapped = Base64.strict_encode64(gzip(json_body)) + envelope = sns_envelope(wrapped) + sig_over_envelope = hmac_hex(api_secret, envelope) + expect { client.verify_and_parse_sns(envelope, sig_over_envelope) } + .to raise_error(StreamChat::WebhookSignatureError, 'invalid webhook signature') + end end end From 488c80ebe97e564748717e9bff2d59ba0b0e8fcd Mon Sep 17 00:00:00 2001 From: nijeesh-stream Date: Mon, 11 May 2026 15:33:06 +0200 Subject: [PATCH 7/7] refactor(webhooks): rename ungzip_payload to gunzip_payload + add golden fixtures (CHA-3071) Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip` command name. The function was added in this PR and not yet released, so this is a straight rename with no back-compat alias. Adds Tommaso's reference fixtures to the test suite as named cases so future SDKs can sanity-check against the same payloads: aGVsbG93b3JsZA== -> helloworld (base64) H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld (base64+gzip) Co-authored-by: Cursor --- .../webhooks_overview/webhooks_overview.md | 2 +- lib/stream-chat/webhook.rb | 8 +++--- spec/webhook_compression_spec.rb | 28 ++++++++++++++----- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/docs/webhooks/webhooks_overview/webhooks_overview.md b/docs/webhooks/webhooks_overview/webhooks_overview.md index e7016da..8a08cb9 100644 --- a/docs/webhooks/webhooks_overview/webhooks_overview.md +++ b/docs/webhooks/webhooks_overview/webhooks_overview.md @@ -173,7 +173,7 @@ end The exact attribute name that carries the signature may vary — refer to the SQS / SNS pages in this section for the up-to-date list. The decoding rules themselves do not change: the signature is always computed over the **uncompressed** JSON. -If you need to build a custom pipeline, the module-level primitives are also exposed: `StreamChat::Webhook.ungzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, `verify_signature`, and `parse_event`. The composite `verify_and_parse_*` methods are thin wrappers on top of these. +If you need to build a custom pipeline, the module-level primitives are also exposed: `StreamChat::Webhook.gunzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, `verify_signature`, and `parse_event`. The composite `verify_and_parse_*` methods are thin wrappers on top of these. ## Webhook types diff --git a/lib/stream-chat/webhook.rb b/lib/stream-chat/webhook.rb index 04f4cc0..b9c4cdc 100644 --- a/lib/stream-chat/webhook.rb +++ b/lib/stream-chat/webhook.rb @@ -16,7 +16,7 @@ module StreamChat # # The composite functions (`verify_and_parse_webhook`, `verify_and_parse_sqs`, # `verify_and_parse_sns`) are the recommended entry points. The primitives - # they compose (`ungzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, + # they compose (`gunzip_payload`, `decode_sqs_payload`, `decode_sns_payload`, # `verify_signature`, `parse_event`) are exposed so callers can build custom # flows or run individual steps in isolation. # @@ -50,7 +50,7 @@ def self.normalize_body(body) # handler correct when middleware - Rack, Rails - auto-decompresses the # request before your code sees it. sig { params(body: T.any(String, T::Array[Integer])).returns(String) } - def self.ungzip_payload(body) + def self.gunzip_payload(body) raw = normalize_body(body) return raw unless raw.start_with?(GZIP_MAGIC) @@ -72,7 +72,7 @@ def self.decode_sqs_payload(body) rescue ArgumentError => e raise WebhookSignatureError, "failed to base64-decode payload: #{e.message}" end - ungzip_payload(decoded) + gunzip_payload(decoded) end # Reverses an SNS HTTP notification envelope. When `notification_body` is a @@ -161,7 +161,7 @@ def self.verify_and_parse_internal(payload, signature, secret) ).returns(T::Hash[String, T.untyped]) end def self.verify_and_parse_webhook(body, signature, secret) - verify_and_parse_internal(ungzip_payload(body), signature, secret) + verify_and_parse_internal(gunzip_payload(body), signature, secret) end # Decode the SQS `Body` (base64, then gzip-if-magic), verify the HMAC diff --git a/spec/webhook_compression_spec.rb b/spec/webhook_compression_spec.rb index 937aa26..3873a66 100644 --- a/spec/webhook_compression_spec.rb +++ b/spec/webhook_compression_spec.rb @@ -41,32 +41,37 @@ def sns_envelope(inner_message) let(:client) { StreamChat::Client.new(api_key, api_secret) } - describe 'StreamChat::Webhook.ungzip_payload' do + describe 'StreamChat::Webhook.gunzip_payload' do it 'passes through plain bytes unchanged' do - expect(StreamChat::Webhook.ungzip_payload(json_body)).to eq(json_body) + expect(StreamChat::Webhook.gunzip_payload(json_body)).to eq(json_body) end it 'inflates gzip-magic bytes' do - expect(StreamChat::Webhook.ungzip_payload(gzip(json_body))).to eq(json_body) + expect(StreamChat::Webhook.gunzip_payload(gzip(json_body))).to eq(json_body) end it 'accepts a body provided as an array of integers' do - expect(StreamChat::Webhook.ungzip_payload(json_body.bytes)).to eq(json_body) + expect(StreamChat::Webhook.gunzip_payload(json_body.bytes)).to eq(json_body) end it 'returns empty input unchanged' do - expect(StreamChat::Webhook.ungzip_payload('')).to eq('') + expect(StreamChat::Webhook.gunzip_payload('')).to eq('') end it 'returns short input below magic length unchanged' do - expect(StreamChat::Webhook.ungzip_payload('ab')).to eq('ab') + expect(StreamChat::Webhook.gunzip_payload('ab')).to eq('ab') end it 'raises on truncated gzip with magic' do bad = "\x1f\x8b\x08\x00\x00\x00".b - expect { StreamChat::Webhook.ungzip_payload(bad) } + expect { StreamChat::Webhook.gunzip_payload(bad) } .to raise_error(StreamChat::WebhookSignatureError, /decompress gzip/i) end + + it 'decompresses the helloworld fixture' do + expect(StreamChat::Webhook.gunzip_payload(Base64.decode64('H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA'))) + .to eq('helloworld') + end end describe 'StreamChat::Webhook.decode_sqs_payload' do @@ -84,6 +89,15 @@ def sns_envelope(inner_message) expect { StreamChat::Webhook.decode_sqs_payload('!!!not-base64!!!') } .to raise_error(StreamChat::WebhookSignatureError, /base64-decode/i) end + + it 'decodes the helloworld base64 fixture' do + expect(StreamChat::Webhook.decode_sqs_payload('aGVsbG93b3JsZA==')).to eq('helloworld') + end + + it 'decodes the helloworld base64+gzip fixture' do + expect(StreamChat::Webhook.decode_sqs_payload('H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA')) + .to eq('helloworld') + end end describe 'StreamChat::Webhook.decode_sns_payload' do