Fix token-list parsing for hop-by-hop request headers

[CypherPotato]

Motivation

• A recent change replaced token-list checks with exact whole-value comparisons for hop-by-hop headers, causing valid comma-separated values such as Transfer-Encoding: gzip, chunked to be mis-parsed and leaving Content-Length trusted.
• This regression can lead to downstream framing disagreements and enable HTTP request smuggling in proxy/backend mismatches, so the parser must treat header values as comma-separated token lists per RFC semantics.

Description

• Restored token-list matching semantics by introducing a helper TokenListContains(ReadOnlySpan<byte> value, ReadOnlySpan<byte> expectedToken) that scans comma-separated tokens with trimming and case-insensitive comparison.
• Replaced exact-value checks with token-list checks for the Connection, Expect, and Transfer-Encoding headers inside HttpRequestReader so Connection: close, Expect: 100-continue, and Transfer-Encoding: ..., chunked are recognized correctly.
• Preserved existing behavior of clearing ContentLength (contentLength = -1) when a chunked transfer-coding is observed and kept the duplicate-TE and Host validations intact in HttpRequestReader.

Testing

• Ran dotnet test initially against an incorrect project path which failed because the specified project file did not exist.
• Ran dotnet test tests/Sisk.Core/tests.csproj --filter CadenteEngineRareTests --nologo, and the project restore/build completed successfully.
• Test execution was aborted because the test host requires .NETCore.App 9.0.0 which is not installed in this environment, so tests could not be executed here (build succeeded, but test runtime was unavailable).

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9b909f95f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reject non-final chunked transfer coding

TokenListContains treats any occurrence of chunked as sufficient, so a request like Transfer-Encoding: chunked, gzip will be accepted as chunked (isChunked = true) instead of rejected. In HTTP/1.1 this is invalid framing (chunked must be the final transfer-coding), and accepting it can create request-boundary mismatches with intermediaries/backends that follow RFC 9112 and return 400, which is a request-smuggling risk on reused connections.

Useful? React with 👍 / 👎.

[CypherPotato]

@codex address those comments

[chatgpt-codex-connector]

Summary

• Addressed the review feedback by tightening Transfer-Encoding parsing: requests are now rejected when chunked appears but is not the final transfer-coding token (for example, chunked, gzip). This is implemented in the case 3 header handling path with a parse-fail reason and early rejection.
• Added a new helper TokenListEndsWith(ReadOnlySpan<byte>, ReadOnlySpan<byte>) to validate the last comma-separated transfer-coding token (with trimming + case-insensitive comparison).
• Kept the existing behavior that sets ContentLength = -1 when a valid chunked transfer-coding is recognized.

Testing

• ⚠️ dotnet test tests/Sisk.Core/tests.csproj --filter CadenteEngineRareTests --nologo (restore/build succeeded, but test execution aborted because Microsoft.NETCore.App 9.0.0 is not installed in this environment).

Git / PR actions performed

• Committed on current branch: Reject non-final chunked transfer-coding in requests (commit 3d28151).
• Created follow-up PR metadata via make_pr with title: “Follow-up: reject non-final chunked transfer-coding in request parsing”.

View task →