feat: run tests on apache only

[theseion]

No description provided.

Add a comment

[theseion]

I'm debugging the failing test.

[theseion]

@azurit I've updated your test to properly work with albedo. I've attached the output of go-ftw below. As you can see, the response is reflected as expected, but the plugin doesn't seem to kick in. To me it looks as if the variable is empty. Output from debug log:

[02/Jun/2024:17:39:42.804830 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][4] Recipe: Invoking rule ffff8e34bb50; [file "/etc/modsecurity.d/owasp-crs/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf"] [line "95"] [id "953120"].
[02/Jun/2024:17:39:42.804959 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][5] Rule ffff8e34bb50: SecRule "RESPONSE_BODY|TX:RESPONSE_BODY_DECOMPRESSED" "@rx (?i)<\\?(?:=|php)?\\s+" "phase:4,log,tag:modsecurity,id:953120,block,capture,t:none,msg:'PHP source code leakage',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}',tag:application-multi,tag:language-php,tag:platform-multi,tag:attack-disclosure,tag:paranoia-level/1,tag:OWASP_CRS,tag:capec/1000/118/116,tag:PCI/6.5.6,ver:OWASP_CRS/4.4.0-dev,severity:ERROR,setvar:tx.outbound_anomaly_score_pl1=+%{tx.error_anomaly_score}"
[02/Jun/2024:17:39:42.805043 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][4] Transformation completed in 0 usec.
[02/Jun/2024:17:39:42.805124 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][4] Executing operator "rx" with param "(?i)<\\?(?:=|php)?\\s+" against RESPONSE_BODY.
[02/Jun/2024:17:39:42.805207 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][9] Target value: "OK"
[02/Jun/2024:17:39:42.805288 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][4] Operator completed in 0 usec.
[02/Jun/2024:17:39:42.805373 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][4] Rule returned 0.
[02/Jun/2024:17:39:42.805453 +0000] [localhost/sid#ffff93eb9018][rid#ffff942120a0][/healthz][9] No match, not chained -> mode NEXT_RULE.

�[90m7:34PM�[0m �[34mTRC�[0m ftw/test: using glob pattern tests/regression/**/*.yaml
�[90m7:34PM�[0m �[34mTRC�[0m ftw/test: found 1 files matching pattern
�[90m7:34PM�[0m �[34mTRC�[0m ftw/output: creating output normal

🛠️  Starting tests!
🚀 Running go-ftw!
👉 executing tests in file 9503110.yaml
	running 9503110-1: �[90m7:34PM�[0m �[34mTRC�[0m Writing header: Accept: */*

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Connection: close

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Host: localhost

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: User-Agent: go-ftw test agent

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: X-CRS-Test: 305fcd37-24ec-450f-a4f8-7279935d9088

�[90m7:34PM�[0m DBG ftw/http: sending data:
GET /status/200 HTTP/1.1
Accept: */*
Connection: close
Host: localhost
User-Agent: go-ftw test agent
X-CRS-Test: 305fcd37-24ec-450f-a4f8-7279935d9088



�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: sending data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: receiving data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: received data - "HTTP/1.1 200 OK\r\nDate: Sun, 02 Jun 2024 17:34:34 GMT\r\nServer: Apache\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Connection: close

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Content-Length: 103

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Content-Type: application/json

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Host: localhost

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: User-Agent: OWASP CRS test agent

�[90m7:34PM�[0m DBG ftw/http: sending data:
POST /reflect HTTP/1.1
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Connection: close
Content-Length: 103
Content-Type: application/json
Host: localhost
User-Agent: OWASP CRS test agent

{
  "encodedBody": "H4sIAJepXGYAA7OxL8goUAAAGCqh2AYAAAA=",
  "headers": {"Content-Encoding": "gzip"}
}


�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: sending data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: receiving data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: received data - "HTTP/1.1 200 OK\r\nDate: Sun, 02 Jun 2024 17:34:34 GMT\r\nServer: Apache\r\nContent-Encoding: gzip\r\nContent-Length: 26\r\nConnection: close\r\n\r\n\x1f\x8b\b\x00\x97\xa9\\f\x00\x03\xb3\xb1/\xc8(P\x00\x00\x18*\xa1\xd8\x06\x00\x00\x00"
�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Accept: */*

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Connection: close

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: Host: localhost

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: User-Agent: go-ftw test agent

�[90m7:34PM�[0m �[34mTRC�[0m Writing header: X-CRS-Test: 305fcd37-24ec-450f-a4f8-7279935d9088

�[90m7:34PM�[0m DBG ftw/http: sending data:
GET /status/200 HTTP/1.1
Accept: */*
Connection: close
Host: localhost
User-Agent: go-ftw test agent
X-CRS-Test: 305fcd37-24ec-450f-a4f8-7279935d9088



�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: sending data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: receiving data
�[90m7:34PM�[0m �[34mTRC�[0m ftw/http: received data - "HTTP/1.1 200 OK\r\nDate: Sun, 02 Jun 2024 17:34:34 GMT\r\nServer: Apache\r\nContent-Length: 0\r\nConnection: close\r\n\r\n"
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: got 5 lines
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: Matching id "953120" in 
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: Matching id "953120" in [Sun Jun 02 17:34:34.980212 2024] [security2:error] [pid 68:tid 281472451408160] [client 192.168.65.1:17112] [client 192.168.65.1] ModSecurity: Warning. Unconditional match in SecAction. [file "/etc/modsecurity.d/owasp-crs/rules/RESPONSE-980-CORRELATION.conf"] [line "98"] [id "980170"] [msg "Anomaly Scores: (Inbound Scores: blocking=10, detection=10, per_pl=5-5-0-0, threshold=5) - (Outbound Scores: blocking=0, detection=0, per_pl=0-0-0-0, threshold=4) - (SQLI=0, XSS=0, RFI=0, LFI=0, RCE=10, PHPI=0, HTTP=0, SESS=0, COMBINED_SCORE=10)"] [ver "OWASP_CRS/4.4.0-dev"] [tag "modsecurity"] [tag "reporting"] [tag "OWASP_CRS"] [hostname "localhost"] [uri "/reflect"] [unique_id "ZlytKuVOXUzFhkDXKf-eQwAAAJE"]
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: Matching id "953120" in [Sun Jun 02 17:34:34.977950 2024] [security2:error] [pid 68:tid 281472451408160] [client 192.168.65.1:17112] [client 192.168.65.1] ModSecurity: Warning. Operator GE matched 5 at TX:blocking_inbound_anomaly_score. [file "/etc/modsecurity.d/owasp-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "233"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 10)"] [ver "OWASP_CRS/4.4.0-dev"] [tag "modsecurity"] [tag "anomaly-evaluation"] [tag "OWASP_CRS"] [hostname "localhost"] [uri "/reflect"] [unique_id "ZlytKuVOXUzFhkDXKf-eQwAAAJE"]
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: Matching id "953120" in [Sun Jun 02 17:34:34.975754 2024] [security2:error] [pid 68:tid 281472451408160] [client 192.168.65.1:17112] [client 192.168.65.1] ModSecurity: Warning. Pattern match "(?i)(?:^|b[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0- ..." at ARGS:headers.Content-Encoding. [file "/etc/modsecurity.d/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "1374"] [id "932236"] [msg "Remote Command Execution: Unix Command Injection (command without evasion)"] [data "Matched Data: gzip found within ARGS:headers.Content-Encoding: gzip"] [severity "CRITICAL"] [ver "OWASP_CRS/4.4.0-dev"] [tag "modsecurity"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/2"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/reflect"] [unique_id "ZlytKuVOXUzFhkDXKf-eQwAAAJE"]
�[90m7:34PM�[0m �[34mTRC�[0m ftw/waflog: Matching id "953120" in [Sun Jun 02 17:34:34.974885 2024] [security2:error] [pid 68:tid 281472451408160] [client 192.168.65.1:17112] [client 192.168.65.1] ModSecurity: Warning. Pattern match "(?i)(?:^|b[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?u[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0-9\\\\?@_a-\\\\{]*)?\\\\x5c?s[\\"'\\\\)\\\\[\\\\x5c]*(?:(?:(?:\\\\|\\\\||&&)[\\\\s\\\\x0b]*)?\\\\$[!#\\\\(\\\\*\\\\-0- ..." at ARGS:headers.Content-Encoding. [file "/etc/modsecurity.d/owasp-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf"] [line "498"] [id "932260"] [msg "Remote Command Execution: Direct Unix Command Execution"] [data "Matched Data: gzip found within ARGS:headers.Content-Encoding: gzip"] [severity "CRITICAL"] [ver "OWASP_CRS/4.4.0-dev"] [tag "modsecurity"] [tag "application-multi"] [tag "language-shell"] [tag "platform-unix"] [tag "attack-rce"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/248/88"] [tag "PCI/6.5.2"] [hostname "localhost"] [uri "/reflect"] [unique_id "ZlytKuVOXUzFhkDXKf-eQwAAAJE"]
💥 failed in 18.064541ms (RTT 58.576375ms)
➕ run 1 total tests in 18.064541ms
⏭  skipped 0 tests
👎 1 test(s) failed to run: ["9503110-1"]
Error: failed 1 tests

[azurit]

@theseion Is it doing the same now? Can you post debug output? Thanks.

[azurit]

I rechecked everything and it should work.

[theseion]

Yes, still the same output:

[02/Jun/2024:18:24:41.180478 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][4] Recipe: Invoking rule ffff7a075b50; [file "/etc/modsecurity.d/owasp-crs/rules/RESPONSE-953-DATA-LEAKAGES-PHP.conf"] [line "95"] [id "953120"].
[02/Jun/2024:18:24:41.180580 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][5] Rule ffff7a075b50: SecRule "RESPONSE_BODY|TX:RESPONSE_BODY_DECOMPRESSED" "@rx (?i)<\\?(?:=|php)?\\s+" "phase:4,log,tag:modsecurity,id:953120,block,capture,t:none,msg:'PHP source code leakage',logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}',tag:application-multi,tag:language-php,tag:platform-multi,tag:attack-disclosure,tag:paranoia-level/1,tag:OWASP_CRS,tag:capec/1000/118/116,tag:PCI/6.5.6,ver:OWASP_CRS/4.4.0-dev,severity:ERROR,setvar:tx.outbound_anomaly_score_pl1=+%{tx.error_anomaly_score}"
[02/Jun/2024:18:24:41.180675 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][4] Transformation completed in 0 usec.
[02/Jun/2024:18:24:41.180766 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][4] Executing operator "rx" with param "(?i)<\\?(?:=|php)?\\s+" against RESPONSE_BODY.
[02/Jun/2024:18:24:41.180863 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][9] Target value: ""
[02/Jun/2024:18:24:41.180955 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][4] Operator completed in 1 usec.
[02/Jun/2024:18:24:41.181039 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][4] Rule returned 0.
[02/Jun/2024:18:24:41.181129 +0000] [localhost/sid#ffff7fc045e8][rid#ffff7ff600a0][/reflect][9] No match, not chained -> mode NEXT_RULE.

[azurit]

I see this in the log:
Executing operator "rx" with param "(?i)<\\?(?:=|php)?\\s+" against RESPONSE_BODY.

But then it is not executing it against TX:RESPONSE_BODY_DECOMPRESSED.