Add searxng-search: web search via self-hosted SearXNG instance - #9
Add searxng-search: web search via self-hosted SearXNG instance#9Fectivnfy112357 wants to merge 4 commits into
Conversation
Agent-plugin contribution for the community registry: plugins/Fectivnfy112357/searxng-search/
aab0f57 to
65cda37
Compare
hetaoBackend
left a comment
hetaoBackend
left a comment
There was a problem hiding this comment.
暂不符合收录门禁：
- base_url 接受任意 http://，同时脚本可能发送 Bearer/Basic Authorization；这会把凭据明文暴露在链路上。请默认强制 HTTPS，仅允许明确的 loopback HTTP（或要求用户显式 unsafe opt-in 并强警告），并补测试。
- “No third-party services / only configured SearXNG instance” 的披露不完整：客户端直连实例没错，但搜索 query 会由该实例继续发往配置的 engines/providers。请区分 direct destination 与 downstream destinations，说明 query、语言、分类等数据可能被哪些搜索引擎处理。
- 配置示例鼓励把 token/password 明文写入文件，但没有文件权限提示。请优先环境变量引用，并检查/警告过宽权限（适用平台上）。
- 请补可复现测试，覆盖 URL 校验、HTTP/HTTPS 规则、auth header、超时、非 JSON/错误响应和不在错误信息中泄露凭据。
修复后我会按新 head 复核。
…ure, env-var, tests 针对 #9 审阅的 4 条 credential / 披露 / 凭据 / 测试门禁要求做修复。 ## 1. HTTPS 强制 + URL 校验 - 新增 `validate_base_url(url, config)`: HTTPS 一律放行；HTTP 仅允许 loopback 主机（127.0.0.1 / ::1 / localhost / 0.0.0.0）; 非 loopback HTTP 需 config 中显式 `allow_insecure_http = true` 并发出强警告 （含凭据明文链路风险），否则 die。 - 在 `load_config()` 末尾调用 `validate_base_url`。 - 不再接受 ftp / file 等非 http(s) scheme。 ## 2. direct vs downstream 披露 - `README.md` "Data and network" 拆为 "Direct destination (the script itself)" 与 "Downstream destinations (the SearXNG instance, not the script)"，列出 引擎名（Google / Bing / DuckDuckGo / Brave / Baidu 等由实例运维者配置）。 - `SKILL.md` 顶部新增 "Network destinations" 节，同一区分。 - `plugin.json` description 同步更新。 ## 3. env-var 优先 + 文件权限 - `references/configuration.md` 顶部加 "Recommended: keep secrets out of the file"，env-var 示例前置；明文示例移到 "Plaintext secrets in the config" 节，并配 `chmod 600` 警告。 - 新增 `check_file_permissions(path)`：POSIX 上 `mode & 0o077` 时 warn； Windows / 缺文件不告警。 - `load_config()` 末尾调用 `check_file_permissions`。 ## 4. 可复现回归测试 - `scripts/tests/test_url_validation.py` (9): HTTPS / loopback / opt-in / non-loopback reject / 非法 scheme / 缺 host。 - `scripts/tests/test_auth_header.py` (8): bearer / basic 头构造，env-var 解析，User-Agent 引用真实仓库，token 不漏到 URL。 - `scripts/tests/test_redaction.py` (5): 5 种凭据形态 + die/warn 路径 不回显凭据。 - `scripts/tests/test_config.py` (11): 缺文件 / 坏 TOML / 缺 base_url / env-var 未设 / 文件权限告警。 - `scripts/tests/test_timeout.py` (2): 默认 30s + 配置覆盖。 - `scripts/tests/test_invalid_response.py` (5): 非 JSON / 结构化 error / HTTPError body 含凭据被 mask / URLError / TimeoutError。 - `scripts/run_tests.py` (unittest discover 入口) + `test/searxng-search.test.mjs` (node --test bridge)。 ## 顺带修 - User-Agent: `XYenon/agents` -> `Fectivnfy112357/MiniMax-Code-Plugins` - 重构 `print(..., file=sys.stderr); sys.exit(1)` 为 `die(msg)` / `warn(msg)` 助手（消除 10+ 处重复）。 - 新增 `__pycache__/` / `*.pyc` 到 plugin-local `.gitignore`。 测试结果: 40/40 OK; node --test pass 1 fail 0; validator OK
The previous implementation always ran `os.stat().st_mode & 0o077`, but on Windows `st_mode` is a synthetic value (typically 0o666) and is not derived from the file's real ACL. The docstring already promised to skip the check on non-POSIX; the implementation now matches by guarding with `os.name != "posix"`. The two POSIX-only tests now self-skip on Windows, and a new `test_skipped_on_non_posix` locks in the no-op behavior. Total test count goes from 40 to 41 (2 skipped on Windows). Found by real test against `http://textvision.top:40001/` (a SearXNG 2026.8.3 instance) where the spurious permission warning fired on every run despite the fix path not being available on Windows.
|
@hetaoBackend 谢谢这次的反馈，4 条都很到位 — credential 链路、direct vs downstream 披露、env-var 优先、可复现测试，确实都是我之前没充分考虑的部分。在 commit 1. HTTPS 强制 + URL 校验：新增 2. direct vs downstream 披露： 3. env-var 优先 + 文件权限： 4. 可复现测试：6 套 stdlib 实测： 顺带：实测时发现 Windows 上 diff 在 |
hetaoBackend
left a comment
hetaoBackend
left a comment
There was a problem hiding this comment.
@Fectivnfy112357 最新 head b31a104 仍有一个已复现的凭据泄露问题：
search.py给请求添加 Authorization 后使用默认 urllib redirect handler。测试中，初始 loopback URL 返回跨端口 302 时，Bearer token 被原样转发到第二个 origin，也可绕过初始 HTTPS/loopback 校验。请禁用 redirect，或逐跳限制同 origin、禁止 HTTPS→HTTP，并补回归测试。- 根测试固定执行
python，在只有python3的受支持环境中npm run check为 26/27；请做跨平台解释器探测。 - 当前形状脱敏不能覆盖实际短 token，服务端 body/error 可能回显配置 secret。请对本次实际 secret 做精确替换，认证错误不要直接回显 body。
修复以上 P1 并跑出 CI/CodeQL 后再复审。
…ion, portable tests Round-2 P1 fixes from reviewer hetaoBackend: - Block all HTTP redirects (_SafeRedirectHandler, replaces urllib's default handler): a 30x response is an error with a safe diagnostic, so the Authorization header can never be forwarded to a different origin or downgraded HTTPS->HTTP. Regression tests + live loopback cross-port 302 check. - Exact-secret redaction: register_secret()/reset_active_secrets() track the actual token/password/base64 used for the request; redact_secrets substitutes them by value before shape-based fallback, catching short or non-shape tokens. 401/403/407 response bodies are suppressed entirely so a server that reflects credentials cannot leak them. - Cross-platform interpreter discovery (py -3 / python / python3, each verified via --version) with an explicit failure when none is found, instead of a misleading 26/27 pass. The node bridge now lives inside the Plugin (plugins/<owner>/<plugin>/test/) so the contribution stays self-contained; the repo-root bridge was removed. Additional hardening found in self-review: - Cap response/error bodies at 10 MiB (MAX_RESPONSE_BYTES). - Validate numeric config fields (timeout, default_max_results) so a string value exits cleanly instead of raising a raw TypeError. - Require a JSON object root from the instance; reject non-UTF-8 bodies with a clean diagnostic instead of a traceback. - Custom config headers now resolve $ENV_VAR references and register resolved values for exact redaction, matching auth.* handling. - Remove unused info() helper; add close() to the HTTPError fixture to silence urllib addbase GC cleanup noise. Docs: SKILL.md/README disclose that redirects are never followed; configuration.md documents $ENV_VAR in headers and a redirect-blocked troubleshooting entry. Test suite: 77 stdlib unittest cases (2 POSIX-only skips), wired into npm test via the in-plugin node bridge; validate.mjs passes.
|
@hetaoBackend 谢谢这次反馈，3 条 P1 都收到，已在 commit 1. redirect 凭据泄露： 2. 跨平台解释器探测：桥接改为按平台探测——Windows 优先 3. 精确脱敏 + 认证错误不回显 body： 顺带加固（自查发现，不在您原话内）：
测试结果： |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
2 participants
What it solves
Web search inside an agent usually means a vendor API with credentials and
quotas. If you already run SearXNG (a privacy-respecting metasearch engine),
this skill gives the agent a first-class search tool against it — one config
file, no vendor SDK, no tracking.
Try it
Install from
/plugins→ Local, configure your instance, then ask:Expected result: a formatted list of results (title, URL, snippet) from
your SearXNG instance, filtered by the requested category / time range /
language.
Direct usage:
Dependencies and platforms
provides it; no public instance is bundled.
~/.config/agents/searxng.toml(or$XDG_CONFIG_HOME);all fields documented in
skills/searxng-search/references/configuration.md.Network and data
base_urlin the local config). No other network destinations.config file; never sent anywhere else.
Test evidence
npm run validate→OK plugin Fectivnfy112357/searxng-search(categories, time range, language and pagination paths in active use).