Skip to content

docs(lark-drive): add answer-oriented search refinement#773

Draft
BytedanceSearch wants to merge 1 commit into
mainfrom
eval-search-auto-pr-2026-05-07T11-35
Draft

docs(lark-drive): add answer-oriented search refinement#773
BytedanceSearch wants to merge 1 commit into
mainfrom
eval-search-auto-pr-2026-05-07T11-35

Conversation

@BytedanceSearch
Copy link
Copy Markdown
Collaborator

Summary

Adds answer-oriented search refinement guidance to drive +search.

The change is intentionally scoped to one search strategy doc:

  • Preserve hard constraints such as business entities, time, product names, and numeric terms.
  • Add query expansions by question type, such as case, policy, metric, and integration questions.
  • Fetch strong candidates instead of stopping after one raw query.
  • Keep suspected eval/process artifacts observable rather than hidden.
  • Require final answers to fill the key slot for the question type.

Eval Search Run

Pollution handling:

  • Tainted search results observed: 43
  • Non-evidential results excluded: 43
  • Tainted fetches: 0
  • Contamination penalty cases: 0

Findings Addressed

  • F-001 / search_strategy / high: add type-aware blind refinement for short queries, metrics, launch summaries, customer cases, and policy searches.
  • F-003 / skill_prompts / high: require slot-complete final answers instead of long source excerpts.

Unhandled:

  • F-002 / tool_capability / medium: message applinks, external webpages, and file/PDF fetch support remain collector/tool capability gaps.

Checks

  • git diff --check
  • node --experimental-strip-types tests/eval-search/eval-search-run.ts --help
  • node --experimental-strip-types tests/eval-search/eval-search-collect-search.ts --help
  • JSON parse for generated summary.json, verdicts.json, and executor_search.json

Change-Id: I85385a12fc24f804f66e9ac7afa1a33a73936eb5
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions github-actions Bot added the size/M Single-domain feat or fix with limited business impact label May 7, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 40f304da-6677-4d52-9c7d-049c1cc03f58

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch eval-search-auto-pr-2026-05-07T11-35

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.40%. Comparing base (7c68639) to head (75f2aec).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #773   +/-   ##
=======================================
  Coverage   65.40%   65.40%           
=======================================
  Files         508      508           
  Lines       46795    46795           
=======================================
  Hits        30605    30605           
  Misses      13548    13548           
  Partials     2642     2642           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Single-domain feat or fix with limited business impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants