Skip to content

feat: add optional You.com search integration - #165

Open
mouse-value-add wants to merge 2 commits into
alphaXiv:mainfrom
mouse-value-add:feat/youcom-web-search
Open

feat: add optional You.com search integration#165
mouse-value-add wants to merge 2 commits into
alphaXiv:mainfrom
mouse-value-add:feat/youcom-web-search

Conversation

@mouse-value-add

Copy link
Copy Markdown

Summary

Add You.com web search as an optional literature source alongside existing alphaXiv, OpenAlex, and bioRxiv sources.

What Changed

  • New literature source: --source youcom for web search capability
  • API integration: Uses You.com search API with optional authentication via YDC_API_KEY
  • Consistent interface: Follows existing CLI patterns: orx lit --source youcom <query>
  • Uniform output: Web results mapped to LitHit format for consistent JSON output
  • Graceful fallback: Uses keyless API when no API key provided

Why Useful

Research often requires context beyond traditional academic papers:

  • Recent developments not yet indexed in academic databases
  • Industry research, blog posts, and technical discussions
  • Broader context around research topics
  • Current community discussions in ML/AI

Usage Examples

# Search current ML research
orx lit "transformer attention mechanisms" --source youcom --limit 5

# Get JSON output for programmatic use  
orx lit "neural networks 2024" --source youcom --json

# View a specific web result
orx paper "https://example.com/article" --source youcom

Environment Setup

# Optional: Set API key for enhanced results
export YDC_API_KEY="your_api_key_here"

# Works without API key using keyless endpoint
orx lit "machine learning" --source youcom

Integration Quality

  • ✅ Compiles and builds successfully
  • ✅ Follows existing code patterns and style
  • ✅ Maintains backward compatibility
  • ✅ Handles errors gracefully (network, API failures)
  • ✅ Optional integration (existing users unaffected)
  • ✅ Tested with both authenticated and keyless API access

Technical Details

  • Uses reqwest HTTP client (already in dependencies)
  • Consistent error handling with existing sources
  • URL-based IDs for web results (makes sense for web content)
  • Falls back to keyless API when no authentication provided
  • Web results don't have publication dates or citations (appropriately marked as None)

The integration is designed to complement, not replace, existing academic sources. It provides researchers with access to current web content while maintaining the same familiar interface.

Tracking

Related to issue: youdotcom-oss/integration-tracking#175

- Add YouCom as a new literature source alongside alphaXiv, OpenAlex, and bioRxiv
- Integrate You.com web search API for current research and broader context
- Optional integration via YDC_API_KEY environment variable (falls back to keyless API)
- Consistent with existing CLI interface: orx lit --source youcom <query>
- Web results mapped to LitHit format for uniform JSON output
- Added handling for You.com URLs in paper command

This enables researchers to search current web content and recent developments
alongside traditional academic sources, particularly useful for:
- Recent research developments not yet in academic indexes
- Industry research and blog posts
- Broader context around research topics
- Current discussions in ML/AI community
…subcommands

Resolves the conflict with the literature-command refactor: You.com is
now a "discover youcom" subcommand (matching Keyword/Embedding/Openalex/
Biorxiv) instead of a "lit --source" flag. Also removes an
accidentally-committed rustup installer script and local test notes.

Verified: cargo build, cargo test (690 passed), and a live smoke test of
"orx discover youcom" against the keyless endpoint returning real
results.
@mouse-value-add

Copy link
Copy Markdown
Author

Resolved the conflict with the recent literature-command refactor and rebased the integration onto the new architecture (0fde75b):

  • You.com is now a discover youcom subcommand, matching the Keyword/Embedding/Openalex/Biorxiv structure, instead of the old lit --source youcom flag that no longer exists.
  • orx paper <url> still recognizes You.com results by URL.
  • Dropped an accidentally-committed rustup installer script and local test notes that shouldn't have been in the diff.

Verified on the branch: cargo build, cargo test (690 passed, 0 failed), and a live smoke test of orx discover youcom "attention mechanisms in transformers" --limit 2 against the keyless endpoint, which returned real results mapped into the LitHit JSON shape.

The integration stays fully opt-in: it only runs when explicitly invoked as discover youcom, uses YDC_API_KEY bearer auth when set, and falls back to the keyless endpoint otherwise. Happy to adjust the subcommand naming or flag set if you'd prefer a different shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant