Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 153 additions & 0 deletions skills/ai-papers-trending/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
name: ai-papers-trending
description: Fetch trending and highly-cited AI/ML research papers via OpenAlex and Semantic Scholar APIs. Use when the user asks for trending AI papers, top-cited machine learning research, SOTA benchmark leaders, or when Papers with Code is mentioned. Papers with Code was decommissioned in 2025; this skill uses OpenAlex (concept-filtered by AI) as primary and Semantic Scholar bulk API as fallback.
---

# AI Papers Trending

Fetch trending or highly-cited AI/ML research papers. Papers with Code (paperswithcode.com) was **shut down and redirected to Hugging Face** in 2025. This skill uses two working alternatives:

- **Primary**: OpenAlex — open scholarly database, no auth, filter by AI concept + year, sort by citations.
- **Fallback**: Semantic Scholar bulk API — larger recall, no auth, sort client-side.

**Verified 2026-07-04**: OpenAlex `api.openalex.org/works` returns AI papers in ~90ms. `paperswithcode.com/api/v1/` HTTP 302 redirects to `huggingface.co/papers/trending` (dead API).

## Quick Start

### Top-cited AI papers of 2025

```bash
curl -s "https://api.openalex.org/works?filter=concepts.id:C154945302,publication_year:2025&sort=cited_by_count:desc&per_page=10"
```

`C154945302` is OpenAlex's concept ID for "Artificial intelligence". Change to:
- `C119857082` — Machine learning
- `C204321447` — Natural language processing
- `C47798520` — Computer vision
- `C50644808` — Deep learning

**Response** (JSON, verified structure):
```json
{
"meta": {
"count": 1340000,
"db_response_time_ms": 91,
"page": 1,
"per_page": 10
},
"results": [
{
"id": "https://openalex.org/W...",
"doi": "https://doi.org/10.xxxx/xxxxx",
"title": "Some LLM paper title",
"publication_date": "2025-06-15",
"cited_by_count": 342,
"authorships": [{ "author": { "display_name": "..." } }],
"concepts": [{ "display_name": "Artificial intelligence", "score": 0.99 }],
"abstract_inverted_index": { "The": [0], "paper": [1], "...": [2] },
"primary_location": { "landing_page_url": "..." }
}
]
}
```

### Reconstruct abstract from inverted index

OpenAlex stores abstracts as inverted indexes (word → positions) for anti-scraping. Rebuild:

```python
import json, subprocess

raw = subprocess.check_output([
'curl', '-s',
'https://api.openalex.org/works?filter=concepts.id:C154945302,publication_year:2025&sort=cited_by_count:desc&per_page=5'
])
data = json.loads(raw)
for w in data['results']:
idx = w.get('abstract_inverted_index') or {}
words = sorted(((pos, word) for word, positions in idx.items() for pos in positions))
abstract = ' '.join(w for _, w in words)
print(f"[⭐{w['cited_by_count']:>4}] {w['title']}")
print(f" {abstract[:200]}...")
print(f" {w.get('doi') or w.get('id')}")
print()
```

## Query Parameters

| Parameter | Purpose | Example |
|-----------|---------|---------|
| `filter=concepts.id:X` | Concept filter | `concepts.id:C154945302` (AI) |
| `filter=publication_year:X` | Year filter | `publication_year:2025` |
| `filter=type:article` | Publication type | `type:article` (excludes datasets, etc.) |
| `sort=cited_by_count:desc` | Order by citations | `cited_by_count:desc` |
| `sort=publication_date:desc` | Order by date | `publication_date:desc` |
| `per_page` | Page size (max 200) | `10`, `50` |
| `page` | 1-based page | `1`, `2` |
| `search=` | Full-text search | `search=large+language+model` |

**Combine filters with commas** (AND logic): `filter=concepts.id:C154945302,publication_year:2025`.

## Fallback: Semantic Scholar

If OpenAlex is unreachable or you need Computer Science–wide sweep:

```bash
curl -s "https://api.semanticscholar.org/graph/v1/paper/search/bulk?query=large+language+model&fieldsOfStudy=Computer+Science&fields=title,year,citationCount,url,abstract,authors&limit=100"
```

**Note**: `search/bulk` returns up to 1000 results per page but **does NOT sort by citations** — sort client-side. The regular `/paper/search` endpoint often returns HTTP 429 (rate limited).

```python
import json, subprocess
raw = subprocess.check_output([
'curl', '-s',
'https://api.semanticscholar.org/graph/v1/paper/search/bulk?query=large+language+model&fieldsOfStudy=Computer+Science&fields=title,year,citationCount,url&limit=100'
])
data = json.loads(raw)
papers = sorted(data.get('data', []), key=lambda p: p.get('citationCount') or 0, reverse=True)
for p in papers[:10]:
print(f"[⭐{p.get('citationCount', 0):>4}] {p['title']} ({p.get('year', '?')})")
```

## Common Workflows

### Recent breakthrough papers (last 6 months, high citations)

```bash
curl -s "https://api.openalex.org/works?filter=concepts.id:C154945302,from_publication_date:2026-01-01&sort=cited_by_count:desc&per_page=10"
```

### Trending topic sweep

```bash
# LLM + Agent + RAG (union of concepts)
curl -s "https://api.openalex.org/works?filter=concepts.id:C154945302,publication_year:2025,default.search:agent+llm+rag&sort=cited_by_count:desc&per_page=15"
```

## Limitations

- **Papers with Code is dead** — do not use `paperswithcode.com/api/v1/*`; it 302 redirects to HF.
- **Hugging Face Papers API works but requires Bearer Token** (free registration). Not covered here; add if the primary sources are insufficient.
- **Citation counts lag by weeks** — brand-new papers won't rank high on `cited_by_count` even if hot on Twitter.
- **OpenAlex abstracts are inverted-indexed** — must be reconstructed as shown above.
- **Semantic Scholar `/search/bulk` doesn't sort** — always sort client-side.
- **No user-agent required** for either API, but consider adding `mailto=you@example.com` to OpenAlex requests for higher rate limits ("polite pool"): `?mailto=you@example.com&filter=...`.

## Failure Modes

| Symptom | Cause | Action |
|---------|-------|--------|
| HTTP 302 to `huggingface.co/papers/trending` | You called `paperswithcode.com` | Migrate to OpenAlex |
| HTTP 429 (Semantic Scholar) | Standard endpoint rate-limited | Use `/search/bulk` instead |
| Empty `results: []` | Concept ID wrong or filter too narrow | Verify concept ID at https://api.openalex.org/concepts?search=... |
| Abstract missing | Some old papers lack it | Skip the abstract field |

## References

- OpenAlex API docs: https://docs.openalex.org/
- OpenAlex works endpoint: `https://api.openalex.org/works` (GET)
- Semantic Scholar API: https://api.semanticscholar.org/api-docs/
- Semantic Scholar bulk endpoint: `https://api.semanticscholar.org/graph/v1/paper/search/bulk` (GET)
- No authentication required for either API.
- Papers with Code (deprecated): https://paperswithcode.com/ → returns 302
139 changes: 139 additions & 0 deletions skills/arxiv-cn-daily/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
---
name: arxiv-cn-daily
description: Fetch latest AI/ML papers from arXiv by category and date. Use when the user asks for arXiv papers, latest AI research, cs.AI cs.CL cs.LG cs.CV papers, arxiv daily digest, or paper monitoring. Uses arXiv's public Atom XML API — no authentication, but weekend skipDays and 3-second rate limit apply.
---

# arXiv Daily Fetcher

Query arXiv (arxiv.org) for the latest AI/ML papers by category, date range, and keyword. Returns Atom XML with paper title, abstract, authors, and PDF link.

**Verified 2026-07-04**: `export.arxiv.org/api/query` works reliably for 15+ years. RSS returns empty on Sat/Sun (arXiv doesn't publish weekends — this is normal, not a failure).

## Quick Start

### Latest cs.AI papers

```bash
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10"
```

**Response** (Atom XML, verified):
```xml
<feed xmlns="http://www.w3.org/2005/Atom">
<opensearch:totalResults>187812</opensearch:totalResults>
<entry>
<id>http://arxiv.org/abs/2607.02514v1</id>
<title>Distributed Attacks in Persistent-State AI Control</title>
<summary>As AI coding agents become more autonomous...</summary>
<published>2026-07-02T17:59:56Z</published>
<updated>2026-07-02T17:59:56Z</updated>
<author><name>Josh Hills</name></author>
<category term="cs.AI"/>
<link href="https://arxiv.org/abs/2607.02514v1" rel="alternate" type="text/html"/>
<link href="https://arxiv.org/pdf/2607.02514v1" rel="related" type="application/pdf"/>
</entry>
</feed>
```

### Multi-category (AI + NLP + LLM + CV)

```bash
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.AI+OR+cat:cs.CL+OR+cat:cs.LG+OR+cat:cs.CV&sortBy=submittedDate&sortOrder=descending&max_results=20"
```

### Parse in Python

```python
import xml.etree.ElementTree as ET
import subprocess

xml = subprocess.check_output([
'curl', '-s',
'https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10'
])

ns = {'atom': 'http://www.w3.org/2005/Atom'}
root = ET.fromstring(xml)
for entry in root.findall('atom:entry', ns):
title = entry.find('atom:title', ns).text.strip().replace('\n', ' ')
published = entry.find('atom:published', ns).text
authors = [a.find('atom:name', ns).text for a in entry.findall('atom:author', ns)]
summary = entry.find('atom:summary', ns).text.strip()
pdf = next((l.get('href') for l in entry.findall('atom:link', ns) if l.get('type') == 'application/pdf'), None)
print(f"[{published[:10]}] {title}")
print(f" Authors: {', '.join(authors[:3])}{' et al.' if len(authors) > 3 else ''}")
print(f" PDF: {pdf}")
print(f" Abstract: {summary[:200]}...")
print()
```

## Query Syntax

| Prefix | Meaning | Example |
|--------|---------|---------|
| `cat:` | Category | `cat:cs.AI` |
| `ti:` | Title | `ti:transformer` |
| `au:` | Author | `au:LeCun` |
| `abs:` | Abstract | `abs:attention` |
| `all:` | Any field | `all:agent` |

**Boolean**: `AND`, `OR`, `ANDNOT`. Wrap in URL — spaces become `+`.

**Date range** (submitted): `submittedDate:[YYYYMMDDTTTT+TO+YYYYMMDDTTTT]` (TTTT = 24h GMT time to minute precision).

**Common AI categories**:
- `cs.AI` — Artificial Intelligence
- `cs.CL` — Computation and Language (NLP)
- `cs.LG` — Machine Learning
- `cs.CV` — Computer Vision
- `cs.NE` — Neural and Evolutionary Computing
- `stat.ML` — Machine Learning (Statistics)

## Common Workflows

### Papers submitted in the last 3 days

```bash
NOW=$(date -u +%Y%m%d0000)
THREE_DAYS_AGO=$(date -v-3d -u +%Y%m%d0000 2>/dev/null || date -u -d "3 days ago" +%Y%m%d0000)
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.AI+AND+submittedDate:%5B${THREE_DAYS_AGO}+TO+${NOW}%5D&sortBy=submittedDate&sortOrder=descending&max_results=30"
```

### Author-focused query

```bash
curl -s "https://export.arxiv.org/api/query?search_query=au:%22Yann+LeCun%22&sortBy=submittedDate&sortOrder=descending&max_results=10"
```

### Keyword within an area

```bash
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.CL+AND+all:%22large+language+model%22&sortBy=submittedDate&sortOrder=descending&max_results=20"
```

## Limitations

- **3-second rate limit** — arXiv's official policy. Enforce `sleep 3` between calls.
- **Weekend skipDays** — arXiv publishes Mon–Fri only. RSS on Sat/Sun returns empty `<channel>` (this is normal).
- **`max_results` ceiling: 2000** per call; total pagination limit: 30,000 results.
- **No server-side rate limit protection**: `max_results=500` returns 1.2 MB uncomplaining. Keep values ≤50 for latency.
- **Delay of a few hours**: newest submissions may take 30 min – 6 h to be searchable via API.
- **Atom XML, not JSON** — parse with `xml.etree.ElementTree` or `feedparser`.

## Failure Modes

| Symptom | Cause | Action |
|---------|-------|--------|
| Empty `<feed>` (no entries) on weekend | arXiv skipDays | Retry Monday, or use API (not RSS) |
| Empty `<feed>` on invalid category | Invalid `cat:` term | Verify category slug |
| HTTP timeout | High traffic / network | Retry with backoff |
| Duplicated results across pages | `sortOrder` mismatch | Ensure consistent `sortBy` + `sortOrder` |
| Feed cut off mid-entry | Response too large | Reduce `max_results` |

## References

- API docs: https://info.arxiv.org/help/api/user-manual.html
- RSS docs: https://info.arxiv.org/help/rss.html
- Category taxonomy: https://arxiv.org/category_taxonomy
- Base URL: `https://export.arxiv.org/api/query` (GET only)
- No authentication required.
Loading