Skip to content

Latest commit

 

History

18 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fetchive

Throw links at it. Get workspace-ready pages back.

An MCP server for people who collect more links than they can process. Paste a KakaoTalk export, a Slack dump, or a bare list of URLs, and Fetchive pulls out every link, skips what you already saved, reads what is behind each one — YouTube transcripts, article text — and hands your assistant a ready-to-save payload for each.

Everything it reads stays searchable, so last month's links can answer this month's question.

It does not write to your workspace. It prepares the page and lets whatever you already have connected — Notion, Obsidian, anything — do the saving. No second login, no second place for your data to sit.


Install

uvx fetchive

Or add it to your MCP client:

{
  "mcpServers": {
    "fetchive": {
      "command": "uvx",
      "args": ["fetchive"]
    }
  }
}

For local speech-to-text on videos with no captions:

uvx --with 'fetchive[stt]' fetchive

That extra needs ffmpeg on your PATH (brew install ffmpeg) and downloads a Whisper model on first use.


Using it

Say this to your assistant:

Here's my link dump from this week, archive it → (paste)

Behind that, one tool call does the work:

  1. archive_batch parses the links, skips what is already saved, extracts transcripts, and starts working in the background
  2. get_batch_status reports progress and returns the finished payloads
  3. Your assistant saves each one with your workspace tool
  4. record_archived tells Fetchive it landed, so next week's dump skips it

Later, ask it what you saved:

지난달에 RAG 청킹 얘기 어디서 봤더라?

search_archive runs full-text search over every transcript and article body it has read — not just titles — and Korean works, which the usual word tokenizer does not manage.

The other tools — ingest_links, extract_youtube_script, check_duplicate, format_for_notion, forget_archived — are the individual steps, there for when you want to inspect or repair something rather than run the whole flow.

A link it cannot read fails rather than becoming an empty page. A row that says only "here is a URL I could not open" looks like the work was done.


How transcripts actually get extracted

This is the hard part of the problem, and no single method survives YouTube for long. Fetchive tries four, in order, and stops at the first that returns words:

Order Method Cost Why it is in the chain
1 youtube-transcript-api, direct free Fastest when it works
2 yt-dlp captions free Different endpoints, so it survives blocks that stop #1
3 youtube-transcript-api via proxy cheap For when your IP itself is the problem
4 yt-dlp audio → local Whisper slow Works on videos with no captions at all

Every attempt is recorded with a reason it failed — blocked, no_captions, unavailable, too_long, and so on — so a run tells you not only what came out but which parts of the pipeline still work. Successful transcripts are cached, so asking twice is free.

Captions are chosen by what exists, not by what you hoped for: a Korean video with only Korean captions gets its Korean transcript rather than being reported as having none.


Configuration

Everything is optional.

Variable Default What it does
FETCHIVE_DB_PATH ~/.fetchive/fetchive.db Where your archive index and transcript cache live
FETCHIVE_LANGUAGES ko,en Preferred caption languages, best first
WEBSHARE_PROXY_USERNAME / _PASSWORD Enables the proxy step. Without these it is skipped
FETCHIVE_ENABLE_STT true Set to false to never fall back to speech-to-text
FETCHIVE_WHISPER_MODEL base Larger is slower and more accurate
FETCHIVE_MAX_STT_DURATION 3600 Refuse to transcribe anything longer, in seconds
FETCHIVE_BATCH_WORKERS 3 Concurrent extractions. YouTube dislikes more
FETCHIVE_QUOTE_WORD_LIMIT 15 Longest verbatim quote allowed out of the server
FETCHIVE_TRANSPORT stdio http for a hosted deployment

About copyright

Transcripts are extracted for your own reading and cached locally. Putting that text into a stored page is a different act, so what Fetchive prepares for your workspace is a summary, a word count, and at most one quote of fifteen words per source — enforced in the payload builder rather than left to whoever is calling. The link to the original always travels with it.

Fetchive is a tool for organising material you have chosen to keep. It is not for republishing anyone's work, and you remain responsible for what you do with what it extracts. If a source's terms do not permit you to store its content, do not archive it.


Your data

Everything stays on your machine, in one SQLite file. Fetchive has no server, no account, and no telemetry. It talks to YouTube to fetch transcripts, and to a proxy only if you configure one. Nothing is ever sent to Fetchive's authors.

To delete everything: rm -rf ~/.fetchive. To remove one item, call forget_archived with its URL.

Full policy: https://github.com/lsmman/fetchive/blob/main/PRIVACY.md


Development

uv sync
uv run pytest
uv run ruff check src tests

Network tests are deselected by default. To run them against real YouTube:

uv run pytest -m network

MIT licensed.

About

Throw a pile of links at it, get workspace-ready pages back — an MCP server that extracts YouTube transcripts, skips duplicates, and prepares payloads for your own workspace tool.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages