Skip to content

resolveworks/mine

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

mine

A pi extension that adds a web_fetch tool for fetching web pages as clean markdown. It launches Chrome via Playwright on a virtual X display (Xvfb).

How it works

  1. Opens the target URL in a headed Chrome instance
  2. Waits for the page's load event (JS-heavy SPAs work fine)
  3. Automatically dismisses cookie consent banners using I Still Don't Care About Cookies
  4. Extracts the page content and converts it to clean markdown using Defuddle

Requirements

mine renders Chrome to a virtual X display so its window never appears on screen. Linux only. You need two things on your system:

  • Google Chrome — Playwright launches the system Chrome (channel: "chrome"). Install from google.com/chrome or via your package manager.
  • Xvfb — virtual framebuffer X server.
Distro Xvfb install
Arch sudo pacman -S xorg-server-xvfb
Debian/Ubuntu sudo apt install xvfb
Fedora/RHEL sudo dnf install xorg-x11-server-Xvfb

The extension manages the Xvfb process itself — start, display assignment, and shutdown are all automatic.

Install

pi install git:github.com/resolveworks/mine

For project-local install:

pi install git:github.com/resolveworks/mine -l

Usage

Once installed, the web_fetch tool is available to pi:

> Fetch the content of https://example.com

License

MIT

About

O(n) archive lookup

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors