Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,10 @@ Extract and Batch scrape. It calls the Spicrawl API directly, not the MCP server
API key (with the `scrape` scope, plus `batch` for Batch scrape) in the plugin's authorization
settings.

[![Spicrawl for Dify: install, add your key, then Scrape, Extract and Batch scrape in workflows and agents](assets/dify-demo.webp)](https://docs.spicrawl.com/agents/dify)

Watch the full video on the [Dify docs page](https://docs.spicrawl.com/agents/dify), or download [`assets/dify-demo.mp4`](assets/dify-demo.mp4).

1. In Dify open Plugins, Marketplace, search for Spicrawl and select Install, or open the
[Spicrawl listing](https://marketplace.dify.ai/plugin/spicrawl/spicrawl). It needs Dify 1.14.2
or later.
Expand Down
Binary file added assets/dify-demo.mp4
Binary file not shown.
Binary file added assets/dify-demo.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 0 additions & 18 deletions dify/_assets/icon-dark.svg

This file was deleted.

4 changes: 2 additions & 2 deletions dify/_assets/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 1 addition & 2 deletions dify/manifest.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version: 0.1.4
version: 0.1.5
type: plugin
author: spicrawl
name: spicrawl
Expand All @@ -11,7 +11,6 @@ description:
zh_Hans: Dify 网页抓取工具:将任意网址抓取为适合 LLM、RAG 和 AI 智能体使用的干净 Markdown,支持 JavaScript 渲染、字段提取,以及一次最多 100 个网址的批量抓取。
ja_JP: Dify 用 Web スクレイパー。任意の Web サイトを LLM・RAG・AI エージェント向けのクリーンな Markdown に変換。JavaScript レンダリング、フィールド抽出、最大 100 件の URL の一括取得に対応。
icon: icon.svg
icon_dark: icon-dark.svg
resource:
memory: 268435456
plugins:
Expand Down
1 change: 0 additions & 1 deletion dify/provider/spicrawl.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ identity:
zh_Hans: Dify 网页抓取工具:将任意网址抓取为适合 LLM、RAG 和 AI 智能体使用的干净 Markdown,支持 JavaScript 渲染、字段提取,以及一次最多 100 个网址的批量抓取。
ja_JP: Dify 用 Web スクレイパー。任意の Web サイトを LLM・RAG・AI エージェント向けのクリーンな Markdown に変換。JavaScript レンダリング、フィールド抽出、最大 100 件の URL の一括取得に対応。
icon: icon.svg
icon_dark: icon-dark.svg
tags:
- search
- utilities
Expand Down
2 changes: 1 addition & 1 deletion dify/tools/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import requests

API_BASE = "https://api.spicrawl.com"
USER_AGENT = "spicrawl-dify-plugin/0.1.4"
USER_AGENT = "spicrawl-dify-plugin/0.1.5"
# Target statuses Spicrawl bills as a success; anything else is returned at 0 credits.
BILLED_STATUSES = (200, 404, 410)

Expand Down
Loading