Skip to content

Smart/Agentic Crawler (Invite Collaboration) #227

Description

@unclecode

I'm planning to add a smart crawler that takes a set of user-defined objectives and continues crawling to satisfy them. Objectives can be a query requiring a sufficient amount of information to answer, a set of actions with specific goals, or other custom tasks. I also plan to incorporate reinforcement learning into this process later. I've created an issue for discussion and invite interested individuals to share ideas and comments. If you're enthusiastic about contributing, please share your email address, and I'll invite you to our Discord channels to collaborate.

Happy Crawling

Activity

self-assigned this
on Nov 4, 2024

zhixiongtan commented on Nov 5, 2024

@zhixiongtan

zhixiongtan94@gmail.com
THANKS FOR YOUR WORK

unclecode commented on Nov 6, 2024

@unclecode
OwnerAuthor

@zhixiongtan sent :)

wakaka6 commented on Dec 18, 2024

@wakaka6
Contributor

unclecode commented on Dec 25, 2024

@unclecode
OwnerAuthor

Sending the links @wakaka6

wakaka6 commented on Dec 25, 2024

@wakaka6
Contributor

Sending the links   发送链接@wakaka6

@unclecode Hi bro, I didn't get the invite link in my email. Also for the sake of other people contributing code better, it is recommended to create a way for git-hook and github action to use ruff and mypy to check and format the code as a way to ensure the style of the code and avoid confusion caused by editor formatting plugins used by different people.

unclecode commented on Dec 26, 2024

@unclecode
OwnerAuthor

@wakaka6 Already sent. and Thank you so much for your suggestion. It has been in our plans for a while. I wanted to engage with everything, like the raw material, to familiarize myself with how people interact with the library and gather ideas for building smart collaboration structures. I completely agree that Ruff is blazing fast, and I believe MyPy is the de facto standard. I love the pre-commit feature because it allows users to check their work on their own machines before submission. We're definitely working on it, and I hope to have it ready by January. May be you join and help me on this too.

berkaygkv commented on Jan 16, 2025

@berkaygkv

This is what I was just thinking. I would be glad to contribute. @unclecode
berkaygokova@gmail.com

unclecode commented on Jan 17, 2025

@unclecode
OwnerAuthor

You are most welcome @berkaygkv

aravindkarnam commented on Jan 17, 2025

@aravindkarnam
Collaborator

@berkaygkv Sent you an invite link! Check your email.

Repository owner locked and limited conversation to collaborators on Jan 17, 2025
converted this issue into a discussion #476 on Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

✨ EnhancementImprovement on an existing feature

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions