New agent environment: an genetic search/elo rating system - #321
New agent environment: an genetic search/elo rating system#321Somasundaram-Rahul wants to merge 24 commits into
Conversation
|
This looks awesome and I look forward to testing it out soon! |
… prompts have been modified accordingly. Also, example has been updated to something simpler
…eadline will automatically lose the round
|
I've fixed the issues I raised in my previous message. This environment seems to work well on my end. The simplest way to get started with this would be the dashboard. Setting up a new 'elo' environment can be done the same way as you would set up a new symposium or teams. I'd love to hear if you have feedback on this. |
awadell1
left a comment
awadell1
left a comment
There was a problem hiding this comment.
On the code:
- A worked example that could land on https://lanl.github.io/ursa/latest/examples/ would be great. Details are here: https://lanl.github.io/ursa/latest/examples/#adding-an-example
- This seems be introducing a lot of primitives that should be on BaseEnvironment
- This really should be creating an invoking a langgraph graph, not doing lots of async calls. The current implementation is going to kill observability
- The config story and integration with the dashboard (or TUI) isn't great (somewhat out of scope. Adding a new thing requires a whole lot of modification to the dashboard code, which won't scale (See the old HITL)
- there are a lot of race-conditions here plus potential for unexpected mutation
- This runs in serial, it really should be using a map-reduce structure so that things can happen in parallel
On the environment:
The procedure seems like a stochastic sort with lots of re-evaluations. It's not clear to me why children or re-evaluations are even needed.
- Why not a deterministic evaluator (or the privatives for one)
- Anything to back the prompt structure?
- There's not a lot of configurability around the judge
- There's not really any thing preventing the judge from mutating a players config
There was a problem hiding this comment.
Can this be:
- moved to it's own folder (or subfolder)
- Get an
example.yamlfile, so it shows up on the examples page - Be more of a walkthrough and use
--<8--to import code instead of inlining it. See examples/environments/README.md
More details on adding examples are in the examples/README.md
| ```yaml | ||
| name: numerical_integration_elo | ||
| group: default | ||
| workspace: ./elo_workspace | ||
|
|
||
| generations: 2 | ||
| initial_rating: 1500 | ||
| k_factor: 32 | ||
| deaths_per_round: 1 | ||
| seed: 12345 | ||
|
|
||
| judge_prompt: > | ||
| Prefer correct, executed, reproducible numerical work. Evaluate accuracy, | ||
| convergence checks, and evidence supporting the conclusions. In later | ||
| generations, reward useful improvements and stronger validation. | ||
|
|
||
| members: | ||
| - name: researcher_1 | ||
| role: Develops and validates numerical integration methods | ||
| agent: ExecutionAgent | ||
| config: | ||
| use_web: false | ||
|
|
||
| - name: researcher_2 | ||
| role: Explores alternative methods and checks numerical accuracy | ||
| agent: ExecutionAgent | ||
| config: | ||
| use_web: false |
There was a problem hiding this comment.
For example this could be:
```yaml
--8<-- "examples/environments/agent_elo.yaml"
```| use_web: false | ||
| ``` | ||
|
|
||
| The repository also includes `examples/environments/agent_elo.yaml` and |
There was a problem hiding this comment.
These should be links
The repository also includes [agent_elo.yaml](agent_elo.yaml)...Paths are relative to the markdown file
| ## Run Elo from Python | ||
|
|
||
| ```python | ||
| from langchain.chat_models import init_chat_model |
There was a problem hiding this comment.
Ditto on the this being an --8<--
| reproduction. Cancellation may leave an already-running blocking subprocess | ||
| active until it exits or reaches its own timeout. | ||
|
|
||
| ## Workspaces, persistence, and lineage |
There was a problem hiding this comment.
I think this stuff belongs more on the docs for agent_elo, than in a worked example.
So basically, it belongs here. And the worked example (the stuff above) should go elsewhere
| else symposium_cache_dir(self.group, name) / "symposium.yaml" | ||
| ) | ||
|
|
||
| if launch.environment_type == "agent_team": |
There was a problem hiding this comment.
Should this be a BaseEnvironment method?
| """ | ||
|
|
||
|
|
||
| ELO_STARTER_YAML = """name: research_elo |
There was a problem hiding this comment.
- This defo doesn't go here
- Probably better to use importlib.resources.files
|
|
||
|
|
||
| @pytest.mark.parametrize("backup_fails", [False, True]) | ||
| def test_persistence_backup_releases_database_handles( |
There was a problem hiding this comment.
I'm not seeing anything creating these tables so idk what's being tested here
| "langchain-openai>=1.0.1", | ||
| "langgraph-checkpoint-sqlite>=3.0.3", | ||
| "aiosqlite>=0.20.0", | ||
| "aiosqlite>=0.20.0,<0.22", |
| llm=self.llm, | ||
| workspace=self.workspace, | ||
| group=self.group, | ||
| use_web=False, |
3 participants
Very much work in progress but would appreciate feedback on this. The idea is to introduce an environment where multiple agents compete with each other in match-ups. Winners get to improve their elo scores. After one iteration, winner(s) with highest elo scores get to reproduce (clones of them are made) and looser(s) with lowest elo scores are removed from the population.
Some things work well, at least when running the test
examples/environment/run_elo.py. But quite a few features are missing: the strategy for pairing up agents against each other, improve the way judging is done, etc. Also this currently does not work with the dashboard.