Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .codedna
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,6 @@ packages:
cross_cutting_patterns: {}

agent_sessions:
- agent: "gpt-5"
provider: "openai"
date: "2026-08-21"
session_id: "s_20260821_gitbook_docs"
task: "Redesign documentation like GitBook"
changed: ["docs/install.html","docs/site.css",".codedna"]
visited: ["docs/install.html","docs/site.css",".codedna"]
message: "Reworked the documentation shell with a GitBook-inspired three-column layout, searchable hierarchical sidebar, keyboard shortcut, breadcrumb, page metadata, sticky outline, pagination, feedback, and responsive spacing. HTML/CSS checks, Ruff, and 284 tests pass."
- agent: "gpt-5"
provider: "openai"
date: "2026-08-21"
Expand Down Expand Up @@ -70,3 +62,11 @@ agent_sessions:
changed: ["codedna_tool/languages/php.py","codedna_tool/languages/base.py","codedna_tool/languages/vbnet.py","codedna_tool/languages/__init__.py","codedna_tool/cli.py","integrations/.roorules","integrations/install.sh","integrations/README.md","docs/languages.md","README.md","README-it.md","CHANGELOG.md","tests/test_issues_4_6_7.py","tests/test_docs.py"]
visited: ["codedna_tool/languages/php.py","codedna_tool/languages/base.py","codedna_tool/languages/csharp.py","codedna_tool/languages/__init__.py","codedna_tool/cli.py","integrations/.clinerules"]
message: "Validated #4/#6/#7 still open. Fixed PHP mixed-HTML injection + full-file has_codedna_header; added VbNetAdapter; added Roo --tools roo. 306 tests pass."
- agent: "composer-2.5"
provider: "cursor"
date: "2026-09-08"
session_id: "s_20260908_001"
task: "Challenge honesty + experimental notes"
changed: ["docs/challenge.md","docs/challenge.it.md","docs/challenge.html","challenge/SUBMISSION_TEMPLATE.md","challenge/SUBMISSION_TEMPLATE.it.md","README.md","README-it.md"]
visited: ["docs/challenge.md","docs/challenge.it.md","docs/challenge.html","challenge/SUBMISSION_TEMPLATE.md","challenge/SUBMISSION_TEMPLATE.it.md","README.md","README-it.md"]
message: "Documented experimental agent×language caveat + honesty/Meet review + public lives in challenge rules."
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ All notable changes to CodeDNA will be documented in this file.

### Added

- **CodeDNA Challenge (€200)** — bilingual rules, required `metrics.json` (languages + frameworks + size/files), docs site page + static board (`docs/challenge.html`, `docs/challenge-board.json`). PR opens enrollment (no separate signup). Separate from historical SWE-bench benchmarks.
- **CodeDNA Challenge (€200)** — bilingual rules, required `metrics.json` (languages + frameworks + size/files ≥25 + `install` + `setup.layout` + `bugs_reported`), optional file-localization metrics + optional judge-agent comparison, real-project bar (no toy sites), same-task control vs CodeDNA via two branches/checkouts/twin projects, copy-paste agent/judge prompts + task list, Meet presentation verification, docs site page + live public board (IT default) updated as valid PRs arrive. PR opens enrollment (no separate signup). Separate from historical SWE-bench benchmarks.

### Added

Expand Down
4 changes: 2 additions & 2 deletions README-it.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ Nessuna infrastruttura. Nessuna pipeline di retrieval. Nessuna memoria esterna.

## CodeDNA Challenge — €200

Valutazione pubblica sul **tuo** progetto (non SWE-bench). Almeno 10 task con/senza CodeDNA, stack L0/L1/L2 a pari livello, PR con metriche — anche se vanno contro CodeDNA.
Valutazione pubblica sul **tuo** progetto (non SWE-bench). Progetto reale funzionante (≥25 file), stack tecnologico + install CodeDNA dichiarati, stessa metodologia per tutti (due branch / checkout / progetti gemelli), ≥10 **stessi** task con/senza CodeDNA, [prompt da copiare per l’agente](docs/challenge-agent-prompt.it.md), PR con metriche — anche se vanno contro CodeDNA. I bug vanno segnalati (sperimentale tra agenti). Possibile presentazione Meet; la bacheca pubblica si aggiorna quando arrivano PR valide.

- Bacheca: [larens94.github.io/codedna/challenge.html](https://larens94.github.io/codedna/challenge.html)
- Classifica / bacheca: [larens94.github.io/codedna/challenge.html](https://larens94.github.io/codedna/challenge.html)
- Regolamento: [`docs/challenge.it.md`](docs/challenge.it.md) (IT) · [`docs/challenge.md`](docs/challenge.md) (EN)
- Submission: `challenge/<handle>/metrics.json` ([esempio](challenge/metrics.example.json)) — **aprire la PR ti iscrive**

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ No infrastructure. No retrieval pipeline. No external memory. The code carries i

## CodeDNA Challenge — €200

Public **own-project** evaluation (not SWE-bench). 10 tasks with/without CodeDNA, fair L0/L1/L2 stack parity, metrics PR — even if results go against CodeDNA.
Public **own-project** evaluation (not SWE-bench). Real working project (≥25 files), declared tech stack + CodeDNA install, same methodology for everyone (two branches / checkouts / twin projects), ≥10 **same** tasks with/without CodeDNA, copy-paste [agent prompt](docs/challenge-agent-prompt.md), metrics PR — even if results go against CodeDNA. Bugs must be reported (experimental across agents). Meet presentation may be required; public board updates as valid PRs arrive.

- Board: [larens94.github.io/codedna/challenge.html](https://larens94.github.io/codedna/challenge.html)
- Ranking / board: [larens94.github.io/codedna/challenge.html](https://larens94.github.io/codedna/challenge.html)
- Rules: [`docs/challenge.md`](docs/challenge.md) (EN) · [`docs/challenge.it.md`](docs/challenge.it.md) (IT)
- Submission: `challenge/<handle>/metrics.json` ([example](challenge/metrics.example.json)) — **opening the PR enrolls you**

Expand Down
13 changes: 13 additions & 0 deletions challenge/AGENT_PROMPT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Agent prompts (copy-paste)

Canonical prompts (also served on GitHub Pages):

- Italiano: [`docs/challenge-agent-prompt.it.md`](../docs/challenge-agent-prompt.it.md)
- English: [`docs/challenge-agent-prompt.md`](../docs/challenge-agent-prompt.md)

Task checklist:

- Italiano: [`TASKS_TEMPLATE.it.md`](./TASKS_TEMPLATE.it.md)
- English: [`TASKS_TEMPLATE.md`](./TASKS_TEMPLATE.md)

On the public page use **Copia per il tuo agente** → https://larens94.github.io/codedna/challenge.html
17 changes: 14 additions & 3 deletions challenge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,30 @@

```text
challenge/<github-handle>/
metrics.json # REQUIRED
metrics.json # REQUIRED (stack + install + setup.layout + same-task control/codedna results)
README.md # optional
notes.md # optional
```

No official signup. Public board (maintainer-updated): [`docs/challenge.html`](../docs/challenge.html) · data [`docs/challenge-board.json`](../docs/challenge-board.json)
No official signup. Public ranking (updated as valid PRs arrive): [`docs/challenge.html`](../docs/challenge.html) · data [`docs/challenge-board.json`](../docs/challenge-board.json)

**Copy for your agent**

- Italiano: [`docs/challenge-agent-prompt.it.md`](../docs/challenge-agent-prompt.it.md)
- English: [`docs/challenge-agent-prompt.md`](../docs/challenge-agent-prompt.md)
- Judge (optional, after both sessions): [`docs/challenge-judge-prompt.it.md`](../docs/challenge-judge-prompt.it.md) · [EN](../docs/challenge-judge-prompt.md)
- Task lists: [`TASKS_TEMPLATE.it.md`](./TASKS_TEMPLATE.it.md) · [`TASKS_TEMPLATE.md`](./TASKS_TEMPLATE.md)
- Pointer: [`AGENT_PROMPT.md`](./AGENT_PROMPT.md)

Methodology (same for everyone): same ≥10 tasks with vs without CodeDNA via `two_branches` / `two_checkouts` / `two_projects`.
File-localization fields (`files_expected`, F1, …) are **optional** — use when known; otherwise a judge agent can still compare sessions.

**Metrics format**

- Example: [`metrics.example.json`](./metrics.example.json)
- Schema: [`metrics.schema.json`](./metrics.schema.json)

Required project fields: `languages`, `frameworks`, `approx_source_files`, `size_band` (`S`/`M`/`L`/`XL`).
Required: `languages`, `frameworks`, `approx_source_files` (≥25), `size_band`, `install`, `setup.layout`, `bugs_reported`.

**Rules**

Expand Down
51 changes: 39 additions & 12 deletions challenge/SUBMISSION_TEMPLATE.it.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,36 +7,59 @@
### Partecipante

- Handle:
- Issue di iscrizione collegata:
- Modalità challenge:
- [ ] Parity (stessi L1/L2 su Control e CodeDNA)
- [ ] Dichiarata `codedna-only` vs stack superiore

### Stack sotto test
### Stack tecnologico (obbligatorio)

- Linguaggio/i:
- Framework (obbligatorio — non solo il linguaggio):
- Agente / modello:
- File sorgente approx. (≥ **25**):
- Size band (`S`/`M`/`L`/`XL`):
- Note stack (DB, monorepo, infra — opzionale):
- URL pubblico (opzionale) / “privato — solo metriche”:
- [ ] È un **progetto reale e funzionante** (non sito giocattolo / hello-world / demo throwaway)

### Install CodeDNA (obbligatorio)

- Agente usato con CodeDNA:
- Passi esatti di install / init (incolla i comandi):
- Valore di `codedna install --tools` (se usato):
- Install + annotation ok? sì / no — note:

### Setup Control vs CodeDNA (obbligatorio)

- [ ] `two_branches` (es. `challenge/control` + `challenge/codedna`)
- [ ] `two_checkouts` (due cartelle)
- [ ] `two_projects` (progetti gemelli)
- Ref Control / CodeDNA:
- [ ] Prompt agente usato: [`docs/challenge-agent-prompt.it.md`](../docs/challenge-agent-prompt.it.md)
- [ ] Lista task congelata: [`TASKS_TEMPLATE.it.md`](./TASKS_TEMPLATE.it.md)

### Stack sotto test (L0/L1/L2)

| Livello | Tool / file | Presente in Control? | Presente in CodeDNA? |
|---|---|---|---|
| L0 CodeDNA | header in-source | no | sì |
| L1 | | | |
| L2 | | | |

### Progetto

- Linguaggio/i:
- Framework (obbligatorio — non solo il linguaggio):
- File sorgente approx.:
- Size band (`S`/`M`/`L`/`XL`):
- URL pubblico (opzionale) / “privato — solo metriche”:

### Metrics JSON (obbligatorio)

- [ ] Presente `challenge/<handle>/metrics.json`
- [ ] Partito da [`metrics.example.json`](./metrics.example.json) / conforme a [`metrics.schema.json`](./metrics.schema.json)
- [ ] `schema_version` = `"1.0"`
- [ ] Compilati `project.languages` + `project.frameworks` + `approx_source_files` + `size_band`
- [ ] Compilati `project.languages` + `project.frameworks` + `approx_source_files` (≥25) + `size_band`
- [ ] Compilati `install.agent` + `install.steps`
- [ ] Compilato `setup.layout` (`two_branches` | `two_checkouts` | `two_projects`)
- [ ] Presente `bugs_reported` (array vuoto OK se nessuno)
- [ ] ≥10 task con mix `difficulty` (easy ≥3, medium ≥3, hard ≥2)
- [ ] Ogni task ha risultati `control` e `codedna`
- [ ] Gli **stessi task** hanno risultati `control` e `codedna`
- [ ] `summary.favors` impostato (`codedna` | `control` | `tie` | `inconclusive`)
- [ ] (Opzionale) `files_expected` / file F1 solo se conosciuti — non inventati
- [ ] (Opzionale) giudice usato: [`docs/challenge-judge-prompt.it.md`](../docs/challenge-judge-prompt.it.md) → `judge` / `tasks[].judge`
- [ ] I risultati possono favorire **oppure** sfavorire CodeDNA (onestà OK)

### Opzionale
Expand All @@ -47,9 +70,13 @@
### Bug trovati in CodeDNA

- Link (oppure “nessuno”) — elencarli anche in `metrics.json` → `bugs_reported`:
- [ ] Se qualcosa si è rotto su agente/linguaggio, ho aperto issue o PR di fix

### Checklist

- [ ] Nessun secret / sorgente proprietaria in questa PR
- [ ] Solo log redatti
- [ ] Modalità + parity di stack dichiarate nel JSON (`mode`, `stack`)
- [ ] Metriche e narrativa sono veritiere (non inventate)
- [ ] Posso presentare progetto e processo di test in una call di review (Meet / simile) se richiesto
- [ ] Capisco che claim inventati o progetti finti = squalifica
51 changes: 39 additions & 12 deletions challenge/SUBMISSION_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,36 +7,59 @@
### Entrant

- Handle:
- Linked entry issue:
- Challenge mode:
- [ ] Parity (same L1/L2 on Control and CodeDNA)
- [ ] Declared `codedna-only` vs higher stack

### Stack under test
### Tech stack (required)

- Language(s):
- Framework(s) (required — not language alone):
- Agent / model:
- Approx. source files (≥ **25**):
- Size band (`S`/`M`/`L`/`XL`):
- Tech stack notes (DB, monorepo, infra — optional):
- Public URL (optional) / “private — metrics only”:
- [ ] This is a **real working project** (not a toy site / hello-world / throwaway demo)

### CodeDNA install (required)

- Agent used with CodeDNA:
- Exact install / init steps (paste commands):
- `codedna install --tools` value (if any):
- Did install + annotation work? yes / no — notes:

### Control vs CodeDNA setup (required)

- [ ] `two_branches` (e.g. `challenge/control` + `challenge/codedna`)
- [ ] `two_checkouts` (two folders)
- [ ] `two_projects` (twin projects)
- Control / CodeDNA refs:
- [ ] Agent prompt used: [`docs/challenge-agent-prompt.md`](../docs/challenge-agent-prompt.md)
- [ ] Frozen task list: [`TASKS_TEMPLATE.md`](./TASKS_TEMPLATE.md)

### Stack under test (L0/L1/L2)

| Level | Tools / files | Present in Control? | Present in CodeDNA? |
|---|---|---|---|
| L0 CodeDNA | in-source headers | no | yes |
| L1 | | | |
| L2 | | | |

### Project

- Language(s):
- Framework(s) (required — not language alone):
- Approx. source files:
- Size band (`S`/`M`/`L`/`XL`):
- Public URL (optional) / “private — metrics only”:

### Metrics JSON (required)

- [ ] `challenge/<handle>/metrics.json` present
- [ ] Copied from [`metrics.example.json`](./metrics.example.json) / matches [`metrics.schema.json`](./metrics.schema.json)
- [ ] `schema_version` = `"1.0"`
- [ ] `project.languages` + `project.frameworks` + `approx_source_files` + `size_band` filled
- [ ] `project.languages` + `project.frameworks` + `approx_source_files` (≥25) + `size_band` filled
- [ ] `install.agent` + `install.steps` filled
- [ ] `setup.layout` filled (`two_branches` | `two_checkouts` | `two_projects`)
- [ ] `bugs_reported` present (empty array OK if none)
- [ ] ≥10 tasks with `difficulty` mix (easy ≥3, medium ≥3, hard ≥2)
- [ ] Each task has both `control` and `codedna` results
- [ ] **Same tasks** each have both `control` and `codedna` results
- [ ] `summary.favors` set (`codedna` | `control` | `tie` | `inconclusive`)
- [ ] (Optional) `files_expected` / file F1 only when known — never invented
- [ ] (Optional) judge used: [`docs/challenge-judge-prompt.md`](../docs/challenge-judge-prompt.md) → `judge` / `tasks[].judge`
- [ ] Results may favor **or** disfavor CodeDNA (honesty OK)

### Optional
Expand All @@ -47,9 +70,13 @@
### Bugs found in CodeDNA

- Links (or “none”) — also list them in `metrics.json` → `bugs_reported`:
- [ ] If something broke on my agent/language, I filed an issue or fix PR

### Checklist

- [ ] No secrets / proprietary source in this PR
- [ ] Redacted logs only
- [ ] Mode + stack parity declared in JSON (`mode`, `stack`)
- [ ] Metrics and narrative are truthful (not invented)
- [ ] I can present the project and test process on a review call (Meet / similar) if asked
- [ ] I understand fabricated claims or fake projects = disqualification
35 changes: 35 additions & 0 deletions challenge/TASKS_TEMPLATE.it.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# CodeDNA Challenge — template lista task

> Congela questa lista **prima** delle run cronometrate. Esegui gli **stessi** ID su Control e su CodeDNA.
> Prompt completo per l’agente: [`docs/challenge-agent-prompt.it.md`](../docs/challenge-agent-prompt.it.md)

## Setup (scegline uno)

- [ ] `two_branches` — es. `challenge/control` + `challenge/codedna`
- [ ] `two_checkouts` — due cartelle dello stesso progetto
- [ ] `two_projects` — progetti gemelli dallo stesso codice di partenza

Agente / modello: _______________
Note layout: _______________

## Task (≥10 — easy ≥3, medium ≥3, hard ≥2)

| ID | Difficoltà | Titolo (specifico del progetto) | Control ok? | CodeDNA ok? | Note |
|---|---|---|---|---|---|
| E1 | easy | | | | |
| E2 | easy | | | | |
| E3 | easy | | | | |
| M1 | medium | | | | |
| M2 | medium | | | | |
| M3 | medium | | | | |
| H1 | hard | | | | |
| H2 | hard | | | | |
| T9 | easy/medium/hard | | | | |
| T10 | easy/medium/hard | | | | |

Extra opzionali: E4, M4, H3…

## Dopo le run

- [ ] `metrics.json` compilato (`install`, `setup.layout`, `bugs_reported`, `summary.favors`)
- [ ] PR aperta con submission template
35 changes: 35 additions & 0 deletions challenge/TASKS_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# CodeDNA Challenge — task list template

> Freeze this list **before** timed runs. Run the **same** IDs on Control and on CodeDNA.
> Italiano: adatta i titoli al tuo repo; scheletro anche in [`docs/challenge-agent-prompt.it.md`](../docs/challenge-agent-prompt.it.md).

## Setup (pick one)

- [ ] `two_branches` — e.g. `challenge/control` + `challenge/codedna`
- [ ] `two_checkouts` — two folders of the same project
- [ ] `two_projects` — twin projects from the same starting code

Agent / model: _______________
Layout notes: _______________

## Tasks (≥10 — easy ≥3, medium ≥3, hard ≥2)

| ID | Difficulty | Title (project-specific) | Control pass? | CodeDNA pass? | Notes |
|---|---|---|---|---|---|
| E1 | easy | | | | |
| E2 | easy | | | | |
| E3 | easy | | | | |
| M1 | medium | | | | |
| M2 | medium | | | | |
| M3 | medium | | | | |
| H1 | hard | | | | |
| H2 | hard | | | | |
| T9 | easy/medium/hard | | | | |
| T10 | easy/medium/hard | | | | |

Optional extras: E4, M4, H3…

## After runs

- [ ] `metrics.json` filled (`install`, `setup.layout`, `bugs_reported`, `summary.favors`)
- [ ] PR opened with submission template
Loading