Skip to content

Addressing DogFooding experiences feedback - #156

Merged
danielholanda merged 1 commit into
mainfrom
lemonade_router_builder_DF_feedback_2
Aug 24, 2026
Merged

Addressing DogFooding experiences feedback#156
danielholanda merged 1 commit into
mainfrom
lemonade_router_builder_DF_feedback_2

Conversation

@sdevinenamd

Copy link
Copy Markdown
Collaborator

Thank you @amd-hasnasir and Matt for testing out the skill and sharing your valuable feedback. Addressed the pointers in this PR.

Hasan's feedback

  • SSN test prompt caused Claude to refuse → Since this is a model behavior and not consistently observed, added an alternative email-only backup command (Step 5). Fyi, the walkthroughs are only for internal devs so this would eventually become obsolete to the public.
  • Steps 5/6 didn't say to use a new session → added "On a new Claude session, run:"
  • Walkthrough expected rule id pii-stays-local, skill defaults to rule-1 → fixed walkthrough to match actual default

Matt's feedback

  • "Write better prompts" doesn't fix silent misrouting - it's a judge-capability issue → router.model now defaults to most capable candidate, not cheapest. Also added a note to switch to smaller models if performance capacity is known
  • Verification step was buried/undersold → promoted to mandatory Step 9
  • Classifier label typos silently score 0 forever, no error → documented + added a verification check
  • Validator can't confirm models exist on target host → noted in Step 1 / Step 8b
  • Mode A trace ids are synthetic (__route_0, …), not author-given → documented in reference.md
  • Empty rationale isn't a reliable fallback signal → clarified default_used is the real one
  • GPU multi model resource gap → Lemonade allows loading >1 models depending on the GPU availability. However, clarifying that level of technicality is out of scope for this skill.

@danielholanda danielholanda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good

@danielholanda
danielholanda merged commit ff4520e into main Aug 24, 2026
25 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants