Skip to content

chore: fix eReputation prompt - #461

Closed
coodos wants to merge 6 commits into
mainfrom
feat/change-evoting-prompt
Closed

coodos wants to merge 6 commits into
mainfrom
feat/change-evoting-prompt

Conversation

@coodos

@coodos coodos commented Nov 24, 2025 •

Copy link
Copy Markdown
Contributor

Description of change

Issue Number

Type of change

  • Fix (a change which fixes an issue)

How the change has been tested

Change checklist

  • I have ensured that the CI Checks pass locally
  • I have removed any unnecessary logic
  • My code is well documented
  • I have signed my commits
  • My code follows the pattern of the application
  • I have self reviewed my code

Summary by CodeRabbit

  • Improvements
    • Clarified and strengthened eVoting scoring guidance to prioritize explicit charter instructions over other group values.
    • Enforced one-to-one matching between chartered values and references; avoid guesses or assumptions and favor caution when uncertain.
    • Standardized handling for members without references by assigning a consistent default score of 1.
    • Made justifications more consistent across bulk assessments and final responses.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Nov 24, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

Expanded buildBulkVotingPrompt in VotingReputationService: added a console log and broadened scoring guidance and rules for evaluating charter values and member references, duplicating those rules across both the bulk prompt and final response sections. No function signatures or control flow were changed. (≤50 words)

Changes

Cohort / File(s) Summary
Voting prompt and guidance
platforms/eReputation-api/src/services/VotingReputationService.ts
Added console.log("Building bulk voting prompt for charter"); expanded scoring guidance in the bulk prompt and final response text to: consider only charter-mentioned values for eVoting, use only one-to-one matches between values and references (avoid guessing), prefer caution when unsure, obey charter-explicit eVoting instructions by ignoring other group values, and assign score 1 for members with no references unless charter states otherwise.

Sequence Diagram(s)

(omitted — changes are prompt content updates, not control-flow or new interactions)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Check that the new prompt wording is unambiguous and consistent across bulk and final sections
  • Verify no inadvertent sensitive or PII wording introduced into prompts
  • Confirm the console.log placement is appropriate and not noisy in production contexts

Suggested reviewers

  • xPathin
  • sosweetham

Poem

🐰 I tunneled through prompts with careful care,

Charters and scores now hop in the air.
When no reference shows, score one will stay,
Unless the charter guides another way.
A rabbit's small cheer for voting today!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is incomplete. While it follows the template structure and includes the checklist, critical sections like 'Description of change' and 'How the change has been tested' are empty, lacking explanation of what was fixed and testing details. Fill in the 'Description of change' section explaining the prompt improvements and the 'How the change has been tested' section with specific testing details or validation performed.
Title check ❓ Inconclusive The title 'chore: fix eReputation prompt' is vague and generic. While it indicates a chore-type change to fix a prompt, it lacks specificity about what aspect of the eReputation prompt was fixed. Consider a more descriptive title that clarifies the specific fix, such as 'chore: improve eReputation voting prompt scoring guidance' to better convey the changes made.
✅ Passed checks (1 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/change-evoting-prompt

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coodos
coodos marked this pull request as ready for review November 24, 2025 09:27

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7952d68 and 34fd963.

📒 Files selected for processing (1)
  • platforms/eReputation-api/src/services/VotingReputationService.ts (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-11-20T12:00:02.259Z
Learnt from: coodos
Repo: MetaState-Prototype-Project/prototype PR: 453
File: platforms/eReputation-api/src/controllers/WebhookController.ts:6-7
Timestamp: 2025-11-20T12:00:02.259Z
Learning: In platforms/eReputation-api/src/controllers/WebhookController.ts, the eager instantiation of VotingReputationService in the constructor (which requires OPENAI_API_KEY at startup) is intentional behavior and works as intended.

Applied to files:

  • platforms/eReputation-api/src/services/VotingReputationService.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build

Comment thread platforms/eReputation-api/src/services/VotingReputationService.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
platforms/eReputation-api/src/services/VotingReputationService.ts (2)

389-389: Consider removing or enhancing this console log.

The static log message provides no useful debugging information. Either remove it or enhance it to include contextual details (e.g., number of members, charter length).

Apply this diff to remove the log:

-        console.log("Building bulk voting prompt for charter");

Or enhance it with useful context:

-        console.log("Building bulk voting prompt for charter");
+        console.log(`Building bulk voting prompt for ${membersData.length} members`);

465-466: Fix typo in deprecated method for consistency.

Line 466 contains the same typo as in the active method: "explcitly" should be "explicitly". While this method is deprecated, fixing the typo maintains consistency.

Apply this diff:

 - If the charter has explicit instructions on what they consider for eVoting, DO NOT consider any other group values to make a reference. 
-- If someone has no references give them a score of 1, unless explcitly stated in the charter that they shall get another score.
+- If someone has no references, give them a score of 1, unless explicitly stated in the charter that they shall get another score.
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 34fd963 and 23a2bbd.

📒 Files selected for processing (1)
  • platforms/eReputation-api/src/services/VotingReputationService.ts (3 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-11-20T12:00:02.259Z
Learnt from: coodos
Repo: MetaState-Prototype-Project/prototype PR: 453
File: platforms/eReputation-api/src/controllers/WebhookController.ts:6-7
Timestamp: 2025-11-20T12:00:02.259Z
Learning: In platforms/eReputation-api/src/controllers/WebhookController.ts, the eager instantiation of VotingReputationService in the constructor (which requires OPENAI_API_KEY at startup) is intentional behavior and works as intended.

Applied to files:

  • platforms/eReputation-api/src/services/VotingReputationService.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build

Comment on lines +416 to +421
- Only consider the values mentioned in the charter for eVoting explicitly do not consider anything else, and only consider the references that are relevant to the values mentioned in the charter for eVoting.
- Focus on voting-relevant reputation factors mentioned in the charter
- Provide a ONE SENTENCE justification explaining each score
- Only consider values and ereferences which match one to one, and do not consider things where you need to make a guess or assumption, if you are not sure always err to caution.
- If the charter has explicit instructions on what they consider for eVoting, DO NOT consider any other group values to make a reference.
- If someone has no references give them a score of 1, unless explcitly stated in the charter that they shall get another score.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix typos and grammar in the prompt instructions.

Several issues in the new prompt rules:

  • Line 416: Awkward phrasing "explicitly do not consider anything else"
  • Line 419: Typo "ereferences" should be "references"
  • Line 419: Incomplete idiom "err to caution" should be "err on the side of caution"
  • Line 421: Typo "explcitly" should be "explicitly"

These errors will be passed directly to the AI and may affect instruction clarity.

Apply this diff to fix all issues:

-- Only consider the values mentioned in the charter for eVoting explicitly do not consider anything else, and only consider the references that are relevant to the values mentioned in the charter for eVoting.
+- Only consider the values mentioned in the charter for eVoting and do not consider anything else; only consider references that are relevant to the values mentioned in the charter for eVoting.
 - Focus on voting-relevant reputation factors mentioned in the charter
 - Provide a ONE SENTENCE justification explaining each score
-- Only consider values and ereferences which match one to one, and do not consider things where you need to make a guess or assumption, if you are not sure always err to caution.
+- Only consider values and references which match one to one, and do not consider things where you need to make a guess or assumption; if you are not sure, always err on the side of caution.
 - If the charter has explicit instructions on what they consider for eVoting, DO NOT consider any other group values to make a reference. 
-- If someone has no references give them a score of 1, unless explcitly stated in the charter that they shall get another score.
+- If someone has no references, give them a score of 1, unless explicitly stated in the charter that they shall get another score.
🤖 Prompt for AI Agents
In platforms/eReputation-api/src/services/VotingReputationService.ts around
lines 416 to 421, fix grammar and typos in the prompt rules: reword the awkward
phrase "explicitly do not consider anything else" to "consider only the values
mentioned in the charter for eVoting and do not consider any other values,"
change "ereferences" to "references," change "err to caution" to "err on the
side of caution," and correct "explcitly" to "explicitly"; keep the meaning that
only voting-relevant charter values and matching references are used, require a
ONE SENTENCE justification per score, and state that a score of 1 applies to
subjects with no references unless the charter explicitly prescribes another
score.

@coodos coodos closed this Dec 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant