fix: show updated agent answer before each feedback prompt when verbose=False - #6386
fix: show updated agent answer before each feedback prompt when verbose=False#6386Julien-ser wants to merge 3 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 WalkthroughWalkthrough
ChangesVerbosity-aware agent answer display in human feedback
Suggested reviewers: Merge Risk: ⚪ Minimal · up to Non-verbose human-feedback prompts now show the current agent answer, including repeat feedback cycles and training mode, so operators can evaluate visible output. The change is covered across synchronous and asynchronous paths with no remaining concrete merge risk. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Description checkExplanation The description explains the related issue, problem, fix, affected code paths, and rebasing context. It does not include an explicit Verification section or test checklist, but the core required information is present. Full details: Linked Issues checkExplanation The changes satisfy issue Full details: Out of Scope Changes checkExplanation The code and regression test changes directly support issue ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
lib/crewai/src/crewai/core/providers/human_input.py (1)
342-386: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRender the supplied answer before branching on training mode.
handle_feedback*now passesanswerfor non-verbose prompts, but_prompt_input*only prints it in the non-training branch. In training mode, the answer is dropped and the user is asked to judge result quality without seeing the result.Proposed fix
try: + if answer is not None: + output_str = HumanInputProvider._get_output_string(answer) + result_content = Text() + result_content.append(output_str, style="green") + formatter.console.print( + Panel( + result_content, + title="✅ Agent Final Answer", + border_style="green", + padding=(1, 2), + ) + ) + if crew and getattr(crew, "_train", False): prompt_text = ( "TRAINING MODE: Provide feedback to improve the agent's performance.\n\n" "This will be used to train better versions of the agent.\n" "Please provide detailed feedback about the result quality and reasoning process." ) title = "🎓 Training Feedback Required" else: - if answer is not None: - output_str = HumanInputProvider._get_output_string(answer) - result_content = Text() - result_content.append(output_str, style="green") - formatter.console.print( - Panel( - result_content, - title="✅ Agent Final Answer", - border_style="green", - padding=(1, 2), - ) - ) prompt_text = (Apply the same move in
_prompt_input_async.Also applies to: 414-457
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/crewai/src/crewai/core/providers/human_input.py` around lines 342 - 386, The answer display in _prompt_input is still gated by the training-mode branch, so handle_feedback callers in training mode never see the agent result before giving feedback. Move the answer-rendering logic (the HumanInputProvider._get_output_string and Panel print) to run before the training/non-training branch, and apply the same fix in _prompt_input_async so both paths show the supplied answer consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@lib/crewai/src/crewai/core/providers/human_input.py`:
- Around line 342-386: The answer display in _prompt_input is still gated by the
training-mode branch, so handle_feedback callers in training mode never see the
agent result before giving feedback. Move the answer-rendering logic (the
HumanInputProvider._get_output_string and Panel print) to run before the
training/non-training branch, and apply the same fix in _prompt_input_async so
both paths show the supplied answer consistently.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a50ddd51-8cb9-4a08-b157-30aa0fda95a7
📒 Files selected for processing (1)
lib/crewai/src/crewai/core/providers/human_input.py
ErenAta16
left a comment
ErenAta16
left a comment
There was a problem hiding this comment.
Reviewed the diff against human_input.py on main (confirmed this is the only file touched, +68/-8, via the PR's files API - my local shallow clone briefly showed a much larger diff against a stale cached main, but that was a clone artifact on my end, not something in this PR).
This fixes a real usability gap (#6072): when verbose=False, the human-in-the-loop feedback loop re-prompts the operator for input after each iteration without ever showing what the agent's updated answer actually was, so the operator is asked to give feedback "blind" on a response they can't see. The fix threads a show_answer/answer flag through _prompt_input (both sync and async paths) that's derived from agent.verbose or crew.verbose, and prints the current answer immediately before the feedback prompt only when verbose is off (when verbose is on, the answer is already visible from the normal execution log, so it correctly avoids double-printing).
Checked that both the sync (_handle_regular_feedback) and async (_handle_regular_feedback_async) iteration loops were updated consistently - they were, including the initial pre-loop prompt and every iteration inside the loop. No existing test file covers this provider directly, but the logic change is small, symmetric across both code paths, and matches the linked issue's exact complaint. LGTM.
|
This PR is stale because it has been open for 45 days with no activity. |
When human_input=True but verbose=False, the feedback panel said 'Provide feedback on the Final Result above' but the result was never displayed — the AgentLogsExecutionEvent is verbose-gated while the human-input gate fires unconditionally. Fix: handle_feedback / handle_feedback_async detect when the executor is non-verbose and pass the AgentFinish to _prompt_input / _prompt_input_async. Both methods now print a green 'Agent Final Answer' panel before the feedback prompt when the caller supplies the answer, so the operator always sees the result they are asked to review. Training mode is unaffected — the answer panel is skipped in that path. Fixes crewAIInc#6072
…verbose=False Addresses CodeRabbit feedback from PR crewAIInc#6073: when iterating through multiple rounds of human feedback in non-verbose mode, only the initial answer was shown. On subsequent rounds, the user was prompted again without seeing the newly generated response. Pass show_answer=not is_verbose to _handle_regular_feedback / _handle_regular_feedback_async so that _prompt_input re-displays the current answer before each feedback prompt whenever verbose output is suppressed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The answer panel added by this PR was rendered only in the non-training branch of _prompt_input / _prompt_input_async, so with crew._train set the operator was still asked to judge output they could not see -- the same gap this PR set out to close, just on the training path. The training prompt explicitly asks for "detailed feedback about the result quality and reasoning process", so the result has to be on screen there as much as in normal mode. Hoist the rendering above the training/normal branch in both the sync and async paths, and extract it into a _render_answer_panel helper rather than repeat the same panel construction a third and fourth time. Adds a parametrized regression test covering _train=True and _train=False; the _train=True case fails without this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JRC5ACRhgrzdagKjPUbYEH
5ed84ef to
51cfc09
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
|
Updated: rebased onto current The gap was real. The answer panel this PR adds was rendered only in the non-training branch of Hoisted the rendering above the training/normal branch in both the sync and async paths, and pulled it into a Added a parametrized regression test over After the fix, both pass, along with |
There was a problem hiding this comment.
🟡 Changes recommended
The new test assertion is likely brittle across Rich versions (title type), and the new answer-panel styling is inconsistent with existing final-answer rendering conventions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR fixes the human_input=True feedback loop UX when verbose=False by ensuring the agent’s latest answer is re-rendered before each subsequent feedback prompt, so operators can see what changed between iterations.
Changes:
- Plumbs a
show_answerflag through the regular feedback loop to re-display the updatedAgentFinishoutput before each new feedback prompt when non-verbose. - Adds a helper to render the “✅ Agent Final Answer” Rich panel from the human-input provider path.
- Adds a unit test asserting the answer panel is rendered by
_prompt_inputin both normal and training modes.
File summaries
| File | Description |
|---|---|
| lib/crewai/src/crewai/core/providers/human_input.py | Adds answer-panel rendering and propagates show_answer so non-verbose human feedback prompts always show the latest answer. |
| lib/crewai/tests/agents/test_agent.py | Adds coverage to ensure _prompt_input renders the answer panel in both training and non-training prompts. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| rendered = [ | ||
| call.args[0] | ||
| for call in formatter.console.print.call_args_list | ||
| if getattr(call.args[0], "title", None) == "✅ Agent Final Answer" | ||
| ] |
| result_content = Text() | ||
| result_content.append( | ||
| HumanInputProvider._get_output_string(answer), style="green" | ||
| ) |
Fixes #6072.
Problem
When
human_input=Trueis set andverbose=False, the agent's initial answer was displayed before the first feedback prompt, but on every subsequent iteration the user was prompted again without seeing what the agent had produced in the latest round. The operator had no way to know what changed between feedback cycles.Fix
Pass
show_answer=not is_verbosethrough to_handle_regular_feedback/_handle_regular_feedback_async. On each loop iteration, after_invoke_loop()/_ainvoke_loop()produces a new answer,_prompt_inputre-renders the updated answer in a Rich panel before asking for the next round of input.Changes
lib/crewai/src/crewai/core/providers/human_input.pyhandle_feedback/handle_feedback_async— passshow_answer=not is_verbose_handle_regular_feedback/_handle_regular_feedback_async— acceptshow_answer: bool = False; on subsequent iterations, passanswer=answer if show_answer else Noneto the prompt helpers_prompt_input/_prompt_input_async— already acceptanswerand render the panel (added in the initial commit)Rebased from
This branch was rebased cleanly onto current
mainfrom the original work in #6073.AI assistance
This fix was developed with the help of Claude Code (
🤖 Generated with Claude Code). All changed lines have been reviewed and understood.Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com