[v1.0] Preserve streaming output for JSON-capable applications

[codeforester]

Goal

Keep normal human and NDJSON output streaming, and bound memory when lifecycle JSON is requested.

Background

When an application merely declares a lifecycle JSON option, run_app() redirects all stdout to an in-memory StringIO before Click parses the option:

base-cli/lib/python/base_cli/_run.py

Lines 66 to 95 in 8a93d22

	output_capture: io.StringIO | None = None
	try:
	try:
	display_command = app.profile.display_command()
	invocation_argv = _effective_invocation_argv(app, args, display_command)
	command = app.click_command
	click = dialect_for_command(command)
	invocation_token = _INVOCATION_ARGV.set(invocation_argv)
	try:
	bypass_token = _INVOCATION_MAIN_BYPASS.set(command)
	# A configured JSON option may be enabled by any Click-supported
	# source (for example ``default_map`` or a combined short flag),
	# so raw argv cannot determine capture eligibility. Buffer the
	# command whenever JSON mode exists and let the parsed lifecycle
	# value decide whether to emit an envelope or replay human text.
	output_capture = io.StringIO() if app.lifecycle_options.json is not None else None
	try:
	if output_capture is None:
	result = command.main(
	args=args,
	prog_name=display_command or app.name,
	standalone_mode=False,
	)
	else:
	with redirect_stdout(output_capture):
	result = command.main(
	args=args,
	prog_name=display_command or app.name,
	standalone_mode=False,
	)

. If JSON was not selected, the entire buffer is replayed only after the command finishes:

base-cli/lib/python/base_cli/_run.py

Lines 150 to 176 in 8a93d22

	try:
	if state.attached_completion:
	if state.json_output:
	_emit_json_success(state, ExitCode.SUCCESS, output_capture)
	return ExitCode.SUCCESS
	exit_code = _normalize_command_result(result)
	if state.json_output:
	if exit_code == ExitCode.SUCCESS:
	_emit_json_success(state, exit_code, output_capture)
	else:
	_emit_json_error(
	state,
	outcome_from_exit_code(exit_code),
	"Command returned a non-zero exit code.",
	output_capture,
	)
	return exit_code
	except TypeError as exc:
	if state.json_output:
	outcome = outcome_from_exception(click, exc)
	_emit_json_error(state, outcome, str(exc), output_capture)
	return outcome.exit_code
	print(f"ERROR: {exc}", file=sys.stderr)
	return ExitCode.FAILURE
	finally:
	if output_capture is not None and not state.json_output:
	sys.stdout.write(output_capture.getvalue())

.

This breaks progress output and pipeline latency for the default human path, defeats NdjsonWriter.write()'s flush guarantee, and makes output memory proportional to command output. It regresses the bounded/streaming commitments in #65 and #192.

Scope

• Preserve the real stdout stream whenever lifecycle JSON is not active.
• Retain support for Click default_map, environment variables, aliases, combined short options, and negated flags.
• Bound or spool any capture required for a single JSON envelope.
• Define overflow and stream-failure behavior.

Acceptance Criteria

• A JSON-capable app with no JSON request streams and flushes output immediately.
• NDJSON records remain observable one at a time and use bounded memory.
• Human terminal behavior is unchanged by declaring JSON capability.
• Active --json still emits exactly one valid v1 envelope with no prose contamination.
• Large captured output has a documented deterministic limit or disk-spooling policy.
• Tests assert write/flush timing, backpressure/error behavior, large output, env/default-map activation, and negation.

Validation

Run JSON-contract, output, invocation-parity, performance, and subprocess tests.

Non-Goals

Do not remove the existing JSON envelope contract.

Project Fields

• Status: Backlog
• Priority: P1
• Area: CLI
• Initiative: v1.0 Readiness
• Size: M

Ownership

• Assignee: @codeforester