[v2] Implement conditional autoprompt importing

[aemous]

Description of changes:

• Refactored autoprompt code so that resolving the autoprompt mode does not require importing autoprompt or instantiating a AutoPromptDriver object. The mode resolution code has been moved to clidriver.py as the standalone functions resolve_auto_prompt_mode and validate_auto_prompt_args_are_mutually_exclusive. AutoPromptDriver is now only imported lazily when autoprompt mode is actually active.
• Moved resolve_mode and validate_auto_prompt_args_are_mutually_exclusive tests from tests/unit/autoprompt/test_core.py to tests/unit/test_clidriver.py to reflect the new location of those functions.
• Separated PrompterKeyboardInterrupt into a new minimal awscli/autoprompt/exceptions.py module, so that awscli.errorhandler can import it without pulling in awscli.autoprompt.factory (which has many top-level prompt_toolkit imports).
• Made the Display import in ecs/monitorexpressgatewayservice.py lazy, deferring the import of prompt_toolkit_display (and therefore prompt_toolkit) until the interactive ECS monitoring display is actually used.
• Made the imports of PTKPrompt and RequiredInputValidator from configure/sso.py in customizations/login/login.py lazy, deferring the import of configure/sso.py until the login region prompt is actually needed.
• Made imports of StartLiveTailCommand and TailCommand in customizations/logs/__init__.py lazy, deferring the import of prompt_toolkit until the aws logs subcommand is invoked (i.e. the logs subcommand table is built).
• Extracted ConfigureSSOCommand, ConfigureSSOSessionCommand, and their shared base class BaseSSOConfigurationCommand into a new customizations/configure/sso_commands.py module, which has no prompt_toolkit imports. This allows configure/configure.py to keep its reference to these SSO commands and defer the import of prompt_toolkit (via configure/sso.py) until aws configure sso is actually invoked.
• Made the import of select_menu from wizard/ui/selectmenu in configure/sso.py lazy, deferring the import of wizard/ui/__init__.py (which imports prompt_toolkit) until ConfigureSSOCommand is actually instantiated.
• Made the import of create_wizard_app from wizard/factory in wizard/devcommands.py lazy, deferring the import of the wizard UI chain (which imports prompt_toolkit) until a wizard dev command is actually run.
• Made the import of wizard/factory in wizard/commands.py lazy by introducing a _get_runner method on TopLevelWizardCommand, deferring the import of the wizard factory and UI chain until a wizard command is actually executed.
• Moved the import of devcommands in awscli/customizations/wizard/commands.py from module-level to within the register_wizard_commands function. Although this currently gives no behavioral difference since register_wizard_commands is always called for every command, in the future when plugin/customization initialization is only called when a relevant command is entered (lazy plugin loading), this change will lead to lazy-importing of devcommands only when it's needed.

Description of tests:

• Successfully ran pre-prod build workflow.
• Benchmarked before and after this patch (results below)

Benchmark Results: Performance Improvement Analysis

The following benchmarking analysis evaluates the impact of the Conditional Autoprompt feature across two standard CLI operations. Tests were performed with 2,000 samples per state to ensure statistical precision.

Tested on m7i.xlarge EC2 instance, x86 architecture, AL2023 os, 64 GB storage, 2,000 iterations each.

Consolidated Performance Gains

CLI Command	Metric	Before Patch	After Patch	Change
aws --version	Mean Time	609.883 ms	528.471 ms	-81.412 ms
	stdev.	14.9329 ms	11.6255 ms	-3.3074 ms
	SE	0.3339 ms	0.2600 ms	—
aws sts get-caller-identity	Mean Time	697.245 ms	615.703 ms	-81.542 ms
	stdev.	9.62246 ms	9.41794 ms	-0.20452 ms
	SE	0.2152 ms	0.2106 ms	—

Statistical Significance & Reliability

To filter out environmental noise (CPU spikes, OS scheduling), we measured the performance shift in terms of Standard Error (SE).

• Distance in Standard Errors: The improvements represent shifts of ~243.8 SEs for aws --version and ~379 SEs for aws sts get-caller-identity.
• P-Value: In both tests, $p < 0.001$. This indicates that the probability of these improvements being a result of random chance is effectively zero.
• Precision: With a sample size of $n=2000$, our margin of error is extremely small ($<0.3$ ms). The performance gains are systemic and statistically "locked-in."

Visual Analysis

Average execution times compared

Significant time savings are shown. We can expect similar improvements across all modeled commands, and some improvement(s) to customization commands.

Probability density functions of execution time

aws --version execution time probability.

aws sts get-caller-identity execution time probability.

The probability density curves in both graphs show no overlap between the "Before" and "After" states. The narrow, tall peaks confirm that the large sample size has provided a high-confidence estimate of the true performance improvement. For the version command, we note that the peak is taller and narrower in the "After" state, signaling that the patch not only improves execution time but also reduces noise.

---

Summary: The feature provides a consistent and significant performance reduction (11.70-13.35%) across different command types with statistical certainty.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[hssyoo]

The measurements in the PR description show a ~5ms improvement, which is far from the ~66ms mean import time we measured. Based on these numbers, it seems highly likely that prompt_toolkit is still being imported elsewhere.

I was able to confirm this by checking sys.modules at the end of a aws --version call.

[aemous]

The measurements in the PR description show a ~5ms improvement, which is far from the ~66ms mean import time we measured. Based on these numbers, it seems highly likely that prompt_toolkit is still being imported elsewhere.

I was able to confirm this by checking sys.modules at the end of a aws --version call.

Great catch! I have analyzed sys.modules repeatedly to find where to switch eager imports to lazy imports across various files, the resulting sys.modules show no more prompt_toolkit for unrelated commands, when autoprompt is off.

[hssyoo]

It's looking better but it looks like there's some gaps:

1. I noticed that aws configure list still imports prompt_toolkit even though it doesn't need it.
2. I'm not seeing anything in the PR that will guard against someone from adding unconditional prompt_toolkit imports in the future.

I'd like to see us step back and first enumerate all cases where we know we'll need to import prompt_toolkit. Knowing all cases upfront will help us verify that the implementation behaves as intended and help us write tests to ensure future changes don't reintroduce unconditional imports.

I'm also not sure if scattering import statements across different files and within methods is the right approach. I'm wondering if what we want instead is an authoritative source for importing prompt_toolkit. For example, say we had 2 modules:

1. _prompt.py - The only module in the AWS CLI repository that's allowed to import anything from prompt_toolkit.
2. prompt.py - A module that exposes factory functions that return objects that need require prompt_toolkit.

The imports from _prompt.py would happen within the factory functions. This is nice because then the rest of the AWS CLI codebase can import from prompt.py, but prompt_toolkit won't be imported until the factory function is actually called.

A single entrypoint for importing prompt_toolkit also enables us to write linting rules that assert that prompt_toolkit is never imported outside of _prompt.py, providing an extra layer of protection.

Thoughts?

[hssyoo]

1. We're only deferring prompt_toolkit import here, right? Let's be specific about that instead of generally claiming "refactor imports".
2. I wouldn't put any concrete numbers in the changelog. This figure changes between hardware and command run. Just say it avoids prompt_toolkit import time for commands that don't need it.

[aemous]

It's looking better but it looks like there's some gaps:

1. I noticed that aws configure list still imports prompt_toolkit even though it doesn't need it.
2. I'm not seeing anything in the PR that will guard against someone from adding unconditional prompt_toolkit imports in the future.

I'd like to see us step back and first enumerate all cases where we know we'll need to import prompt_toolkit. Knowing all cases upfront will help us verify that the implementation behaves as intended and help us write tests to ensure future changes don't reintroduce unconditional imports.

I'm also not sure if scattering import statements across different files and within methods is the right approach. I'm wondering if what we want instead is an authoritative source for importing prompt_toolkit. For example, say we had 2 modules:

1. _prompt.py - The only module in the AWS CLI repository that's allowed to import anything from prompt_toolkit.
2. prompt.py - A module that exposes factory functions that return objects that need require prompt_toolkit.

The imports from _prompt.py would happen within the factory functions. This is nice because then the rest of the AWS CLI codebase can import from prompt.py, but prompt_toolkit won't be imported until the factory function is actually called.

A single entrypoint for importing prompt_toolkit also enables us to write linting rules that assert that prompt_toolkit is never imported outside of _prompt.py, providing an extra layer of protection.

Thoughts?

Configure list

Regarding aws configure list, good catch. I have completed an audit of the codebase for prompt_toolkit usage, and root-caused it to the following import chain:

configure.py imports sso_commands.py (module-level)
  → ConfigureSSOCommand.__init__ imports sso.py (lazy, but fires during subcommand table build)
    → sso.py has module-level prompt_toolkit imports

In summary, every aws configure command will import prompt_toolkit for this exact reason. It should be easy to apply a patch to fix this so that the module is only imported for the 2 SSO commands that use it (aws configure sso and aws configure sso-session). Sounds like we should only proceed with this patch if we decide against your proposal in part 2.

Single point of importing prompt_toolkit (with linting enforcement)

prompt_toolkit audit results

Below is the audit results for when prompt_toolkit is definitely needed:

1. aws configure sso / aws configure sso-session

Always needed — these are inherently interactive commands that always prompt the user

Current bug (as you pointed out): imported during init of BaseSSOConfigurationCommand, which fires during subcommand table build for any aws configure subcommand (including list, get, set, etc.)

Fix: move imports from init to _run_main

2. aws <service> wizard (iam, lambda, dynamodb, events, configure)
   Always needed — wizards are inherently interactive TUIs

Currently correct: prompt_toolkit is deferred to _get_runner() → _run_wizard(). Help and subcommand table build don't trigger it.

3. aws logs start-live-tail
   Always needed — it's a TUI command

Current bug: startlivetail.py has module-level prompt_toolkit imports, and it's imported during building-command-table.logs to inject the command. So every aws logs command (tail, describe-log-groups, etc.) pays the prompt_toolkit cost.

Fix: defer prompt_toolkit imports in startlivetail.py to method level, or restructure so the command class can be instantiated without importing prompt_toolkit

4. aws ecs monitor-express-gateway-service (with --mode INTERACTIVE)

Conditionally needed — only when display_mode == 'INTERACTIVE' (default when TTY is present, explicit via --mode)

Currently correct: prompt_toolkit_display.py is lazily imported inside the display strategy factory method. Other aws ecs commands are clean.

5. Autoprompt mode (--cli-auto-prompt / config cli_auto_prompt)

Conditionally needed — only when autoprompt mode resolves to 'on' or 'on-partial'

Currently correct: AutoPromptDriver is lazily imported inside _do_main only when the mode is active.

6. aws login (when no region is configured)
   Conditionally needed — only when _resolve_region falls through to _prompt_for_region (no --region flag, no region in config/profile)

Currently correct: lazy import inside _prompt_for_region.

In summary, there are two places on this current branch that import prompt_toolkit when it is not needed. One is all of the aws configure commands (as you pointed out), and the other is all aws logs commands.

Thoughts on your factory-with-enforcement proposal

I'm also not sure if scattering import statements across different files and within methods is the right approach. I'm wondering if what we want instead is an authoritative source for importing prompt_toolkit. For example, say we had 2 modules:

While I see your point, I don't see how the proposed factory approach solves the concern. So instead of importing prompt_toolkit, devs will need to call the factory functions that import prompt_toolkit. But the devs still have to make sure they call the factory function an optimal place. For example, a dev can just as easily call the factory function in the ConfigureCommand's init function, so it seems to replace the problem of where to scatter the imports with where to scatter the factory function calls.

A single entrypoint for importing prompt_toolkit also enables us to write linting rules that assert that prompt_toolkit is never imported outside of _prompt.py, providing an extra layer of protection.

I don't understand why forcing prompt_toolkit to be imported by a single module is a good restriction to have, since, as stated in the previous paragraph, the factory function calls still need to be placed well to avoid unnecessarily importing prompt_toolkit.

---

A lint rule saying "only these files may import prompt_toolkit" is just an allowlist — it prevents new files from adding prompt_toolkit imports, but it doesn't prevent the actual problem (those allowed files being imported too early).

The rule we actually want to enforce seems more like "prompt_toolkit must not be in sys.modules after create_clidriver() + driver.main() for commands that don't need it." That seems more of a runtime property, not a static lint check. We can't determine it from source analysis alone because it depends on which code paths execute.

What are your thoughts on adding a test for a subset of commands that asserts that prompt_toolkit is not in sys.modules while executing these commands? It would prevent future regression of the module sneaking back in over time:

def test_no_prompt_toolkit_imported_for_basic_commands():
    for cmd in ['s3 ls', 'configure list', 'logs describe-log-groups', 'ecs describe-services']:
        # run in subprocess, assert prompt_toolkit not in sys.modules