Skip to content

Fix exponential memory use from independent Fn::If conditions and YAML aliases - #442

Merged
satyakigh merged 9 commits into
mainfrom
perfo
Oct 13, 2026
Merged

satyakigh merged 9 commits into
mainfrom
perfo

Conversation

@satyakigh

@satyakigh satyakigh commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A value with N independent Fn::If conditions made the core enumerate all 2^N deployment scenarios as deep copies, retain several copies for the validation, and run a solver query per copy. An S3 bucket with 16 optional tags used 1.8 GB. CloudFormation places no limit on conditions, so lowering a count limit is not a valid fix. This change makes the cost polynomial in the number of independent conditions, in shared layers used by both engines.

Independent conditions no longer multiply. Consumers now ask only the question they need. Whole-Properties consumers project to the properties they read before expansion. The schema validator checks property presence and minItems, maxItems, and uniqueItems from member-presence conditions instead of enumerating every list. Cached scenarios are shared behind Arc rather than copied on every read.

Remaining joint expansions are bounded. Products are enumerated in full up to 4,096 combinations and 2^20 materialized nodes. Larger products use a pairwise-covering set that includes every member alternative and every pair of condition states. Checks that need larger coordinated worlds use dedicated member-presence or projection paths.

YAML expansion is bounded. Nested aliases can multiply a small document into millions of copied nodes. Alias resolution now has a document-wide budget of 2^20 copied nodes, and aliases that would exceed the structural nesting limit are rejected. Both cases return a located parse error instead of exhausting memory.

Performance

Measured with the optimized native CLI; 262 MB is the engine's fixed baseline:

Template Before After
S3 bucket, 16 optional tags 1.8 GB, 3.0 s, solver budget exhausted 262 MB, 0.17 s
Lambda function, 16 optional properties 1.56 GB, 0.74 s 262 MB, 0.12 s
Lambda function, 16 optional environment variables 1.42 GB, budget exhausted 262 MB, 0.16 s
IAM policy, 16 optional statements 1.47 GB, 22 s, budget exhausted 261 MB, 0.15 s
YAML alias expansion, 9^8 nodes over 3 GB, killed parse error, 371 MB

The regression fixtures show the same result with debug builds: a 17-subnet cardinality template exceeded 1.5 GB on main, while this branch completed in 1.1 s at 264 MB. Other projection fixtures completed 5-27× faster and peaked at 26-72% of main's memory.

Validation behavior changes

  • W2530 now evaluates every reachable SnapStart.ApplyOn branch and reports at the ApplyOn key. It is cleared only by an AWS::Lambda::Version whose FunctionName targets the function in a deployment where both exist. A Version that only depends on or describes the function, exists under an exclusive condition, or targets another function does not count. Unreachable branches do not warn. This intentionally differs from cfn-lint, which accepts any Version that is a graph child and reports warnings on unreachable branches.
  • F3037 now finds duplicates in arrays that mix literal and Ref items.
  • Templates that previously exhausted memory or the scenario budget now complete, so existing checks can report their findings. The new fixtures cover Route53 record projections, Fargate properties, item cardinality, SnapStart, and condition reachability.
  • YAML documents whose aliases exceed the node or expanded-depth limit receive a located parse error.
  • Above the covering threshold, the covering set does not represent findings that require three or more condition states jointly. Built-in checks with that need bypass it: schema item counts and uniqueness use member presence, while the Route53, ECS, RDS, and DynamoDB checks use projected properties. Every covering scenario is reachable, so the covering cannot introduce a false positive.

Related issue

None.

Checklist

  • Validation behavior changes are covered by focused templates and agree across all three engine selectors.
  • The code adheres to the contributing and design guidelines.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

@satyakigh
satyakigh merged commit d15effd into main Oct 13, 2026
17 of 19 checks passed
@satyakigh
satyakigh deleted the perfo branch October 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant