Skip to content

fix(spanner): do not cache retryable errors in AbstractLazyInitializer - #14752

Open
sakthivelmanii wants to merge 5 commits into
mainfrom
fix/lazy-initializer-retryable-errors
Open

sakthivelmanii wants to merge 5 commits into
mainfrom
fix/lazy-initializer-retryable-errors

Conversation

@sakthivelmanii

@sakthivelmanii sakthivelmanii commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When AbstractLazyInitializer.get() is invoked and initialize() throws an exception, the failure was previously permanently cached in this.error with this.initialized = true regardless of whether the error was transient or retryable. In MultiplexedSessionDatabaseClient.getDialect() and DatabaseClientImpl.getStatementFactory(), an initial transient failure during session creation or query execution (DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, or UNAVAILABLE) caused all subsequent calls to immediately rethrow the cached exception without retrying, even after multiplexed session creation succeeded on retry.

Solution

Update AbstractLazyInitializer to introduce a protected boolean shouldCacheError(Exception) method that returns true by default, while allowing subclasses and client components to override it to avoid caching transient or retryable errors.

  • In AbstractLazyInitializer, define protected boolean shouldCacheError(Exception) returning true by default so retryable errors can be selectively retried on subsequent get() calls without permanent caching.
  • In MultiplexedSessionDatabaseClient (dialectSupplier), override shouldCacheError(Exception) to not cache retryable SpannerException errors matching RETRYABLE_ERROR_CODES (DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE).
  • In DatabaseClientImpl, simplify statementFactorySupplier to call new StatementFactory(getDialect()) directly and override shouldCacheError(Exception) to return false (delegating error caching and retry lifecycle entirely to getDialect()), and remove unused getDialectAsync() and unused imports.
  • Add unit tests in LazySpannerInitializerTest and DatabaseClientImplTest covering retryable error recovery, non-retryable error caching, and concurrent retries.

When AbstractLazyInitializer.get() is called and initialize() fails with an exception, the failure was previously cached permanently regardless of whether the underlying error was transient or retryable. In MultiplexedSessionDatabaseClient and DatabaseClientImpl, this caused calls to getDialect() or getStatementFactory() to permanently cache transient startup or query errors (such as DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, or UNAVAILABLE) and continually rethrow the cached exception even after session creation succeeded on subsequent retries.

This change modifies AbstractLazyInitializer to selectively cache only resource-not-found exceptions (DatabaseNotFoundException, InstanceNotFoundException, SessionNotFoundException) and non-retryable errors. Retryable SpannerExceptions (with error codes matching MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES: DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE) are rethrown without setting initialized = true, allowing subsequent get() calls to retry initialization.

In addition:
- Relaxed visibility of MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES from private to package-private to share the retryable error classification.
- In DatabaseClientImpl.statementFactorySupplier, unwrapped ExecutionException to surface the underlying cause to SpannerExceptionFactory.asSpannerException, and converted TimeoutException via SpannerExceptionFactory.propagateTimeout to produce DEADLINE_EXCEEDED.
- Added comprehensive unit tests in LazySpannerInitializerTest and DatabaseClientImplTest verifying retry behavior on transient errors and concurrent retry handling.
@sakthivelmanii
sakthivelmanii requested review from a team as code owners October 23, 2026 15:18

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates AbstractLazyInitializer to allow retrying initialization when transient, retryable errors occur, rather than caching them indefinitely. It also refactors exception handling in DatabaseClientImpl when retrieving the database dialect and exposes RETRYABLE_ERROR_CODES from MultiplexedSessionDatabaseClient. Comprehensive tests have been added to verify recovery behavior. The review feedback suggests simplifying the shouldCacheError method in AbstractLazyInitializer by removing redundant checks for specific exception subclasses, as they are already covered by the more general SpannerException check.

@@ -49,6 +55,14 @@ public T get() throws Exception {
return object;
}

private static boolean shouldCacheError(Exception e) {
if (e instanceof SpannerException) {
return !MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES.contains(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is a generic abstract class that in theory could be used for any kind of lazy initialization. It should not contain such specific code for the MultiplexedSessionDatabaseClient. You should rather design this by adding a method shouldCacheError(Exception) that by default returns false, and then extend this class in MultiplexedSessionClient and override the method there, and then add this logic in that method implementation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made shouldCacheError(Exception) a protected method on AbstractLazyInitializer (returning true by default to preserve existing behavior for other initializers) and overridden it in MultiplexedSessionDatabaseClient and DatabaseClientImpl.

* cached, the exception is rethrown without marking this initializer as initialized, allowing
* subsequent calls to retry {@link #initialize()}.
*
* <p>Defaults to {@code true} for backward compatibility. Subclasses can override this method to

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove 'for backward compatibility.' That sentence only makes sense in the context of the current PR, but in one year from now, it will not make sense to a reader anymore.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed 'for backward compatibility' from the Javadoc.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we can simplify this implementation to this:

  private final AbstractLazyInitializer<StatementFactory> statementFactorySupplier =
      new AbstractLazyInitializer<StatementFactory>() {
        @Override
        protected StatementFactory initialize() {
          return new StatementFactory(getDialect());
        }

        @Override
        protected boolean shouldCacheError(Exception e) {
          // getDialect() already decides whether to cache or retry dialect errors.
          // Never cache errors here so that statementFactorySupplier always delegates
          // the retry/caching lifecycle entirely to getDialect().
          return false;
        }
      };

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified statementFactorySupplier to call getDialect() directly and return false in shouldCacheError(Exception).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants