Repository navigation
fix(spanner): do not cache retryable errors in AbstractLazyInitializer - #14752
sakthivelmanii wants to merge 5 commits into
Conversation
When AbstractLazyInitializer.get() is called and initialize() fails with an exception, the failure was previously cached permanently regardless of whether the underlying error was transient or retryable. In MultiplexedSessionDatabaseClient and DatabaseClientImpl, this caused calls to getDialect() or getStatementFactory() to permanently cache transient startup or query errors (such as DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, or UNAVAILABLE) and continually rethrow the cached exception even after session creation succeeded on subsequent retries. This change modifies AbstractLazyInitializer to selectively cache only resource-not-found exceptions (DatabaseNotFoundException, InstanceNotFoundException, SessionNotFoundException) and non-retryable errors. Retryable SpannerExceptions (with error codes matching MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES: DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE) are rethrown without setting initialized = true, allowing subsequent get() calls to retry initialization. In addition: - Relaxed visibility of MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES from private to package-private to share the retryable error classification. - In DatabaseClientImpl.statementFactorySupplier, unwrapped ExecutionException to surface the underlying cause to SpannerExceptionFactory.asSpannerException, and converted TimeoutException via SpannerExceptionFactory.propagateTimeout to produce DEADLINE_EXCEEDED. - Added comprehensive unit tests in LazySpannerInitializerTest and DatabaseClientImplTest verifying retry behavior on transient errors and concurrent retry handling.
There was a problem hiding this comment.
Code Review
This pull request updates AbstractLazyInitializer to allow retrying initialization when transient, retryable errors occur, rather than caching them indefinitely. It also refactors exception handling in DatabaseClientImpl when retrieving the database dialect and exposes RETRYABLE_ERROR_CODES from MultiplexedSessionDatabaseClient. Comprehensive tests have been added to verify recovery behavior. The review feedback suggests simplifying the shouldCacheError method in AbstractLazyInitializer by removing redundant checks for specific exception subclasses, as they are already covered by the more general SpannerException check.
| @@ -49,6 +55,14 @@ public T get() throws Exception { | |||
| return object; | |||
| } | |||
|
|
|||
| private static boolean shouldCacheError(Exception e) { | |||
| if (e instanceof SpannerException) { | |||
| return !MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES.contains( | |||
There was a problem hiding this comment.
This class is a generic abstract class that in theory could be used for any kind of lazy initialization. It should not contain such specific code for the MultiplexedSessionDatabaseClient. You should rather design this by adding a method shouldCacheError(Exception) that by default returns false, and then extend this class in MultiplexedSessionClient and override the method there, and then add this logic in that method implementation.
There was a problem hiding this comment.
Made shouldCacheError(Exception) a protected method on AbstractLazyInitializer (returning true by default to preserve existing behavior for other initializers) and overridden it in MultiplexedSessionDatabaseClient and DatabaseClientImpl.
| * cached, the exception is rethrown without marking this initializer as initialized, allowing | ||
| * subsequent calls to retry {@link #initialize()}. | ||
| * | ||
| * <p>Defaults to {@code true} for backward compatibility. Subclasses can override this method to |
There was a problem hiding this comment.
nit: remove 'for backward compatibility.' That sentence only makes sense in the context of the current PR, but in one year from now, it will not make sense to a reader anymore.
There was a problem hiding this comment.
Removed 'for backward compatibility' from the Javadoc.
There was a problem hiding this comment.
I think that we can simplify this implementation to this:
private final AbstractLazyInitializer<StatementFactory> statementFactorySupplier =
new AbstractLazyInitializer<StatementFactory>() {
@Override
protected StatementFactory initialize() {
return new StatementFactory(getDialect());
}
@Override
protected boolean shouldCacheError(Exception e) {
// getDialect() already decides whether to cache or retry dialect errors.
// Never cache errors here so that statementFactorySupplier always delegates
// the retry/caching lifecycle entirely to getDialect().
return false;
}
};There was a problem hiding this comment.
Simplified statementFactorySupplier to call getDialect() directly and return false in shouldCacheError(Exception).
Problem
When
AbstractLazyInitializer.get()is invoked andinitialize()throws an exception, the failure was previously permanently cached inthis.errorwiththis.initialized = trueregardless of whether the error was transient or retryable. InMultiplexedSessionDatabaseClient.getDialect()andDatabaseClientImpl.getStatementFactory(), an initial transient failure during session creation or query execution (DEADLINE_EXCEEDED,RESOURCE_EXHAUSTED, orUNAVAILABLE) caused all subsequent calls to immediately rethrow the cached exception without retrying, even after multiplexed session creation succeeded on retry.Solution
Update
AbstractLazyInitializerto introduce aprotected boolean shouldCacheError(Exception)method that returnstrueby default, while allowing subclasses and client components to override it to avoid caching transient or retryable errors.AbstractLazyInitializer, defineprotected boolean shouldCacheError(Exception)returningtrueby default so retryable errors can be selectively retried on subsequentget()calls without permanent caching.MultiplexedSessionDatabaseClient(dialectSupplier), overrideshouldCacheError(Exception)to not cache retryableSpannerExceptionerrors matchingRETRYABLE_ERROR_CODES(DEADLINE_EXCEEDED,RESOURCE_EXHAUSTED,UNAVAILABLE).DatabaseClientImpl, simplifystatementFactorySupplierto callnew StatementFactory(getDialect())directly and overrideshouldCacheError(Exception)to returnfalse(delegating error caching and retry lifecycle entirely togetDialect()), and remove unusedgetDialectAsync()and unused imports.LazySpannerInitializerTestandDatabaseClientImplTestcovering retryable error recovery, non-retryable error caching, and concurrent retries.