Skip to content

325: Add a Hilt screenshot test sample - #334

Open
DanielJette wants to merge 2 commits into
mainfrom
325-hilt-sample-test
Open

DanielJette wants to merge 2 commits into
mainfrom
325-hilt-sample-test

Conversation

@DanielJette

@DanielJette DanielJette commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What does this change accomplish?

Fixes #325

The Hilt recipe in #321 was approved but held back for having no compiled example behind it. This adds
one, so the recipe has the same footing as #318's FullscreenCaptureExampleTests.kt.

It also gives #207 an executable regression test. Comment out @AndroidEntryPoint on the host
activity and the test reproduces that issue's exception verbatim:

java.lang.IllegalStateException: Given component holder class
dev.testify.samples.flix.hilt.HiltComposableTestActivity does not implement interface
dagger.hilt.internal.GeneratedComponent or interface dagger.hilt.internal.GeneratedComponentManager

How have you achieved it?

A new Samples/Flix/FlixHilt library module with:

  • HiltComposableTestActivity — an @AndroidEntryPoint subclass of ComposableTestActivity.
  • HiltTestRunner — substitutes HiltTestApplication.
  • SampleViewModel — an @HiltViewModel with no dependencies, so the render is deterministic and a
    failure means the injection broke rather than the data changed.
  • HiltComposableScreenshotTest — @HiltAndroidTest with HiltAndroidRule ordered before
    ComposableScreenshotScenarioRule, plus its recorded baseline.

ComposableScreenshotRule can't be used: it hardcodes activityClass = ComposableTestActivity.
ComposableScreenshotScenarioRule lets the test choose the host.

Added to the Bitrise test_flix workflow, which is the only thing that exercises it.
flix_sample.yml's Samples/Flix/** path filter triggers that workflow but its steps only assemble
and run :FlixSample, so it does not cover this module.

Why a separate module, rather than adding this to :FlixSample

I tried :FlixSample first, since it already has Hilt, KSP and hiltViewModel() in use. It breaks
two existing tests, and the reason is worth recording:

testInstrumentationRunner is module-wide, so pointing Flix at HiltTestRunner replaces
FlixApplication with HiltTestApplication for every test in the module. FlixApplication
implements ImageLoaderFactory and supplies the Coil ImageLoader that FoundationModule builds on
an IdlingThreadPoolExecutor. Without it, Coil falls back to its default loader on an ordinary
background dispatcher, and the image-backed tests capture before their images have drawn:

$ ./gradlew FlixSample:screenshotTest      # with the runner switched
Tests run: 23,  Failures: 3
  HiltComposableScreenshotTest   - no baseline yet (expected)
  CastMemberScreenshotTest       - ScreenshotIsDifferentException
  MoviePosterScreenshotTest      - ScreenshotIsDifferentException

Both pass on main, individually and as a suite, so this was caused by the runner change and not
pre-existing. HomeScreenTest survived only because #327 gave it its own Dispatchers.Unconfined
loader in @Before.

A separate module leaves that suite alone. The host activity is in src/debug/java with
debugCompileOnly, matching the recipe step for step, so #321 can point at this as a worked example
without caveats. A library module's test APK is the application, so androidTest with an
androidTest manifest and no compileOnly also works — debug is chosen to mirror the recipe, not
because it is the only arrangement.

Scope of Impact and Testing instructions

Purely additive — a new sample module and one line in settings.gradle. No library, plugin or
existing sample code is touched, so no CHANGELOG entry.

Verified locally on an API 37 emulator:

$ ./gradlew FlixHilt:screenshotRecord
  ✓ Recording baseline for HiltComposableScreenshotTest_default.
$ ./gradlew FlixHilt:screenshotTest
  OK (1 test)
$ ./gradlew FlixSample:screenshotTest
  OK (22 tests)          # unchanged, which is the point

FlixHilt:ktlintCheck reports the same "Initial star should align" violation on the license header
that every other sample file has, including FlixSample's on main. Samples are not gated on
ktlint; I kept the house header rather than diverging from it.

@AndroidTestifyBot AndroidTestifyBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested

The sample builds, the baseline matches, and the reasoning for a separate module is sound. But it doesn't exercise the recipe it exists to back, one of its stated reasons is wrong, and the Bitrise edit changes an unrelated step.

1. It doesn't verify what #325 asked for

#325 lists six unverified steps from the recipe. Steps 1–4 are: the activity in src/debug/java, the debugCompileOnly pair, that compileOnly suffices for a subclass declared in the debug manifest, and the src/debug/AndroidManifest.xml declaration. This sample puts the activity in main with plain compileOnly, so those four are still unverified, while #321 now describes this module as "this recipe, compiled and running in CI".

A library module has a debug source set too. Moving the activity and its manifest entry to src/debug and using debugCompileOnly would make the sample mirror the recipe exactly, and "Fixes #325" would then be true.

2. "cannot reference test sources" is not true here

HiltComposableTestActivity's KDoc says it lives in main "because the activity has to be declared in a merged manifest, which cannot reference test sources". I moved the activity to src/androidTest/java, declared it in src/androidTest/AndroidManifest.xml, deleted both compileOnly lines, and ran FlixHilt:screenshotTest:

OK (1 test)

Same committed baseline, pixel-identical. For a library module the test APK is the app, so the androidTest manifest is merged into it. The main + compileOnly arrangement is a choice, not a constraint; please fix the comment (and the matching sentence in #321), whichever layout you keep.

3. The Bitrise edit moves verbose: true off the FlixLibrary step

The new block is inserted between FlixLibrary's test_runner and its - verbose: true, so that input now belongs to the FlixHilt step and the FlixLibrary step silently loses it. Insert after the verbose line and give the new step its own.

4. GitHub Actions does not cover it

flix_sample.yml already covers it through its Samples/Flix/** path filter.

The filter only triggers the workflow. Its steps assemble and run :FlixSample; FlixHilt is never built or tested there. Bitrise test_flix is the only coverage.

Smaller points

  • New files carry Copyright (c) 2023.
  • androidx.test:core-ktx:1.4.0 sits next to rules:1.7.0 and ext:junit:1.3.0.

Verified

  • ./gradlew FlixHilt:screenshotTest on a clean install, API 37 1080x2220@440dp: OK (1 test) against the committed baseline.
  • The APK lands at Samples/Flix/FlixHilt/build/outputs/apk/androidTest/debug/FlixHilt-debug-androidTest.apk, matching the Bitrise paths.
  • settings.gradle merges cleanly with #333.

@DanielJette

Copy link
Copy Markdown
Contributor Author

All four points addressed. Agreed on every one; two were defects I introduced.

1. The sample now mirrors the recipe

You were right that leaving the activity in main meant steps 1–4 of #325 stayed unverified while #321 claimed otherwise. Moved HiltComposableTestActivity to src/debug/java, its declaration to src/debug/AndroidManifest.xml, and both Testify artifacts to debugCompileOnly. So the sample is now step-for-step what the recipe tells a reader to do, and Fixes #325 is true.

Verified after uninstalling the test APK first, so it is a clean install rather than a leftover one:

$ adb uninstall dev.testify.samples.flix.hilt.test
Success
$ ./gradlew FlixHilt:screenshotTest
OK (1 test)

Same committed baseline, pixel-identical — git status shows no change to the PNG.

2. The "cannot reference test sources" claim is gone

Your src/androidTest experiment disproves it, and I should not have stated a constraint I had not tested. The KDoc now says debug is used to match the recipe, and explicitly notes that in a library module the activity works equally well from androidTest with an androidTest manifest and no compileOnly at all.

3. The Bitrise verbose: true

A real bug on my part — inserting the new block before that line silently moved FlixLibrary's input onto the FlixHilt step. FlixLibrary has its verbose: true back and FlixHilt has its own.

4. GitHub Actions

Correct, and my description was wrong: the path filter triggers flix_sample.yml but its steps only assemble and run :FlixSample. Bitrise test_flix is the only coverage. Description corrected.

Smaller points

  • Copyright years are now 2026 on all four new files.
  • Left androidx.test:core-ktx at 1.4.0: it is the version Samples/Flix and Samples/Legacy both pin, so matching them seemed better than this module being the only one on a different version. Happy to bump all three together if you would rather, but that felt like it belonged in its own change.

@AndroidTestifyBot AndroidTestifyBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

All four points addressed and verified.

  • The host activity is now in src/debug/java, declared in src/debug/AndroidManifest.xml, with both Testify artifacts as debugCompileOnly. The sample mirrors recipe steps 1–4, so Fixes #325 holds.
  • The KDoc no longer claims a constraint, and it records the androidTest alternative accurately.
  • FlixLibrary's Bitrise step has its verbose: true back, and FlixHilt has its own.
  • The description now credits Bitrise test_flix as the only coverage.
  • Keeping core-ktx:1.4.0 in line with the other two samples is fine by me.

Local, on the new head, API 37 emulator, test APK uninstalled first:

$ ./gradlew FlixHilt:screenshotTest     OK (1 test)
$ ./gradlew FlixSample:screenshotTest   OK (22 tests)

Re-run CI before merging. Run Screenshot Tests 0 and 1 on flix_sample.yml failed with INSTRUMENTATION_RESULT: shortMsg=Process crashed. while running the FlixSample suite. Your new commit only touches Samples/Flix/FlixHilt/** and bitrise.yml, neither of which that workflow builds. The previous head passed the same workflow, and FlixSample passes locally on this head, so it looks like an emulator flake, but it should be green before merge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a Hilt screenshot test example to the sample suite

2 participants