Found while fixing #245, and only partly addressed there.
runProcess reads the process's standard output and nothing else (Plugins/Gradle/src/main/kotlin/dev/testify/internal/ClientUtilities.kt):
val process = Runtime.getRuntime().exec(command)
val result = streamData.handleInputStream(process.inputStream)
process.waitFor()
return result
Two signals are thrown away:
- Standard error. Anything
adb or the device-side command writes there is invisible to the caller.
- The exit code.
waitFor()'s return value is discarded, so a command that failed outright looks the same as one that succeeded.
The concrete case this produced: with the application under test not installed, screenshotTest on a com.android.test module printed INSTRUMENTATION_STATUS: Error=Unable to find instrumentation target package, ran zero tests, and reported BUILD SUCCESSFUL. #333 fixes that one path by giving runProcess an opt-in redirectErrorStream that the am instrument call uses.
Every other call site still has the gap. screenshotPull, screenshotClear, reportPull and the device-setup tasks all go through Adb.execute(), so an adb failure in any of them is silent — a pull that copies nothing looks like a pull with nothing to copy.
Suggested fix: check the exit code in runProcess and surface a non-zero one. That is the better signal than string-matching stderr, but it needs checking per call site first: some commands are expected to fail benignly, and listFiles already appends 2>/dev/null precisely because a missing directory is normal. A blanket throw would break those.
Worth doing as its own change, with the call sites audited, rather than widening #333.
Found while fixing #245, and only partly addressed there.
runProcessreads the process's standard output and nothing else (Plugins/Gradle/src/main/kotlin/dev/testify/internal/ClientUtilities.kt):Two signals are thrown away:
adbor the device-side command writes there is invisible to the caller.waitFor()'s return value is discarded, so a command that failed outright looks the same as one that succeeded.The concrete case this produced: with the application under test not installed,
screenshotTeston acom.android.testmodule printedINSTRUMENTATION_STATUS: Error=Unable to find instrumentation target package, ran zero tests, and reportedBUILD SUCCESSFUL. #333 fixes that one path by givingrunProcessan opt-inredirectErrorStreamthat theam instrumentcall uses.Every other call site still has the gap.
screenshotPull,screenshotClear,reportPulland the device-setup tasks all go throughAdb.execute(), so anadbfailure in any of them is silent — a pull that copies nothing looks like a pull with nothing to copy.Suggested fix: check the exit code in
runProcessand surface a non-zero one. That is the better signal than string-matching stderr, but it needs checking per call site first: some commands are expected to fail benignly, andlistFilesalready appends2>/dev/nullprecisely because a missing directory is normal. A blanket throw would break those.Worth doing as its own change, with the call sites audited, rather than widening #333.