diff --git a/Runner/suites/Multimedia/CDSP/fastrpc_test/fastrpc_test_README.md b/Runner/suites/Multimedia/CDSP/fastrpc_test/fastrpc_test_README.md index 300365d62..313217565 100644 --- a/Runner/suites/Multimedia/CDSP/fastrpc_test/fastrpc_test_README.md +++ b/Runner/suites/Multimedia/CDSP/fastrpc_test/fastrpc_test_README.md @@ -36,6 +36,8 @@ Supported capabilities: `--domain`, `--domain-name`, `--pd-mode`, `--unsigned-pd`, `--verbose`) - Auto-discovery of system libraries and DSP skeletons for Yocto, Debian, Ubuntu, and RPM layouts - Runtime domain and PD discovery without SoC-name filtering +- Automatic readiness checks for enabled FastRPC DT declarations, independent of + successful remoteproc registration ## Prerequisites @@ -66,6 +68,8 @@ Have these on the target (or specify paths with the flags below): - Yocto, CentOS, and distributions without an exact package mapping remain image-managed and report SKIP when the binary is absent. A missing binary in an explicitly selected `--bin-dir` also reports SKIP without package changes. + These skips apply only after expected kernel registration passes the preflight + below. A registration failure takes precedence over a missing test payload. - Missing optional image assets are reported as SKIP unless the operator explicitly selected a domain whose required runtime library is absent. @@ -116,7 +120,7 @@ Env: ### Quick start ```bash -# Default: dynamically discover runnable domains and supported PD modes +# Default: validate DT-declared domains and discover runnable domains/PD modes ./run.sh # With repeat and timeout: @@ -171,6 +175,53 @@ package version. When runtime fallback discovers a domain but the installed `fas reports that it cannot represent that domain, the invocation is classified as FAIL so the image or package gap remains visible until a matching payload is provisioned. Other domains still execute. +Automatic selection also reads enabled FastRPC declarations from the running DT, +using the public `qcom,fastrpc` and `qcom,kaanapali-fastrpc` driver match entries +and their `label` property. Newer compatibles with the kaanapali fallback are +covered. A context-bank node or a generic DSP node alone is not an expectation. +The FastRPC node and every ancestor must have absent, `okay`, or `ok` status. +Disabled ancestors therefore do not create false failures. + +A mapped DT-declared domain is checked even if healthcheck omits it or reports it +offline/unsupported, or remoteproc registration and the endpoint are absent. +Missing readiness, endpoints, or required runtime libraries record FAIL while +other healthy domains continue. Missing optional shared test assets cannot +overwrite an already recorded readiness failure with SKIP. With no DT or no +enabled FastRPC declarations, the existing healthcheck/runtime fallback remains +in effect. Failed DT traversal, unreadable properties, or a missing required +FastRPC label fail discovery. Labels outside the suite's protocol map are logged +as untestable, not guessed or mapped to another DSP. + +Explicit CLI or environment selection checks only the requested domain, without +requiring DT discovery. No SoC/board lists, distro gates, new parameters, or new +package installation paths are introduced. + +When the test binary is absent, a read-only registration preflight runs before +the missing-payload SKIP or existing package-provider handling. It checks every +DT-expected domain, or only the explicitly selected domain, for a running +remoteproc and a normal or secure FastRPC endpoint. Missing/offline registration, +missing endpoints, invalid explicit selection and DT discovery failures report +FAIL even without `fastrpc_test`. Healthy registration is logged as readiness, +never as functional PASS. If registration is healthy or no domain is expected, +the existing binary/package policy proceeds, including SKIP for missing optional +payloads on image-managed systems. This preflight neither loads modules nor +starts remoteprocs, creates endpoints, or installs packages. + +The installed-binary path is unchanged and still prefers `fastrpc-healthcheck` +for execution capability, with the existing remoteproc fallback when absent. +The missing-binary preflight checks kernel registration only, it cannot prove +client support, PD support or functional RPC execution. + +Automatic mode uses only the running system's enabled FastRPC DT declarations +and discovered runtime capabilities. If CDSP is absent from both DT and runtime, +other discovered domains continue, or the suite reports SKIP when none apply. +Removed or disabled DT nodes do not create historical or board-specific domain +requirements. A domain still exposed by runtime discovery remains eligible even +without an enabled DT declaration. Triage determines whether absent hardware was +intentional. Enabled DT declarations with missing/offline registration still fail. +Explicit `--domain-name cdsp` continues to check CDSP only and fails when it is +unavailable. No mandatory-domain CI policy or new requirement option is introduced. + ### LAVA integration example ``` @@ -210,10 +261,11 @@ package gap remains visible until a matching payload is provisioned. Other domai executed, its raw output is retained. A normalized six-column table containing domain, state, signed-PD support, unsigned-PD support, normalized FastRPC support, and the support reason is retained only when strict parsing succeeds. -- **Offline automatic domain**: Logged and excluded from automatic selection. Other runnable - domains continue. -- **Unsupported automatic domain**: Logged with the healthcheck reason and excluded. Other - runnable domains continue. +- **Offline or unsupported automatic domain**: FAIL when an enabled FastRPC DT declaration + expects it, otherwise logged and excluded. Other runnable domains continue. +- **DT expectation evidence**: `fastrpc_dt_expected_domains.log` records numeric ID, label, + full node path and compatible list as tab-separated fields. The checked traversal paths + and errors are retained in `fastrpc_dt_compatible_paths.log` and `fastrpc_dt_scan_errors.log`. - **Test payload lacks a runtime-ready domain**: A retained invocation log proving that the installed client cannot represent the selected numeric domain is classified as FAIL. This is feature probing, not a package-version check, so it behaves the same on Yocto, Debian, Ubuntu, @@ -222,12 +274,14 @@ package gap remains visible until a matching payload is provisioned. Other domai - **Missing runtime artifacts**: When the capability source is `fastrpc-healthcheck`, a domain missing its endpoint or required runtime library fails the suite — healthcheck has declared it supported, so the absence indicates a broken installation. When the capability source is the - runtime fallback, such a domain is logged and excluded while other runnable domains continue. - Missing shared test libraries or DSP skeletons skip the suite before execution. + runtime fallback, such a domain fails if declared by enabled FastRPC DT, otherwise it is + logged and excluded while other runnable domains continue. Missing shared test libraries + or DSP skeletons skip execution but do not clear prior readiness failures. - **Binary resolved to /bin/fastrpc_test**: Blocked by default. Set `ALLOW_BIN_FASTRPC=1` or use `--bin-dir` to a non-`/bin` path. -- **Missing binary on any distribution**: Package installation is not attempted. The suite reports - SKIP with the expected image-provided binary path. +- **Missing binary**: Expected kernel registration is checked first and any failure + is reported before optional-payload handling. Otherwise the existing package + policy described under Prerequisites applies, with SKIP on image-managed paths. - **Session create errors with -U 1**: If unsigned PD returns `0x80000416`, confirm your image includes unsigned shells/policies (or use `--pd-mode signed-only`). - **Domain not discovered**: Check `dmesg` for remoteproc firmware load errors. The test @@ -257,6 +311,34 @@ package gap remains visible until a matching payload is provisioned. Other domai takes precedence over runtime discovery. - If `fastrpc_test` is not in the default path, use `--bin-dir` to specify its location. +## Host regression checks + +From the repository root: + +```sh +dash scripts/tests/fastrpc-domain-validation.sh +# Focused current-DT/runtime absence policy checks: +dash scripts/tests/fastrpc-domain-validation.sh --absence-only +dash scripts/tests/fastrpc-startup-validation.sh dash +``` + +These host-only fixtures exercise DT expectations and the actual runner selection +and result-finalization blocks. Remoteproc, endpoint, library, and invocation +boundaries are mocked. Coverage includes missing/offline CDSP with healthy ADSP, +healthcheck omission and unsupported reports, missing endpoints/libraries, +disabled ancestors, explicit overrides, no-DT fallback, optional asset skips, +compatible fallback lists, symlink roots, and malformed labels. They do not +perform DUT operations or establish hardware/cross-distribution validation. +The absence-only subset checks ADSP continuing without CDSP, explicit absent-CDSP +failure, runtime CDSP without a DT declaration, and SKIP with no applicable domain. + +The startup harness additionally executes an unchanged copy of the full runner +with a temporary bootstrap and mocked hardware/package boundaries. It checks +missing-binary precedence, CLI/environment selection, absent/offline remoteprocs, +missing endpoints, discovery errors, preservation of the installed-binary path, +and existing provider outcomes. No package installation or real RPC invocation +is performed by these host checks. + ## License SPDX-License-Identifier: BSD-3-Clause diff --git a/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh b/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh index b48ec7a0a..af83bab43 100755 --- a/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh +++ b/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh @@ -308,6 +308,15 @@ log_info "Kernel: $(uname -a 2>/dev/null || echo N/A)" log_info "Date(UTC): $(date -u 2>/dev/null || echo N/A)" log_soc_info +# -------------------- Logging root ----------------------------- +TS="$(date +%Y%m%d-%H%M%S)" +LOG_ROOT="./logs_${TESTNAME}_${TS}" +if ! mkdir -p "$LOG_ROOT"; then + test_result_finish "FAIL" "$TESTNAME FAIL - cannot create log directory: $LOG_ROOT" +fi + +log_info "Evidence directory: $test_path/$LOG_ROOT" + # -------------------- Binary directory resolution ----------------- if [ -n "$BIN_DIR" ]; then : @@ -315,6 +324,18 @@ else BIN_DIR="/usr/bin" fi +RUN_DIR="$BIN_DIR" +RUN_BIN="$RUN_DIR/fastrpc_test" + +# Missing optional payloads must not hide expected kernel registration failures. +# Keep package/layout handling and the installed-binary execution path unchanged. +if [ ! -x "$RUN_BIN" ]; then + log_info "FastRPC test binary is unavailable at $RUN_BIN, checking expected kernel registration first" + if ! fastrpc_registration_preflight "$LOG_ROOT"; then + test_result_finish + fi +fi + case "$BIN_DIR" in /bin) if [ "${ALLOW_BIN_FASTRPC:-0}" != "1" ]; then @@ -323,9 +344,6 @@ case "$BIN_DIR" in ;; esac -RUN_DIR="$BIN_DIR" -RUN_BIN="$RUN_DIR/fastrpc_test" - if [ ! -x "$RUN_BIN" ]; then if [ "$RUN_DIR" = "/usr/bin" ] && command -v pkg_ensure_host_distro_package_set_present >/dev/null 2>&1; then @@ -351,15 +369,6 @@ if [ ! -x "$RUN_BIN" ]; then fi fi -# -------------------- Logging root ----------------------------- -TS="$(date +%Y%m%d-%H%M%S)" -LOG_ROOT="./logs_${TESTNAME}_${TS}" -if ! mkdir -p "$LOG_ROOT"; then - test_result_finish "FAIL" "$TESTNAME FAIL - cannot create log directory: $LOG_ROOT" -fi - -log_info "Evidence directory: $test_path/$LOG_ROOT" - fastrpc_capture_qrtr_evidence "$LOG_ROOT" # Prepare the image-provided DSP artifacts before healthcheck records capability. @@ -432,6 +441,19 @@ fi DOMAINS_TO_TEST="$(resolve_domains_to_test)" +# Expectations must not disappear when remoteproc or FastRPC registration fails. +# Only current enabled DT declarations add expectations, not historical board data. +# Explicit CLI/environment selection retains its scope and does not require DT. +EXPECTED_DOMAINS="" +if [ "$SELECTION_SOURCE" = "automatic" ]; then + if ! EXPECTED_DOMAINS=$(fastrpc_capture_expected_domains "$LOG_ROOT"); then + test_result_finish "FAIL" "$TESTNAME FAIL - cannot discover expected FastRPC DT domains, check DT access and labels, artifact=$LOG_ROOT/fastrpc_dt_scan_errors.log" + fi + for d in $EXPECTED_DOMAINS; do + DOMAINS_TO_TEST=$(fastrpc_append_word_unique "$DOMAINS_TO_TEST" "$d") + done +fi + if [ -z "$DOMAINS_TO_TEST" ] && [ "$SELECTION_SOURCE" != "automatic" ]; then test_result_finish "FAIL" "$TESTNAME FAIL - explicit domain selection is invalid or unavailable, source=$SELECTION_SOURCE" fi @@ -444,6 +466,15 @@ fi RUNNABLE_DOMAINS="" REQUESTED_PDS="$(requested_pds)" for d in $DOMAINS_TO_TEST; do + domain_required=0 + if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then + domain_required=1 + fi + case " $EXPECTED_DOMAINS " in + *" $d "*) + domain_required=1 + ;; + esac fastrpc_domain_ready "$d" domain_ready_status=$? @@ -451,6 +482,10 @@ for d in $DOMAINS_TO_TEST; do if [ "$SELECTION_SOURCE" != "automatic" ]; then test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") is not ready, selection=$SELECTION_SOURCE" fi + if [ "$domain_required" -eq 1 ]; then + test_result_record "FAIL" "$(domain_to_name "$d") is expected but not ready, check remoteproc firmware and FastRPC registration, capability_source=$CAPABILITY_SOURCE DT=$LOG_ROOT/fastrpc_dt_expected_domains.log" + continue + fi log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=runtime-not-ready" continue @@ -458,8 +493,9 @@ for d in $DOMAINS_TO_TEST; do if ! fastrpc_domain_endpoint_available "$d"; then label="$(domain_to_endpoint_label "$d")" - if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then - test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") has no usable endpoint, expected=/dev/fastrpc-${label} or /dev/fastrpc-${label}-secure capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE" + if [ "$domain_required" -eq 1 ]; then + test_result_record "FAIL" "$(domain_to_name "$d") has no usable endpoint, expected=/dev/fastrpc-${label} or /dev/fastrpc-${label}-secure, check FastRPC driver registration, capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE" + continue fi log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=endpoint-absent expected=/dev/fastrpc-${label}[-secure]" @@ -469,8 +505,9 @@ for d in $DOMAINS_TO_TEST; do domain_library="$(fastrpc_domain_system_library_path "$d" || true)" if [ -z "$domain_library" ]; then required_library="$(fastrpc_domain_field "$d" 7)" - if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then - test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") runtime library is missing, expected=$required_library checked=$FASTRPC_LIB_SYS_DIRS_CHECKED capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE" + if [ "$domain_required" -eq 1 ]; then + test_result_record "FAIL" "$(domain_to_name "$d") runtime library is missing, provision $required_library in the image, checked=$FASTRPC_LIB_SYS_DIRS_CHECKED capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE" + continue fi log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=runtime-library-missing expected=$required_library checked=$FASTRPC_LIB_SYS_DIRS_CHECKED" @@ -495,11 +532,13 @@ done DOMAINS_TO_TEST="$RUNNABLE_DOMAINS" if [ -z "$DOMAINS_TO_TEST" ]; then - test_result_finish "SKIP" "$TESTNAME SKIP - no automatically discovered domain has complete readiness, endpoint, library, and requested-PD support" + test_result_record "SKIP" "No selected domain has complete readiness, endpoint, library, and requested-PD support" + test_result_finish fi if ! fastrpc_validate_runtime_artifacts; then - test_result_finish "SKIP" "$TESTNAME SKIP - shared FastRPC test artifacts are incomplete: $FASTRPC_ARTIFACT_ERROR" + test_result_record "SKIP" "Shared FastRPC test artifacts are incomplete: $FASTRPC_ARTIFACT_ERROR" + test_result_finish fi log_info "Domain mode: $DOMAIN_MODE" @@ -546,6 +585,9 @@ log_info "[FASTRPC-MATRIX] combinations=$matrix_combinations repeats=$REPEAT inv # -------------------- Run loop --------------------------------- # Nested loop over domains and PDs RUNTIME_FAILURE_SEEN=0 +if [ "$TEST_RESULT_FAIL_COUNT" -gt 0 ]; then + RUNTIME_FAILURE_SEEN=1 +fi # Track per-domain/per-PD results during execution # Format: "DOMAIN:PD:pass_count:fail_count:subtests_total:subtests_pass:subtests_fail:subtests_skip" @@ -776,7 +818,7 @@ EOF log_info "$SUMMARY_SEP" _total_count=$((TEST_RESULT_PASS_COUNT + TEST_RESULT_FAIL_COUNT)) overall_inv_line="$(printf '%-20s Total:%6d | Passed:%6d | Failed:%6d' \ - 'Overall invocations:' "$_total_count" "$TEST_RESULT_PASS_COUNT" "$TEST_RESULT_FAIL_COUNT")" + 'Overall checks:' "$_total_count" "$TEST_RESULT_PASS_COUNT" "$TEST_RESULT_FAIL_COUNT")" log_info "$overall_inv_line" overall_sub_line="$(printf '%-20s Total:%6d | Passed:%6d | Failed:%6d | Skipped:%6d' \ diff --git a/Runner/utils/lib_fastrpc.sh b/Runner/utils/lib_fastrpc.sh index 1e09a6fac..2c6e28db5 100755 --- a/Runner/utils/lib_fastrpc.sh +++ b/Runner/utils/lib_fastrpc.sh @@ -1234,19 +1234,12 @@ EOF return 2 } -# fastrpc_domain_ready DOMAIN_ID -# Validate domain readiness from parsed healthcheck data or remoteproc fallback evidence. -# Input: numeric domain ID and optional FASTRPC_HEALTHCHECK_TSV. Output: no stdout contract. -# Returns: 0 when ready, 1 when unhealthy, 2 when runtime evidence is absent. -# Side effects: emits failure diagnostics. -fastrpc_domain_ready() { +# fastrpc_remoteproc_domain_ready DOMAIN_ID +# Check exact-domain kernel registration without requiring a userspace test payload. +# Input: numeric domain ID. Output: diagnostic logging, no data on stdout. +# Returns 0 when running, 1 when unhealthy, 2 when absent. Read-only probes. +fastrpc_remoteproc_domain_ready() { domain_id="$1" - - if [ -s "${FASTRPC_HEALTHCHECK_TSV:-}" ]; then - fastrpc_healthcheck_domain_ready "$domain_id" - return $? - fi - remoteproc_record="$(fastrpc_remoteproc_domain_state "$domain_id")" if [ -z "$remoteproc_record" ]; then log_fail "[FASTRPC-CAPABILITY] domain=$(domain_to_name "$domain_id") expected=runtime-remoteproc observed=not-found" @@ -1265,6 +1258,161 @@ EOF return 0 } +# fastrpc_domain_ready DOMAIN_ID +# Validate execution readiness from healthcheck or remoteproc fallback evidence. +# Input: numeric domain ID and optional FASTRPC_HEALTHCHECK_TSV. Output: diagnostics. +# Returns 0 when ready, 1 when unhealthy, 2 when absent. Read-only probes. +fastrpc_domain_ready() { + if [ -s "${FASTRPC_HEALTHCHECK_TSV:-}" ]; then + fastrpc_healthcheck_domain_ready "$1" + else + fastrpc_remoteproc_domain_ready "$1" + fi +} + +# fastrpc_registration_preflight RESULT_DIR +# Check expected kernel registration before a missing test binary can cause SKIP. +# Uses CLI/environment selection when explicit, otherwise enabled FastRPC DT labels. +# This is a read-only kernel inventory check, not a healthcheck/client capability +# substitute. The normal healthcheck-based execution path remains unchanged. +# Inputs: result directory and existing selection globals. Output: diagnostics. +# Returns 0 when expectations are met or absent, 1 on discovery/readiness failure. +# Records only FAIL results, never functional PASS. Writes DT evidence in RESULT_DIR. +fastrpc_registration_preflight() { + frp_result_dir="$1" + frp_failed=0 + + if [ -n "${CLI_DOMAIN_NAME:-}" ] || [ -n "${CLI_DOMAIN:-}" ] || + [ -n "${FASTRPC_DOMAIN_NAME:-}" ] || [ -n "${FASTRPC_DOMAIN:-}" ] || + [ "${DOMAIN_MODE:-all-supported}" = "single" ]; then + frp_domains=$(resolve_domains_to_test) + if [ -z "$frp_domains" ]; then + test_result_record "FAIL" "Explicit FastRPC domain selection is invalid, provide a supported --domain or --domain-name even when the test binary is absent" + return 1 + fi + else + if ! frp_domains=$(fastrpc_capture_expected_domains "$frp_result_dir"); then + test_result_record "FAIL" "Cannot discover expected FastRPC domains before checking the test payload, check DT access and labels, evidence=$frp_result_dir" + return 1 + fi + fi + + for frp_domain in $frp_domains; do + if ! fastrpc_remoteproc_domain_ready "$frp_domain"; then + test_result_record "FAIL" "$(domain_to_name "$frp_domain") is expected but not registered and running, check remoteproc firmware and driver startup, missing fastrpc_test does not waive this requirement, evidence=$frp_result_dir" + frp_failed=1 + elif ! fastrpc_domain_endpoint_available "$frp_domain"; then + frp_label=$(domain_to_endpoint_label "$frp_domain") + test_result_record "FAIL" "$(domain_to_name "$frp_domain") is expected but its FastRPC endpoint is absent, check driver registration for /dev/fastrpc-${frp_label} or /dev/fastrpc-${frp_label}-secure, evidence=$frp_result_dir" + frp_failed=1 + else + log_info "[FASTRPC-PREFLIGHT] domain=$(domain_to_name "$frp_domain") kernel_registration=ready, functional validation still requires the test payload" + fi + done + + [ "$frp_failed" -eq 0 ] +} + +# fastrpc_dt_node_available NODE ROOT +# Check NODE and every ancestor up to the resolved DT ROOT using OF status rules. +# Inputs: absolute paths. No stdout. Returns 0 when enabled, 1 when disabled, +# or 2 on unreadable status or a path outside ROOT. Diagnostics go to stderr. +fastrpc_dt_node_available() { + fdna_node="$1" + fdna_root="$2" + case "$fdna_node/" in + "$fdna_root/"*) + ;; + *) + return 2 + ;; + esac + + while :; do + if [ -e "$fdna_node/status" ]; then + if ! fdna_status=$(dt_property_text "$fdna_node" status); then + log_fail "Cannot read DT status at $fdna_node, check DT access permissions" >&2 + return 2 + fi + case "$fdna_status" in + okay|ok) + ;; + *) + log_info "[FASTRPC-DT] excluded=$1 ancestor=$fdna_node status=$fdna_status" >&2 + return 1 + ;; + esac + fi + [ "$fdna_node" = "$fdna_root" ] && return 0 + fdna_node=${fdna_node%/*} + done +} + +# fastrpc_capture_expected_domains RESULT_DIR +# Capture enabled FastRPC declarations independently of successful registration. +# Reuses DT root/property helpers. The generic compatible enumerator does not +# check ancestors or propagate traversal errors, so retain a checked scan here. +# Output: space-separated expected domain IDs on stdout, diagnostics on stderr. Returns 0 on a +# complete scan (including absent DT), 1 on discovery or artifact failure. +# Writes fastrpc_dt_{compatible_paths,expected_domains,scan_errors}.log in RESULT_DIR. +# Expected rows are domain ID, label, node path and compatible list, tab-separated. +fastrpc_capture_expected_domains() { + fced_dir="$1" + fced_paths="$fced_dir/fastrpc_dt_compatible_paths.log" + fced_table="$fced_dir/fastrpc_dt_expected_domains.log" + fced_errors="$fced_dir/fastrpc_dt_scan_errors.log" + : >"$fced_table" || return 1 + : >"$fced_paths" || return 1 + : >"$fced_errors" || return 1 + + if ! fced_root=$(dt_runtime_root); then + log_info "[FASTRPC-DT] Runtime DT is absent, retaining healthcheck or remoteproc discovery" >&2 + return 0 + fi + fced_root=$(readlink -f "$fced_root") || return 1 + if ! find "$fced_root" -type f -name compatible >"$fced_paths" 2>"$fced_errors"; then + log_fail "Cannot enumerate runtime DT at $fced_root, check DT access, artifact=$fced_errors" >&2 + return 1 + fi + + while IFS= read -r fced_path; do + fced_node=${fced_path%/*} + if ! fced_compat=$(dt_property_text "$fced_node" compatible); then + log_fail "Cannot read DT compatible at $fced_node, check DT access permissions" >&2 + return 1 + fi + # Both are public driver match entries. Newer SoC bindings include the + # kaanapali fallback. Context-bank compatibles must not match here. + case " $fced_compat " in + *" qcom,fastrpc "*|*" qcom,kaanapali-fastrpc "*) + ;; + *) + continue + ;; + esac + if fastrpc_dt_node_available "$fced_node" "$fced_root"; then + : + else + fced_status=$? + [ "$fced_status" -eq 1 ] && continue + return 1 + fi + if ! fced_label=$(dt_property_text "$fced_node" label) || [ -z "$fced_label" ]; then + log_fail "Enabled FastRPC DT node lacks a readable label at $fced_node, fix the image DT" >&2 + return 1 + fi + fced_domain=$(name_to_domain "$fced_label") + if [ -z "$fced_domain" ]; then + log_warn "[FASTRPC-DT] label=$fced_label node=$fced_node is outside the test client's domain map, extend domain support before testing it" >&2 + continue + fi + printf '%s\t%s\t%s\t%s\n' "$fced_domain" "$fced_label" "$fced_node" "$fced_compat" >>"$fced_table" || return 1 + log_info "[FASTRPC-DT] expected=$(domain_to_name "$fced_domain") label=$fced_label node=$fced_node compatible=$fced_compat" >&2 + done <"$fced_paths" + + awk -F '\t' '!seen[$1]++ {printf "%s%s", separator, $1; separator=" "}' "$fced_table" +} + # discover_supported_domains # Dynamically select runnable FastRPC domains from healthcheck or runtime fallback data. # Inputs: healthcheck globals, DT, remoteproc, and FastRPC endpoint devices. diff --git a/scripts/tests/fastrpc-domain-validation.sh b/scripts/tests/fastrpc-domain-validation.sh new file mode 100644 index 000000000..8ad876370 --- /dev/null +++ b/scripts/tests/fastrpc-domain-validation.sh @@ -0,0 +1,308 @@ +#!/bin/sh +# Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. +# SPDX-License-Identifier: BSD-3-Clause +# Host-only checks: sh scripts/tests/fastrpc-domain-validation.sh [--absence-only]. +# Hardware boundaries are mocked, DT parsing, domain policy and result helpers are real. +# shellcheck disable=SC2034 # Globals are consumed by sourced production helpers/runner blocks. +set -eu + +validation_mode=${1:-all} +case "$validation_mode" in + all|--absence-only) + ;; + *) + printf 'Usage: %s [--absence-only]\n' "$0" >&2 + exit 2 + ;; +esac + +validation_repo=$(CDPATH='' cd -- "$(dirname "$0")/../.." && pwd) +validation_tmp=$(mktemp -d "${TMPDIR:-/tmp}/fastrpc-domain-validation.XXXXXX") + +# validation_cleanup +# Remove only this process's mktemp fixture tree. No stdout, returns rm status. +validation_cleanup() { + case "$validation_tmp" in + "${TMPDIR:-/tmp}"/fastrpc-domain-validation.*) + rm -rf -- "$validation_tmp" + ;; + esac +} +trap validation_cleanup EXIT +trap 'exit 1' HUP INT TERM + +# shellcheck disable=SC1091 +. "$validation_repo/Runner/utils/functestlib.sh" +# shellcheck disable=SC1091 +. "$validation_repo/Runner/utils/lib_fastrpc.sh" + +# Execute the actual runner selection/early-exit block, not a copied policy. +validation_runner="$validation_repo/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh" +awk ' + /^# -------------------- Domain and PD selection/ {emit=1} + /^# -------------------- Buffering tool availability/ {emit=0} + emit {print} +' "$validation_runner" >"$validation_tmp/selection.sh" +awk ' + /^_total_count=/ {print} + /^# Final result determination/ {emit=1} + emit {print} +' "$validation_runner" >"$validation_tmp/final.sh" +test -s "$validation_tmp/selection.sh" +test -s "$validation_tmp/final.sh" + +# dt_runtime_root +# Test seam for the production root resolver. Output fixture path or status 1. +dt_runtime_root() { + [ -d "$mock_dt" ] || return 1 + printf '%s\n' "$mock_dt" +} + +# get_remoteproc_by_firmware ALIAS +# Supply exact-domain remoteproc records including offline states on stdout. +get_remoteproc_by_firmware() { + mock_id=$(name_to_domain "$1") + case " $mock_remoteprocs " in + *" $mock_id "*) + mock_state=running + [ "$mock_id" != "$mock_offline" ] || mock_state=offline + mock_name=$(domain_to_endpoint_label "$mock_id") + printf '/sys/class/remoteproc/remoteproc%s|%s|%s.mdt|%s\n' \ + "$mock_id" "$mock_state" "$mock_name" "$mock_name" + ;; + esac +} + +# dt_has_remoteproc_fw ALIAS +# Generic firmware DT evidence is deliberately absent in these fixtures. +dt_has_remoteproc_fw() { + return 1 +} + +# fastrpc_domain_endpoint_available ID +# Mock normal/secure endpoint inventory. Return success for listed domains. +fastrpc_domain_endpoint_available() { + case " $mock_endpoints " in + *" $1 "*) + return 0 + ;; + esac + return 1 +} + +# fastrpc_domain_system_library_path ID +# Mock installed runtime libraries. Print a path only for listed domains. +fastrpc_domain_system_library_path() { + case " $mock_libraries " in + *" $1 "*) + printf '/mock/lib/%s\n' "$(fastrpc_domain_field "$1" 7)" + ;; + esac +} + +# fastrpc_validate_runtime_artifacts +# Test optional shared-asset gating without loading a DSP payload. +fastrpc_validate_runtime_artifacts() { + FASTRPC_ARTIFACT_ERROR="mock optional skeleton absent" + [ "$mock_assets" = yes ] +} + +# validation_node PATH LABEL COMPATIBLES +# Build NUL-delimited DT properties below the fixture root. Returns write status. +validation_node() { + mkdir -p "$mock_dt/$1" + printf '%b\000' "$3" >"$mock_dt/$1/compatible" + printf '%s\000' "$2" >"$mock_dt/$1/label" +} + +# validation_case NAME EXPECTED_RESULT +# Run real selection and final classification with mocked successful invocations. +# Assert verdict plus a healthy ADSP's continued selection where applicable. +validation_case() { + validation_name="$1" + validation_expected="$2" + LOG_ROOT="$validation_tmp/$validation_name" + mkdir -p "$LOG_ROOT" + ( + set +e + TESTNAME=fastrpc_test + test_result_init "$TESTNAME" "$LOG_ROOT/fastrpc_test.res" + # shellcheck disable=SC1091 + . "$validation_tmp/selection.sh" + for validation_domain in $DOMAINS_TO_TEST; do + test_result_record PASS "mock invocation domain=$validation_domain" + done + # shellcheck disable=SC1091 + . "$validation_tmp/final.sh" + ) >"$LOG_ROOT/stdout.log" 2>&1 + if [ "$(cat "$LOG_ROOT/fastrpc_test.res")" != "fastrpc_test $validation_expected" ]; then + cat "$LOG_ROOT/stdout.log" + printf 'FAIL: %s expected %s\n' "$validation_name" "$validation_expected" >&2 + exit 1 + fi + if [ "$mock_expect_adsp" = yes ]; then + grep -q 'mock invocation domain=0' "$LOG_ROOT/stdout.log" + fi + printf 'PASS: %s\n' "$validation_name" +} + +CLI_DOMAIN="" +CLI_DOMAIN_NAME="" +FASTRPC_DOMAIN="" +FASTRPC_DOMAIN_NAME="" +FASTRPC_UNSIGNED_PD=0 +UNSIGNED_PD_FLAG=0 +DOMAIN_MODE=all-supported +PD_MODE=both +VERBOSE=0 +FASTRPC_LIB_SYS_DIRS_CHECKED=/mock/lib +FASTRPC_HEALTHCHECK_TSV="" +CAPABILITY_SOURCE=runtime-fallback +mock_dt="$validation_tmp/dt" +mock_remoteprocs="0 3" +mock_endpoints="0 3" +mock_libraries="0 3" +mock_offline="" +mock_assets=yes +mock_expect_adsp=yes + +# Current-DT expectations must not invent a removed domain. Use real DT parsing +# and runner selection with healthcheck inventory that contains only ADSP. +mock_dt="$validation_tmp/adsp-only-dt" +mock_remoteprocs=0 +mock_endpoints=0 +mock_libraries=0 +FASTRPC_HEALTHCHECK_TSV="$validation_tmp/absence-healthcheck.tsv" +CAPABILITY_SOURCE=fastrpc-healthcheck +validation_node soc/adsp/fastrpc adsp qcom,fastrpc +printf 'ADSP\tonline\tyes\tno\tyes\tsupported\n' >"$FASTRPC_HEALTHCHECK_TSV" +validation_case absent-cdsp-adsp-continues PASS +if grep -q 'mock invocation domain=3' "$LOG_ROOT/stdout.log"; then + exit 1 +fi +test "$(wc -l <"$LOG_ROOT/fastrpc_dt_expected_domains.log")" -eq 1 + +CLI_DOMAIN_NAME=cdsp +mock_expect_adsp=no +validation_case explicit-cdsp-absent-from-dt-and-runtime FAIL +CLI_DOMAIN_NAME="" + +# A runtime-exposed domain is still tested even without its DT declaration. +mock_endpoints="0 3" +mock_libraries="0 3" +mock_expect_adsp=yes +printf 'CDSP\tonline\tyes\tno\tyes\tsupported\n' >>"$FASTRPC_HEALTHCHECK_TSV" +validation_case runtime-cdsp-without-dt PASS +grep -q 'mock invocation domain=3' "$LOG_ROOT/stdout.log" + +# An existing DT with no FastRPC nodes is not a discovery error. Non-supported +# healthcheck inventory plus no declarations must produce SKIP, not invented FAIL. +mock_dt="$validation_tmp/no-fastrpc-dt" +mkdir -p "$mock_dt" +mock_remoteprocs="" +mock_endpoints="" +mock_expect_adsp=no +printf 'ADSP\toffline\tno\tno\tno\tunsupported\n' >"$FASTRPC_HEALTHCHECK_TSV" +validation_case no-declared-or-supported-domains SKIP + +if [ "$validation_mode" = --absence-only ]; then + printf 'FastRPC absence-policy checks passed. No DUT operations were executed.\n' + exit 0 +fi + +mock_dt="$validation_tmp/dt" +mock_remoteprocs="0 3" +mock_endpoints="0 3" +mock_libraries="0 3" +mock_expect_adsp=yes +FASTRPC_HEALTHCHECK_TSV="" +CAPABILITY_SOURCE=runtime-fallback +validation_node soc/adsp/fastrpc adsp qcom,fastrpc +validation_node soc/cdsp/fastrpc cdsp qcom,fastrpc +validation_case healthy-dt PASS +grep -q 'mock invocation domain=3' "$LOG_ROOT/stdout.log" + +mock_remoteprocs=0 +mock_endpoints=0 +validation_case missing-cdsp-other-domain-passes FAIL +mock_remoteprocs="0 3" +mock_endpoints="0 3" +mock_offline=3 +validation_case offline-cdsp FAIL +mock_offline="" +mock_endpoints=0 +validation_case missing-cdsp-endpoint FAIL +mock_endpoints="0 3" +mock_libraries=0 +validation_case missing-cdsp-runtime-library FAIL +mock_libraries="0 3" + +mock_remoteprocs=0 +mock_endpoints=0 +printf 'disabled\000' >"$mock_dt/soc/cdsp/status" +validation_case disabled-ancestor PASS +printf 'okay\000' >"$mock_dt/soc/cdsp/status" +printf 'reserved\000' >"$mock_dt/soc/cdsp/fastrpc/status" +validation_case reserved-node PASS +printf 'ok\000' >"$mock_dt/soc/cdsp/fastrpc/status" +validation_case enabled-again FAIL + +CLI_DOMAIN_NAME=adsp +validation_case cli-scopes-out-broken-cdsp PASS +CLI_DOMAIN_NAME="" +FASTRPC_DOMAIN_NAME=adsp +validation_case environment-scopes-out-broken-cdsp PASS +FASTRPC_DOMAIN_NAME="" +CLI_DOMAIN_NAME=cdsp +mock_expect_adsp=no +validation_case explicit-missing-cdsp FAIL +CLI_DOMAIN_NAME="" + +mock_assets=no +validation_case assets-cannot-hide-readiness-failure FAIL +mock_assets=yes +mock_remoteprocs="" +mock_endpoints="" +validation_case no-runnable-expected-domains FAIL + +mock_remoteprocs=0 +mock_endpoints=0 +mock_expect_adsp=yes +FASTRPC_HEALTHCHECK_TSV="$validation_tmp/healthcheck.tsv" +CAPABILITY_SOURCE=fastrpc-healthcheck +printf 'ADSP\tonline\tyes\tno\tyes\tsupported\n' >"$FASTRPC_HEALTHCHECK_TSV" +validation_case healthcheck-omits-cdsp FAIL +printf 'CDSP\toffline\tno\tno\tno\toffline\n' >>"$FASTRPC_HEALTHCHECK_TSV" +validation_case healthcheck-offline-cdsp FAIL +printf 'ADSP\tonline\tyes\tno\tyes\tsupported\nCDSP\tonline\tno\tno\tno\tunsupported\n' >"$FASTRPC_HEALTHCHECK_TSV" +validation_case healthcheck-unsupported-expected-cdsp FAIL +printf 'disabled\000' >"$mock_dt/soc/cdsp/status" +validation_case healthcheck-unsupported-optional-cdsp PASS + +FASTRPC_HEALTHCHECK_TSV="" +CAPABILITY_SOURCE=runtime-fallback +mock_dt="$validation_tmp/absent-dt" +validation_case no-dt-runtime-fallback PASS +mock_remoteprocs="" +mock_endpoints="" +mock_expect_adsp=no +validation_case no-dt-no-domains SKIP + +mock_dt="$validation_tmp/new-dt" +mock_remoteprocs="0 3" +mock_endpoints="0 3" +mock_expect_adsp=yes +validation_node soc/cdsp/fastrpc cdsp 'qcom,glymur-fastrpc\000qcom,kaanapali-fastrpc' +validation_node soc/cdsp/fastrpc/cb@1 ignored qcom,fastrpc-compute-cb +validation_case fallback-compatible PASS +test "$(wc -l <"$LOG_ROOT/fastrpc_dt_expected_domains.log")" -eq 1 +ln -s "$mock_dt" "$validation_tmp/dt-link" +mock_dt="$validation_tmp/dt-link" +validation_case symlink-dt-root PASS + +mock_dt="$validation_tmp/malformed-dt" +validation_node soc/fastrpc '' qcom,fastrpc +mock_expect_adsp=no +validation_case missing-label FAIL + +printf 'All FastRPC domain regression checks passed. No DUT operations were executed.\n' diff --git a/scripts/tests/fastrpc-startup-validation.sh b/scripts/tests/fastrpc-startup-validation.sh new file mode 100644 index 000000000..bed7d8f8a --- /dev/null +++ b/scripts/tests/fastrpc-startup-validation.sh @@ -0,0 +1,217 @@ +#!/bin/sh +# Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries. +# SPDX-License-Identifier: BSD-3-Clause +# Run with sh scripts/tests/fastrpc-startup-validation.sh [shell-under-test]. +# This fixture mode is sourced only by the temporary test tree below. +if [ "${startup_fixture_mode:-0}" = 1 ]; then + # find_test_case_by_name NAME + # Resolve the temporary runner directory, never a real DUT suite. + find_test_case_by_name() { + printf '%s\n' "$SCRIPT_DIR" + } + + # log_soc_info + # Avoid host hardware queries in this startup-only check. + log_soc_info() { + : + } + + # fastrpc_capture_expected_domains RESULT_DIR + # Mock DT inventory. Real DT parsing is covered by fastrpc-domain-validation.sh. + fastrpc_capture_expected_domains() { + printf 'DT_INVENTORY\n' >&2 + [ "$startup_dt_error" = 0 ] || return 1 + printf '%s' "$startup_domains" + } + + # fastrpc_remoteproc_domain_state DOMAIN_ID + # Mock kernel records, leaving the real readiness/status validator in use. + fastrpc_remoteproc_domain_state() { + case " $startup_missing " in + *" $1 "*) + return 2 + ;; + esac + case " $startup_offline " in + *" $1 "*) + printf 'offline|/mock/remoteproc%s|mock.mdt|mock\n' "$1" + ;; + *) + printf 'running|/mock/remoteproc%s|mock.mdt|mock\n' "$1" + ;; + esac + } + + # fastrpc_domain_endpoint_available DOMAIN_ID + # Mock the device boundary without creating real FastRPC device nodes. + fastrpc_domain_endpoint_available() { + case " $startup_no_endpoint " in + *" $1 "*) + return 1 + ;; + esac + return 0 + } + + # pkg_ensure_host_distro_package_set_present SET + # Simulate existing package-provider outcomes without installing anything. + pkg_ensure_host_distro_package_set_present() { + printf 'PACKAGE_PROVIDER %s\n' "$1" + if [ "$startup_package_rc" = 0 ]; then + # shellcheck disable=SC2034 # Consumed by the real runner after this mock returns. + RUN_BIN="$startup_fixture_binary" + fi + return "$startup_package_rc" + } + + # fastrpc_capture_qrtr_evidence RESULT_DIR + # Stop immediately after the real runner's prerequisite gate, before any DUT work. + fastrpc_capture_qrtr_evidence() { + printf 'NORMAL_EXECUTION_REACHED\n' + test_result_finish SKIP "Host fixture reached the existing execution path" + } + return 0 +fi + +set -eu +startup_repo=$(CDPATH='' cd -- "$(dirname "$0")/../.." && pwd) +startup_script="$startup_repo/scripts/tests/fastrpc-startup-validation.sh" +startup_shell=${1:-sh} +startup_tmp=$(mktemp -d "${TMPDIR:-/tmp}/fastrpc-startup.XXXXXX") + +# startup_cleanup +# Remove this process's successful fixture tree, retaining failed-run evidence. +startup_cleanup() { + startup_cleanup_rc=$? + if [ "$startup_cleanup_rc" -ne 0 ]; then + printf 'Startup fixture failed or was interrupted, evidence=%s\n' "$startup_tmp" >&2 + return 0 + fi + case "$startup_tmp" in + "${TMPDIR:-/tmp}"/fastrpc-startup.*) + rm -rf -- "$startup_tmp" + ;; + esac +} +trap startup_cleanup EXIT +trap 'exit 1' HUP INT TERM + +mkdir -p "$startup_tmp/utils" "$startup_tmp/bin" +printf "TOOLS='%s/utils'\n" "$startup_tmp" >"$startup_tmp/init_env" +for startup_lib in functestlib lib_fastrpc; do + printf '. "%s/Runner/utils/%s.sh"\nstartup_fixture_mode=1\n. "%s"\n' \ + "$startup_repo" "$startup_lib" "$startup_script" >"$startup_tmp/utils/$startup_lib.sh" +done +# These libraries are not needed until after the gate. No real provider is sourced. +: >"$startup_tmp/utils/lib_pkg_provider.sh" +: >"$startup_tmp/utils/lib_diag.sh" +startup_fixture_binary="$startup_tmp/bin/fastrpc_test" +printf '#!/bin/sh\nexit 99\n' >"$startup_fixture_binary" +chmod +x "$startup_fixture_binary" +export startup_fixture_binary + +# startup_case NAME VERDICT BIN_MODE [RUNNER_ARGS...] +# Run a complete, unchanged run.sh copy with temporary bootstrap and boundary mocks. +# Verify the .res verdict and reject any invented functional PASS. Output: case status. +startup_case() { + startup_name="$1" + startup_verdict="$2" + startup_mode="$3" + shift 3 + startup_case_dir="$startup_tmp/$startup_name" + mkdir -p "$startup_case_dir" + cp "$startup_repo/Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh" "$startup_case_dir/run.sh" + case "$startup_mode" in + missing) + startup_bin_dir="$startup_tmp/not-installed" + ;; + installed) + startup_bin_dir="$startup_tmp/bin" + ;; + default) + startup_bin_dir=/usr/bin + ;; + esac + if ! "$startup_shell" "$startup_case_dir/run.sh" --bin-dir "$startup_bin_dir" "$@" >"$startup_case_dir/stdout.log" 2>&1; then + cat "$startup_case_dir/stdout.log" + printf 'FAIL %s: runner exited before normal result publication\n' "$startup_name" >&2 + return 1 + fi + if [ "$(cat "$startup_case_dir/fastrpc_test.res")" != "fastrpc_test $startup_verdict" ] || + grep -q '\[PASS\]' "$startup_case_dir/stdout.log"; then + cat "$startup_case_dir/stdout.log" + printf 'FAIL %s\n' "$startup_name" >&2 + exit 1 + fi + printf 'PASS %s\n' "$startup_name" +} + +# Fixture values are test-only boundaries, not new product environment options. +startup_domains="0 3" +startup_missing=3 +startup_offline="" +startup_no_endpoint="" +startup_dt_error=0 +startup_package_rc=2 +FASTRPC_DOMAIN="" +FASTRPC_DOMAIN_NAME="" +export startup_domains startup_missing startup_offline startup_no_endpoint +export startup_dt_error startup_package_rc FASTRPC_DOMAIN FASTRPC_DOMAIN_NAME + +startup_case missing-binary-and-cdsp FAIL missing +grep -q 'CDSP.*expected but not registered' "$startup_case_dir/stdout.log" +startup_case explicit-cdsp-without-binary FAIL missing --domain-name cdsp +startup_case explicit-adsp-ignores-cdsp SKIP missing --domain-name adsp +if grep -q DT_INVENTORY "$startup_case_dir/stdout.log"; then + exit 1 +fi +startup_case invalid-explicit-domain FAIL missing --domain-name invalid +FASTRPC_DOMAIN_NAME=cdsp +startup_case environment-required-cdsp FAIL missing +FASTRPC_DOMAIN_NAME="" +startup_missing="" +startup_offline=3 +startup_case offline-cdsp-without-binary FAIL missing +startup_offline="" +startup_no_endpoint=3 +startup_case missing-endpoint-without-binary FAIL missing +startup_no_endpoint="" +startup_case healthy-registration-no-binary SKIP missing +startup_domains=0 +startup_missing=3 +startup_case absent-cdsp-adsp-only-no-binary SKIP missing +grep -q 'domain=ADSP kernel_registration=ready' "$startup_case_dir/stdout.log" +if grep -q 'CDSP.*expected' "$startup_case_dir/stdout.log"; then + exit 1 +fi +startup_missing="" +startup_domains="" +startup_case no-expectation-no-binary SKIP missing +startup_dt_error=1 +startup_case discovery-error-no-binary FAIL missing +startup_dt_error=0 +startup_missing=3 +startup_domains="0 3" +startup_case installed-binary-bypasses-early-preflight SKIP installed +grep -q NORMAL_EXECUTION_REACHED "$startup_case_dir/stdout.log" +if grep -q DT_INVENTORY "$startup_case_dir/stdout.log"; then + exit 1 +fi + +if [ ! -x /usr/bin/fastrpc_test ]; then + startup_case registration-fails-before-package-provider FAIL default + if grep -q PACKAGE_PROVIDER "$startup_case_dir/stdout.log"; then + exit 1 + fi + startup_missing="" + startup_case image-managed-missing-binary SKIP default + grep -q 'PACKAGE_PROVIDER fastrpc' "$startup_case_dir/stdout.log" + startup_package_rc=1 + startup_case existing-package-provider-failure FAIL default + startup_package_rc=0 + startup_case existing-package-provider-success SKIP default + grep -q NORMAL_EXECUTION_REACHED "$startup_case_dir/stdout.log" +else + printf 'SKIP default-path provider mocks, host already has /usr/bin/fastrpc_test\n' +fi +printf 'FastRPC full-runner startup checks passed, no DUT operations executed.\n'