Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 91 additions & 9 deletions Runner/suites/Multimedia/CDSP/fastrpc_test/fastrpc_test_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ Supported capabilities:
`--domain`, `--domain-name`, `--pd-mode`, `--unsigned-pd`, `--verbose`)
- Auto-discovery of system libraries and DSP skeletons for Yocto, Debian, Ubuntu, and RPM layouts
- Runtime domain and PD discovery without SoC-name filtering
- Automatic readiness checks for enabled FastRPC DT declarations, independent of
successful remoteproc registration

## Prerequisites

Expand Down Expand Up @@ -66,6 +68,8 @@ Have these on the target (or specify paths with the flags below):
- Yocto, CentOS, and distributions without an exact package mapping remain
image-managed and report SKIP when the binary is absent. A missing binary in
an explicitly selected `--bin-dir` also reports SKIP without package changes.
These skips apply only after expected kernel registration passes the preflight
below. A registration failure takes precedence over a missing test payload.
- Missing optional image assets are reported as SKIP unless the operator explicitly selected a
domain whose required runtime library is absent.

Expand Down Expand Up @@ -116,7 +120,7 @@ Env:
### Quick start

```bash
# Default: dynamically discover runnable domains and supported PD modes
# Default: validate DT-declared domains and discover runnable domains/PD modes
./run.sh

# With repeat and timeout:
Expand Down Expand Up @@ -171,6 +175,53 @@ package version. When runtime fallback discovers a domain but the installed `fas
reports that it cannot represent that domain, the invocation is classified as FAIL so the image or
package gap remains visible until a matching payload is provisioned. Other domains still execute.

Automatic selection also reads enabled FastRPC declarations from the running DT,
using the public `qcom,fastrpc` and `qcom,kaanapali-fastrpc` driver match entries
and their `label` property. Newer compatibles with the kaanapali fallback are
covered. A context-bank node or a generic DSP node alone is not an expectation.
The FastRPC node and every ancestor must have absent, `okay`, or `ok` status.
Disabled ancestors therefore do not create false failures.

A mapped DT-declared domain is checked even if healthcheck omits it or reports it
offline/unsupported, or remoteproc registration and the endpoint are absent.
Missing readiness, endpoints, or required runtime libraries record FAIL while
other healthy domains continue. Missing optional shared test assets cannot
overwrite an already recorded readiness failure with SKIP. With no DT or no
enabled FastRPC declarations, the existing healthcheck/runtime fallback remains
in effect. Failed DT traversal, unreadable properties, or a missing required
FastRPC label fail discovery. Labels outside the suite's protocol map are logged
as untestable, not guessed or mapped to another DSP.

Explicit CLI or environment selection checks only the requested domain, without
requiring DT discovery. No SoC/board lists, distro gates, new parameters, or new
package installation paths are introduced.

When the test binary is absent, a read-only registration preflight runs before
the missing-payload SKIP or existing package-provider handling. It checks every
DT-expected domain, or only the explicitly selected domain, for a running
remoteproc and a normal or secure FastRPC endpoint. Missing/offline registration,
missing endpoints, invalid explicit selection and DT discovery failures report
FAIL even without `fastrpc_test`. Healthy registration is logged as readiness,
never as functional PASS. If registration is healthy or no domain is expected,
the existing binary/package policy proceeds, including SKIP for missing optional
payloads on image-managed systems. This preflight neither loads modules nor
starts remoteprocs, creates endpoints, or installs packages.

The installed-binary path is unchanged and still prefers `fastrpc-healthcheck`
for execution capability, with the existing remoteproc fallback when absent.
The missing-binary preflight checks kernel registration only, it cannot prove
client support, PD support or functional RPC execution.

Automatic mode uses only the running system's enabled FastRPC DT declarations
and discovered runtime capabilities. If CDSP is absent from both DT and runtime,
other discovered domains continue, or the suite reports SKIP when none apply.
Removed or disabled DT nodes do not create historical or board-specific domain
requirements. A domain still exposed by runtime discovery remains eligible even
without an enabled DT declaration. Triage determines whether absent hardware was
intentional. Enabled DT declarations with missing/offline registration still fail.
Explicit `--domain-name cdsp` continues to check CDSP only and fails when it is
unavailable. No mandatory-domain CI policy or new requirement option is introduced.

### LAVA integration example

```
Expand Down Expand Up @@ -210,10 +261,11 @@ package gap remains visible until a matching payload is provisioned. Other domai
executed, its raw output is retained. A normalized six-column table containing domain, state,
signed-PD support, unsigned-PD support, normalized FastRPC support, and the support reason is
retained only when strict parsing succeeds.
- **Offline automatic domain**: Logged and excluded from automatic selection. Other runnable
domains continue.
- **Unsupported automatic domain**: Logged with the healthcheck reason and excluded. Other
runnable domains continue.
- **Offline or unsupported automatic domain**: FAIL when an enabled FastRPC DT declaration
expects it, otherwise logged and excluded. Other runnable domains continue.
- **DT expectation evidence**: `fastrpc_dt_expected_domains.log` records numeric ID, label,
full node path and compatible list as tab-separated fields. The checked traversal paths
and errors are retained in `fastrpc_dt_compatible_paths.log` and `fastrpc_dt_scan_errors.log`.
- **Test payload lacks a runtime-ready domain**: A retained invocation log proving that the
installed client cannot represent the selected numeric domain is classified as FAIL. This is
feature probing, not a package-version check, so it behaves the same on Yocto, Debian, Ubuntu,
Expand All @@ -222,12 +274,14 @@ package gap remains visible until a matching payload is provisioned. Other domai
- **Missing runtime artifacts**: When the capability source is `fastrpc-healthcheck`, a domain
missing its endpoint or required runtime library fails the suite — healthcheck has declared it
supported, so the absence indicates a broken installation. When the capability source is the
runtime fallback, such a domain is logged and excluded while other runnable domains continue.
Missing shared test libraries or DSP skeletons skip the suite before execution.
runtime fallback, such a domain fails if declared by enabled FastRPC DT, otherwise it is
logged and excluded while other runnable domains continue. Missing shared test libraries
or DSP skeletons skip execution but do not clear prior readiness failures.
- **Binary resolved to /bin/fastrpc_test**: Blocked by default. Set `ALLOW_BIN_FASTRPC=1` or
use `--bin-dir` to a non-`/bin` path.
- **Missing binary on any distribution**: Package installation is not attempted. The suite reports
SKIP with the expected image-provided binary path.
- **Missing binary**: Expected kernel registration is checked first and any failure
is reported before optional-payload handling. Otherwise the existing package
policy described under Prerequisites applies, with SKIP on image-managed paths.
- **Session create errors with -U 1**: If unsigned PD returns `0x80000416`, confirm your image
includes unsigned shells/policies (or use `--pd-mode signed-only`).
- **Domain not discovered**: Check `dmesg` for remoteproc firmware load errors. The test
Expand Down Expand Up @@ -257,6 +311,34 @@ package gap remains visible until a matching payload is provisioned. Other domai
takes precedence over runtime discovery.
- If `fastrpc_test` is not in the default path, use `--bin-dir` to specify its location.

## Host regression checks

From the repository root:

```sh
dash scripts/tests/fastrpc-domain-validation.sh
# Focused current-DT/runtime absence policy checks:
dash scripts/tests/fastrpc-domain-validation.sh --absence-only
dash scripts/tests/fastrpc-startup-validation.sh dash
```

These host-only fixtures exercise DT expectations and the actual runner selection
and result-finalization blocks. Remoteproc, endpoint, library, and invocation
boundaries are mocked. Coverage includes missing/offline CDSP with healthy ADSP,
healthcheck omission and unsupported reports, missing endpoints/libraries,
disabled ancestors, explicit overrides, no-DT fallback, optional asset skips,
compatible fallback lists, symlink roots, and malformed labels. They do not
perform DUT operations or establish hardware/cross-distribution validation.
The absence-only subset checks ADSP continuing without CDSP, explicit absent-CDSP
failure, runtime CDSP without a DT declaration, and SKIP with no applicable domain.

The startup harness additionally executes an unchanged copy of the full runner
with a temporary bootstrap and mocked hardware/package boundaries. It checks
missing-binary precedence, CLI/environment selection, absent/offline remoteprocs,
missing endpoints, discovery errors, preservation of the installed-binary path,
and existing provider outcomes. No package installation or real RPC invocation
is performed by these host checks.

## License

SPDX-License-Identifier: BSD-3-Clause
Expand Down
80 changes: 61 additions & 19 deletions Runner/suites/Multimedia/CDSP/fastrpc_test/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -308,13 +308,34 @@ log_info "Kernel: $(uname -a 2>/dev/null || echo N/A)"
log_info "Date(UTC): $(date -u 2>/dev/null || echo N/A)"
log_soc_info

# -------------------- Logging root -----------------------------
TS="$(date +%Y%m%d-%H%M%S)"
LOG_ROOT="./logs_${TESTNAME}_${TS}"
if ! mkdir -p "$LOG_ROOT"; then
test_result_finish "FAIL" "$TESTNAME FAIL - cannot create log directory: $LOG_ROOT"
fi

log_info "Evidence directory: $test_path/$LOG_ROOT"

# -------------------- Binary directory resolution -----------------
if [ -n "$BIN_DIR" ]; then
:
else
BIN_DIR="/usr/bin"
fi

RUN_DIR="$BIN_DIR"
RUN_BIN="$RUN_DIR/fastrpc_test"

# Missing optional payloads must not hide expected kernel registration failures.
# Keep package/layout handling and the installed-binary execution path unchanged.
if [ ! -x "$RUN_BIN" ]; then
log_info "FastRPC test binary is unavailable at $RUN_BIN, checking expected kernel registration first"
if ! fastrpc_registration_preflight "$LOG_ROOT"; then
test_result_finish
fi
fi

case "$BIN_DIR" in
/bin)
if [ "${ALLOW_BIN_FASTRPC:-0}" != "1" ]; then
Expand All @@ -323,9 +344,6 @@ case "$BIN_DIR" in
;;
esac

RUN_DIR="$BIN_DIR"
RUN_BIN="$RUN_DIR/fastrpc_test"

if [ ! -x "$RUN_BIN" ]; then
if [ "$RUN_DIR" = "/usr/bin" ] &&
command -v pkg_ensure_host_distro_package_set_present >/dev/null 2>&1; then
Expand All @@ -351,15 +369,6 @@ if [ ! -x "$RUN_BIN" ]; then
fi
fi

# -------------------- Logging root -----------------------------
TS="$(date +%Y%m%d-%H%M%S)"
LOG_ROOT="./logs_${TESTNAME}_${TS}"
if ! mkdir -p "$LOG_ROOT"; then
test_result_finish "FAIL" "$TESTNAME FAIL - cannot create log directory: $LOG_ROOT"
fi

log_info "Evidence directory: $test_path/$LOG_ROOT"

fastrpc_capture_qrtr_evidence "$LOG_ROOT"

# Prepare the image-provided DSP artifacts before healthcheck records capability.
Expand Down Expand Up @@ -432,6 +441,19 @@ fi

DOMAINS_TO_TEST="$(resolve_domains_to_test)"

# Expectations must not disappear when remoteproc or FastRPC registration fails.
# Only current enabled DT declarations add expectations, not historical board data.
# Explicit CLI/environment selection retains its scope and does not require DT.
EXPECTED_DOMAINS=""
if [ "$SELECTION_SOURCE" = "automatic" ]; then
if ! EXPECTED_DOMAINS=$(fastrpc_capture_expected_domains "$LOG_ROOT"); then
test_result_finish "FAIL" "$TESTNAME FAIL - cannot discover expected FastRPC DT domains, check DT access and labels, artifact=$LOG_ROOT/fastrpc_dt_scan_errors.log"
fi
for d in $EXPECTED_DOMAINS; do
DOMAINS_TO_TEST=$(fastrpc_append_word_unique "$DOMAINS_TO_TEST" "$d")
done
fi

if [ -z "$DOMAINS_TO_TEST" ] && [ "$SELECTION_SOURCE" != "automatic" ]; then
test_result_finish "FAIL" "$TESTNAME FAIL - explicit domain selection is invalid or unavailable, source=$SELECTION_SOURCE"
fi
Expand All @@ -444,22 +466,36 @@ fi
RUNNABLE_DOMAINS=""
REQUESTED_PDS="$(requested_pds)"
for d in $DOMAINS_TO_TEST; do
domain_required=0
if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then
domain_required=1
fi
case " $EXPECTED_DOMAINS " in
*" $d "*)
domain_required=1
;;
esac
fastrpc_domain_ready "$d"
domain_ready_status=$?

if [ "$domain_ready_status" -ne 0 ]; then
if [ "$SELECTION_SOURCE" != "automatic" ]; then
test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") is not ready, selection=$SELECTION_SOURCE"
fi
if [ "$domain_required" -eq 1 ]; then
test_result_record "FAIL" "$(domain_to_name "$d") is expected but not ready, check remoteproc firmware and FastRPC registration, capability_source=$CAPABILITY_SOURCE DT=$LOG_ROOT/fastrpc_dt_expected_domains.log"
continue
fi

log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=runtime-not-ready"
continue
fi

if ! fastrpc_domain_endpoint_available "$d"; then
label="$(domain_to_endpoint_label "$d")"
if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then
test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") has no usable endpoint, expected=/dev/fastrpc-${label} or /dev/fastrpc-${label}-secure capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE"
if [ "$domain_required" -eq 1 ]; then
test_result_record "FAIL" "$(domain_to_name "$d") has no usable endpoint, expected=/dev/fastrpc-${label} or /dev/fastrpc-${label}-secure, check FastRPC driver registration, capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE"
continue
fi

log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=endpoint-absent expected=/dev/fastrpc-${label}[-secure]"
Expand All @@ -469,8 +505,9 @@ for d in $DOMAINS_TO_TEST; do
domain_library="$(fastrpc_domain_system_library_path "$d" || true)"
if [ -z "$domain_library" ]; then
required_library="$(fastrpc_domain_field "$d" 7)"
if [ "$SELECTION_SOURCE" != "automatic" ] || [ "$CAPABILITY_SOURCE" = "fastrpc-healthcheck" ]; then
test_result_finish "FAIL" "$TESTNAME FAIL - $(domain_to_name "$d") runtime library is missing, expected=$required_library checked=$FASTRPC_LIB_SYS_DIRS_CHECKED capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE"
if [ "$domain_required" -eq 1 ]; then
test_result_record "FAIL" "$(domain_to_name "$d") runtime library is missing, provision $required_library in the image, checked=$FASTRPC_LIB_SYS_DIRS_CHECKED capability_source=$CAPABILITY_SOURCE selection=$SELECTION_SOURCE"
continue
fi

log_info "[FASTRPC-DISCOVERY] domain=$(domain_to_name "$d") eligible=no reason=runtime-library-missing expected=$required_library checked=$FASTRPC_LIB_SYS_DIRS_CHECKED"
Expand All @@ -495,11 +532,13 @@ done
DOMAINS_TO_TEST="$RUNNABLE_DOMAINS"

if [ -z "$DOMAINS_TO_TEST" ]; then
test_result_finish "SKIP" "$TESTNAME SKIP - no automatically discovered domain has complete readiness, endpoint, library, and requested-PD support"
test_result_record "SKIP" "No selected domain has complete readiness, endpoint, library, and requested-PD support"
test_result_finish
fi

if ! fastrpc_validate_runtime_artifacts; then
test_result_finish "SKIP" "$TESTNAME SKIP - shared FastRPC test artifacts are incomplete: $FASTRPC_ARTIFACT_ERROR"
test_result_record "SKIP" "Shared FastRPC test artifacts are incomplete: $FASTRPC_ARTIFACT_ERROR"
test_result_finish
fi

log_info "Domain mode: $DOMAIN_MODE"
Expand Down Expand Up @@ -546,6 +585,9 @@ log_info "[FASTRPC-MATRIX] combinations=$matrix_combinations repeats=$REPEAT inv
# -------------------- Run loop ---------------------------------
# Nested loop over domains and PDs
RUNTIME_FAILURE_SEEN=0
if [ "$TEST_RESULT_FAIL_COUNT" -gt 0 ]; then
RUNTIME_FAILURE_SEEN=1
fi

# Track per-domain/per-PD results during execution
# Format: "DOMAIN:PD:pass_count:fail_count:subtests_total:subtests_pass:subtests_fail:subtests_skip"
Expand Down Expand Up @@ -776,7 +818,7 @@ EOF
log_info "$SUMMARY_SEP"
_total_count=$((TEST_RESULT_PASS_COUNT + TEST_RESULT_FAIL_COUNT))
overall_inv_line="$(printf '%-20s Total:%6d | Passed:%6d | Failed:%6d' \
'Overall invocations:' "$_total_count" "$TEST_RESULT_PASS_COUNT" "$TEST_RESULT_FAIL_COUNT")"
'Overall checks:' "$_total_count" "$TEST_RESULT_PASS_COUNT" "$TEST_RESULT_FAIL_COUNT")"
log_info "$overall_inv_line"

overall_sub_line="$(printf '%-20s Total:%6d | Passed:%6d | Failed:%6d | Skipped:%6d' \
Expand Down
Loading
Loading