Skip to content

Add activity index calculation from accelerometer data (fixes #642) - #859

Open
samerzumot wants to merge 3 commits into
brainflow-dev:masterfrom
samerzumot:feature/issue-642-activity-index
Open

Add activity index calculation from accelerometer data (fixes #642)#859
samerzumot wants to merge 3 commits into
brainflow-dev:masterfrom
samerzumot:feature/issue-642-activity-index

Conversation

@samerzumot

@samerzumot samerzumot commented Aug 23, 2026

Copy link
Copy Markdown
  • Implement get_activity_index in C++ core using multi-axis epoch variance
  • Add get_activity_index bindings across C++, Python, Java, C#, TypeScript, Rust, Swift, Julia, and MATLAB
  • Add tests activity_index.py

…ow-dev#642)

- Implement get_activity_index in C++ core using multi-axis epoch variance
- Add get_activity_index bindings across C++, Python, Java, C#, TypeScript, Rust, Swift, Julia, and MATLAB
- Add automated test activity_index.py

Copy link
Copy Markdown
Member

Thanks for working on this. I found several blockers that should be addressed before merge:

  1. The Rust binding does not compile. BrainFlowError::InvalidArguments does not exist, and the Windows CI job fails with E0599. Please use the existing error pattern, e.g. Err(Error::BrainFlowError(BrainFlowError::InvalidArgumentsError)), at both validation sites.

  2. The implementation is not the Activity Index referenced by add activity index calculated from accelerometer data #642. The core currently returns sqrt((var_x + var_y + var_z) / 3), while the referenced BIOBSS/Bai definition subtracts device/systematic-noise variance before clamping to zero. No baseline/noise parameter is exposed, so a stationary but noisy sensor produces non-zero activity. The published metric also aggregates longer epochs by summing adjacent one-second AIs instead of recomputing one variance over the full period.

  3. The scope/API does not match the design discussed in add activity index calculated from accelerometer data #642. That discussion places low-level IMU feature helpers in DataFilter and the final analytical/ML activity calculation in MLModel. Either re-scope/rename this API as an accelerometer variability helper and remove fixes #642, or implement the complete AI contract.

  4. Input validation is unsafe/inconsistent. The MATLAB binding does not check equal axis lengths before passing accel_x's length to native code, so shorter Y/Z buffers can be read out of bounds. Empty inputs divide by zero or throw unrelated exceptions in most bindings, and period > data_len is handled differently in C++ and the other languages. MATLAB/TypeScript should also reject non-integer periods.

  5. The new Python test is not run by CI. activity_index.py is not referenced from run_unix.yml. Please wire it into a workflow and cover a non-zero baseline/noise case, empty/invalid inputs, and epoch semantics.

Confirmed Rust CI failure: https://github.com/brainflow-dev/brainflow/actions/runs/32657395247/job/100051356398

References:

…iod validation, and baseline noise variance support
@samerzumot

Copy link
Copy Markdown
Author

Thanks for the detailed review and guidance, @Andrey1994! I have addressed all 5 blockers:

1. Rust Compilation Fix

  • Replaced BrainFlowError::InvalidArguments with Err(Error::BrainFlowError(BrainFlowError::InvalidArgumentsError)) at both validation sites in rust_package/brainflow/src/data_filter.rs.

2. Complete Bai et al. (2016) / BIOBSS Formula

  • Added sampling_rate and per-axis baseline rest noise variances (noise_var_x, noise_var_y, noise_var_z) across the C++ core and all language bindings.
  • Subdivided data into 1-second slices ($f_s$ samples, where $f_s$ is the sampling rate).
  • Subtracted device/systematic noise variance from each axis variance before zero-clamping:
    $$AI_t = \sqrt{ \max\left(0, \frac{1}{3} \sum_{i \in {x, y, z}} (\sigma_{i,t}^2 - \bar{\sigma}_i^2)\right) }$$
  • Implemented epoch aggregation by summing adjacent 1-second AIs over the epoch duration ($AI_{\text{epoch}} = \sum_{t=0}^{H-1} AI_t$), strictly following the Bai et al. (2016) additivity property instead of recomputing a single variance over the whole epoch.

3. Scope/API design and Issue #642

Between rescoping this as a variability helper or implementing the complete AI contract, **I chose to implement the complete AI contract in DataFilter.

Here is my reasoning (Let me know what you think):

  • In BrainFlow, all deterministic analytical metrics computed directly from continuous multi-channel raw sensor streams and sampling rates live in DataFilter / DataHandler for example, DataFilter.get_heart_rate(ppg_ir, ppg_red, sampling_rate, ...) and DataFilter.get_oxygen_level(ppg_ir, ppg_red, sampling_rate, ...).
  • MLModel is designed around evaluating pre-extracted 1D feature vectors (double *data, int data_len) using trained machine learning models (like MindfulnessClassifier with logistic regression coefficients or ONNX models). It does not take multi-channel raw data, sampling rates, or baseline noise parameters.
  • The analytical Activity Index from Bai et al. (2016) and BIOBSS (acc_activityindex.py) is fundamentally a signal-processing metric on raw accelerometer streams. By implementing the complete contract (per-axis baseline noise variance subtraction, 1-second base epoch variance, and epoch summing aggregation) this function fully provides the analytical Activity Index calculation requested in add activity index calculated from accelerometer data #642. If a trained classifier (e.g., limb-position-specific activity classification) is desired in the future, that can be added as a separate metric in MLModel, but DataFilter.get_activity_index delivers the complete analytical contract.

4. Input Validation Consistency & Safety

  • MATLAB: Added axis-length equality checks (size(accel_x, 2) == size(accel_y, 2) == size(accel_z, 2)) to eliminate any buffer overruns; rejects non-integer periods and sampling rates.
  • TypeScript: Enforced integer checks on period and samplingRate; rejects empty arrays.
  • Consistency across all languages: Empty inputs, sampling_rate <= 0, negative noise variances, and period > data_len are now uniformly rejected with INVALID_ARGUMENTS_ERROR across C++, Python, Rust, MATLAB, TypeScript, Java, C#, Swift, and Julia.

5. CI Workflow & Test Suite

  • Wired activity_index.py into .github/workflows/run_unix.yml.
  • Expanded activity_index.py to cover:
    1. Stationary noise cancellation: Verified that a stationary sensor with noise produces $AI \approx 0.0$ when baseline noise parameters are supplied.
    2. Theoretical accuracy: Verified known signal variance minus noise variance matches theoretical calculation.
    3. Epoch summing semantics: Verified that a 2-second epoch equals $AI_1 + AI_2$ and is distinct from recomputing variance across the combined 2-second window.
    4. Invalid argument coverage: Tested empty arrays, shape mismatches, negative noise variances, zero sampling rates, and period > data_len.

@Andrey1994 Andrey1994 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] The Node.js package does not compile.
functions.types.ts:450–451 declares getActivityIndex with six arguments, but the updated call passes ten. The declaration is missing samplingRate and the three noise variance parameters. Both Ubuntu and macOS CI fail with TS2554: Expected 6 arguments, but got 10, so the new Python test is subsequently skipped. Please update the declaration to match the native signature. CI failure.
[P2] Invalid samples silently become zero activity.
In data_handler.cpp:1784–1785, std::max(0.0, NaN) returns zero. I reproduced this: a valid signal produces AI ≈ 0.57735, but replacing one sample with NaN produces AI = 0.0 with a success status. Passing noise_var_x=NaN has the same effect. This makes invalid data indistinguishable from inactivity. Please reject non-finite inputs or explicitly preserve an invalid-result indication.

from brainflow.exit_codes import BrainFlowError, BrainFlowExitCodes


def test_activity_index():

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make the tests easier and smaller, its too big now, also tests here are meant to be more like examples for docs, keep them simple, check for example tests for signal filtering

- update Node.js DataHandlerFunctions.getActivityIndex signature to match 10 parameters
- add std::isfinite checks to reject NaN/Inf in accelerometer samples and noise variances
- simplify activity_index.py to a concise BoardShim demonstration test
- remove leftover PNG artifacts from repository root
- add Activity Index test step to Windows CI workflow
@samerzumot

Copy link
Copy Markdown
Author

Thanks for the review @Andrey1994! I've addressed your feedback with the following changes:

  1. Updated DataHandlerFunctions.getActivityIndex in functions.types.ts to include all 10 parameters; verified npm run build passes cleanly.
  2. Added std::isfinite checks in C++ for accelerometer samples and baseline noise variances, returning INVALID_ARGUMENTS_ERROR on any non-finite input.
  3. Simplified the unit tests in activity_index.py with a script using BoardShim, matching signal_filtering.py.
  4. Cleaned up the leftover PNG images from the repo root and added the test step to Windows CI.

@Andrey1994 Andrey1994 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous TypeScript signature and non-finite-input bugs are fixed, and Unix/Windows CI now pass. Two blockers remain:

  • The rewritten activity_index.py is only a smoke/demo script and contains no assertions. It no longer tests noise subtraction, known numerical output, aggregation semantics, or invalid arguments, despite those being the core behavior added by this PR. Please keep the example small, but add focused regression coverage in the appropriate test suite (or a few compact assertions here).
  • Clang Format is currently failing in src/data_handler/data_handler.cpp.

The API example also currently describes a different output granularity from what its call produces.

accel_channels = BoardShim.get_accel_channels(board_id)
print(f'Accel channels: {accel_channels}, Sampling rate: {sampling_rate}')

# demo activity index calculation per 1-second epoch

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] This says “per 1-second epoch,” but omitting period sets it to the full integer-second length of the recording, so this call returns one value containing the sum of all 1-second AIs. Either pass period=sampling_rate to produce per-second outputs, or describe this as an aggregate over the complete recording.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants