Feature/docker isolated testing and CI runner selection - #106
Open
martinrichtersupervag wants to merge 5 commits into
Open
martinrichtersupervag wants to merge 5 commits into
martinrichtersupervag wants to merge 5 commits into
Conversation
- Add Dockerfile based on Ubuntu 22.04 with Python 3, Node.js (xunit-viewer), and required test suite dependencies. - Add docker-compose.yml and .dockerignore. - Add run_in_docker.sh and run_tests_in_docker.sh automation scripts. - Add generate_reports.sh to merge JUnit XML and produce XLSX/HTML reports. - Update run_tests.sh and .gitignore to archive previous reports. - Document Docker execution and network isolation in README.md. This provides a fully reproducible and containerized test execution environment. By utilizing Docker's default bridge network, RTPS multicast discovery (239.255.0.1 and 224.0.0.0/4) is contained inside the container, preventing discovery packets from leaking into the host LAN or interfering with concurrent test instances (especially in LXC or VM environments).
…ibility - Add runner input (defaults to ubuntu-latest) allowing execution on self-hosted runners. - Set fail-fast: false to ensure all matrix test pairs complete even if one fails. - Specify repository: 'omg-dds/dds-rtps' for release-downloader so tests can run in forks. - Use python3 -m junitparser to merge XML reports reliably across systems. - Add --break-system-packages fallback for pip install and continue-on-error for setup-python.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When running tests directly on a host machine, virtual machine, or LXC container, RTPS Simple Participant Discovery Protocol (SPDP) broadcasts UDP discovery packets (
239.255.0.1:7400on domain 0) across the local physical network.If other DDS applications (such as ROS 2 or other local test runners) exist on the same local subnet and domain, mutual discovery occurs. This leads to unexpected endpoint matching, port collisions, and intermittent test failures.
Eliminates the need for contributors to manually set up Python virtual environments, Node.js (
xunit-viewer), or modify host system settings.Enables clean, isolated execution on self-hosted runners without requiring host
sysctlorip routemodifications.Changes
1. Docker Isolation & Tooling
Dockerfile: Pre-configured Ubuntu 22.04 image with Python 3, Node.js 20 LTS,xunit-viewer, and required test dependencies.docker-compose.yml&.dockerignore: Container definition utilizing Docker's default bridge network to contain all RTPS multicast discovery.run_tests_in_docker.sh: One-command automation script that archives previous reports, builds the image if needed, executes tests, merges reports, and cleans up.run_in_docker.sh: Flexible helper for interactive shell or custom test runs (includes automated non-interactive TTY detection for CI).generate_reports.sh: Standalone helper to merge JUnit XMLs viapython3 -m junitparserand generate Excel (.xlsx) and HTML (index.html) reports.run_tests.sh&.gitignore: Added automatic archiving of previous test runs intoarchive_reports/.2. CI Workflow Improvements (
1_run_interoperability_tests.yml)runnerinput parameter (defaults toubuntu-latest, also supportsself-hosted).RUNNER_ENVIRONMENT == "self-hosted"), while keeping direct execution on GitHub-hosted cloud runners.fail-fast: false): Prevents GitHub Actions from aborting the remaining 63 matrix test pairs when one vendor pair fails.repository: "omg-dds/dds-rtps"torelease-downloaderso workflows can run in forked repositories.junitparser mergetopython3 -m junitparser mergeto avoid issues when the binary is not in system$PATH.3. Documentation & Legal
README.md: Added Section 3.5 documenting Docker test execution and added a runner environment selection note in Section 4.CLA/CLA_Martin_Richter.md: Added signed Contributor License Agreement according toCONTRIBUTING.md.How to test
1. Test Single Test / Custom Pair inside Docker
Run an ad-hoc test between two vendors inside the isolated Docker container:
./run_in_docker.sh python3 interoperability_report.py \ -P executables/eprosima_fastdds*shape_main_linux \ -S executables/opendds*shape_main_linux \ -t basic_default_test