Use robot-reported RTDE field types and improved memory copy performance

.github/workflows/ci.yml

@@ -46,6 +46,40 @@ jobs:
           retention-days: 5
           archive: false
 
+  unit_coverage:
+    name: unit_coverage
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - name: Install build-tools
+        run: sudo apt-get update && sudo apt-get install -y build-essential cmake gcovr
+      - name: configure
+        run: >
+          mkdir build &&
+          cd build &&
+          cmake ..
+          -DBUILDING_TESTS=1
+          -DINTEGRATION_TESTS=0
+          -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
+        env:
+          CXXFLAGS: -g -O2 -fprofile-arcs -ftest-coverage
+          CFLAGS: -g -O2 -fprofile-arcs -ftest-coverage
+          LDFLAGS: -fprofile-arcs -ftest-coverage
+      - name: build
+        run: cmake --build build --config Debug
+      - name: test
+        run: cd build && ctest --output-on-failure --output-junit junit.xml
+      - name: gcovr
+        run: cd build && gcovr -r .. --xml coverage.xml --gcov-ignore-parse-errors negative_hits.warn_once_per_file --exclude "../3rdparty"
+      - name: Upload coverage reports to Codecov with GitHub Action
+        uses: codecov/codecov-action@v7
+        with:
+          fail_ci_if_error: true
+          files: build/coverage.xml
+          flags: unit
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
+
   run_tests:
     timeout-minutes: 60
     runs-on: ubuntu-latest

doc/architecture/rtde_client.rst

@@ -36,10 +36,33 @@ the :ref:`rtde_client_example` for an example of the blocking read method.
    {
      if (my_client.getDataPackage(data_pkg, READ_TIMEOUT))
      {
-       std::cout << data_pkg->toString() << std::endl;
+       std::cout << data_pkg.toString() << std::endl;
      }
    }
 
+.. note::
+
+  **Recommended:** Construct a ``DataPackage`` from ``getOutputRecipe()`` after ``init()`` and
+  reuse it in your control loop. With a matching recipe, the normal data receive path of
+  ``getDataPackage()`` and ``getDataPackageBlocking()`` does not allocate.
+
+  **Still supported, but not recommended:** The older flow that lets the client allocate a
+  package remains available for compatibility. The
+  ``std::unique_ptr<rtde_interface::DataPackage> getDataPackage(timeout)`` overload (Deprecated,
+  use ``getDataPackage(DataPackage&, timeout)`` instead) allocates a new package on each call, and
+  passing a null unique pointer to either read method also allocates a package. Passing a package
+  with a foreign recipe is supported through automatic repair, which may allocate. The null-pointer and foreign-recipe paths log
+  warnings; these warn about allocation, not unsupported usage. Prefer a reusable, matching-recipe
+  package for new code, especially in real-time loops.
+
+  The allocation-free guarantee applies only to the normal data receive path with that reused
+  package. It does not cover error handling, non-data messages or reconnection.
+
+  A recipe only lists field names. The data types belonging to them are reported by the robot when
+  it acknowledges the recipe, and the first read applies them to your ``DataPackage`` without
+  allocation. Until that has happened ``getData()`` throws ``std::bad_variant_access``. See
+  `Field data types`_ for how to ask a package what type it gave a field.
+
 Upon construction, two recipe files have to be given, one for the RTDE inputs, one for the RTDE
 outputs. Please refer to the `RTDE
 guide <https://www.universal-robots.com/articles/ur-articles/real-time-data-exchange-rtde-guide/>`_
@@ -63,12 +86,113 @@ following will work
 Reading data
 ------------
 
-After calling ``my_client.start()``, data can be read from the
-``RTDEClient`` by calling ``getDataPackage()`` (with background thread running) or ``getDataPackageBlocking()`` (without background thread running) respectively.
+After calling ``my_client.start()``, data can be read from the ``RTDEClient``. Which read method
+to use depends on how the client was started:
+
+With background reading, ``start(true)`` (the default):
+
+- ``bool getDataPackage(DataPackage& data_package, std::chrono::milliseconds timeout)``:
+  copies the latest package received by the background reader thread into the caller's package.
+- ``bool getDataPackage(std::unique_ptr<DataPackage>& data_package, std::chrono::milliseconds timeout)``:
+  same, for a caller-owned pointer; a null pointer is allocated (with a warning) only on success.
+- ``std::unique_ptr<DataPackage> getDataPackage(std::chrono::milliseconds timeout)``
+  (Deprecated, use ``getDataPackage(DataPackage& data_package, std::chrono::milliseconds timeout)``
+  instead): allocates a new package on every call. It will be removed in May 2027.
+
+Without background reading, ``start(false)``:
+
+- ``bool getDataPackageBlocking(std::unique_ptr<DataPackage>& data_package)``:
+  waits for the next package from the robot and parses it straight into the caller's package.
+
+Calling a method from the other group logs an error and returns ``false``.
 
 Remember that, when not using a background thread, data has to be polled regularly, as the robot
 will shutdown RTDE communication if the receiving side doesn't empty its buffer.
 
+Reusing a package with the negotiated recipe keeps the normal data receive path free of memory
+allocations; prefer an overload that fills an existing, reusable package, especially for real-time
+use.
+
+Always check the return value before using received data. A background read returns ``false`` on
+timeout or when stopping or reconnecting cancels the pending read; restarting the reader does not
+make a cancelled read succeed with stale data. Both unique-pointer overloads retain caller
+ownership on failure and assign a previously null pointer only on success. A failed blocking read
+can still partially update an existing package's values if the incoming data is malformed.
+
+Pacing a loop with the robot
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Both read modes can pace an application loop at the robot's negotiated RTDE output frequency,
+without a separate fixed-period sleep. Choose the mode according to how directly the loop should
+follow incoming packages:
+
+- With ``start(false)``, ``getDataPackageBlocking()`` waits for the next package when no data is
+  already buffered. Calling it at the start of each iteration lets packet arrival pace the loop,
+  coupling it directly to the RTDE stream without a background-reader handoff. This is useful
+  when each iteration should read data and then compute and submit a response. See the
+  :ref:`rtde_roundtrip_example`.
+- With ``start(true)``, ``getDataPackage(package, timeout)`` can also pace the loop: after the
+  latest sample has been consumed, it waits for the background reader to publish another one,
+  up to the timeout. If a newer sample is already available, it returns immediately. This mode
+  decouples socket reading from application work and favors the latest sample; intermediate
+  samples can be overwritten when the application is slower than the stream. It suits loops
+  that need less direct synchronization and do not need to process every received sample.
+
+For background reads used as a loop clock, allow enough timeout for the expected RTDE period
+and scheduling jitter, and handle a ``false`` return instead of processing old data. In either
+mode, the loop must keep up with the negotiated frequency for steady pacing. Buffered data can
+make synchronous reads return immediately, and network or scheduling delays can make arrivals
+irregular. Neither mode guarantees phase synchronization with the robot's internal control cycle
+or receipt of a command in the next cycle. Use the output ``timestamp`` to track robot time and
+detect gaps between samples.
+
+Field data types
+~~~~~~~~~~~~~~~~
+
+``getData()`` has to be given a variable of the field's own type. A missing name returns
+``false``; a type mismatch throws ``std::bad_variant_access``.
+Rather than hardcoding which type a field has, ask the package: ``getDataType()`` reports the
+``DataType`` a field currently holds. A successful client read applies the robot's negotiated
+types to an output package; ``init()`` alone does not type application-owned packages.
+``createInputDataPackage()`` returns an input package with the negotiated types already applied.
+On a recipe-only package, ``setData()`` establishes an untyped field's type from the value written;
+subsequent writes must match that type. An untouched field has no type. This is useful for code
+that has to handle whatever recipe it is configured with, such as a bridge to another middleware:
+
+.. code-block:: c++
+
+   const std::optional<rtde_interface::DataType> type = data_pkg.getDataType(field_name);
+   if (!type)
+   {
+     // Not part of the recipe, or the field has no type yet
+     return;
+   }
+
+   // For "actual_q" this prints "VECTOR6D", the same spelling the RTDE guide uses
+   std::cout << field_name << " is a " << rtde_interface::toString(*type) << std::endl;
+
+   switch (*type)
+   {
+     case rtde_interface::DataType::DOUBLE:
+     {
+       double value;
+       data_pkg.getData(field_name, value);
+       break;
+     }
+     case rtde_interface::DataType::VECTOR6D:
+     {
+       vector6d_t value;
+       data_pkg.getData(field_name, value);
+       break;
+     }
+     // ... remaining types
+   }
+
+``DataType`` covers the complete set the protocol defines: ``BOOL``, ``UINT8``, ``UINT32``,
+``UINT64``, ``INT32``, ``DOUBLE``, ``VECTOR3D``, ``VECTOR6D``, ``VECTOR6INT32`` and
+``VECTOR6UINT32``. Switching over it exhaustively means the compiler will point out any case a
+future protocol addition leaves unhandled.
+
 Writing data
 ------------
 
@@ -105,11 +229,11 @@ an empty input recipe, like this:
    // Alternatively, pass an empty filename when using recipe files
    // rtde_interface::RTDEClient my_client(ROBOT_IP, notifier, OUTPUT_RECIPE_FILE, "");
    my_client.init();
-   auto data_pkg = std::make_unique<rtde_interface::DataPackage>(my_client->getOutputRecipe());
+   auto data_pkg = std::make_unique<rtde_interface::DataPackage>(my_client.getOutputRecipe());
    my_client.start();
    while (true)
    {
-     if (my_client.getDataPackage(data_package, READ_TIMEOUT))
+     if (my_client.getDataPackage(data_pkg, READ_TIMEOUT))
      {
        std::cout << data_pkg->toString() << std::endl;
      }
@@ -121,12 +245,58 @@ RTDEWriter
 The ``RTDEWriter`` class provides an interface to write data to the RTDE interface. Data fields that
 should be written have to be defined inside the ``INPUT_RECIPE`` as noted above.
 
-The class offers specific methods for every RTDE input possible to write.
+.. important::
+
+  **Use RTDEClient to initialize and access RTDEWriter (recommended).** Create an ``RTDEClient``
+  with a non-empty input recipe, call ``init()``, then use its writer through ``getWriter()``.
+  The client handles the connection, protocol negotiation, input recipe setup and writer
+  initialization, including the field types and recipe ID reported by the robot.
+
+  Constructing and using ``RTDEWriter`` directly is still supported, but is a lower-level option
+  for applications that manage the RTDE connection and handshake themselves. It is not the
+  recommended approach for normal application code.
+
+The class offers convenience methods for common inputs and ``sendPackage()`` for a complete input
+recipe.
+
+Data is sent asynchronously to the RTDE interface. A successful ``sendPackage()`` or ``send...()``
+call updates the pending send buffer and notifies the writer thread; it does not confirm delivery
+or processing by the robot. This is not a FIFO queue of calls: multiple updates before the writer
+consumes the pending buffer can be coalesced, and a later ``sendPackage()`` can replace an earlier
+pending package. Separate helper calls may be transmitted separately or coalesced, depending on
+when the writer runs. Use ``sendPackage()`` to submit related fields together in one buffer update,
+not to guarantee a distinct transmission for every call.
+
+To write several fields at once, ask the client for a package that already carries the data types
+the robot reported for the input recipe. Call ``createInputDataPackage()`` after a successful
+``init()`` with a non-empty input recipe, fill the fields you care about and pass it to
+``sendPackage()``. The new package starts with zero values; when reusing it, fields retain their
+previous values unless explicitly changed or reset. Because the package is already typed,
+``setData()`` reports a value written with the wrong type immediately:
+
+.. code-block:: c++
+
+   rtde_interface::DataPackage input_pkg = my_client.createInputDataPackage();
+   input_pkg.setData("speed_slider_mask", uint32_t{ 1 });
+   input_pkg.setData("speed_slider_fraction", 0.5);
+   my_client.getWriter().sendPackage(input_pkg);
+
+A package constructed from ``getInputRecipe()`` still works. Its types are taken from the values
+written to it and are checked when the package is submitted to ``sendPackage()``. The field names
+and order must match the negotiated input recipe. Fields that remain untyped are sent as typed
+zeros, while incompatible types cause ``sendPackage()`` to return ``false``. See the
+:ref:`rtde_roundtrip_example` for a complete example.
 
-Data is sent asynchronously to the RTDE interface.
+If direct ``RTDEWriter`` use is required instead of the recommended ``RTDEClient`` flow, perform the RTDE handshake
+and configure the stopped writer with ``setProtocolVersion(negotiated_version)`` and
+``setRecipeTypes(acknowledged_types)`` before calling ``init(recipe_id)`` with the acknowledged
+input recipe ID. Constructing the writer or calling ``init(recipe_id)`` alone does not establish
+the field types. ``RTDEClient::init()`` handles these steps automatically.
 
 .. note::
 
    The ``RTDEWriter`` will return ``false`` on any writing attempts for fields that have not been
    setup in the ``INPUT_RECIPE``. When no input recipe was provided, all write operations will
-   return ``false``.
+   return ``false``. No writer thread is started in that case, and ``createInputDataPackage()``
+   throws ``UrException`` even after successful client initialization. The factory also throws
+   before input recipe negotiation or while the writer is stopped.

doc/examples.rst

@@ -23,6 +23,7 @@ may be running forever until manually stopped.
    examples/primary_pipeline
    examples/primary_pipeline_calibration
    examples/rtde_client
+   examples/rtde_roundtrip
    examples/external_fts_through_rtde
    examples/script_command_interface
    examples/script_sender

doc/examples/rtde_client.rst

@@ -56,15 +56,20 @@ fetch data synchronously. Hence, we pass ``false`` to the ``start()`` method.
    :start-at: auto data_pkg = std::make_unique<rtde_interface::DataPackage>(my_client.getOutputRecipe());
    :end-before: // Change the speed slider
 
+The loop reuses a package built from the negotiated output recipe, keeping the normal data receive
+path allocation-free. Null pointers allocate a package, and foreign-recipe repair may allocate;
+error handling, non-data messages and reconnection are outside this guarantee. The recipe only
+names the fields, so the first read applies their negotiated types in place without allocation.
+
 In our main loop, we wait for a new data package to arrive using the blocking read method. Once
 received, data from the received package can be accessed using the ``getData()`` method of the
 ``DataPackage`` object. This method takes the key of the data to be accessed as a parameter and
 returns the corresponding value.
 
 .. note:: The key used to access data has to be part of the output recipe used to initialize the RTDE
-   client. Passing a string literal, e.g. ``"actual_q"``, is possible but not recommended as it is
-   converted to an ``std::string`` automatically, causing heap allocations which should be avoided
-   in Real-Time contexts.
+   client. ``getData()`` returns ``false`` for an unknown key. If the type of the passed
+   variable doesn't match the type the robot reported for that field, it throws
+   ``std::bad_variant_access``.
 
 Writing Data to the RTDE client
 -------------------------------
@@ -91,7 +96,9 @@ initialize the RTDE client has to contain the keys necessary to send that specif
    <https://www.universal-robots.com/articles/ur/interface-communication/real-time-data-exchange-rtde-guide/>`_
    for more information.
 
-.. note:: Every ``send...`` call to the RTDEWriter triggers a package sent to the robot. If you
-   want to modify more than one input at a time, it is recommended to use the ``sendPackage()``
-   method. That allows setting up the complete data package with its input recipe and sending that
-   to the robot at once.
+.. note:: Every successful ``send...`` call updates the pending buffer and notifies the writer
+   thread. Calls may be coalesced before transmission; they are not queued as separate packages.
+   To submit several inputs together, use ``createInputDataPackage()`` after ``init()``, fill the
+   fields and pass the package to ``sendPackage()``. Separate helper calls can otherwise be
+   transmitted between updates. Neither API confirms delivery to the robot; see the
+   :ref:`rtde_roundtrip_example` for verification using robot outputs.