Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion src/plugins/plugin_utils/hand_injector.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,34 @@
#include <cstring>
#include <stdexcept>
#include <string>
#include <system_error>

namespace plugin_utils
{

namespace
{
//! Callers that cannot link plugin_utils recognize it by name() and read code().value().
class XrResultCategory final : public std::error_category
{
public:
const char* name() const noexcept override
{
return "XrResult";
}
std::string message(int value) const override
{
return std::to_string(value);
}
};

void CheckXrResult(XrResult result, const char* message)
{
static const XrResultCategory category;
if (XR_FAILED(result))
{
throw std::runtime_error(std::string(message) + " failed with XrResult: " + std::to_string(result));
// what() reads "<message> failed with XrResult: <code>".
throw std::system_error(result, category, std::string(message) + " failed with XrResult");
}
}
} // namespace
Expand Down
7 changes: 5 additions & 2 deletions src/plugins/remote_device_bridge/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ SPDX-License-Identifier: Apache-2.0
- `session/protocol_frame.*` and `session/protocol_payloads.*` are the only serialization points. Every frame is encoded and decoded there and nowhere else; changing the wire format means changing them and `tests/cpp/plugins/remote_device_bridge/test_wire_codec.cpp` together. Sessions hold an `ITransport` (`session/inc/remote_plugin_session/transport.hpp`) and only call `send/receive/shutdown` on complete frames; TCP framing lives in `TcpTransport`.
- WebSocket is a second transport, not a second protocol: `WebSocketTransport` carries exactly one wire frame per binary message (no TLS; terminate `wss://` in a proxy). Clients pick it with `--remote=ws://host:port[/path]` via `parse_remote`/`connect_transport`; the bridge serves it on a separate `--ws-listen` port sharing the fork-per-client loop and `--max-clients`. Never sniff the protocol on the TCP port. The server-side HTTP upgrade runs in the first `receive()` so the accept loop never blocks on a client.
- Never add `OpenXR::openxr_loader`, `Teleop::plugin_utils` or DeviceIO session libraries to `Teleop::remote_plugin_session`: a plugin using `--remote` must run without an OpenXR runtime.
- ERROR on channel 0 ends the session; ERROR on a channel id refuses only that OPEN and the session and its other channels continue. Refuse per channel whenever the session itself is still sound.
- `Teleop::remote_device_bridge_server` (`server/`) gets each client's session from a `SessionFactory`; only `app/main.cpp` builds the OpenXR session, with `plugin_utils::make_openxr_plugin_session_factory`. Keep the OpenXR loader and `plugin_utils` out of `server/` so `tests/cpp/plugins/remote_device_bridge_server` can run the real fork-per-client bridge against a fake session.
- ERROR on channel 0 ends the session; ERROR on a channel id refuses only that OPEN (schema above `--max-schema-bytes`, a hand someone else owns) and the session and its other channels continue. Refuse per channel whenever the session itself is still sound.
- `RemotePluginSession` never throws on the sample path for transport failures (`push`, `push_buffer`, `query`, `update`): after one they become no-ops; only channel creation and the constructor throw. Caller bugs (null joints or buffer) still throw `std::invalid_argument`.
- Timestamps cross the wire in the sender's `CLOCK_MONOTONIC`; the receiver maps them with `ClockOffsetEstimator`, which prefers PING/PONG exchanges (minimum RTT) over the one-way minimum of `local_recv - remote_send`; the one-way minimum hides the minimum network delay. Do not push raw remote timestamps into OpenXR.
- The bridge (`server/`) talks to OpenXR from one thread per client (the thread running that `ClientSession`). Socket reading happens on a helper thread that only parses frames into a queue. Keep it that way; `OpenXRPluginSession` and its channels are not thread-safe.
- Timestamps cross the wire in the sender's `CLOCK_MONOTONIC`; the receiver maps them with `ClockOffsetEstimator`. The bridge's offset comes from PING/PONG (the minimum-RTT recent exchange, `--clock-source=two-way`); the one-way minimum of `local_recv - remote_send` hides the minimum network delay and is only the fallback (no PONG in time) or the `--clock-source=one-way` A/B baseline. Do not push raw remote timestamps into OpenXR.
- One bridge process serves N clients. The accept loop never creates threads or OpenXR objects so it may `fork()` a child per client (the isolation of N local plugin processes). Do not serve clients on threads: the OpenXR loader allows one `XrInstance` per process (a second one fails with -10). State the children must share (hand ownership) lives in a `MAP_SHARED` page mapped before the first fork; the parent clears a dead child's entries when it reaps it. Sample paths are lossy by design: per-channel latest-wins queues on the client, coalescing and a staleness guard on the bridge; control frames are never dropped or reordered.
6 changes: 6 additions & 0 deletions src/plugins/remote_device_bridge/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,9 @@ if(WIN32)
endif()

add_subdirectory(session)
add_subdirectory(server)
add_subdirectory(app)

install(FILES plugin.yaml
DESTINATION plugins/remote_device_bridge
)
18 changes: 18 additions & 0 deletions src/plugins/remote_device_bridge/app/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

add_executable(remote_device_bridge
main.cpp
)

target_link_libraries(remote_device_bridge
PRIVATE
Teleop::plugin_utils
Teleop::remote_device_bridge_server
Teleop::remote_plugin_session
log_bridge::log_bridge_core
)

install(TARGETS remote_device_bridge
RUNTIME DESTINATION plugins/remote_device_bridge
)
188 changes: 188 additions & 0 deletions src/plugins/remote_device_bridge/app/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

#include <log_bridge/logger.hpp>
#include <plugin_utils/openxr_plugin_session.hpp>
#include <remote_device_bridge/bridge_server.hpp>
#include <remote_plugin_session/remote_address.hpp>

#include <atomic>
#include <chrono>
#include <csignal>
#include <cstdlib>
#include <iostream>
#include <limits>
#include <optional>
#include <stdexcept>
#include <string>
#include <string_view>

namespace
{

static_assert(ATOMIC_BOOL_LOCK_FREE == 2, "lock-free atomic bool is required for signal safety");

std::atomic<bool> g_stop_requested{ false };

void signal_handler(int signal)
{
if (signal == SIGINT || signal == SIGTERM)
{
g_stop_requested.store(true, std::memory_order_relaxed);
}
}

std::optional<std::string> flag_value(std::string_view arg, std::string_view name)
{
if (arg.starts_with(name) && arg.size() > name.size() && arg[name.size()] == '=')
{
return std::string(arg.substr(name.size() + 1));
}
return std::nullopt;
}

//! Whole-string integer in [min, max of T]: std::stoi accepts trailing junk and std::stoul wraps "-1".
template <typename T>
T parse_integer(std::string_view flag, const std::string& value, T min)
{
size_t used = 0;
long long parsed = 0;
try
{
parsed = std::stoll(value, &used);
}
catch (const std::exception&)
{
used = 0;
}
if (used == 0 || used != value.size() || parsed < static_cast<long long>(min) ||
static_cast<unsigned long long>(parsed) > static_cast<unsigned long long>(std::numeric_limits<T>::max()))
{
throw std::invalid_argument(std::string(flag) + " must be an integer >= " + std::to_string(min) + ", got '" +
value + "'");
}
return static_cast<T>(parsed);
}

void print_usage()
{
const remote_device_bridge::BridgeOptions defaults;
std::cout
<< "Usage: remote_device_bridge [--listen=<ip:port>] [--ws-listen=<ip:port>] [--app-name=<str>]\n"
" [--wrist-rate-hz=<n>] [--max-clients=<n>] [--max-sample-age-ms=<n>]\n"
" [--max-schema-bytes=<n>] [--clock-source=two-way|one-way]\n"
" [--stats-csv=<path>]\n"
<< " --listen bind address (default " << defaults.listen.host << ":" << defaults.listen.port << ")\n"
<< " --ws-listen also accept WebSocket clients (--remote=ws://...) here; no TLS (default off,\n"
<< " port " << remote_plugin_session::kDefaultBridgeWsPort << " when omitted)\n"
<< " --app-name OpenXR app name; defaults to the name the client sends in HELLO\n"
<< " --wrist-rate-hz wrist sample rate towards the client (default " << defaults.wrist_rate_hz << ")\n"
<< " --max-clients concurrent clients; further ones get HELLO_ACK 'bridge full' (default "
<< defaults.max_clients << ")\n"
<< " --max-sample-age-ms drop samples older than this instead of pushing them; 0 = off (default "
<< defaults.max_sample_age->count() << ")\n"
<< " --max-schema-bytes refuse schema channels whose sample size exceeds this (default "
<< defaults.max_schema_bytes << ", the\n"
<< " largest the runtime accepts)\n"
<< " --clock-source two-way: PING/PONG min-RTT clock offset, one-way if the client never answers\n"
<< " in time; one-way: frame-header minimum, no PINGs (default two-way)\n"
<< " --stats-csv write one row per pushed hand sample to this file (truncated at start)\n";
}

remote_device_bridge::BridgeOptions parse_args(int argc, char** argv)
{
remote_device_bridge::BridgeOptions options;
for (int i = 1; i < argc; ++i)
{
const std::string arg = argv[i];
if (auto value = flag_value(arg, "--listen"))
{
options.listen = remote_plugin_session::parse_endpoint(*value, remote_plugin_session::kDefaultBridgePort);
}
else if (auto value = flag_value(arg, "--ws-listen"))
{
options.ws_listen =
remote_plugin_session::parse_endpoint(*value, remote_plugin_session::kDefaultBridgeWsPort);
}
else if (auto value = flag_value(arg, "--app-name"))
{
if (value->empty())
{
throw std::invalid_argument("--app-name must not be empty");
}
options.app_name = *value;
}
else if (auto value = flag_value(arg, "--wrist-rate-hz"))
{
options.wrist_rate_hz = parse_integer<int>("--wrist-rate-hz", *value, 1);
}
else if (auto value = flag_value(arg, "--max-clients"))
{
options.max_clients = parse_integer<int>("--max-clients", *value, 1);
}
else if (auto value = flag_value(arg, "--max-sample-age-ms"))
{
const int ms = parse_integer<int>("--max-sample-age-ms", *value, 0);
options.max_sample_age =
ms == 0 ? std::nullopt : std::optional<std::chrono::milliseconds>(std::chrono::milliseconds(ms));
}
else if (auto value = flag_value(arg, "--max-schema-bytes"))
{
options.max_schema_bytes = parse_integer<size_t>("--max-schema-bytes", *value, 1);
}
else if (auto value = flag_value(arg, "--clock-source"))
{
if (*value != "two-way" && *value != "one-way")
{
throw std::invalid_argument("--clock-source must be two-way or one-way");
}
options.clock_source = *value == "two-way" ? remote_device_bridge::ClockSource::TwoWay :
remote_device_bridge::ClockSource::OneWay;
}
else if (auto value = flag_value(arg, "--stats-csv"))
{
options.stats_csv = *value;
}
else if (flag_value(arg, "--plugin-root-id"))
{
// Injected by the PluginManager; not used by the bridge.
}
else if (arg == "--help" || arg == "-h")
{
print_usage();
std::exit(0);
}
else
{
throw std::invalid_argument("unknown argument '" + arg + "'");
}
}
return options;
}

} // namespace

int main(int argc, char** argv)
try
{
const remote_device_bridge::BridgeOptions options = parse_args(argc, argv);

std::signal(SIGINT, signal_handler);
std::signal(SIGTERM, signal_handler);

remote_device_bridge::BridgeServer server(options, plugin_utils::make_openxr_plugin_session_factory);
server.run(g_stop_requested);
return 0;
}
catch (const std::exception& e)
{
auto logger = isaaccapture::Logger::get("isaaccapture.plugins.remote_device_bridge.main");
logger->error("{}: {}", argv[0], e.what());
return 1;
}
catch (...)
{
auto logger = isaaccapture::Logger::get("isaaccapture.plugins.remote_device_bridge.main");
logger->error("{}: Unknown error occurred", argv[0]);
return 1;
}
14 changes: 14 additions & 0 deletions src/plugins/remote_device_bridge/plugin.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

name: remote_device_bridge
description: "Mirrors a remote plugin session (hand tracking, schema push, wrist pull) onto the local OpenXR runtime"
command: "./remote_device_bridge"
version: "0.1.0"
devices:
- path: "/hand/left"
type: "hand_tracker"
description: "Left hand from a remote plugin"
- path: "/hand/right"
type: "hand_tracker"
description: "Right hand from a remote plugin"
26 changes: 26 additions & 0 deletions src/plugins/remote_device_bridge/server/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

find_package(Threads REQUIRED)

add_library(remote_device_bridge_server STATIC
bridge_server.cpp
inc/remote_device_bridge/bridge_server.hpp
latency_stats.cpp
inc/remote_device_bridge/latency_stats.hpp
)

target_include_directories(remote_device_bridge_server
INTERFACE
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/inc>
)

target_link_libraries(remote_device_bridge_server
PUBLIC
Teleop::remote_plugin_session
PRIVATE
log_bridge::log_bridge_core
Threads::Threads
)

add_library(Teleop::remote_device_bridge_server ALIAS remote_device_bridge_server)
Loading
Loading