diff --git a/electron/native/README.md b/electron/native/README.md index 5b38de88e..7c9deeb0b 100644 --- a/electron/native/README.md +++ b/electron/native/README.md @@ -85,7 +85,9 @@ The current helper implementation supports display/window video capture, system Encoder selection: by default the helper keeps the existing sink-writer path first. If that path fails while setting up H.264, it retries with the Microsoft software H.264 encoder (`mfh264enc.dll`). The key of this retry is registering that encoder locally in the helper process via `MFTRegisterLocalByCLSID`, which makes a software H.264 encoder available even when the machine's hardware encoders are missing or broken; hardware transforms are disabled for the retry only as a secondary guard so the sink writer prefers the locally registered software encoder, not as the fallback mechanism itself. Set `preferSoftwareEncoder: true` in the helper JSON, or set `OPENSCREEN_WGC_PREFER_SOFTWARE_ENCODER=true` before launching Electron, to force the software path from the first attempt. -The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`). When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way. +Frame input path: the helper feeds the encoder from the GPU when it can. On that path it copies the WGC frame across a keyed-mutex bridge to a second D3D11 device, converts BGRA to NV12 with the D3D11 video processor, and submits an allocator-owned DXGI sample to the hardware H.264 encoder, so no frame ever passes through system memory. The alternative is the original path: a staging texture, `Map(D3D11_MAP_READ)`, and a row-by-row copy into an `IMFMediaBuffer` — which is where a driver stall costs a recording (issue #252). The GPU path is a preference, never a requirement: it is skipped outright for `preferSoftwareEncoder` and for inline webcam PiP (both need the frame in system memory), and it degrades to the CPU path on its own if the encoding device, the NV12 video processor, the shared bridge texture, the DXGI sample allocator, or the hardware sink writer is unavailable. Set `OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1` to force the CPU path. Because the two paths land on different encoders and hardware MFTs default to constant bitrate, the GPU path asks for VBR through `ICodecAPI`; without it a static screen spends the full configured budget (measured 16.9 Mbps against 1.95 for the same desktop). + +The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`; `videoInput` is `dxgi-nv12` or `cpu-rgb32`, and reports what the encoder settled on rather than what was asked for). On the GPU path the helper also prints one `[frame-drops] gpu_bridge_contended=` line to stderr at stop: a frame the bridge was too busy to take is skipped rather than failing the recording, and a large count there is the first thing to look at in a report about missing frames. When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way. Encoder diagnostic on final sink-writer failure: when the final `MFCreateSinkWriterFromURL` attempt fails, the helper logs the registered H.264 video encoder MFT count (via `MFTEnumEx`), the registered AAC encoder count when audio was requested, and the hex HRESULT. If no H.264 encoder is registered, it additionally emits the four-bullet actionable error (missing Media Feature Pack / GPU driver registration / empty `HKLM:\SOFTWARE\Microsoft\Windows Media Foundation\Transforms` / reboot). If an H.264 encoder IS registered but the sink writer still failed, it logs a hint pointing at invalid output path, missing MP4 mux, or GPU driver incompatibility. There is still no fail-fast pre-flight gate because `MFTEnumEx` and `MFCreateSinkWriterFromURL` can disagree about which H.264 encoders are available in non-interactive / Session 0 contexts. diff --git a/electron/native/wgc-capture/src/main.cpp b/electron/native/wgc-capture/src/main.cpp index 63cbcbba7..bc96b150a 100644 --- a/electron/native/wgc-capture/src/main.cpp +++ b/electron/native/wgc-capture/src/main.cpp @@ -607,6 +607,15 @@ int main(int argc, char* argv[]) { MFEncoderOptions encoderOptions{}; encoderOptions.preferSoftwareEncoder = config.preferSoftwareEncoder; encoderOptions.injectDefaultSinkWriterFailureOnce = injectDefaultSinkWriterFailureOnce; + // Keep the CPU path for software encoding and inline webcam PiP: both need + // the frame in system memory, which is the one thing the DXGI path does not + // produce. The env var is the escape hatch for a machine where the GPU path + // misbehaves in a way the encoder's own probes do not catch -- a support + // answer instead of a hotfix. + encoderOptions.useDxgiInput = + !config.preferSoftwareEncoder && + (!webcamActive || writeSeparateWebcam) && + readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0; MFEncoder encoder; if (!encoder.initialize( @@ -622,8 +631,14 @@ int main(int argc, char* argv[]) { std::cerr << "ERROR: Failed to initialize Media Foundation encoder" << std::endl; return 1; } + // `videoInput` reports what the encoder settled on, not what was asked for: + // it silently degrades to the CPU readback on any machine the GPU path does + // not fit, and a bug report that cannot tell the two apart is a bug report + // about the wrong path. + const bool usesDxgiInput = encoder.usesDxgiInput(); std::cout << "{\"event\":\"encoder-selection\",\"schemaVersion\":2,\"video\":\"" << encoder.videoEncoderSelection() + << "\",\"videoInput\":\"" << (usesDxgiInput ? "dxgi-nv12" : "cpu-rgb32") << "\",\"preferSoftwareEncoder\":" << (config.preferSoftwareEncoder ? "true" : "false") << "}" << std::endl; @@ -631,6 +646,7 @@ int main(int argc, char* argv[]) { if (writeSeparateWebcam) { MFEncoderOptions webcamEncoderOptions = encoderOptions; webcamEncoderOptions.injectDefaultSinkWriterFailureOnce = false; + webcamEncoderOptions.useDxgiInput = false; const int webcamPixels = std::max(1, webcamCapture.width()) * std::max(1, webcamCapture.height()); const int webcamBitrate = webcamPixels >= 1280 * 720 ? 8'000'000 : 4'000'000; if (!webcamEncoder.initialize( @@ -652,6 +668,10 @@ int main(int argc, char* argv[]) { CaptureControl control; std::atomic firstFrameWritten = false; std::atomic encodeFailed = false; + // Frames the GPU bridge was too busy to take. Reported at stop rather than + // per frame: a handful over a recording is normal contention, a stream of + // them is the next bug report, and neither is worth a log line each. + std::atomic contendedFrames = 0; Microsoft::WRL::ComPtr latestFrameTexture; int64_t latestFrameTimestampHns = 0; int64_t firstFrameTimestampHns = -1; @@ -802,22 +822,40 @@ int main(int argc, char* argv[]) { std::this_thread::sleep_for(std::chrono::milliseconds(testStallReadbackMs)); } if (latestFrameTexture) { - // captureVideoSample performs the GPU readback - // (CopyResource/Map) from latestFrameTexture, which must - // stay serialized (via `mutex`) against the WGC - // frame-arrival callback above, which writes new data - // into the same texture on another thread. - hasVideoSample = encoder.captureVideoSample( - latestFrameTexture.Get(), - frameTimestampHns, - !writeSeparateWebcam && webcamFrame.data ? &webcamFrame : nullptr, - videoSample); - if (!hasVideoSample) { + // Both entry points do their GPU work on latestFrameTexture, + // which must stay serialized (via `mutex`) against the WGC + // frame-arrival callback above, which writes new data into + // the same texture on another thread. Which one is live is + // the encoder's answer, not this struct's request: it falls + // back to the CPU path on its own when the GPU path does + // not fit the machine. + bool captured = false; + if (usesDxgiInput) { + captured = encoder.captureDxgiSample( + latestFrameTexture.Get(), + frameTimestampHns, + videoSample); + } else { + captured = encoder.captureVideoSample( + latestFrameTexture.Get(), + frameTimestampHns, + !writeSeparateWebcam && webcamFrame.data ? &webcamFrame : nullptr, + videoSample); + } + if (!captured) { encodeFailed = true; control.requestStop(); break; } - lastEncodedVideoTimestampHns = frameTimestampHns; + // The DXGI path returns success with no sample when the + // GPU bridge was momentarily busy. That costs one frame, + // which beats ending a recording that is otherwise fine. + hasVideoSample = videoSample != nullptr; + if (hasVideoSample) { + lastEncodedVideoTimestampHns = frameTimestampHns; + } else { + contendedFrames += 1; + } } } @@ -1080,8 +1118,12 @@ int main(int argc, char* argv[]) { // sleep still be killed. if (stopElapsedMs() >= currentStepDeadlineMs.load() && !shutdownComplete.load()) { const char* step = currentStopStep.load(); + // The encoder stage is what turns "video-writer-join was + // abandoned" into something actionable: it names the call the + // writer thread is sitting in, instead of leaving the next + // report to guess the way issue #252 had to. std::cerr << "[stop-timing] step=" << step << " elapsed_ms=" << stopElapsedMs() - << " phase=abandoned" << std::endl; + << " phase=abandoned encode_stage=" << encoder.encodeStage() << std::endl; std::cout << "{\"event\":\"stop-timeout\",\"schemaVersion\":2,\"step\":\"" << step << "\"}" << std::endl; std::cout.flush(); @@ -1124,6 +1166,9 @@ int main(int argc, char* argv[]) { beginStopStep("video-writer-join", stepBudgetMs); stopVideoWriter(); logStopStep("video-writer-join"); + if (usesDxgiInput) { + std::cerr << "[frame-drops] gpu_bridge_contended=" << contendedFrames.load() << std::endl; + } // No frame lock here, and the ordering above is what makes that safe rather // than incidental: stopVideoWriter() joined the only thread that calls into // the encoder's GPU readback, and audioMixer->stop() joined the only other diff --git a/electron/native/wgc-capture/src/mf_encoder.cpp b/electron/native/wgc-capture/src/mf_encoder.cpp index 60f82e9f5..046a63e22 100644 --- a/electron/native/wgc-capture/src/mf_encoder.cpp +++ b/electron/native/wgc-capture/src/mf_encoder.cpp @@ -2,6 +2,10 @@ #include "audio_sample_utils.h" +#include +#include +#include +#include #include #include #include @@ -134,6 +138,7 @@ enum class SinkWriterCreateStage { SoftwareEncoderRegistration, CreateAttributes, DisableHardwareTransforms, + ConfigureDxgiManager, CreateSinkWriter, }; @@ -179,6 +184,7 @@ HRESULT ensureSoftwareH264EncoderRegisteredForProcess() { HRESULT createSinkWriterFromUrl( const std::wstring& outputPath, bool forceSoftwareEncoder, + IMFDXGIDeviceManager* dxgiDeviceManager, bool injectDefaultSinkWriterFailureOnce, bool& injectedDefaultSinkWriterFailure, Microsoft::WRL::ComPtr& sinkWriter, @@ -218,6 +224,26 @@ HRESULT createSinkWriterFromUrl( failedStage = SinkWriterCreateStage::DisableHardwareTransforms; return hr; } + } else if (dxgiDeviceManager != nullptr) { + HRESULT hr = MFCreateAttributes(&attributes, 3); + if (FAILED(hr)) { + std::cerr << "ERROR: MFCreateAttributes(DXGI sink writer) failed (hr=0x" + << std::hex << hr << std::dec << ")" << std::endl; + failedStage = SinkWriterCreateStage::CreateAttributes; + return hr; + } + hr = attributes->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, TRUE); + if (FAILED(hr)) { + failedStage = SinkWriterCreateStage::ConfigureDxgiManager; + return hr; + } + hr = attributes->SetUnknown(MF_SINK_WRITER_D3D_MANAGER, dxgiDeviceManager); + if (FAILED(hr)) { + std::cerr << "ERROR: Set MF_SINK_WRITER_D3D_MANAGER failed (hr=0x" + << std::hex << hr << std::dec << ")" << std::endl; + failedStage = SinkWriterCreateStage::ConfigureDxgiManager; + return hr; + } } failedStage = SinkWriterCreateStage::CreateSinkWriter; @@ -332,6 +358,43 @@ const char* MFEncoder::videoEncoderSelection() const { return videoEncoderSelection_; } +bool MFEncoder::usesDxgiInput() const { + return useDxgiInput_; +} + +const char* MFEncoder::encodeStage() const { + return encodeStage_.load(); +} + +int64_t MFEncoder::nextSampleTime(int64_t timestampHns, int64_t sampleDuration) { + // On `timestampMutex_` and not `writerMutex_`, deliberately. Every caller + // of this runs under main.cpp's frame lock, and `writerMutex_` is held + // across IMFSinkWriter::WriteSample by both submitVideoSample and + // writeAudio. Taking it here put a synchronous encode inside the frame + // lock: an audio WriteSample would stall the video writer, the video + // writer would stall every WGC callback waiting on that lock, and stop + // would find wgc-quiesce undrained and the writer unjoinable. That is the + // system-audio reproduction in the issue #252 follow-up. Nothing else + // touches these two fields, so they get a lock of their own that no + // blocking call is ever held across. + // + // No sinkWriter_/finalized_ check any more either: that check was the only + // other reason to be on writerMutex_, and submitVideoSample already makes + // it before writing. A sample built for a writer that has since gone away + // is discarded there, which costs one wasted buffer on a path that is + // shutting down anyway. + std::scoped_lock lock(timestampMutex_); + if (firstTimestampHns_ < 0) { + firstTimestampHns_ = timestampHns; + } + int64_t sampleTime = timestampHns - firstTimestampHns_; + if (sampleTime <= lastTimestampHns_) { + sampleTime = lastTimestampHns_ + sampleDuration; + } + lastTimestampHns_ = sampleTime; + return sampleTime; +} + bool MFEncoder::initialize( const std::wstring& outputPath, int width, @@ -347,12 +410,26 @@ bool MFEncoder::initialize( fps_ = std::max(1, fps); device_ = device; context_ = context; + captureDevice_ = device; + captureContext_ = context; + // The injected failure exists to prove the software fallback still works. + // Leaving the GPU path on would make it prove something else: the DXGI + // attempt would eat the injection and the run would land on the plain CPU + // encoder, never reaching the software encoder the knob is aimed at. + useDxgiInput_ = options.useDxgiInput && !options.injectDefaultSinkWriterFailureOnce; videoEncoderSelection_ = kVideoEncoderSelectionDefault; if (!succeeded(MFStartup(MF_VERSION), "MFStartup")) { return false; } + if (useDxgiInput_ && !initializeDxgiPipeline()) { + std::cerr << "WARNING: The GPU DXGI encode path is unavailable on this machine; " + << "using the CPU readback path." << std::endl; + releaseDxgiPipeline(); + useDxgiInput_ = false; + } + Microsoft::WRL::ComPtr outputType; if (!succeeded(MFCreateMediaType(&outputType), "MFCreateMediaType(output)")) { return false; @@ -369,13 +446,58 @@ bool MFEncoder::initialize( if (!succeeded(MFCreateMediaType(&inputType), "MFCreateMediaType(input)")) { return false; } - inputType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video); - inputType->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_RGB32); - inputType->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive); - inputType->SetUINT32(MF_MT_DEFAULT_STRIDE, static_cast(width_ * 4)); - setFrameSize(inputType.Get(), static_cast(width_), static_cast(height_)); - setFrameRate(inputType.Get(), static_cast(fps_)); - setPixelAspectRatio(inputType.Get()); + // Rebuilt rather than built once, because falling back to the CPU path + // after the sink writer has already refused the NV12 type has to leave a + // type the RGB32 path would have produced from scratch. Every attribute + // one mode sets is deleted by the other; nothing carries over. + auto configureVideoInputType = [&](bool dxgi) { + inputType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video); + inputType->SetGUID(MF_MT_SUBTYPE, dxgi ? MFVideoFormat_NV12 : MFVideoFormat_RGB32); + inputType->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive); + if (dxgi) { + inputType->DeleteItem(MF_MT_DEFAULT_STRIDE); + // The video processor below converts full-range BGRA into + // studio-range BT.709, so say so. Left untagged, the encoder and + // the player each pick their own default (BT.601 is the common + // one) and the recording comes back with shifted colours the CPU + // path does not have. + inputType->SetUINT32(MF_MT_VIDEO_NOMINAL_RANGE, MFNominalRange_16_235); + inputType->SetUINT32(MF_MT_YUV_MATRIX, MFVideoTransferMatrix_BT709); + } else { + inputType->SetUINT32(MF_MT_DEFAULT_STRIDE, static_cast(width_ * 4)); + inputType->DeleteItem(MF_MT_VIDEO_NOMINAL_RANGE); + inputType->DeleteItem(MF_MT_YUV_MATRIX); + } + setFrameSize(inputType.Get(), static_cast(width_), static_cast(height_)); + setFrameRate(inputType.Get(), static_cast(fps_)); + setPixelAspectRatio(inputType.Get()); + }; + + // Carried on the H.264 type as well so the MP4 sink writes the matching + // colour tags instead of leaving players to guess from the frame size. + auto configureOutputColorTags = [&](bool dxgi) { + if (dxgi) { + outputType->SetUINT32(MF_MT_VIDEO_NOMINAL_RANGE, MFNominalRange_16_235); + outputType->SetUINT32(MF_MT_YUV_MATRIX, MFVideoTransferMatrix_BT709); + } else { + outputType->DeleteItem(MF_MT_VIDEO_NOMINAL_RANGE); + outputType->DeleteItem(MF_MT_YUV_MATRIX); + } + }; + + // The allocator is the last thing that can refuse the GPU path, and it can + // only be built once the NV12 type exists. Falling back here costs nothing + // but the type rewrite, because no sink writer has been created yet. + configureVideoInputType(useDxgiInput_); + configureOutputColorTags(useDxgiInput_); + if (useDxgiInput_ && !initializeSampleAllocator(inputType.Get())) { + std::cerr << "WARNING: The DXGI sample allocator is unavailable on this machine; " + << "using the CPU readback path." << std::endl; + releaseDxgiPipeline(); + useDxgiInput_ = false; + configureVideoInputType(false); + configureOutputColorTags(false); + } bool injectedDefaultSinkWriterFailure = false; @@ -397,6 +519,7 @@ bool MFEncoder::initialize( const HRESULT sinkWriterHr = createSinkWriterFromUrl( outputPath, forceSoftwareEncoder, + forceSoftwareEncoder ? nullptr : dxgiDeviceManager_.Get(), options.injectDefaultSinkWriterFailureOnce, injectedDefaultSinkWriterFailure, sinkWriter_, @@ -430,6 +553,9 @@ bool MFEncoder::initialize( "SetInputMediaType")) { return false; } + if (useDxgiInput_) { + applyHardwareRateControl(std::max(1, bitrate)); + } if (!succeeded(sinkWriter_->BeginWriting(), "BeginWriting")) { return false; } @@ -449,6 +575,25 @@ bool MFEncoder::initialize( return true; } + if (useDxgiInput_) { + // The GPU path exists to dodge a CPU readback, not to be a requirement. + // Drop it and retry the exact chain a machine without it would have + // taken -- the software encoder cannot accept DXGI samples, so without + // this the fallback below would be unreachable for every recording that + // asked for the GPU path, which is all of them by default. + std::cerr + << "WARNING: Hardware DXGI H.264 encoder setup failed; " + << "retrying on the CPU readback path." + << std::endl; + releaseDxgiPipeline(); + useDxgiInput_ = false; + configureVideoInputType(false); + configureOutputColorTags(false); + if (configureSinkWriterAttempt(false, kVideoEncoderSelectionDefault, false)) { + return true; + } + } + std::cerr << "WARNING: Default Media Foundation H.264 encoder setup failed; " << "retrying with the Microsoft software H.264 encoder." @@ -603,32 +748,437 @@ bool MFEncoder::copyBgraFrameToBuffer(const BgraFrameView& frame, BYTE* destinat return true; } -bool MFEncoder::captureVideoSample( +bool MFEncoder::initializeDxgiPipeline() { + return initializeDxgiEncodingDevice() && + succeeded( + MFCreateDXGIDeviceManager(&dxgiResetToken_, &dxgiDeviceManager_), + "MFCreateDXGIDeviceManager") && + succeeded( + dxgiDeviceManager_->ResetDevice(device_.Get(), dxgiResetToken_), + "IMFDXGIDeviceManager::ResetDevice") && + initializeVideoProcessor(); +} + +void MFEncoder::releaseDxgiPipeline() { + bridgeInputView_.Reset(); + encoderBridgeMutex_.Reset(); + encoderBridgeTexture_.Reset(); + captureBridgeMutex_.Reset(); + captureBridgeTexture_.Reset(); + videoProcessor_.Reset(); + videoProcessorEnumerator_.Reset(); + videoContext_.Reset(); + videoDevice_.Reset(); + videoSampleAllocator_.Reset(); + dxgiDeviceManager_.Reset(); + dxgiResetToken_ = 0; + // Put the encoder back on the capture device. initializeDxgiEncodingDevice + // overwrites device_/context_ with the second device it creates, and the + // CPU path's staging texture has to live on the same device the WGC frames + // do or its CopyResource silently does nothing. + device_ = captureDevice_; + context_ = captureContext_; +} + +bool MFEncoder::initializeDxgiEncodingDevice() { + Microsoft::WRL::ComPtr captureDxgiDevice; + if (!succeeded(captureDevice_.As(&captureDxgiDevice), "Query capture IDXGIDevice")) { + return false; + } + Microsoft::WRL::ComPtr adapter; + if (!succeeded(captureDxgiDevice->GetAdapter(&adapter), "Get capture DXGI adapter")) { + return false; + } + + const UINT flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_VIDEO_SUPPORT; + D3D_FEATURE_LEVEL featureLevels[] = { + D3D_FEATURE_LEVEL_11_1, + D3D_FEATURE_LEVEL_11_0, + D3D_FEATURE_LEVEL_10_1, + D3D_FEATURE_LEVEL_10_0, + }; + D3D_FEATURE_LEVEL featureLevel{}; + if (!succeeded( + D3D11CreateDevice( + adapter.Get(), + D3D_DRIVER_TYPE_UNKNOWN, + nullptr, + flags, + featureLevels, + ARRAYSIZE(featureLevels), + D3D11_SDK_VERSION, + &device_, + &featureLevel, + &context_), + "D3D11CreateDevice(encoder)")) { + return false; + } + + Microsoft::WRL::ComPtr multithread; + if (!succeeded(context_.As(&multithread), "Query encoder ID3D10Multithread")) { + return false; + } + multithread->SetMultithreadProtected(TRUE); + return true; +} + +bool MFEncoder::initializeVideoProcessor() { + if (!succeeded(device_.As(&videoDevice_), "Query ID3D11VideoDevice")) { + return false; + } + if (!succeeded(context_.As(&videoContext_), "Query ID3D11VideoContext")) { + return false; + } + + D3D11_VIDEO_PROCESSOR_CONTENT_DESC contentDesc{}; + contentDesc.InputFrameFormat = D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE; + contentDesc.InputFrameRate = {static_cast(fps_), 1}; + contentDesc.InputWidth = static_cast(width_); + contentDesc.InputHeight = static_cast(height_); + contentDesc.OutputFrameRate = {static_cast(fps_), 1}; + contentDesc.OutputWidth = static_cast(width_); + contentDesc.OutputHeight = static_cast(height_); + contentDesc.Usage = D3D11_VIDEO_USAGE_PLAYBACK_NORMAL; + + if (!succeeded( + videoDevice_->CreateVideoProcessorEnumerator( + &contentDesc, + &videoProcessorEnumerator_), + "CreateVideoProcessorEnumerator")) { + return false; + } + + UINT nv12Support = 0; + if (!succeeded( + videoProcessorEnumerator_->CheckVideoProcessorFormat( + DXGI_FORMAT_NV12, + &nv12Support), + "CheckVideoProcessorFormat(NV12)")) { + return false; + } + if ((nv12Support & D3D11_VIDEO_PROCESSOR_FORMAT_SUPPORT_OUTPUT) == 0) { + std::cerr << "ERROR: D3D11 video processor does not support NV12 output" << std::endl; + return false; + } + + if (!succeeded( + videoDevice_->CreateVideoProcessor( + videoProcessorEnumerator_.Get(), + 0, + &videoProcessor_), + "CreateVideoProcessor")) { + return false; + } + + // Processor state, not per-blt arguments. Nothing below changes for the + // life of the recording -- the capture size is fixed at initialize() -- + // so setting it once keeps four driver round trips out of every frame. + const RECT frameRect{0, 0, width_, height_}; + videoContext_->VideoProcessorSetOutputTargetRect(videoProcessor_.Get(), TRUE, &frameRect); + videoContext_->VideoProcessorSetStreamFrameFormat( + videoProcessor_.Get(), + 0, + D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE); + videoContext_->VideoProcessorSetStreamSourceRect(videoProcessor_.Get(), 0, TRUE, &frameRect); + videoContext_->VideoProcessorSetStreamDestRect(videoProcessor_.Get(), 0, TRUE, &frameRect); + + // Screen pixels are full-range sRGB and H.264 in an MP4 is conventionally + // studio-range BT.709. Say both out loud: the driver's default is BT.601 + // limited, which a player then decodes as BT.709 at 1080p and above, and + // the recording comes back with visibly shifted colours. The matching tags + // go on the media types in initialize(). + D3D11_VIDEO_PROCESSOR_COLOR_SPACE inputColorSpace{}; + inputColorSpace.RGB_Range = 0; // full range, 0-255 + inputColorSpace.Nominal_Range = D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_0_255; + videoContext_->VideoProcessorSetStreamColorSpace(videoProcessor_.Get(), 0, &inputColorSpace); + + D3D11_VIDEO_PROCESSOR_COLOR_SPACE outputColorSpace{}; + outputColorSpace.YCbCr_Matrix = 1; // BT.709 + outputColorSpace.Nominal_Range = D3D11_VIDEO_PROCESSOR_NOMINAL_RANGE_16_235; + videoContext_->VideoProcessorSetOutputColorSpace(videoProcessor_.Get(), &outputColorSpace); + return true; +} + +void MFEncoder::applyHardwareRateControl(int bitrate) { + // The D3D manager switches the sink writer onto a hardware MFT, and those + // default to constant bitrate: a static desktop then spends the full + // configured budget doing nothing, 16.9 Mbps measured against the 1.95 the + // software encoder the CPU path lands on produced for the same screen. Same + // budget, opposite reading of it. Ask for VBR so the GPU path spends what + // the picture costs, which is what users have been getting all along. + // + // Best effort on purpose. An encoder that exposes neither knob still + // produces a valid recording, and a bitrate we could not pin down is not + // worth failing a capture over. + Microsoft::WRL::ComPtr codecApi; + if (FAILED(sinkWriter_->GetServiceForStream( + videoStreamIndex_, + GUID_NULL, + IID_PPV_ARGS(&codecApi)))) { + std::cerr << "WARNING: The hardware H.264 encoder exposes no ICodecAPI; " + << "its default bitrate applies." << std::endl; + return; + } + + VARIANT value{}; + value.vt = VT_UI4; + value.ulVal = eAVEncCommonRateControlMode_UnconstrainedVBR; + if (FAILED(codecApi->SetValue(&CODECAPI_AVEncCommonRateControlMode, &value))) { + std::cerr << "WARNING: Could not select VBR on the hardware H.264 encoder" << std::endl; + } + // Kept as the mean rather than lowered: the configured value is the budget + // for a busy screen, and under VBR a quiet one no longer has to spend it. + value.ulVal = static_cast(bitrate); + if (FAILED(codecApi->SetValue(&CODECAPI_AVEncCommonMeanBitRate, &value))) { + std::cerr << "WARNING: Could not set the hardware H.264 encoder bitrate" << std::endl; + } +} + +bool MFEncoder::initializeSampleAllocator(IMFMediaType* inputType) { + if (!succeeded( + MFCreateVideoSampleAllocatorEx( + __uuidof(IMFVideoSampleAllocatorEx), + reinterpret_cast(videoSampleAllocator_.GetAddressOf())), + "MFCreateVideoSampleAllocatorEx")) { + return false; + } + if (!succeeded( + videoSampleAllocator_->SetDirectXManager(dxgiDeviceManager_.Get()), + "IMFVideoSampleAllocator::SetDirectXManager")) { + return false; + } + Microsoft::WRL::ComPtr allocatorAttributes; + if (!succeeded(MFCreateAttributes(&allocatorAttributes, 2), "MFCreateAttributes(allocator)")) { + return false; + } + allocatorAttributes->SetUINT32(MF_SA_D3D11_USAGE, D3D11_USAGE_DEFAULT); + allocatorAttributes->SetUINT32( + MF_SA_D3D11_BINDFLAGS, + D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE); + return succeeded( + videoSampleAllocator_->InitializeSampleAllocatorEx(4, 30, allocatorAttributes.Get(), inputType), + "IMFVideoSampleAllocatorEx::InitializeSampleAllocatorEx"); +} + +MFEncoder::Nv12ConvertResult MFEncoder::convertBgraTextureToNv12( + ID3D11Texture2D* texture, + ID3D11Texture2D* outputTexture) { + // Short on purpose. This runs on the video-writer thread while it holds + // main.cpp's frame lock, and that lock is what issue #252 was about: any + // multi-second wait taken under it is a multi-second wait the shutdown + // watchdog counts against its 8s step budget. A few frame intervals is + // long enough for a busy GPU and short enough that a stuck bridge costs a + // dropped frame instead of the recording. + const DWORD acquireTimeoutMs = static_cast(std::max(50, 4000 / fps_)); + + if (!captureBridgeTexture_) { + D3D11_TEXTURE2D_DESC bridgeDesc{}; + texture->GetDesc(&bridgeDesc); + bridgeDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET; + bridgeDesc.CPUAccessFlags = 0; + bridgeDesc.Usage = D3D11_USAGE_DEFAULT; + bridgeDesc.MiscFlags = D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX; + if (!succeeded( + captureDevice_->CreateTexture2D(&bridgeDesc, nullptr, &captureBridgeTexture_), + "CreateTexture2D(capture bridge)")) { + return Nv12ConvertResult::Failed; + } + if (!succeeded(captureBridgeTexture_.As(&captureBridgeMutex_), "Query capture bridge mutex")) { + return Nv12ConvertResult::Failed; + } + + Microsoft::WRL::ComPtr bridgeResource; + if (!succeeded(captureBridgeTexture_.As(&bridgeResource), "Query capture bridge resource")) { + return Nv12ConvertResult::Failed; + } + HANDLE sharedHandle = nullptr; + if (!succeeded(bridgeResource->GetSharedHandle(&sharedHandle), "Get capture bridge handle")) { + return Nv12ConvertResult::Failed; + } + if (!succeeded( + device_->OpenSharedResource( + sharedHandle, + __uuidof(ID3D11Texture2D), + reinterpret_cast(encoderBridgeTexture_.GetAddressOf())), + "Open encoder bridge texture")) { + return Nv12ConvertResult::Failed; + } + if (!succeeded(encoderBridgeTexture_.As(&encoderBridgeMutex_), "Query encoder bridge mutex")) { + return Nv12ConvertResult::Failed; + } + + // The bridge is the only input this processor ever reads, so its view + // is built once here rather than per frame. Views describe a resource, + // they do not read it, so this needs no keyed-mutex ownership. + D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC inputViewDesc{}; + inputViewDesc.FourCC = 0; + inputViewDesc.ViewDimension = D3D11_VPIV_DIMENSION_TEXTURE2D; + inputViewDesc.Texture2D.MipSlice = 0; + inputViewDesc.Texture2D.ArraySlice = 0; + if (!succeeded( + videoDevice_->CreateVideoProcessorInputView( + encoderBridgeTexture_.Get(), + videoProcessorEnumerator_.Get(), + &inputViewDesc, + &bridgeInputView_), + "CreateVideoProcessorInputView")) { + return Nv12ConvertResult::Failed; + } + } + + // Key 0 is the capture side's, key 1 the encoder's. Timing out here leaves + // key 0 exactly where it was, so the next frame simply tries again; that + // is the whole reason this one is recoverable and the one below is not. + encodeStage_ = "bridge-acquire-capture"; + if (FAILED(captureBridgeMutex_->AcquireSync(0, acquireTimeoutMs))) { + encodeStage_ = "idle"; + return Nv12ConvertResult::Contended; + } + encodeStage_ = "bridge-copy"; + captureContext_->CopyResource(captureBridgeTexture_.Get(), texture); + encodeStage_ = "bridge-release-capture"; + if (!succeeded(captureBridgeMutex_->ReleaseSync(1), "Release capture bridge")) { + return Nv12ConvertResult::Failed; + } + encodeStage_ = "bridge-acquire-encoder"; + // Key 1 was just handed over by this same thread and nothing else in the + // process can hold it, so a failure here means the bridge is broken rather + // than busy, and no later frame could recover it. + if (!succeeded(encoderBridgeMutex_->AcquireSync(1, acquireTimeoutMs), "Acquire encoder bridge")) { + return Nv12ConvertResult::Failed; + } + const auto releaseEncoderBridge = [&]() { + return succeeded(encoderBridgeMutex_->ReleaseSync(0), "Release encoder bridge"); + }; + + // Recreated per frame because the allocator hands out a different texture + // from its pool each time. The input view and every processor setting are + // hoisted out; this one call is what is genuinely per-frame. + D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC outputViewDesc{}; + outputViewDesc.ViewDimension = D3D11_VPOV_DIMENSION_TEXTURE2D; + outputViewDesc.Texture2D.MipSlice = 0; + + Microsoft::WRL::ComPtr outputView; + encodeStage_ = "output-view"; + if (!succeeded( + videoDevice_->CreateVideoProcessorOutputView( + outputTexture, + videoProcessorEnumerator_.Get(), + &outputViewDesc, + &outputView), + "CreateVideoProcessorOutputView")) { + releaseEncoderBridge(); + return Nv12ConvertResult::Failed; + } + + D3D11_VIDEO_PROCESSOR_STREAM stream{}; + stream.Enable = TRUE; + stream.OutputIndex = 0; + stream.InputFrameOrField = 0; + stream.PastFrames = 0; + stream.FutureFrames = 0; + stream.pInputSurface = bridgeInputView_.Get(); + + encodeStage_ = "video-processor-blt"; + const bool converted = succeeded( + videoContext_->VideoProcessorBlt( + videoProcessor_.Get(), + outputView.Get(), + 0, + 1, + &stream), + "VideoProcessorBlt"); + encodeStage_ = "bridge-release-encoder"; + const bool released = releaseEncoderBridge(); + encodeStage_ = "idle"; + return converted && released ? Nv12ConvertResult::Ok : Nv12ConvertResult::Failed; +} + +bool MFEncoder::captureDxgiSample( ID3D11Texture2D* texture, int64_t timestampHns, - const BgraFrameView* webcamFrame, Microsoft::WRL::ComPtr& outSample) { outSample.Reset(); + if (!texture) { + return false; + } - const int64_t sampleDuration = 10'000'000LL / fps_; - int64_t sampleTime = 0; - { - std::scoped_lock writerLock(writerMutex_); - if (!sinkWriter_ || finalized_) { + D3D11_TEXTURE2D_DESC desc{}; + texture->GetDesc(&desc); + if (desc.Width != static_cast(width_) || + desc.Height != static_cast(height_) || + desc.Format != DXGI_FORMAT_B8G8R8A8_UNORM) { + std::cerr << "ERROR: Unexpected WGC DXGI texture format or dimensions" << std::endl; + return false; + } + + Microsoft::WRL::ComPtr sample; + encodeStage_ = "allocate-sample"; + if (!succeeded(videoSampleAllocator_->AllocateSample(&sample), "Allocate DXGI video sample")) { + encodeStage_ = "idle"; + return false; + } + + Microsoft::WRL::ComPtr buffer; + if (!succeeded(sample->GetBufferByIndex(0, &buffer), "Get DXGI video buffer")) { + return false; + } + + Microsoft::WRL::ComPtr dxgiBuffer; + if (!succeeded(buffer.As(&dxgiBuffer), "Query IMFDXGIBuffer")) { + return false; + } + Microsoft::WRL::ComPtr nv12Texture; + if (!succeeded( + dxgiBuffer->GetResource( + __uuidof(ID3D11Texture2D), + reinterpret_cast(nv12Texture.GetAddressOf())), + "IMFDXGIBuffer::GetResource")) { + return false; + } + switch (convertBgraTextureToNv12(texture, nv12Texture.Get())) { + case Nv12ConvertResult::Ok: + break; + case Nv12ConvertResult::Contended: + // No sample, no failure. Leaving outSample empty tells the caller + // to skip this pass. + return true; + case Nv12ConvertResult::Failed: return false; - } + } - if (firstTimestampHns_ < 0) { - firstTimestampHns_ = timestampHns; - } + // Stamped only once the frame exists. Doing this first, as the CPU path + // does, would let every dropped frame still advance lastTimestampHns_ and + // stretch the timeline by the frames that were never written. + const int64_t sampleDuration = 10'000'000LL / fps_; + const int64_t sampleTime = nextSampleTime(timestampHns, sampleDuration); - sampleTime = timestampHns - firstTimestampHns_; - if (sampleTime <= lastTimestampHns_) { - sampleTime = lastTimestampHns_ + sampleDuration; - } - lastTimestampHns_ = sampleTime; + DWORD maximumLength = 0; + if (!succeeded(buffer->GetMaxLength(&maximumLength), "IMFMediaBuffer::GetMaxLength(DXGI)")) { + return false; + } + if (!succeeded( + buffer->SetCurrentLength(maximumLength), + "IMFMediaBuffer::SetCurrentLength(DXGI)")) { + return false; } + sample->SetSampleTime(sampleTime); + sample->SetSampleDuration(sampleDuration); + outSample = sample; + return true; +} + +bool MFEncoder::captureVideoSample( + ID3D11Texture2D* texture, + int64_t timestampHns, + const BgraFrameView* webcamFrame, + Microsoft::WRL::ComPtr& outSample) { + outSample.Reset(); + + const int64_t sampleDuration = 10'000'000LL / fps_; + const int64_t sampleTime = nextSampleTime(timestampHns, sampleDuration); + // The GPU readback below (copyFrameToBuffer -> CopyResource/Map on // `texture`) is not internally synchronized here. Callers must hold their // own lock around this call that also serializes against whatever thread @@ -674,23 +1224,7 @@ bool MFEncoder::captureBgraSample( outSample.Reset(); const int64_t sampleDuration = 10'000'000LL / fps_; - int64_t sampleTime = 0; - { - std::scoped_lock writerLock(writerMutex_); - if (!sinkWriter_ || finalized_) { - return false; - } - - if (firstTimestampHns_ < 0) { - firstTimestampHns_ = timestampHns; - } - - sampleTime = timestampHns - firstTimestampHns_; - if (sampleTime <= lastTimestampHns_) { - sampleTime = lastTimestampHns_ + sampleDuration; - } - lastTimestampHns_ = sampleTime; - } + const int64_t sampleTime = nextSampleTime(timestampHns, sampleDuration); Microsoft::WRL::ComPtr buffer; const DWORD frameBytes = static_cast(width_ * height_ * 4); @@ -734,11 +1268,15 @@ bool MFEncoder::submitVideoSample(IMFSample* sample) { // encode synchronously on the calling thread. Callers must NOT hold any // lock shared with a thread that needs to make timely progress (e.g. a // stop-request check) across this call. + encodeStage_ = "write-sample"; std::scoped_lock writerLock(writerMutex_); if (!sinkWriter_ || finalized_) { + encodeStage_ = "idle"; return false; } - return succeeded(sinkWriter_->WriteSample(videoStreamIndex_, sample), "WriteSample"); + const bool written = succeeded(sinkWriter_->WriteSample(videoStreamIndex_, sample), "WriteSample"); + encodeStage_ = "idle"; + return written; } bool MFEncoder::writeAudio(const BYTE* data, DWORD byteCount, int64_t timestampHns, int64_t durationHns) { diff --git a/electron/native/wgc-capture/src/mf_encoder.h b/electron/native/wgc-capture/src/mf_encoder.h index e5fbd74c8..fd2797006 100644 --- a/electron/native/wgc-capture/src/mf_encoder.h +++ b/electron/native/wgc-capture/src/mf_encoder.h @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -29,6 +30,12 @@ struct AudioInputFormat { struct MFEncoderOptions { bool preferSoftwareEncoder = false; bool injectDefaultSinkWriterFailureOnce = false; + // A request, never a requirement. Every step of the GPU path degrades to + // the CPU readback rather than failing the recording, so a machine without + // a hardware H.264 encoder, without NV12 video-processor output, or with a + // driver that refuses shared keyed-mutex textures records exactly as it did + // before the path existed. Ask usesDxgiInput() for what actually happened. + bool useDxgiInput = false; }; constexpr const char* kVideoEncoderSelectionDefault = "default"; @@ -67,6 +74,10 @@ class MFEncoder { int64_t timestampHns, const BgraFrameView* webcamFrame, Microsoft::WRL::ComPtr& outSample); + bool captureDxgiSample( + ID3D11Texture2D* texture, + int64_t timestampHns, + Microsoft::WRL::ComPtr& outSample); bool captureBgraSample( const BgraFrameView& frame, int64_t timestampHns, @@ -75,8 +86,35 @@ class MFEncoder { bool writeAudio(const BYTE* data, DWORD byteCount, int64_t timestampHns, int64_t durationHns); bool finalize(); const char* videoEncoderSelection() const; + // Which video input path initialize() actually settled on, which is not + // necessarily the one that was asked for. Callers must read this rather + // than their own MFEncoderOptions to decide which capture entry point to + // call, or a machine that fell back would be fed DXGI samples the sink + // writer was never configured for. + bool usesDxgiInput() const; + // A breadcrumb, not state: safe to read from another thread at any time. + const char* encodeStage() const; private: + // Contended is not Failed: the bridge is a two-key handshake and a missed + // acquire costs one frame, which is a better outcome than ending a + // recording that is otherwise healthy. + enum class Nv12ConvertResult { + Ok, + Contended, + Failed, + }; + + bool initializeDxgiPipeline(); + void releaseDxgiPipeline(); + bool initializeDxgiEncodingDevice(); + bool initializeVideoProcessor(); + bool initializeSampleAllocator(IMFMediaType* inputType); + void applyHardwareRateControl(int bitrate); + int64_t nextSampleTime(int64_t timestampHns, int64_t sampleDuration); + Nv12ConvertResult convertBgraTextureToNv12( + ID3D11Texture2D* texture, + ID3D11Texture2D* outputTexture); bool ensureStagingTexture(ID3D11Texture2D* texture); bool copyFrameToBuffer( ID3D11Texture2D* texture, @@ -89,8 +127,34 @@ class MFEncoder { Microsoft::WRL::ComPtr sinkWriter_; Microsoft::WRL::ComPtr device_; Microsoft::WRL::ComPtr context_; + Microsoft::WRL::ComPtr captureDevice_; + Microsoft::WRL::ComPtr captureContext_; + Microsoft::WRL::ComPtr captureBridgeTexture_; + Microsoft::WRL::ComPtr captureBridgeMutex_; + Microsoft::WRL::ComPtr encoderBridgeTexture_; + Microsoft::WRL::ComPtr encoderBridgeMutex_; + Microsoft::WRL::ComPtr bridgeInputView_; Microsoft::WRL::ComPtr stagingTexture_; + Microsoft::WRL::ComPtr dxgiDeviceManager_; + Microsoft::WRL::ComPtr videoSampleAllocator_; + Microsoft::WRL::ComPtr videoDevice_; + Microsoft::WRL::ComPtr videoContext_; + Microsoft::WRL::ComPtr videoProcessorEnumerator_; + Microsoft::WRL::ComPtr videoProcessor_; + UINT dxgiResetToken_ = 0; + // Guards the sink writer, and is held across IMFSinkWriter::WriteSample -- + // a synchronous encode. Only threads that can afford to wait out an encode + // may take it, which rules out anything holding a caller's frame lock. std::mutex writerMutex_; + // Guards the sample clock alone, so the capture* entry points (which do run + // under a caller's frame lock) never queue behind an encode. Splitting this + // out is what stops an audio WriteSample from wedging the video writer, and + // through it the WGC callbacks, at stop (issue #252 follow-up). + std::mutex timestampMutex_; + // Where the encoder is right now, for the shutdown watchdog to name when a + // step overruns. `video-writer-join phase=abandoned` says which thread is + // stuck; this says which call it is stuck in. + std::atomic encodeStage_{"idle"}; DWORD videoStreamIndex_ = 0; DWORD audioStreamIndex_ = 0; bool hasAudioStream_ = false; @@ -100,5 +164,6 @@ class MFEncoder { int64_t firstTimestampHns_ = -1; int64_t lastTimestampHns_ = -1; bool finalized_ = false; + bool useDxgiInput_ = false; const char* videoEncoderSelection_ = kVideoEncoderSelectionDefault; }; diff --git a/electron/native/wgc-capture/src/wgc_session.cpp b/electron/native/wgc-capture/src/wgc_session.cpp index ccab06727..76649a990 100644 --- a/electron/native/wgc-capture/src/wgc_session.cpp +++ b/electron/native/wgc-capture/src/wgc_session.cpp @@ -1,6 +1,7 @@ #include "wgc_session.h" #include +#include #include #include #include @@ -63,7 +64,7 @@ WgcSession::~WgcSession() { } bool WgcSession::createD3DDevice() { - UINT flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT; + UINT flags = D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_VIDEO_SUPPORT; #if defined(_DEBUG) flags |= D3D11_CREATE_DEVICE_DEBUG; #endif @@ -109,6 +110,12 @@ bool WgcSession::createD3DDevice() { return false; } + Microsoft::WRL::ComPtr multithread; + if (!succeeded(d3dContext_.As(&multithread), "Query ID3D10Multithread")) { + return false; + } + multithread->SetMultithreadProtected(TRUE); + Microsoft::WRL::ComPtr dxgiDevice; if (!succeeded(d3dDevice_.As(&dxgiDevice), "Query IDXGIDevice")) { return false; diff --git a/technical-documentation/architecture/recording.md b/technical-documentation/architecture/recording.md index 01337f210..a56bf451f 100644 --- a/technical-documentation/architecture/recording.md +++ b/technical-documentation/architecture/recording.md @@ -68,7 +68,8 @@ Cursor samples are persisted as cursor telemetry rather than baked into editable ## Known gaps - A window with odd client dimensions can produce black video: H.264 encoding requires even dimensions (`electron/native/wgc-capture/src/wgc_session.cpp:38`). -- The Windows helper's frame lock (`electron/native/wgc-capture/src/main.cpp`) is still held across blocking, uninterruptible D3D11 work: the WGC callback's `CopyResource`, and the video writer's `Map(D3D11_MAP_READ)` readback in `mf_encoder.cpp`. A driver that stalls inside either one still costs the recording. What no longer happens is a hang: stop detection runs on `CaptureControl::stopMutex`, which no frame thread ever touches, and a shutdown watchdog force-exits the helper when a step overruns its budget, naming the step it died in. Each step gets `OPENSCREEN_WGC_STEP_BUDGET_MS` (8s by default) and that is the bound which normally fires; the whole shutdown is capped by `OPENSCREEN_WGC_STOP_BUDGET_MS` (50s by default), which the encoder-finalize step alone is allowed to spend in full because a long software-encoder finalize legitimately takes seconds (issue #34). Getting the readback out of the lock, and picking the D3D adapter that actually drives the captured monitor instead of adapter 0, are the outstanding fixes (issue #252). +- The Windows helper's frame lock (`electron/native/wgc-capture/src/main.cpp`) is still held across blocking, uninterruptible D3D11 work: the WGC callback's `CopyResource`, and whatever the video writer does with the frame. A driver that stalls inside either one still costs the recording. What no longer happens is a hang: stop detection runs on `CaptureControl::stopMutex`, which no frame thread ever touches, and a shutdown watchdog force-exits the helper when a step overruns its budget, naming the step it died in. Each step gets `OPENSCREEN_WGC_STEP_BUDGET_MS` (8s by default) and that is the bound which normally fires; the whole shutdown is capped by `OPENSCREEN_WGC_STOP_BUDGET_MS` (50s by default), which the encoder-finalize step alone is allowed to spend in full because a long software-encoder finalize legitimately takes seconds (issue #34). Picking the D3D adapter that actually drives the captured monitor instead of adapter 0 is still outstanding. +- The video writer has two ways to get a frame to the encoder, and which one runs is a per-machine outcome, not a setting. The GPU path (`videoInput: "dxgi-nv12"`) copies the frame across a keyed-mutex bridge to a second D3D11 device, converts BGRA to NV12 with the D3D11 video processor, and hands the hardware H.264 encoder a DXGI sample; it never touches system memory. The CPU path (`videoInput: "cpu-rgb32"`) is the original staging-texture `Map(D3D11_MAP_READ)` readback, and is what a `Map`/`Unmap` that never returns wedges (issue #252: Windows 10, WDDM 2.7, multi-adapter). The GPU path is the default and degrades to the CPU one on its own at every step — no hardware encoder, no NV12 video-processor output, no shared keyed-mutex texture, no DXGI sample allocator — so a machine it does not fit records exactly as it did before it existed. It is skipped outright for `preferSoftwareEncoder` and for inline webcam PiP, both of which need the frame in system memory, and `OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1` forces it off. The two paths land on different encoders, so the GPU one asks for VBR explicitly through `ICodecAPI`: hardware MFTs default to constant bitrate and would spend the full configured budget on a static screen (measured 16.9 Mbps against 1.95 for the same desktop). - Linux/Wayland can produce no usable frames on the `getDisplayMedia` fallback because Chromium initializes Vulkan against the Ozone Wayland backend. The PipeWire helper path is unaffected. - On Linux the compositor's source picker appears on every recording. That is deliberate — see "Why Linux sends no source identity" — but it is an interruption, and there is currently no way to reuse a previous choice without also making it impossible to change. - Holding a portal session across the countdown means the compositor's "screen is being shared" indicator is up before recording begins. That is honest — access really has been granted — but the user can click it to revoke, or close the window they picked. The helper's exit surfaces as a rejected `waitUntilSourceSelected`; the session is not yet subscribed to the portal's `Session::Closed` signal, so a revocation is reported as a failed start rather than a specific message.