Skip to content

Support DiffSinger pitch local retaking#2183

Open
KakaruHayate wants to merge 4 commits into
openutau:masterfrom
KakaruHayate:ds-pitch-retake
Open

Support DiffSinger pitch local retaking#2183
KakaruHayate wants to merge 4 commits into
openutau:masterfrom
KakaruHayate:ds-pitch-retake

Conversation

@KakaruHayate

Copy link
Copy Markdown
Contributor

No description provided.

yqzhishen and others added 2 commits June 5, 2026 21:02
Pull the per-frame retake mask construction (DsPitch.Process) and the
absolute-tick -> phrase-local note-index mapping (LoadRenderedPitch
overload) out into pure static helpers in DiffSingerRetake so the
padding-shift indexing logic and the position lookup can be unit tested
without instantiating RenderPhrase / a singer / ONNX.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a user-facing preference and supporting plumbing to enable “local retake” when loading DiffSinger-rendered pitch, so pitch generation/editing can be scoped to selected notes rather than always reprocessing the whole phrase.

Changes:

  • Adds a new preference (DiffSingerLocalRetaking) and exposes it in the Preferences UI + string resources.
  • Extends pitch loading to accept selected note positions and propagates a per-frame retake mask through RenderPitchResult.
  • Updates the “Load rendered pitch” batch edit to only write curve points for retaken frames; adds unit tests for retake helpers.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
OpenUtau/Views/PreferencesDialog.axaml Adds a toggle for DiffSinger pitch local retaking.
OpenUtau/ViewModels/PreferencesViewModel.cs Wires the new toggle to persisted preferences.
OpenUtau/Strings/Strings.axaml Adds EN string for the new toggle label.
OpenUtau/Strings/Strings.zh-CN.axaml Adds zh-CN string for the new toggle label.
OpenUtau.Core/Util/Preferences.cs Persists the new boolean preference with defaults.
OpenUtau.Core/Render/IRenderer.cs Adds retake mask to pitch result and a pitch-load overload that accepts selected note positions.
OpenUtau.Core/Editing/NoteBatchEdits.cs Uses the retake mask to skip writing curve points for non-retaken frames.
OpenUtau.Core/DiffSinger/DiffSingerRetake.cs Adds helper utilities for mapping selection → note indexes and building a frame mask.
OpenUtau.Core/DiffSinger/DiffSingerRenderer.cs Implements selection-aware pitch loading for DiffSinger when enabled by preference.
OpenUtau.Core/DiffSinger/DiffSingerPitch.cs Adds support for partial retake in pitch predictor (retake mask + optional existing pitch).
OpenUtau.Test/Core/DiffSinger/DiffSingerRetakeTest.cs Adds unit tests for the new retake helper logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread OpenUtau.Core/DiffSinger/DiffSingerPitch.cs
Comment thread OpenUtau.Core/DiffSinger/DiffSingerPitch.cs Outdated
Comment thread OpenUtau.Core/DiffSinger/DiffSingerRetake.cs
Comment thread OpenUtau.Core/Render/IRenderer.cs Outdated
Comment thread OpenUtau.Core/Editing/NoteBatchEdits.cs Outdated
Comment thread OpenUtau.Test/Core/DiffSinger/DiffSingerRetakeTest.cs Outdated
Comment thread OpenUtau.Core/DiffSinger/DiffSingerRetake.cs
- Fix retake frame mask incorrectly indexing past gap-rest segments
  inserted in noteDurMsList; callers now supply a paddedToRealNoteIndex
  array kept in lockstep with the padded duration list. Gap rests
  follow the preceding real note for continuity.
- Mark nullable parameters on Process / BuildRetakeFrameMask /
  MapSelectedPositionsToNoteIndexes and the retakeMask field.
- Guard retakeMask bounds in NoteBatchEdits to tolerate shorter masks.
- Move retake tests into OpenUtau.Core.DiffSinger namespace and add
  coverage for gap-rest mapping and the -1 sentinel.
yqzhishen
yqzhishen previously approved these changes Jun 6, 2026
The existing Ctrl+R keyboard shortcut for LoadRenderedPitch has
no visible menu hint, making it undiscoverable. Set InputGesture on
the MenuItemViewModel so the menu displays 'Ctrl+R' next to the item.
@KakaruHayate

Copy link
Copy Markdown
Contributor Author

English description:

Support DiffSinger pitch local retaking

When users edit pitch of specific notes in DiffSinger, the pitch predictor currently regenerates pitch for the entire phrase. This PR introduces "pitch local retaking" — only the edited notes are re-predicted; unchanged notes keep their existing pitch frames, significantly reducing edit-to-audible feedback latency.

How it works

  • DsPitch.Process() now accepts optional retakeNoteIndexes (which real-pitch notes changed) and existingPitch (the last rendered pitch buffer).
  • When provided, it builds a "retake frame mask" via DiffSingerRetake.BuildRetakeFrameMask(), mapping padded segments back to real note indexes. Only frames belonging to the given note indexes are regenerated; the rest are copied from existingPitch.
  • LoadRenderedPitch (NoteBatchEdit) now passes the currently selected note indexes as retake candidates.
  • A retakeMask is included in the returned RenderPitchResult, so downstream consumers (e.g., note batch edits that process rendered pitch) can filter out unchanged frames.

Configuration

  • A new preference DiffSingerPitchRetake (default: true) controls whether the feature is active. Disabling it restores the original full-phrase re-prediction behavior.
  • The feature requires DiffSinger tensor cache to be enabled for reasonable performance (re-fetching previously computed encoder outputs from cache instead of re-running the encoder for unchanged frames).

Files changed (11 files, +318 / -6)

  • OpenUtau.Core/DiffSinger/DiffSingerRetake.cs (new) — retake frame mask builder
  • OpenUtau.Core/DiffSinger/DiffSingerPitch.cs — retake parameter plumbing + mask integration
  • OpenUtau.Core/DiffSinger/DiffSingerRenderer.cs — expose FrameMs for mask building
  • OpenUtau.Core/Editing/NoteBatchEdits.cs — pass selected note indexes to LoadRenderedPitch
  • OpenUtau.Core/Render/IRenderer.cs — add retakeMask to RenderPitchResult
  • OpenUtau.Core/Util/Preferences.cs — add DiffSingerPitchRetake setting
  • OpenUtau.Test/Core/DiffSinger/DiffSingerRetakeTest.cs (new) — unit tests
  • OpenUtau/ViewModels/PreferencesViewModel.cs — wire up the toggle
  • OpenUtau/Views/PreferencesDialog.axaml — add toggle UI
  • OpenUtau/Strings/Strings.axaml, Strings.zh-CN.axaml — localized strings

Additional change

  • The LoadRenderedPitch item in the Notes menu now displays its Ctrl+R keyboard shortcut hint (previously hidden), making the shortcut discoverable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants