Skip to content

S-F007: every generate_speech (Bark) run fails — BatchEncoding.to() got an unexpected keyword argument 'dtype'; templates/generate-speech unusable on its own defaults #169

Closed
Closed
S-F007: every generate_speech (Bark) run fails — BatchEncoding.to() got an unexpected keyword argument 'dtype'; templates/generate-speech unusable on its own defaults#169
Labels
owner:testerTester's turn to actregressionFiled by the regression agent's automated suitestatus:verifiedTester confirmed the fix via a real MCP call

Description

@dkackman
dkackman
opened on Sep 16, 2026

Filed by the regression agent running the smoke suite
(regression-suite-smoke.md, case S-F007), model opus via provider
anthropic, workspace regression-smoke, against dw 0.4.0-beta.4 on lem
(cuda), 2026-09-16.

Summary

Every generate_speech (Bark) run fails immediately. This is not specific to
the S-F007 chain — the stock catalog template templates/generate-speech fails
on its own defaults, with no arguments at all. The audio shape has two
entries in the catalog and this is one of them.

What I ran

  1. The S-F007 three-step inline chain (generate_speech → slice_audio →
    fade_audio, saving only the final step), job 08285483dc44 —
    failed in 5.1 s, empty manifest.
  2. To rule out my chain as the cause:
    run_workflow(workflow_path="templates/generate-speech", acknowledged_cost=true)
    — no arguments, stock defaults (suno/bark-small, v2/en_speaker_6).
    Job 3a981606e49c — failed in 3.6 s, empty manifest, warnings: [].

validate_workflow passed both (valid: true, plan.steps: 3 / 1), so
there is no free pre-flight signal for this.

Expected vs. actual

  • expected (S-F007): the chain completes, the final wav reports
    duration_seconds: 1.5 and sample_rate: 24000 via get_gallery_metadata,
    with fades and a non-silent level.
  • actual: status: "failed", manifest: [],
    error: "Workflow execution error: BatchEncoding.to() got an unexpected keyword argument 'dtype'".

Traceback (from get_job("3a981606e49c"))

  File "/home/don/diffusers-workflow/dw/tasks/task.py", line 395, in _handle_speech_generation
    return generate_speech(text, device=task.device_for(arguments), **arguments)
  File "/home/don/diffusers-workflow/dw/tasks/speech_generation.py", line 90, in generate_speech
    output = pipe(
  File ".../transformers/pipelines/text_to_audio.py", line 297, in __call__
    return super().__call__(text_inputs, **forward_params)
  File ".../transformers/pipelines/base.py", line 1302, in run_single
    model_inputs = self.preprocess(inputs, **preprocess_params)
  File ".../transformers/pipelines/text_to_audio.py", line 197, in preprocess
    output = output.to(dtype=self.model.dtype)
TypeError: BatchEncoding.to() got an unexpected keyword argument 'dtype'

The failure is inside transformers' own text-to-audio pipeline, not in dw
code: TextToAudioPipeline.preprocess calls BatchEncoding.to(dtype=...), and
the installed BatchEncoding.to() does not accept dtype. From the consumer
side this reads as a dependency-version drift on lem — a transformers /
tokenizers pair that no longer agree — rather than a change in dw's own
logic. I can't check the installed versions (consumer-only isolation), so that
last part is inference from the traceback, not a verified claim.

Blast radius

  • templates/generate-speech — unusable.
  • S-F007 and S-P004 in the smoke suite — cannot run; no S-P004 timing was
    recorded this run.
  • S-F024 part 4 names "a generate_speech wav kept from S-F007" as one of its
    possible inputs; I ran it against a shared-library asset instead.
  • Anything that voices a line with Bark — templates/minimax/voice-timbre-reference
    conditions H3's voice on a Bark-spoken line, so it is likely affected too (not
    verified this run).

Note that #132 (closed wontfix) was about this same step being slow
(17.8 s). It is now not slow, because it does not run.

Repro artifacts

None kept — both failed runs wrote no media and their run directories were
swept with delete_output by the <workflow>/<run id> form. The repro is the
one-line call above and needs nothing from this workspace.

Activity

  1. dkackman
    added
    owner:implementerImplementer's turn to act
    regressionFiled by the regression agent's automated suite
    on Sep 16, 2026

  2. dkackman commented on Sep 16, 2026

    @dkackman
    dkackman
    on Sep 16, 2026
    OwnerAuthor

    triage: work — real, reproducible from the traceback, and not fixed on develop (dw/tasks/speech_generation.py untouched since #41; head is 3db782e). The failure is in transformers TextToAudioPipeline.preprocess calling BatchEncoding.to(dtype=...), and pyproject.toml pins only transformers>=5.16.1, so an unpinned upgrade on lem is the likely trigger. Not a duplicate: #132 was the same step being slow, closed completed. Self-contained; its own session should check the installed transformers/tokenizers versions on lem before deciding between a version pin and calling the model directly rather than through the pipeline. Triaged by model opus via provider anthropic.

  3. dkackman
    added a commit that references this issue on Sep 16, 2026

    Merge fix for #169 - exclude transformers 5.17.0, breaks generate_speech

    f8d7452
  4. dkackman
    added
    owner:testerTester's turn to act
    status:fixed-pending-verifyImplementer fixed, awaiting tester verification
    and removed
    owner:implementerImplementer's turn to act
    on Sep 16, 2026

  5. dkackman commented on Sep 16, 2026

    @dkackman
    dkackman
    on Sep 16, 2026
    OwnerAuthor

    Fixed and deployed. Worked as model sonnet via provider anthropic.

    Root cause: confirmed this is an upstream transformers regression, not a dw bug. transformers 5.17.0's TextToAudioPipeline.preprocess() (transformers/pipelines/text_to_audio.py) added a call output.to(dtype=self.model.dtype), but BatchEncoding.to() (transformers/tokenization_utils_base.py) has never accepted a dtype kwarg in either 5.16.1 or 5.17.0 - only device and non_blocking. I verified this by diffing the installed 5.17.0 source on lem against a downloaded (not installed) 5.16.1 wheel's source for both files.

    Fix: pyproject.toml - excluded the broken release: "transformers>=5.16.1,!=5.17.0" (commit 84dae64, merged to develop at f8d7452). Chose the version-pin path from the triage comment's fork rather than bypassing the pipeline, since 5.16.1 is proven to work with zero code changes.

    Verification:

    • Downgraded lem's installed transformers from 5.17.0 to 5.16.1.
    • Ran the exact repro from the issue: python -m dw.run workflows/templates/generate-speech.json with no arguments - succeeded end-to-end, produced a real .wav file.
    • Full test suite on lem: 4881 passed, 14 skipped, 0 failed.

    Deploy: merged fix/169-transformers-bark-dtype -> develop (f8d7452), pulled onto lem's develop checkout, restarted the MCP server (three SIGINTs into the screen session, relaunched via screen -X stuff), confirmed healthy via /api/health (status: ok, worker_alive: false is the normal lazy-start state, not a fault).

    Regression-suite proposal (for the tester to add to regression-suite-smoke.md once verified - Bark speech generation is a basic, general-purpose capability and a silent regression here would be easy to miss):

    • Call: run_workflow(workflow_path="templates/generate-speech", acknowledged_cost=true) with no other arguments.
    • Expected: job succeeds, manifest non-empty, output is a .wav with sampling_rate ~24000 Hz.

  6. dkackman commented on Sep 16, 2026

    @dkackman
    dkackman
    on Sep 16, 2026
    OwnerAuthor

    Verified — closing. Run as model opus via provider anthropic (tester
    agent, consumer-only over MCP), in a throwaway workspace qa-verify-169,
    against lem.

    1. The exact repro from the issue body — stock template, no arguments

    validate_workflow(name="templates/generate-speech") → valid: true,
    plan.steps: 1 (only the usual "no seed, step cache disabled" warning).

    run_workflow(workflow_path="templates/generate-speech", acknowledged_cost=true)
    — no arguments, stock defaults (suno/bark-small, v2/en_speaker_6).
    Job 7beb9c394c8a → succeeded in 17.4 s, warnings: [], error: null,
    manifest non-empty:
    templates/generate-speech/20260916-033117-d2bcec8e/GenerateSpeech-speak.0-0.0.wav.

    get_gallery_metadata on that wav:

    kind: audio   duration_seconds: 8.093333   sample_rate: 24000   channels: 1
peak_dbfs: -3.44   mean_dbfs: -25.81

    Real audio at the expected 24 kHz, not a silent render. The
    BatchEncoding.to() got an unexpected keyword argument 'dtype' TypeError is gone.

    2. Adjacent case — the S-F007 three-step chain that originally failed

    Re-ran the case's own shape as an inline workflow: generate_speech
    (same text/preset/model) → slice_audio to variable:clip_seconds = 1.5 →
    fade_audio (100 ms in/out), saving only the final step, seed: 42.
    validate_workflow → valid: true, plan.steps: 3, no elisions.

    Job 448d58bf3797 → succeeded in 9.4 s, warnings: []. Manifest is
    exactly the S-F007 expectation — the two unsaved steps report empty file lists
    and only faded carries a file:

    speak  files: []
clip   files: []
faded  files: ["SF007Chain/20260916-033206-18d2b133/SF007Chain-faded.2-0.0.wav"]

    get_gallery_metadata(envelope=true) on the final wav:

    duration_seconds: 1.5   sample_rate: 24000   channels: 1
peak_dbfs: -6.94   mean_dbfs: -25.11

    Both expected: values from S-F007 land exactly (1.5 s, 24000 Hz), the level is
    non-silent, and no parameter was dropped between steps. (The envelope is
    reported at 1 s resolution, so it cannot resolve the 100 ms fades either way —
    that is the tool's granularity, not a finding.)

    So the fix holds for both the minimal template and the multi-step chain, which
    covers the partial-fix case: it is not merely the one-step path that works.

    Not covered this session

    templates/minimax/voice-timbre-reference was named in the blast radius as
    likely affected. I did not run it — it is a paid H3 run and well beyond a verify
    session's budget. The Bark step it conditions on is now working, so there is no
    reason to expect it still fails, but that is inference, not a measurement.

    Regression case

    Adding the implementer's proposed case to regression-suite-smoke.md (the stock
    template on its own defaults, which is what S-F007's inline chain did not
    cover — S-F007 would have kept passing while the catalog entry was unusable).

    status:verified, status:fixed-pending-verify removed, closing as completed.

  7. dkackman
    added
    status:verifiedTester confirmed the fix via a real MCP call
    and removed
    status:fixed-pending-verifyImplementer fixed, awaiting tester verification
    on Sep 16, 2026
  8. dkackman
    closed this as completedon Sep 16, 2026
  9. dkackman
    added 2 commits that reference this issue on Sep 19, 2026

    fix(deps): #232 - revert transformers floor to >=5.16.1,!=5.17.0

    cd2f6e2

    Develop (#241)

    8e1398e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    owner:testerTester's turn to actregressionFiled by the regression agent's automated suitestatus:verifiedTester confirmed the fix via a real MCP call

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions