Skip to content

Add music generation agent tool - #5

Open
octo-patch wants to merge 1 commit into
lingyuanli:masterfrom
octo-patch:octo/20260817-music-generation-tool-recvsf1NJoSEWB
Open

Add music generation agent tool#5
octo-patch wants to merge 1 commit into
lingyuanli:masterfrom
octo-patch:octo/20260817-music-generation-tool-recvsf1NJoSEWB

Conversation

@octo-patch

Copy link
Copy Markdown
Contributor

Reason: The multimodal tool registry covers image, video, TTS, voice and audio mixing but has no music generation tool.

Changes

  • multimodal_core.py: added a generate_music capability alongside the existing voice methods, reusing the current region resolution so both the global and CN /v1/music_generation endpoints are supported, together with the shared Bearer/JSON header helper.
  • Request body is built by _minimax_music_payload, which sends model (the only mandatory field) plus prompt, lyrics, output_format, audio_setting.format, is_instrumental, lyrics_optimizer and an explicit non-streaming stream flag. Optional fields are omitted when the caller does not supply them, and aigc_watermark is only attached on the CN region.
  • Output-format handling supports both documented forms: hex payloads are decoded to bytes, url payloads are downloaded before the temporary link expires, and both are stored under /storage/audios like the other audio tools. Audio format accepts mp3, wav and pcm; unsupported values are rejected before any request is sent.
  • Response parsing is isolated in _extract_minimax_music_audio: it checks base_resp.status_code, only reads data.audio for the completed status, and reports a clear message for the in-progress status and for empty audio.
  • New minimax_music.py tool class exposing minimax_music_generation, registered in the planner/ReAct flow tool list, in the multimodal branch of agent_task_runner, and in the UI compact tool-content set so its results render like the sibling audio tools.
  • New settings in .env.example: MINIMAX_MUSIC_MODEL (default music-3.0), MINIMAX_MUSIC_OUTPUT_FORMAT, MINIMAX_MUSIC_AUDIO_FORMAT and MINIMAX_MUSIC_AIGC_WATERMARK. No model name or host is hard-coded at the call site; the existing MINIMAX_REGION / MINIMAX_BASE_URL settings continue to select the host.

Checks

  • python3 -m pytest tests/test_minimax_music_tool.py tests/test_minimax_voice_tool.py --noconftest -q — 14 passed (10 new tests plus the 4 existing voice tests). --noconftest was needed because tests/conftest.py imports the FastAPI app, whose database dependencies are not installed in this environment.
  • python3 -m py_compile on every changed Python module.
  • Drove the tool end-to-end against a stubbed HTTP layer to confirm the request reaches /v1/music_generation with the Bearer header, that a completed hex response is decoded to a stored file, that a url response is downloaded and stored, and that an in-progress response fails with a readable message.

New tests cover the tool schema registration, endpoint resolution for both regions, payload defaults and optional fields, the CN-only watermark field, completed / in-progress / error / empty-audio response parsing, the missing API key guard, the prompt-or-lyrics requirement, and format validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant