docs(vllm-omni): refresh for v1.5 (0.26.0) — changelog, bidi streaming, s3tokenizer - #6510
Open
Yadan-Wei wants to merge 1 commit into
Open
docs(vllm-omni): refresh for v1.5 (0.26.0) — changelog, bidi streaming, s3tokenizer#6510Yadan-Wei wants to merge 1 commit into
Yadan-Wei wants to merge 1 commit into
Conversation
…g, s3tokenizer - Add v1.5.0 changelog entry (vLLM-Omni 0.26.0 / vLLM v0.26.0): Rust frontend, SageMaker bidirectional WebSocket streaming, s3tokenizer/CosyVoice3, FlashInfer 0.6.14. - Document SageMaker Bidirectional Streaming (WebSocket) in deployment/sagemaker.md: capability LABEL, the 6 WS routes, slashless model_invocation_path, inference_ami_version requirement. - Add a runnable, end-to-end-verified bidi example (examples/vllm-omni/bidi/deploy_bidi_stream.py) — deploy → stream TTS over the bidi WebSocket → teardown. Verified against the released prod image (219 KB audio streamed back). - Add s3tokenizer to "What's Included". Auto-generated by skill: vllm-omni-doc-update (coverage-gap example verified end-to-end via verify_examples).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refreshes the vLLM-Omni AL2023 DLC docs for the v1.5 / vLLM-Omni 0.26.0 release and documents the new SageMaker bidirectional WebSocket streaming feature. Generated by the
vllm-omni-doc-updateaudit skill (drift + fact-check + coverage-gap), baseline v1.4.0 / omni v0.21.0rc1.Changes
changelog/index.md— new v1.5.0 entry (vLLM-Omni 0.26.0 / vLLM v0.26.0): Rust frontend (vllm-rs), SageMaker bidirectional WebSocket streaming,s3tokenizer/CosyVoice3 fix, FlashInfer 0.6.14.deployment/sagemaker.md— new Bidirectional Streaming (WebSocket) section: the capability LABEL, the 6 supported WS routes (matched againstWS_ALLOWED_PATHS), slashlessmodel_invocation_path, and theinference_ami_versionrequirement.index.md— addeds3tokenizerto "What's Included".examples/vllm-omni/bidi/deploy_bidi_stream.py— new runnable end-to-end example (deploy → stream TTS over the bidi WebSocket → teardown).Verification
The bidi example was verified end-to-end against the released prod image (
vllm:omni-sagemaker-cuda-v1): deployed a real SageMaker endpoint, streamed 253 KB of audio back over the bidirectional WebSocket, and tore down cleanly. Re-verified under profile-only credentials (no env-var creds) to confirm the credential bootstrap path works for profile/SSO/role users, not just env-var users.Doc claims were fact-checked against the code:
WS_ALLOWED_PATHSinscripts/docker/vllm/omni_sagemaker_serve.pydocker/vllm_omni/Dockerfile.amzn2023test/vllm-omni/sagemaker/test_sm_omni_bidi_endpoint.py.github/config/image/vllm-omni/sagemaker-amzn2023.ymlPunch list for a human (NOT addressed here)
docs/vllm-omni/deployment/ec2.mdusesdocker run -p 8080:8080+curl localhost:8080, but the EC2 entrypointscripts/docker/vllm/omni_dockerd_entrypoint.shisexec vllm serve --omni "$@"with no--port, so the container binds vLLM's default 8000. SageMaker sets--port 8080explicitly; EC2 does not. Either add--port 8080to the EC2 entrypoint (recommended — makes the docs correct as written) or change every EC2 doc example to-p 8080:8000. Left for an owner to decide since it touches the entrypoint.vllm:omni-cuda(unversioned) vs the config'svllm:omni-cuda-v1; likely an intentional rolling tag — flagging only.Test plan
pre-commit run --all-filespasses (passed locally)