Cherry pick rocjpeg and rocjpeg fallback

[apinge]

Motivation

Cherry pick zejunchen-zejun#211 and zejunchen-zejun#257

Modifications

Accuracy Tests

#!/bin/bash
curl --request POST \
     --url "http://localhost:9080/v1/chat/completions" \
     --header "Content-Type: application/json" \
     --data '{
         "model": "/model/Qwen3.5-397B-A17B-PTPC-FP8",
         "messages": [
             {
             "role": "user",
             "content": [
                 {
                 "type": "image_url",
                 "image_url": {
                     "url": "/sgl-workspace/evaluation5/ali_uc/page-crop-00001.jpg"
                     }
                 },
		 {
                 "type": "image_url",
                 "image_url": {
                     "url": "/sgl-workspace/evaluation5/ali_uc/page-crop-00001.jpg"
                     }
                 },
		 {
                 "type": "image_url",
                 "image_url": {
                     "url": "/sgl-workspace/evaluation5/ali_uc/page-crop-00001.jpg"
                     }
                 },
		 {
                 "type": "image_url",
                 "image_url": {
                     "url": "/sgl-workspace/evaluation5/ali_uc/page-crop-00001.jpg"
                     }
                 },
		 {
                 "type": "image_url",
                 "image_url": {
                     "url": "/sgl-workspace/evaluation5/ali_uc/page-crop-00001.jpg"
                     }
                 },
                 {
                 "type": "text",
                 "text": "Please briefly describe what the image contains."
                 }
             ]
             }
         ],
         "temperature": 0.0,
         "top_p": 0.7,
         "top_k": 3,
         "max_tokens": 500
     }'

{"id":"87e44fbb3e5c4d3d8fe2aab791dbe4d6","object":"chat.completion","created":1785328348,"model":"/model/Qwen3.5-397B-A17B-PTPC-FP8","choices":[{"index":0,"message":{"role":"assistant","content":"","reasoning_content":"The user wants a brief description of the image.\n\n1.  **Analyze the image:**\n    *   **Subject:** A cartoon boy.\n    *   **Action:** He is sitting and reading a book.\n    *   **Setting:** Outdoors, under a tree. There is green grass.\n    *   **Appearance:**\n        *   He has brown hair.\n        *   He is wearing a red hat (bucket hat style).\n        *   He is wearing a blue shirt with long sleeves.\n        *   He is wearing red pants.\n        *   He has tan shoes.\n        *   He has rosy cheeks.\n    *   **Object:** An open blue book.\n    *   **Background:** A large tree trunk to his right, green leaves above him forming a canopy. Green grass below.\n\n2.  **Synthesize the description:**\n    *   Start with the main subject: A cartoon boy.\n    *   Describe what he's doing: Reading a blue book.\n    *   Describe where he is: Sitting on the grass under a tree.\n    *   Add details about his clothing: Red hat, blue shirt, red pants.\n\n3.  **Draft the response:**\n    The image shows a cartoon illustration of a young boy sitting on the grass under a tree. He is wearing a red hat, a blue long-sleeved shirt, and red pants. He is holding an open blue book and appears to be reading. The tree has a brown trunk and green leaves that canopy over him.\n\n4.  **Refine for brevity:**\n    \"A cartoon boy wearing a red hat, blue shirt, and red pants is sitting on the grass under a tree, reading a blue book.\" -> This is concise and accurate.\n\n5.  **Final Polish:** Just ensure it covers the key elements.\n    *   Boy reading.\n    *   Under a tree.\n    *   Red hat/clothes.\n    *   Blue book.\n\n    Let's go with a slightly more descriptive version but keep it brief.\n    \"The image features a cartoon of a young boy sitting on the grass beneath a large tree. He is wearing a red hat, a blue shirt, and red pants. He is holding an open blue book and reading.\"\n\n    Wait, looking closely at the provided images, they are","tool_calls":null},"logprobs":null,"finish_reason":"length","matched_stop":null}],"usage":{"prompt_tokens":378,"total_tokens":878,"completion_tokens":500,"prompt_tokens_details":{"cached_tokens":0,"image_tokens":350},"reasoning_tokens":500},"metadata":{"weight_version":"default"}}

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ❌ Run #30451825788
Latest PR Test (Extra): ❌ Run #30451825715

[Copilot]

Pull request overview

This PR cherry-picks upstream changes to improve multimodal image ingestion by enabling JPEG batch decoding (rocjpeg path) with a CPU/OpenCV fallback, and wiring that into the multimodal processor data-loading fast/legacy paths.

Changes:

• Added JPEG-format detection plus new load_image_tensor() and batch_decode_jpeg_gpu() utilities to support “return bytes for JPEG, decode otherwise” and batch decode.
• Updated BaseMultimodalProcessor image loading to return (img_tensor_bytes, "jpeg") for JPEGs and batch-decode them after the parallel load stage.
• Adjusted file path handling for file:// inputs in get_image_bytes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File	Description
python/sglang/srt/utils/common.py	Adds JPEG detection, batch JPEG decode helper with fallback, new tensor-oriented image loader, and improves file:// path normalization.
python/sglang/srt/multimodal/processors/base_processor.py	Switches image load path to the new tensor/bytes API and adds JPEG batch decode in both fast and legacy multimodal loading.

Suppressed comments (1)

python/sglang/srt/multimodal/processors/base_processor.py:1065

• Same as fast path: batch_decode_jpeg_gpu() can yield None entries, and decoded_img.to(...) will fail with an attribute error. Consider surfacing a clearer error (or skipping/handling failures) before storing into final_results.

        # Batch decode all JPEG images
        if jpeg_bytes_list:
            decoded_images = batch_decode_jpeg_gpu(
                jpeg_bytes_list, device=target_device
            )

            # Put decoded images back to their original positions
            for img_idx, decoded_img in zip(jpeg_indices, decoded_images):
                final_results[img_idx] = decoded_img.to(target_device)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.