[JENKINS-72520] Http Request plugin Multipart form does not set boundaries for multipart

[jenkins-infra-bot]

I'm trying to POST an XML file (OWASP ZAP Report) to an external service (OWASP Defect Dojo), using a shared Jenkins library. A curl-command (see below) for POST-ing the XML file works fine. But I have a hard time translating that into Jenkins' Http Request plugin, so we can include it in a reusable Jenkins shared library.

This results in the following response:

 

Response Code: HTTP/1.1 400 Bad Request
Response: 
{"detail":"Multipart form parse error - Invalid boundary in multipart: None"} 

Is this a bug or am I missing something?

 

 

curl command:

 

curl -X 'POST' \
  'http::8080/api/v2/import-scan/' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'X-CSRFToken: ' \
  -H 'Authorization: Token ' \
  -F 'scan_date=2023-11-20' \
  -F 'minimum_severity=Info' \
  -F 'active=true' \
  -F 'verified=true' \
  -F 'scan_type=ZAP Scan' \
  -F 'file=@​zapreport.xml;type=text/xml' \
  -F 'product_name=Test project' \
  -F 'engagement_name=Test engagement' \
  -F 'close_old_findings=false' \
  -F 'close_old_findings_product_scope=false' \
  -F 'push_to_jira=false' \
  -F 'create_finding_groups_for_all_findings=true' 

 

 

Jenkins Groovy code:

ResponseContentSupplier response = steps.httpRequest url: "${env.DEFECT_DOJO_HOST_URL}${api}",
acceptType: 'APPLICATION_JSON',
contentType: 'APPLICATION_FORM_DATA',
httpMode: 'POST',
uploadFile: 'zapreport.xml',
multipartName: 'zapreport.xml',
customHeaders: customHeadersDummy(authHeader, headers), // Includes auth-token
formData: [[body: '''{
    "product_name": "Dummy project",
    "engagement_name": "Dummy Engagement",
    "scan_date": "2023-11-20",
    "minimum_severity": "Info",
    "active": "true",
    "verified": "true",
    "scan_type": "ZAP Scan",
    "close_old_findings": "false",
    "close_old_findings_product_scope": "false",
    "push_to_jira": "false",
    "create_finding_groups_for_all_findings": "true"
}''',
 contentType: 'text/xml',
 fileName: 'zapreport.xml',
 name: 'zapreport.xml',
 uploadFile: 'zapreport.xml']],
 consoleLogResponseBody: true,
responseHandle: 'NONE',
timeout: null
wrapAsMultipart: false 

 

---

Originally reported by brampat, imported from: Http Request plugin Multipart form does not set boundaries for multipart

• assignee: janario
• status: Open
• priority: Blocker
• component(s): http-request-plugin
• resolution: Unresolved
• votes: 0
• watchers: 8
• imported: 20260702-081740

Raw content of original issue

I'm trying to POST an XML file (OWASP ZAP Report) to an external service (OWASP Defect Dojo), using a shared Jenkins library. A curl-command (see below) for POST-ing the XML file works fine. But I have a hard time translating that into Jenkins' Http Request plugin, so we can include it in a reusable Jenkins shared library.

This results in the following response:

 

Response Code: HTTP/1.1 400 Bad Request
Response: 
{"detail":"Multipart form parse error - Invalid boundary in multipart: None"} 

Is this a bug or am I missing something?
 
 
curl command:
 

curl -X 'POST' \
  'http:<server>:8080/api/v2/import-scan/' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -H 'X-CSRFToken: <token>' \
  -H 'Authorization: Token <token>' \
  -F 'scan_date=2023-11-20' \
  -F 'minimum_severity=Info' \
  -F 'active=true' \
  -F 'verified=true' \
  -F 'scan_type=ZAP Scan' \
  -F 'file=@zapreport.xml;type=text/xml' \
  -F 'product_name=Test project' \
  -F 'engagement_name=Test engagement' \
  -F 'close_old_findings=false' \
  -F 'close_old_findings_product_scope=false' \
  -F 'push_to_jira=false' \
  -F 'create_finding_groups_for_all_findings=true' 

 
 
Jenkins Groovy code:

        ResponseContentSupplier response = steps.httpRequest url: "${env.DEFECT_DOJO_HOST_URL}${api}",
                acceptType: 'APPLICATION_JSON',
                contentType: 'APPLICATION_FORM_DATA',
                httpMode: 'POST',
                uploadFile: 'zapreport.xml',
                multipartName: 'zapreport.xml',
                customHeaders: customHeadersDummy(authHeader, headers), // Includes auth-token
                formData: [[body: '''{
    "product_name": "Dummy project",
    "engagement_name": "Dummy Engagement",
    "scan_date": "2023-11-20",
    "minimum_severity": "Info",
    "active": "true",
    "verified": "true",
    "scan_type": "ZAP Scan",
    "close_old_findings": "false",
    "close_old_findings_product_scope": "false",
    "push_to_jira": "false",
    "create_finding_groups_for_all_findings": "true"
}''',
                 contentType: 'text/xml',
                 fileName: 'zapreport.xml',
                 name: 'zapreport.xml',
                 uploadFile: 'zapreport.xml']],
                 consoleLogResponseBody: true,
                responseHandle: 'NONE',
                timeout: null
                wrapAsMultipart: false 

 

• environment: Jenkins 2.393

[jenkins-infra-bot]

markewaite:

• Original comment link
• Raw content of original comment:

  Since you already have it working as a curl command, why not just use the curl command in your Pipeline?  Curl is available for multiple platforms and is known to work well.
  

Since you already have it working as a curl command, why not just use the curl command in your Pipeline? Curl is available for multiple platforms and is known to work well.

[jenkins-infra-bot]

brampat:

• Original comment link
• Raw content of original comment:

  Well.... is that even possible from a Groovy shared library? And can I catch the response?
  
  But aside from that: I'm using the httpRequest plugin for integrating / calling the Defect Dojo REST API. So for consistency I'ld like to do that for the import-scan call as well.
  
  Why are we making shared libraries for this? Because we're tailoring reusable (share) libraries for our development teams to, well.... reuse. Encapsulating the complex logic into easy to use function-calls (facade pattern?) is what we aim for instead of instructions to let them copy-paste a curl command. So much issues with that. What if Defect Dojo moves to another server? Where would they get auth-tokens from? How should they inject them correctly into the pipeline without exposing them? etc.
  
  2nd aside: switching to curl / sh scripts might result in command injection issues, which I'ld like to prevent.
  

Well.... is that even possible from a Groovy shared library? And can I catch the response?

But aside from that: I'm using the httpRequest plugin for integrating / calling the Defect Dojo REST API. So for consistency I'ld like to do that for the import-scan call as well.

Why are we making shared libraries for this? Because we're tailoring reusable (share) libraries for our development teams to, well.... reuse. Encapsulating the complex logic into easy to use function-calls (facade pattern?) is what we aim for instead of instructions to let them copy-paste a curl command. So much issues with that. What if Defect Dojo moves to another server? Where would they get auth-tokens from? How should they inject them correctly into the pipeline without exposing them? etc.

2nd aside: switching to curl / sh scripts might result in command injection issues, which I'ld like to prevent.

[jenkins-infra-bot]

markewaite:

• Original comment link
• Raw content of original comment:

  Well.... is that even possible from a Groovy shared library? 
  
  Yes, it is possible to call sh and bat and powershell from a Pipeline shared library.
  
  And can I catch the response?
  
  Yes, it is possible to catch the return code or stdout or stderr from sh and bat and powershell in a Pipeline shared library..
  
  Because we're tailoring reusable (share) libraries for our development teams to, well.... reuse.
  
  I think that's a very good thing.  Placing the curl call inside the Pipeline shared library will give you the same reuse benefit without requiring that you wait for someone to implement this request in the http-request plugin.
  
  If you're submitting the improvement as an http-request plugin pull request with tests that confirm the code changes are working well, then the maintainers of the http-request plugin (including me) may be willing to review the pull request, evaluate it, merge it, and release it.  If you're volunteering to become one of the maintainers of the http-request plugin, then you could submit the pull request, evaluate it, merge it, and release it yourself.  If you're not submitting the improvement, then you are at the mercy of the plugin maintainers.  I suggested that you use curl because I assumed that you weren't ready to submit the pull request that implements the feature.
  

Well.... is that even possible from a Groovy shared library?

Yes, it is possible to call sh and bat and powershell from a Pipeline shared library.

And can I catch the response?

Yes, it is possible to catch the return code or stdout or stderr from sh and bat and powershell in a Pipeline shared library..

Because we're tailoring reusable (share) libraries for our development teams to, well.... reuse.

I think that's a very good thing. Placing the curl call inside the Pipeline shared library will give you the same reuse benefit without requiring that you wait for someone to implement this request in the http-request plugin.

If you're submitting the improvement as an http-request plugin pull request with tests that confirm the code changes are working well, then the maintainers of the http-request plugin (including me) may be willing to review the pull request, evaluate it, merge it, and release it. If you're volunteering to become one of the maintainers of the http-request plugin, then you could submit the pull request, evaluate it, merge it, and release it yourself. If you're not submitting the improvement, then you are at the mercy of the plugin maintainers. I suggested that you use curl because I assumed that you weren't ready to submit the pull request that implements the feature.

[jenkins-infra-bot]

brampat:

• Original comment link
• Raw content of original comment:

  Thank you for your response.
  
  I'm a bit confused. What do you mean with "Implement this feature"? Is it currently not possible to do a POST with form-data and file-upload, similar to my pasted curl command? Or is it not possible to POST to Defect Dojo? Since the Http Request plugin is a generic tool, I'ld  like to assume that any Defect Dojo specific feature doesn't make sense. But I expected it to support any form of Http Requests. Is that incorrect?
  

Thank you for your response.

I'm a bit confused. What do you mean with "Implement this feature"? Is it currently not possible to do a POST with form-data and file-upload, similar to my pasted curl command? Or is it not possible to POST to Defect Dojo? Since the Http Request plugin is a generic tool, I'ld  like to assume that any Defect Dojo specific feature doesn't make sense. But I expected it to support any form of Http Requests. Is that incorrect?

[jenkins-infra-bot]

markewaite:

• Original comment link
• Raw content of original comment:

  What do you mean with "Implement this feature"?
  
  It would have been clearer if I'd used the phrase "fix this bug" instead of "implement this feature".  Jerry Weinberg's definition of a bug is "anything that bugs someone".  This bugs you, so it is a bug.
  
  I mean that you are requesting a change in the behavior of the http-request plugin.  You've defined this change in behavior of the http-request plugin as a blocker.  I've offered a workaround so that your efforts won't be blocked.
  
  I don't plan to work on this issue.  I suspect that the other maintainers of the http-request plugin are unlikely to work on this issue either.  If you decide to submit a pull request to fix the bug and if you include tests that verify the fix, then I'll put the review of that fix into my review queue.
  
  But I expected it to support any form of Http Requests. Is that incorrect?
  
  I agree that the http-request plugin should support any form of http request that makes sense in the context of a Jenkins controller or Jenkins agent.  I don't know if the http-request plugin support multi-part form submission of files.  I assume from your very thorough description of the issue that it includes some support for multi-part form submission of files, but that support is incorrect or insufficient.
  

What do you mean with "Implement this feature"?

It would have been clearer if I'd used the phrase "fix this bug" instead of "implement this feature". Jerry Weinberg's definition of a bug is "anything that bugs someone". This bugs you, so it is a bug.

I mean that you are requesting a change in the behavior of the http-request plugin. You've defined this change in behavior of the http-request plugin as a blocker. I've offered a workaround so that your efforts won't be blocked.

I don't plan to work on this issue. I suspect that the other maintainers of the http-request plugin are unlikely to work on this issue either. If you decide to submit a pull request to fix the bug and if you include tests that verify the fix, then I'll put the review of that fix into my review queue.

But I expected it to support any form of Http Requests. Is that incorrect?

I agree that the http-request plugin should support any form of http request that makes sense in the context of a Jenkins controller or Jenkins agent. I don't know if the http-request plugin support multi-part form submission of files. I assume from your very thorough description of the issue that it includes some support for multi-part form submission of files, but that support is incorrect or insufficient.

[jenkins-infra-bot]

brampat:

• Original comment link
• Raw content of original comment:

  Thank you for clearing this up. This does save me a lot of time trying to trial and error my issue using the current http-request plugin.
  
  I don't think I'm able to fix this issue in the source of http-request plugin, due to time constraints. So I will first try to implement the workaround in a Groovy shared library. If I can spend the time I will see if and how I can implement the bug-fix eventually, but I do realise that this might be a niche use-case. Although, it is something that's essential to integrating OWASP Defect Dojo's import scan API-calls.
  

Thank you for clearing this up. This does save me a lot of time trying to trial and error my issue using the current http-request plugin.

I don't think I'm able to fix this issue in the source of http-request plugin, due to time constraints. So I will first try to implement the workaround in a Groovy shared library. If I can spend the time I will see if and how I can implement the bug-fix eventually, but I do realise that this might be a niche use-case. Although, it is something that's essential to integrating OWASP Defect Dojo's import scan API-calls.

[jenkins-infra-bot]

jsolaa:

• Original comment link
• Raw content of original comment:

  I had a similar problem here, and AI offered a workaround that seems to work: building the request body (with boundaries) manually.
  
  Dropping here, in case someone finds it useful:
  
   
  
  Map upload(String arg1, String arg2, String filename, String fileContent) {
      try {
          // Build multipart form data with manual boundaries
          def boundary = '----WebKitFormBoundary' + UUID.randomUUID().toString().replaceAll('-', '')
          
          def body = buildMultipartBody(boundary, [
              'arg1': arg1,
              'arg2': arg2
          ], filename, fileContent)
          
          def requestConfig = [
              httpMode: 'POST',
              url: ...
              requestBody: body,
              customHeaders: [
                  [name: 'Content-Type', value: "multipart/form-data; boundary=${boundary}"]
              ]
          ]
  
          def response = httpRequest(requestConfig)
          ...        
      } catch (Exception e) {
          ...
      }
  }
  
  String buildMultipartBody(String boundary, Map<String, String> fields, String filename, String fileContent) {
      def body = ''
  
      // Add form fields
      fields.each { key, value ->
          body += "--${boundary}\r\n"
          body += "Content-Disposition: form-data; name=\"${key}\"\r\n\r\n"
          body += "${value}\r\n"
      }
  
      // Add file
      body += "--${boundary}\r\n"
      body += "Content-Disposition: form-data; name=\"article\"; filename=\"${new File(filename).name}\"\r\n"
      body += "Content-Type: text/plain\r\n\r\n"
      body += "${fileContent}\r\n"
      body += "--${boundary}--\r\n"
  
      return body
  }
  
   
   
  

I had a similar problem here, and AI offered a workaround that seems to work: building the request body (with boundaries) manually.

Dropping here, in case someone finds it useful:

 

Map upload(String arg1, String arg2, String filename, String fileContent) {
    try {
// Build multipart form data with manual boundaries
def boundary = '----WebKitFormBoundary' + UUID.randomUUID().toString().replaceAll('-', '')

def body = buildMultipartBody(boundary, [
    'arg1': arg1,
    'arg2': arg2
], filename, fileContent)

def requestConfig = [
    httpMode: 'POST',
    url: ...
    requestBody: body,
    customHeaders: [
[name: 'Content-Type', value: "multipart/form-data; boundary=${boundary}"]
    ]
]

def response = httpRequest(requestConfig)
...
    } catch (Exception e) {
...
    }
}

String buildMultipartBody(String boundary, Map<String, String> fields, String filename, String fileContent) {
    def body = ''

    // Add form fields
    fields.each { key, value ->
body += "--${boundary}\r\n"
body += "Content-Disposition: form-data; name=\"${key}\"\r\n\r\n"
body += "${value}\r\n"
    }

    // Add file
    body += "--${boundary}\r\n"
    body += "Content-Disposition: form-data; name=\"article\"; filename=\"${new File(filename).name}\"\r\n"
    body += "Content-Type: text/plain\r\n\r\n"
    body += "${fileContent}\r\n"
    body += "--${boundary}--\r\n"

    return body
}