Relative references in $ref are not supported

[tnmtechnologies]

Version

4.5.8

Context

Relative references in $ref are not supported (see OpenAPI spec https://spec.openapis.org/oas/latest.html#relative-references-in-uris) while it worked well with previous OpenAPI library.

The following exception is thrown:

java.lang.UnsupportedOperationException: Can't resolve 'the relative reference to a OpenAPI file ', only internal refs are supported.
at io.vertx.json.schema.impl.JsonRef.resolveUri(JsonRef.java:344)
at io.vertx.json.schema.impl.JsonRef.resolve(JsonRef.java:211)
at io.vertx.json.schema.impl.SchemaRepositoryImpl.resolve(SchemaRepositoryImpl.java:243)
at io.vertx.openapi.contract.OpenAPIVersion.lambda$resolve$3(OpenAPIVersion.java:86)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:191)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:279)
at io.vertx.core.impl.ContextImpl.lambda$internalExecuteBlocking$2(ContextImpl.java:210)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)

We work with more than one hundred of OpenAPI files. Most of them use relative references to components in other OpenAPI files.
They are provided by a third party.

It would be nice to support relative references from a configuration option.
It would avoid to update OpenAPI files in order to replace relative references by fake absolute ones and to add them by hand as additionalContractFiles in OpenAPIContract.from method.

Do you have a reproducer?

No

• Link to github project/gist

Steps to reproduce

Extra

• Anything that can be relevant such as OS version, JVM version

[pk-work]

Due to security reasons we decided that files must be provided upfront. Maybe we could add a method which parses the OpenAPI files, follow the relative links and create the required additionalContractFiles Map. But I don't think that is something I will develop in the future.

If you want to provide such a feature, I would support it.

[tsegismont]

@pk-work as discussed off-list, I will a give a shot to implementing this feature. Below is a summary of the plan as I understand it.

We should parse the contract, follow the relative links and create the required additionalContractFiles map. But we should also modify the content of the contract, because vert.x json schema will tolerate only internal references.

For example, we could transform a relative reference like this nested/schema.yaml into absolute file urls like this one: file:///path/to/my/openapi/files/nested/schema.yaml (assuming the relative reference is resolved when in /path/to/my/openapi/files)

[pk-work]

Yes, the contract must be modified, because the relative link must be replaced with the "key" from the additional contract map. Here is an example [1] yaml and the related test [2] in which the map is created.

[1] https://github.com/eclipse-vertx/vertx-openapi/blob/main/src/test/resources/io/vertx/tests/contract/from_with_path_and_additional_files/petstore.yaml

[2]

vertx-openapi/src/test/java/io/vertx/tests/contract/OpenAPIContractBuilderTest.java

Line 72 in 1efa73c

void should_create_contract_when_valid_contract_path_and_additional_contract_paths_are_provided(Vertx vertx,

[tsegismont]

Thanks @pk-work

I believe it should work if the replacement is an absolute file URI like file:///path/to/my/openapi/files/nested/schema.yaml
Is it fine with you or can you think about any problem (for example exposing a file system hierarchy publicly) ?

[pk-work]

If you would expose the full path publicly it may is an issue, because it can tell an attacker which OS or web server you use. But in this case we replace relative links with absolute ones during the contract "construction". I'm pretty sure this is not exposed somewhere, so it shouldn't be any problem.

[tsegismont]

@tnmtechnologies have you tried a bundler tool like Redocly:

npx @redocly/cli bundle contract.yaml --output bundled.yaml

@pk-work any experience with that?

[tsegismont]

In fact, with swagger-parser:

<dependencies>
    <dependency>
        <groupId>io.swagger.parser.v3</groupId>
        <artifactId>swagger-parser</artifactId>
        <version>2.1.20</version>
    </dependency>
</dependencies>

Creating a bundled contract is relatively straightforward:

package org.example.myproject;

import io.swagger.v3.core.util.Yaml;
import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.parser.OpenAPIV3Parser;
import io.swagger.v3.parser.core.models.ParseOptions;
import io.swagger.v3.parser.core.models.SwaggerParseResult;

import java.nio.file.Files;
import java.nio.file.Paths;

public class OpenApiBundler {
  public static void main(String[] args) throws Exception {
    String inputPath = "src/test/resources/io/vertx/tests/contract/relative_refs/cross_refs/openapi.yaml";

    ParseOptions options = new ParseOptions();
    // This resolves external files and moves them into the 'components' section
    options.setResolve(true);

    // This keeps the $ref but points it to the new local component.
    options.setResolveFully(false);

    SwaggerParseResult result = new OpenAPIV3Parser().readLocation(inputPath, null, options);
    OpenAPI openAPI = result.getOpenAPI();

    if (openAPI != null) {
      String bundledYaml = Yaml.pretty().writeValueAsString(openAPI);
      Files.write(Paths.get("bundled-openapi.yaml"), bundledYaml.getBytes());
    } else {
      System.err.println("Errors found: " + result.getMessages());
    }
  }
}

And then this could be embedded in a Maven plugin (after updating the code to take arguments):

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>3.1.0</version>
    <executions>
        <execution>
            <id>bundle-openapi</id>
            <phase>generate-resources</phase>
            <goals>
                <goal>java</goal>
            </goals>
            <configuration>
                <mainClass>org.example.myproject</mainClass>
                <arguments>
                    <argument>src/main/resources/openapi.yaml</argument>
                    <argument>${project.build.directory}/bundled-openapi.yaml</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>

[pk-work]

Yes, I have experience with redocly. I think I even created some test resources [1] in vertx-json-schema with redocly.

But calling redocly from Java during contract construction is most probably not an option. I also don't get completely how a maven plugin can solve the issue. Because the contract is not always available on build time.

[1] https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/test/resources/resolve/guestbook_bundle.json

[tsegismont]

Because the contract is not always available on build time.

Oh right, that's a very good point. Having a dependency on Swagger parser at build time is not big deal IMO, but since the contracts may not be available until the application starts, we can't rely on this.