Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Relative reference in subdirectory OpenApi document fails to load #1674

Open
dldl-cmd opened this issue May 27, 2024 · 10 comments · Fixed by #2133 · May be fixed by #2243
Open

Relative reference in subdirectory OpenApi document fails to load #1674

dldl-cmd opened this issue May 27, 2024 · 10 comments · Fixed by #2133 · May be fixed by #2243
Labels
priority:p2 Medium. Generally has a work-around and a smaller sub-set of customers is affected. SLA <=30 days type:bug A broken experience WIP
Milestone
v2-Preview17

Comments

@dldl-cmd
Copy link
Contributor

Relative reference in subdirectory OpenApi document fails to load

When loading a document which is splitted into multiple documents relative references are failing to load when a parent document is in a subdirectory.

What happens is, that a System.IO.FileNotFoundException is thrown:

Could not find file '<path to root of document structure>\Pets.json'.
StackTrace:
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.File.OpenRead(String path)
   at Microsoft.OpenApi.Readers.Services.DefaultStreamLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\DefaultStreamLoader.cs:line 47
   at Microsoft.OpenApi.Readers.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\OpenApiWorkspaceLoader.cs:line 46
   at Microsoft.OpenApi.Readers.OpenApiYamlDocumentReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiYamlDocumentReader.cs:line 105
   at Microsoft.OpenApi.Readers.OpenApiTextReaderReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiTextReaderReader.cs:line 83
   at Microsoft.OpenApi.Readers.OpenApiStreamReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiStreamReader.cs:line 71
   at Microsoft.OpenApi.Readers.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\OpenApiWorkspaceLoader.cs:line 47
   at Microsoft.OpenApi.Readers.OpenApiYamlDocumentReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiYamlDocumentReader.cs:line 105
   at Microsoft.OpenApi.Readers.OpenApiTextReaderReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiTextReaderReader.cs:line 83
   at Microsoft.OpenApi.Readers.OpenApiStreamReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiStreamReader.cs:line 71
   at Program.<<Main>$>d__0.MoveNext() in C:\Users\user\source\repos\OpenApiReferencesInSubDirectories\OpenApiReferencesInSubDirectories\Program.cs:line 17

Files to reproduce

OpenApi.NET Version: 1.6.14

using Microsoft.OpenApi.Readers;

OpenApiStreamReader openApiStreamReader = new OpenApiStreamReader(new OpenApiReaderSettings
{
    BaseUrl = new Uri("<path to root of document structure>"),
    ReferenceResolution = ReferenceResolutionSetting.ResolveLocalReferences,
    LoadExternalRefs = true,
});

FileStream fileStream = new FileStream("Root.json", FileMode.Open, FileAccess.Read);

var document =  await openApiStreamReader.ReadAsync(fileStream);

./Root.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore",
    "license": {
      "name": "MIT"
    }
  },
  "servers": [
    {
      "url": "http://petstore.swagger.io/v1"
    }
  ],
  "paths": {
    "/pets": {
      "get": {
        "summary": "List all pets",
        "operationId": "listPets",
        "responses": {
          "200": {
            "description": "An array of pets",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "./Directory/AllPets.json#/components/schemas/AllPets"
                }
              }
            }
          }
        }
      }
    }
  }
}

./Directory/AllPets.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore"
  },
  "paths": {
  },
  "components": {
    "schemas": {
      "AllPets": {
        "$ref": "./Pets.json#/components/schemas/Pets"
      }
    }
  }
}

./Directory/Pets.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore"
  },
  "paths": {
  },
  "components": {
    "schemas": {
      "Pets": {
        "type": "array",
        "items": {
          "$ref": "./Pet.json#/components/schemas/Pet"
        }
      },
      "Pet": {
        "required": [
          "id",
          "name"
        ],
        "properties": {
          "id": {
            "type": "integer",
            "format": "int64"
          },
          "name": {
            "type": "string"
          },
          "tag": {
            "type": "string"
          }
        }
      }
    }
  }
}
@MaggieKimani1 MaggieKimani1 added the type:bug A broken experience label Jun 3, 2024
This was referenced Feb 4, 2025
Closed
Merged
@baywet baywet added this to the v2 - Preview8 milestone Feb 7, 2025
@baywet
Copy link
Member

baywet commented Feb 7, 2025

closing since #2133 was merged

@baywet baywet closed this as completed Feb 7, 2025
@dldl-cmd
Copy link
Contributor Author

@baywet The issue is still present. As the parameter LoadExternalRefs is set to true. If the parameter is set to false, the issue would be solved by the PR #2133 as it only fixed that the reference could be parsed correctly.

With the current main branch, the following exception is thrown:


System.IO.FileNotFoundException
  HResult=0x80070002
  Message=Could not find file 'C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Pets.json'.
  Source=System.Private.CoreLib
  StackTrace:
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.File.OpenRead(String path)
   at Microsoft.OpenApi.Reader.Services.DefaultStreamLoader.<LoadAsync>d__3.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\DefaultStreamLoader.cs:line 45
   at Microsoft.OpenApi.Reader.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\OpenApiWorkspaceLoader.cs:line 48
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadExternalRefsAsync>d__12.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 272
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<InternalLoadAsync>d__11.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 251
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadAsync>d__6.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 139
   at Microsoft.OpenApi.Models.OpenApiDocument.<LoadAsync>d__67.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Models\OpenApiDocument.cs:line 564
   at Microsoft.OpenApi.Reader.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\OpenApiWorkspaceLoader.cs:line 49
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadExternalRefsAsync>d__12.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 272
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<InternalLoadAsync>d__11.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 251
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadAsync>d__6.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 139
   at Microsoft.OpenApi.Models.OpenApiDocument.<LoadAsync>d__67.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Models\OpenApiDocument.cs:line 564
   at ReferencesToSubFolders.Program.<Main>d__0.MoveNext() in C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Program.cs:line 21
   at ReferencesToSubFolders.Program.<Main>d__0.MoveNext() in C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Program.cs:line 27
   at ReferencesToSubFolders.Program.<Main>(String[] args)

This exception occurs because the Uri is build from OpenApiReaderSettings.BaseUrl and appends the relative path. This construction happens in OpenApiModelFactory in the method LoadExternalRefsAsync.

// Create workspace for all documents to live in.
var baseUrl = settings.BaseUrl ?? new Uri(OpenApiConstants.BaseRegistryUri);
var openApiWorkSpace = new OpenApiWorkspace(baseUrl);

// Load this root document into the workspace
var streamLoader = new DefaultStreamLoader(settings.BaseUrl);
var workspaceLoader = new OpenApiWorkspaceLoader(openApiWorkSpace, settings.CustomExternalLoader ?? streamLoader, settings);

My expectation would be that the baseUrl is the location of the current document. This would allow to use relative references to the current file instead of the settings base url.

I wanted to start implementing a fix also for this issue, but I would like to get some understanding of the expected behavior with a few classes:

  1. Should OpenApiDocument.BaseUri be the location of the current document or https://openapi.net/<guid>?
  2. Should OpenApiModelFactory.LoadExternalRefsAsync reuse the existing OpenApiWorkspace as all documents should be loaded into one workspace?
  3. How should a relative reference be resolved as there is no ResolveReference on the OpenApiDocument only on the OpenApiWorkspace?

@baywet
Copy link
Member

baywet commented Feb 13, 2025

@dldl-cmd which version of the library are you testing with?

@dldl-cmd
Copy link
Contributor Author

@baywet v2 - I tested with latest main branch (c4bdccc)

@baywet baywet reopened this Feb 13, 2025
@baywet
Copy link
Member

baywet commented Feb 13, 2025

Thanks for confirming!
Since you're already setup to test this and build locally, Is this something you'd like to submit a pull request for provided some guidance?

@dldl-cmd
Copy link
Contributor Author

dldl-cmd commented Feb 13, 2025 •
edited
Loading

@baywet Yes, I would like to do a PR on this issue. I already look at some code pieces therefore I came up with the above questions.

@baywet
Copy link
Member

baywet commented Feb 13, 2025

To answer your questions:

  1. I think this is a placeholder when the document is not being loaded from a "remote source", maybe we should put the path there instead? @MaggieKimani1 to provide more context.
  2. I believe reusing the existing workspace to load additional documents is the desired design here.
  3. You should be able to navigate from the document to the workspace using the workspace property? I'm not sure I understand the question.

Let us know if you have any additional comments or questions.

@dldl-cmd
Copy link
Contributor Author

  1. I think the solution could be to have a method e.g. on the OpenApiReference or the OpenApiDocument to turn the relative reference inside of a document to an absolute reference, which could then be given to OpenApiWorkspace.ResolveReference(<absolute path>).

    With the OpenApiReference.HostDocument property it should be possible on a reference itself.

@baywet
Copy link
Member

baywet commented Feb 13, 2025

I think it'd make sense, feel free to startup a PR, seeing the code will make things a bit more concrete at this point.

@darrelmiller
Copy link
Member

darrelmiller commented Feb 19, 2025 •
edited
Loading

Relative references should be resolved in any implementation of IStreamLoader, not in an OpenAPIDocument.

@baywet baywet added the priority:p2 Medium. Generally has a work-around and a smaller sub-set of customers is affected. SLA <=30 days label Feb 19, 2025
dldl-cmd added a commit to dldl-cmd/OpenAPI.NET that referenced this issue Mar 10, 2025
@dldl-cmd
Fix: relative references in subdirectory documents are not loading mi…
Loading
Loading status checks…
7cb9192 
…crosoft#1674

Use OpenApiDocuments BaseUri as location of the document. This allows to have during loading further documents a base Url for retrieval, which can be combined with a relative Uri to get an absolute.
dldl-cmd added a commit to dldl-cmd/OpenAPI.NET that referenced this issue Mar 10, 2025
@dldl-cmd
Fix: relative references in subdirectory documents are not loading mi…
Loading
Loading status checks…
4d295d8 
…crosoft#1674

Use OpenApiDocuments BaseUri as location of the document. This allows to have during loading further documents a base Url for retrieval, which can be combined with a relative Uri to get an absolute.
dldl-cmd added a commit to dldl-cmd/OpenAPI.NET that referenced this issue Mar 10, 2025
@dldl-cmd
Fix: relative references in subdirectory documents are not loading mi…
Loading
Loading status checks…
962d6fa 
…crosoft#1674

Use OpenApiDocuments BaseUri as location of the document. This allows to have during loading further documents a base Url for retrieval, which can be combined with a relative Uri to get an absolute.
@MaggieKimani1 MaggieKimani1 linked a pull request Mar 14, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority:p2 Medium. Generally has a work-around and a smaller sub-set of customers is affected. SLA <=30 days type:bug A broken experience WIP
Projects
None yet
Milestone
v2-Preview17
4 participants
@darrelmiller @baywet @MaggieKimani1 @dldl-cmd