Clean up /schemas on main branch

[duncanbeevers]

🗣 Discussion

The main branch contains a /schemas folder with files we don't want end-users consuming directly. In order to shield users from these files, the same way we do with /src, they should be present on the dev branch and not on the main branch.

• Identify and remove build process dependencies on /schemas artifacts
  • dev: schemas live in src folder #4361
• enable schema build & publish process on vX.Y-dev branches
  • Update schema-publish.sh #4376
  • dev: fix schema publish.yaml #4378
• Disable schema-publish workflow by syncing it from dev to main
  • cherry-pick commits from dev and test that this will not mess up the sync flow from main to dev
• How and where do we want to maintain the v3.0 schema and its test cases?
  • Proposal: we won't, we'll park them together with the v1.2 and v2.0 schemas
• Where do we want to park the v1.2 and v2.0 schemas before deleting the /schemas folder?
  Proposal: we park them in a new /_archive_ folder
• clean up schemas on main
  • move /schemas folder on main into /_archive_
  • optional: delete 3.1 schemas and add README explaining that maintenance will happen in v3.1-dev and that the ready-to-use schemas are on spec.openapis.org
  • move 3.0 schema tests into /_archive_/schemas/v3.0
  • delete 3.1 schema tests
  • test that these changes will not mess up the sync flow from main to dev or corrupt the 3.1 schema tests on dev

[notEthan]

What? Why?
I don't know who would need to be "shielded" or why these shouldn't be consumed. My own OpenAPI tooling is built with these schemas pretty much as its foundation.
I'm confused the intention here. Is the plan to not publish schemas anymore? If so, that's pretty terrible. If not, it seems like this just makes it harder for anybody who wants these schemas to find them - they won't even exist outside of development branch(es)? They've lived where they are for a decade, why make it difficult for people?

[handrews]

@notEthan Schemas are officially published in JSON form with correct $ids to the spec.openapis.org site via the gh-pages branch. The files on main are YAML sources with placeholder WORK-IN-PROGRESS values in $id and are not official publications. This is (obviously) confusing, so we're having sources in one place (src on various branches) and published schemas in one place (spec.openapis.org, which is our official publication location for both spec and schemas).

[notEthan]

Okay, I was thrown by interpreting/misinterpreting that the schemas in /schemas are not to be consumed, but that just meant the stuff under development (I did look for other discussion to see if I had this wrong but failed to find it). Pardon my confusion on that.

I'm still not sure the motivation for removing the existing published schemas - v2.0 in particular, still significantly in use but I don't see its schema on spec.openapis.org. Does this have a home? (1.x too but I haven't seen any 1.x in a very long time, don't know that it has significant use anymore.)

I feel like having published versions on main of both the specifications and schemas is valuable, seems preferable to navigating the web site or accessing the relatively obscure gh-pages branch. But I'll deal with it if that doesn't work for OAI.

[handrews]

@notEthan good catch on 2.0 and 1.2. We don't currently publish those spec versions on spec.openapis.org (perhaps we should). I don't have a strong feeling about leaving those schemas on main, as long as 3.0+ (which are placeholders from an outdated development workflow that was always confusing to both contributors and users) get removed. I slightly prefer moving everything to the spec site, but that's just my personal love of consistency and should be balanced against community needs. I'm also fine with "1.2 and 2.0 aren't broken, so dont fix them" and just leave a README explaining the 3.x situation (by pointing to CONTRIBUTING.md).

[ralfhandl]

2.0 and 1.2. We don't currently publish those spec versions on spec.openapis.org (perhaps we should)

The 2.0 schema's id is

OpenAPI-Specification/schemas/v2.0/schema.json

Line 3 in 8e3015a

"id": "http://swagger.io/v2/schema.json#",

and it can be fetched from http://swagger.io/v2/schema.json#, so apparently the OAI doesn't own this schema. Also the README points to the NPM package swagger-schema-official which is also not owned by the OAI.

If we want to add it despite the ownership problem, we should also show the 2.0 spec:

• gh-pages: show OAS 2.0 schema #4366
• gh-pages: show OAS 2.0 spec #4365

The 1.2 schemas have a surprising structure, so I would leave them out.

[duncanbeevers]

@ralfhandl Is there anything further blocking removal of /schemas from main?
I think it would be very nice, because it would eliminate one of the merge conflicts affecting #4396

[ralfhandl]

The merge conflict can be resolved with

• v3.1-dev: resolve conflicts that prevent a direct merge from dev #4417

[ralfhandl]

Before deleting the schemas folder we need to decide where it's content should go.

[karenetheridge]

@notEthan This may be of some help (feel free to copy!) -- I bundle the schemas with my implementation, and at installation time check that they have not been altered from the expected content. At runtime, the bundled schemas are used -- they are not fetched live from the website.

• publish-time script to fetch the schemas and compute checksums: https://metacpan.org/release/ETHER/OpenAPI-Modern-0.083/source/update-schemas
• installation-time test to confirm checksums match: https://metacpan.org/release/ETHER/OpenAPI-Modern-0.083/source/t/checksums.t

[baywet]

main -> dev -> dev-version -> release -> main makes a roundtrip, we want to be careful deleting those files.
The approach should have been cherry-picking, to avoid those type of issues.
Creating release branches remains an issue even with cherry picking.

Proposal:

1. move the schemas to an archive folder, on main branch
2. remove them from the dev and 3.1-dev, etc...

@ralfhandl to proceed as agreed on today's call.