Issue 5607 - Federations should work without bouncer

[sebjf]

This fixes #5607

Description

This PR updates the backend so that Federations function without the scene collection having to be maintained by bouncer.

The way it achieves this is to use the existing model settings document to get the Containers in a Federation, in place of the ref nodes. Any properties that are not in this document, but expected by the frontend/viewer in the responses, are generated procedurally. The history collection is still maintained, but by the backend directly.

Generally, the changes are cleanly split between v4 and v5.

The v4 functions are all concerned with getting the submodels for a model, for the purposes of sequences, groups, metadata etc. These functions have been updated to use the v4 getSubModels array, which in turn has been updated to use the model settings document. getRefNodes is now no longer used, though it still exists as it is referenced in a couple of methods that do not seem to be used (all automated & manual tests are passing).

On the v5 side, the changes are concerned with updating a Federation. Instead of building the parameters object and submitting it to the queue, the new functionality updates the model settings document directly and inserts a new revision node into the history*. The old logic has been removed. The new revision node has a containers array, but this is for historical information only - the authoritative list of containers is still expected to be the model settings. This ensures backwards compatibility without a migration.

(*For this, a stub that imports the v4 history as a v5 module has been created, similar to other utils entries in v5, presumably before they are updated & replaced in the future.)

As above, no migration is needed. If this change functions for a release without issue, a future migration script could remove the redundant scene and ref collections for all legacy federations. Similarly, there no required changes to bouncer or the deployments - that functionality simply won't be used, so can be decommissioned at will in the future.

Finally, not related to the Federations, but the travis.yml has been updated to only check out the required SHA instead of the entire history of the tests repo, saving 2-3 minutes for each build.

Acceptance Criteria

The acceptance criteria is from this ticket, which all passes currently: #5607

• Container with Hidden Geometry works (show hidden geometry in tree)
  • Ifc Spaces
  • Sequences
• Federation of two Containers with Hidden Geometry works (both are hidden, tree toggle works)
• Federation of two Containers, only one with Hidden Geometry, works (geometry is hidden, tree toggle works)
• Federations can be created, have models added, have models removed, and deleted
• A legacy Federation can be loaded
• A legacy Federation can be updated and loaded
• Federation updates result in live updates to all Admin users
• Federation updates result in live updates to all Viewer users
• A new Federation's revisions.json response should show a user and time accurate log of its modification history
• A legacy Federation's revisions.json response should show a user and time accurate log of its modification history
• A mixed Federation's revisions.json response should show a user and time accurate log of its modification history
• A Federation containing two Containers each with a Sequence shall show both Sequences
• A Federation containing two Containers each with a Sequence shall only show Sequences for models to which the User has access (https://github.com/3drepo/3D-Repo-Product-Team/issues/848)
• A Federation containing a Container inaccessible to User can be viewed by User (minus that Container). The Container count should reflect all models in the Federation. (https://github.com/3drepo/3D-Repo-Product-Team/issues/849)
• A Federation containing no Containers accessible to a User should show "Federation is Empty" message
• A Federation containing only voided accessible revisions should show "Federation is Empty" message
• A Federation containing a Container with voided Container Revisions will open the first non-void revision for that Container
• A Federation containing an Empty, Queued or Processing Container will load (the Tree should show an empty entry for the Container)
• A Federation containing a Queued or Processing Container with an existing revision will show the latest completed revision and a warning.
• A Federation containing Grouped Containers loads all Containers under the appropriate Group
• A Federation containing Grouped Containers without valid revisions loads and shows empty entries under the appropriate Group in the Tree
• A Federation containing only a Group with only inaccessible containers should show the "Federation is Empty" message
• A Federation containing a Group consisting only of inaccessible Containers should load for a User (https://github.com/3drepo/3D-Repo-Product-Team/issues/850)
• Show/Hide/Isolate buttons in Tree should work for Groups and their children
• A Federation's Containers Groups can be edited
• A Federation containing a Normal Group can be reloaded (https://github.com/3drepo/3D-Repo-Product-Team/issues/847)
• A Federation containing a Normal Group can be loaded by a User and all Objects are included
• A Federation containing a Normal Group with objects crossing accessible and inaccessible Containers functions as expected (inaccessible objects not shown, others functioning)
• A Federation containing a Normal Group with objects crossing only inaccessible Containers functions as expected (inaccessible objects not shown)
• A Federation containing a Smart Group can be reloaded
• A Federation containing a Smart Group can be loaded by a User and all Objects are included
• A Federation containing a Smart Group with objects crossing accessible and inaccessible Containers functions as expected (inaccessible objects not shown, others functioning) (https://github.com/3drepo/3D-Repo-Product-Team/issues/851)
• A Federation containing a Smart Group with objects crossing only inaccessible Containers functions as expected (inaccessible objects not shown)
• Groups persist between Container Revisions
• Groups persist between Federation Revisions
• External Id Groups created in a Federation at a given Container Revision continue to work when a new Container Revision has been uploaded
• Views can be saved within a Container and reloaded
  • Default view functions as expected
• Views can be saved within a Federation and reloaded
  • Default view functions as expected
• Views can be saved by with highlighted and colour overridden objects that cover accessible and inaccessible objects and then loaded by User
• Tickets can be saved in Containers
• Tickets persist between Container Revisions
• Tickets can be saved in Federation
• Tickets persist between Federation Revisions
• Tree functions to highlight, show/hide and isolate within Container
• Tree functions to highlight, show/hide and isolate within Federation
• Tree functions to highlight, show/hide and isolate within Federation with mixed permissions for User
• Diff/Clash can open within a Federation
• Diff/Clash can function when used by User with mixed access

[sebjf]

Hi @carmenfan, this is passing all tests now for when you are back!

[carmenfan]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[carmenfan]

looks fine, passing it to @Didi-Kwan for QA

Didi we just changed the way federation is created here - so when you edit a federation it should now be instantly updated, this could affect things like tree/group selections so we'd like a general behaviour check on federations