Add design for dotnetup installation tracking#52834
Add design for dotnetup installation tracking#52834dsplaisted wants to merge 2 commits intodotnet:release/dnupfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a design document that describes how dotnetup will track installations, handle updates, and support uninstalls. The design is based on a discussion from the runtime installation design PR (#52409) and proposes using a shared manifest to track install specifications and installed components.
Changes:
- Added a new design document describing the installation tracking mechanism for dotnetup
- Defined the structure for install specs, installations, and subcomponents in the shared manifest
- Outlined the implementation approach for install, update, delete, and garbage collection operations
|
|
||
| ## Desired behavior | ||
|
|
||
| When a user installs a .NET SDK or runtime with dotnetup, we will call the information about what the requested to be installed the "Install Spec". This includes the component that should be installed as well as the version or channel that should be installed. An install spec may also be derived from a global.json file, in which case the spec should also include the path to the corresponding global.json. |
There was a problem hiding this comment.
Grammatical error: "what the requested to be installed" should be "what they requested to be installed" or "what was requested to be installed".
| When a user installs a .NET SDK or runtime with dotnetup, we will call the information about what the requested to be installed the "Install Spec". This includes the component that should be installed as well as the version or channel that should be installed. An install spec may also be derived from a global.json file, in which case the spec should also include the path to the corresponding global.json. | |
| When a user installs a .NET SDK or runtime with dotnetup, we will call the information about what was requested to be installed the "Install Spec". This includes the component that should be installed as well as the version or channel that should be installed. An install spec may also be derived from a global.json file, in which case the spec should also include the path to the corresponding global.json. |
|
|
||
| ### Subcomponent | ||
|
|
||
| Subcomponents are sub-pieces of an installation. We need to represent these because different installed components or versions may have overlapping subcomponents. So for each installation, we will keeep a list of subcomponents that are part of that installation. |
There was a problem hiding this comment.
Spelling error: "keeep" should be "keep".
| Subcomponents are sub-pieces of an installation. We need to represent these because different installed components or versions may have overlapping subcomponents. So for each installation, we will keeep a list of subcomponents that are part of that installation. | |
| Subcomponents are sub-pieces of an installation. We need to represent these because different installed components or versions may have overlapping subcomponents. So for each installation, we will keep a list of subcomponents that are part of that installation. |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
||
| ### Installing a component | ||
|
|
||
| - Is there already already a matching install spec in the shared manifest? |
There was a problem hiding this comment.
Interesting question: How do we define 'matching?' Is 9.0.101 'matching' an explicit install spec which was a version, '9.0.102'? Or do we only do exact match ('latest' == 'latest'), ('9.0' == '9.0') Do we allow installing 9.0.101 if 9.0.102 is installed? The admin installer made the decision to not allow this, though I wonder if @joeloff has the context as to why. (probably downgrade attack prevention)
I think for this tool we probably do an exact match.
If I track an install 'spec' of '9.0.1xx' and '9.0' it probably means I want 9.0.1xx and then the latest 9.0 version.
There was a problem hiding this comment.
In this case I think it would be an exact match. If you have a spec for 'latest' and then you request an install of '11.0', then even if those resolve to the same version then it should add the new spec. Likewise if you have a spec for 10.0 and then you ask it to install 10.0.105 - it should add the new spec.
Specs can also differ based on whether they came from global.json or not. If there's a global.json spec for 10.0.105 and you request an install for 10.0.105, it should also add the new spec.
|
|
||
| ## Implementation | ||
|
|
||
| ### Installing a component |
There was a problem hiding this comment.
We may want to note that all operations against the manifest shall require a lock, or we should think about the designing the manifest in a way that enables concurrency more effectively (e.g. last writer wins.)
There are many ways to design that lock:
- Simplest : Doing any installation at all requires holding the same lock
- Directory Specific Lock - the specific hive/directory determines the lock - multiple readers/writers to the same manifest but they are working in different directories.
- Install version specific lock - enable concurrency for numerous installs across process (or instead across 1 install)
The check for which installs exist should never try to operate without holding the lock as well, so another interesting conversation is whether we want to hold read/write specific locks. (e.g. read only lock checks existing installs, but we can't get the read only lock if the write lock is taken.)
There was a problem hiding this comment.
I think we would start simply, by having an exclusive lock. I don't know if there's much user need for multiple installs happening at once, and that would make it a lot more complicated.
We may want a reader/writer lock though so that multiple windows can list what's installed at once. But maybe it's OK if we don't, the commands should execute quickly so it would be OK if they were serialized.
| - Component | ||
| - Version (this is the exact version that is installed) | ||
| - Dotnet root | ||
| - Subcomponents |
There was a problem hiding this comment.
Architecture is also important here
There was a problem hiding this comment.
Good point. I haven't figured out how we'll handle architecture. I think probably each dotnet root is architecture-specific, so maybe the architecture is stored or associated with the dotnet root.
| ### Installation | ||
| - Component | ||
| - Version (this is the exact version that is installed) | ||
| - Dotnet root |
There was a problem hiding this comment.
What is the advantage of recording dotnet root here?
There was a problem hiding this comment.
I see it now but remark below that I think we should remove this
| - Component | ||
| - Version (this is the exact version that is installed) | ||
| - Dotnet root | ||
| - Subcomponents |
There was a problem hiding this comment.
I believe we may also want to record the hive in which it was installed so we can properly support --directory installs. That could point to the global json path instead to reduce the amount of properties and indicate the hive is controlled by the global.json (either default hive if no sdks path specified or that path in the global.json)
| - For install specs that came from a global.json file, update the versions in them if the global.json file has changed. Delete those specs if the global.json file has been deleted (or no longer specifies a version). | ||
| - For each install spec, find the latest installation record that matches. Mark that installation record to keep for this garbage collection. | ||
| - Delete all installation records from the manifest which weren't marked. | ||
| - Iterate through all components installed in the dotnet root. Remove any which are no longer listed under an installation record in the manifest. |
There was a problem hiding this comment.
If dotnet_root is set to a path including installs also managed by the user, this may cause us to delete those installs. I think we likely shouldn't do this step.
There was a problem hiding this comment.
Several related issues which I haven't really figured out or included in this design:
- Should we include data for multiple dotnet roots in the same manifest or have a different manifest for each root? Having everything in one manifest would let us keep track of all installs centrally. But maybe it would be better to have the manifest in each dotnet root so that the paths can be relative and things don't break if you move the dotnet root folder around.
- Should we support installing into folders with an installation that was not tracked by dotnetup? We probably wouldn't be able to handle upgrades and installations well. Some options:
- Don't support installing into folders with existing non-dnup installs at all
- Install, but only in an "untracked" mode which lays down the files but doesn't add them to a manifest and doesn't support update and uninstall.
- Create a manifest and add information to it about all the existing files. Those files should never be deleted as part of garbage collection (unless the user chooses to delete all the pre-existing installs).
|
|
||
| - Go through all install specs in the manifest. | ||
| - For install specs that came from a global.json file, update the versions in them if the global.json file has changed. Delete those specs if the global.json file has been deleted (or no longer specifies a version). | ||
| - For each install spec, find the latest installation record that matches. Mark that installation record to keep for this garbage collection. |
There was a problem hiding this comment.
I like this approach to simplify the algorithm for deleting old versions!
| - `packs/Microsoft.AspNetCore.App.Ref/10.0.2` - 3 levels | ||
| - `sdk-manifests/10.0.100/microsoft.net.sdk.android/36.1.2` - 4 levels | ||
|
|
||
| ## Implementation |
There was a problem hiding this comment.
A manifest version should also likely be included for compat / breaking changes.
|
|
||
| ## Dotnetup shared manifest contents | ||
|
|
||
| ### Install specs |
There was a problem hiding this comment.
We should likely provide a json schema for the manifest file as well.
| - `packs/Microsoft.AspNetCore.App.Ref/10.0.2` - 3 levels | ||
| - `sdk-manifests/10.0.100/microsoft.net.sdk.android/36.1.2` - 4 levels | ||
|
|
||
| ## Implementation |
There was a problem hiding this comment.
Question: Do we fail if the manifest doesn't exist? I think we don't. We can try to recreate the manifest from a hive but it will never be the same - not including custom directories nor the intent of a user (e.g. explicit runtime install request vs sdk install runtime)
I also wonder if we should keep backup copies of the manifests, say in the temp folder, in case something goes wrong so we can restore the manifest. Then we get to the interesting question of 'history' like workload history but we don't need to do that for now 😁 This is probably better
2 participants
Based on discussion we had on the runtime installation design, here's a proposal for how dotnetup should handle installation tracking, updates, and uninstalls.