API v2 endpoints for managing CoCalc as an admin

[haraldschilly]

The following list of desired activities might already be implemented, in some way or another. The goal is to make it possible to do this via the API (v2). If applicable, I'm mentioning how to do this via the UI.

• list accounts: similar to searching for accounts in Admin → User Search, there should be a way to get all accounts, with a limit like 1000 and pagination, and optionally them by email, (partial) name or account_id (i.e. then this just returns 0 or 1 entries).
  • each output should contain a few basic facts: account_id, first_name, last_name, "user name", email_address, deleted, banned, last_active, created

For a given account_id:

• update an account: allow admins to change the first_name, last_name, email_address (+ a check if it exists, and throws an error if the new email address already exists), and that "user name"
• banning an account: set/clear the "ban" flag
• get projects belonging to an account: for a given account_id, list all projects that user is a collaborator of. For simplicity, I think it should be like in the admin interface (which also lists projects for a given account) and list these projects with project_id, title, list of collaborator IDs (?), last_active, created, name, and state (i.e. is the project running or not).
• deleting an account: set/clear the "delete" flag

For a given project_id:

• updating a project: allow admins to the change the project's title, description, name, etc. i.e. these basic fields.
• manage licenses: A bit more complex is adding/removing licenses.
  • list licenses (their IDs) applied to a project
  • add/remove a license_id
• manage collaborators:
  • list collabs on a project (like, list accounts) but by belonging to a project
  • add/remove an account_id as a collaborator
• stopping a project: this the analog of "project stop" in the UI
• starting a project: similar to start
• blocking a project: I think that's not exposed anywhere, but there is a "block" flag for projects (which causes them to stop and make it impossible to start them again)
• deleting a project: setting the delete flag

License management:

I don't know how much of this is implemented already. In the Admin panel, there is a way to create/edit/delete licenses. This is similar to accounts, in some sense.

• list licenses: either paginated, or with a filter (by title/description/license_id)
• create license: this does not need to be complex. Just a way to create one, and returning the ID. The data for a license contains its title/description, configuration, start, and (optional) end date, and a run_limit (at least 1). There is also a "manager" of a license, which means there are account_id's to add or remove.
• edit license: probably similar to create
• delete/invalidate: licenses shouldn't be deleted, otherwise it might break assumptions. In any case, invalidating means to set the end-date to the current time (because outside the start/end date range, a license is invalid).

[schrodingersket]

Currently going through the API to see what we already have. Here's what I think is the current state (happy to have feedback from William or Harald on this in case I missed/misinterpreted something):

• List Accounts: This endpoint is very similar to what's needed for this, but requires some search parameters. Perhaps the best bet for this one would be to simply add pagination and results limits and call it a day.
• Update Account: We have an endpoint to fetch a user's profile, but it cannot currently update it. I think a new endpoint is necessary here. Edit: PR here
• Ban/Unban: Done here and here.
• List Account Projects: No endpoint exists. PR here
• Delete Account: Done here.
• Project Update: No endpoint exists. PR here
• Project Collaborator Management: No endpoint exists. PR here
• License Management: The converse is implemented (i.e., fetching all projects associated with a given license), but we do not yet have an endpoint with which to fetch, add, and remove licenses associated with a project. The first of these can be added to the /licenses/get-licenses endpoint.
• Project Start/Stop: Completed here and here.
• Project Block: No endpoint exists.
• Delete Project: No endpoint exists.
• List Licenses: No endpoint exists. Individual licenses may be queried. Note that when this is implemented, it should support filtering by project id in order to support the License Management requirement.
• Create License: No endpoint exists.
• Edit License: No endpoint exists.
• Invalidate License: No endpoint exists. This could be completed either via the Edit License API endpoint, or by calling the same backend code as that endpoint but with the license end time set to the current time.

[williamstein]

Perhaps the best bet for this one would be to simply add pagination and results limits and call it a day.

That would be very nice to have anyways. Of course, also add the ability to use an empty search (for admins), if it isn't there already.

Update account

What does this mean? You can update anything except email address through the user_query mechanism.

List Account Projects

This is currently done via user_query and maintained on the client right now via a "changefeed" (LISTEN/NOTIFY) that was tricky to make fast. The main issue here is that some users can have 10K+ projects, so some care must be taken. The actual postgresql query is slightly tricky, because it's a join involving JSONB, and we don't want it to be slow or overload the database. But indeed a different version of this with paging would be more straightforward and good to add!

Project update

again, can do via user_query

we do not yet have an endpoint with which to fetch, add, and remove licenses associated with a project.

I think we do, but it's maybe just via user_query. I.e., you can look in the frontend code that literally does add/remove etc. licenses and see what it does. Remember that any new endpoint you add has potentially a lot of subtle maintenance implications (and also abuse), e.g., when adding a license to a project, typically the project needs to be restarted. These should all be handled via the user_query codepath, so you MUST be aware of that if you add another endpoint (and make sure there is only one thing in the code doing the actual work).

Project Collaborator Management: No endpoint exists.

It does exist, but maybe it is a v1 endpoint? Again, I would look at our frontend client code that does add/remove collaborators and see what's up with it.

It could be you end up writing numerous endpoints that are just lightweight wrappers around user_query.

Project Block: No endpoint exists.

This could be nice to have through the UI. Right now we do this via some CLI scripts on the backend that directly edit the database. It would also be nice if the frontend UI told users if their project was blocked. We use this entirely for temporary subtle maintenance issues.

Delete Project: No endpoint exists.

Again, it does, via user_query.

List Licenses: No endpoint exists. Edit License: No endpoint exists.

It does, via user_query.

The user_query functionality is basically like a "GraphQL API" for cocalc. We now need a traditional HTTP style api... but I think the way to do that is in some cases to just proxy calls that go to user_query, instead of trying to do the same thing as user_query again, since the user_query code has various hooks builtin to deal with what constraints.

[schrodingersket]

Sounds like user_query is going to be my best friend for a few days. (:

[williamstein]

The bad news - it is basically the last thing still written in coffeescript..:

…

-- William Stein

On Mon, Jul 22, 2024 at 8:16 AM Jonathan Thompson ***@***.***> wrote: Sounds like user_query is going to be my best friend for a few days. (: — Reply to this email directly, view it on GitHub <#7665 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJXS5QJOSQYJBG3VE4Q6VDZNUO37AVCNFSM6AAAAABKMWNEPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGIYTEMJQGQ> . You are receiving this because you commented.Message ID: ***@***.***>