Skip to content

Commit

Permalink
[omdb] Add db vmm list and db vmm info (#6936)
Browse files Browse the repository at this point in the history
`omdb db` has commands for listing and displaying details regarding
customer VM instances. These commands reference the Propolis VMM
processes whose states are recorded in the `vmm` table, including their
IDs. However, there are no commands for directly querying that table,
which is unfortunate. While debugging an issue on the colo rack last
week, @augustuswm and I wanted to be able to list all VMMs in the
`Failed` state, but the only way to do this was to find a CockroachDB
zone and run our own manual SQL queries. Debugging this type of issue
would be easier if there were OMDB commands for querying the VMM table.

This commit adds two new commands to OMDB, `omdb db vmm list` (aliases
`db vmm ls`, `db vmms`), which lists entries from the `vmm` table
(potentially filtered by state), and `omdb db vmm info` (alias
`db vmm show`), which looks up a VMM record by UUID and displays its
contents. In addition, I factored out some code for displaying VMM
records
that was shared between the `db vmm info` command and the
`db instance info` command.

Closes #6928

## Examples

<details open>
<summary>Querying a VMM record with `omdb db vmm info`:</summary>

```console
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm info 040678cf-2e31-4bc6-baa2-79b20f40ca83 2>/dev/null

== VMM =========================================================================
                  ID: 040678cf-2e31-4bc6-baa2-79b20f40ca83
         instance ID: 8ee8bf04-9435-49bb-a8e7-b1368315359a
          created at: 2024-10-23 20:02:34.980041 UTC
               state: running
          updated at: 2024-10-23T20:02:50.078970Z (generation 4)
    propolis address: fd00:1122:3344:104::1:417:12400
             sled ID: 0c7011f7-a4bf-4daf-90cc-1c2410103300
         sled serial: BRM42220057

== SLED RESOURCE RESERVATIONS ==================================================
    hardware threads: 2
             RSS RAM: 0 B
       reservoir RAM: 8 GiB

root@oxz_switch1:~#
```

</details>
<details open>
<summary>Listing VMMs with `omdb db vmm list`:</summary>

```console
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm ls 2>/dev/null
INSTANCE_ID                          ID                                   STATE    GEN SLED
8ee8bf04-9435-49bb-a8e7-b1368315359a 040678cf-2e31-4bc6-baa2-79b20f40ca83 running  4   BRM42220057
6dd05e98-7a36-4799-8a0b-97b7ab101a05 6ff33c2d-3291-4d61-a3b5-43e8203c1b41 running  4   BRM42220057
6ead53ad-e997-4b85-86b8-342905a68ada ae6e42a2-37af-4bbe-80b2-e4515482018d running  4   BRM42220057
1c8803ac-d1a1-4e2f-a827-fca538d5cbb3 c24058bd-c747-44dd-906a-d2d1bc46755d running  4   BRM42220057
228b79fd-b4ff-4f50-97a5-286c949e695d c70b5536-8bdc-4bdf-810b-4d3836ea2947 running  4   BRM42220057
e97e9fb9-62c3-4745-9d91-b0b6fa2baeba fa525216-9f81-4a8f-8fca-f7858994dce2 starting 3   BRM42220057
5fad8b64-1631-4c7e-8cb1-ed089830d406 6e637a89-2bb1-4327-89ac-6f55757fd3ec running  4   BRM44220011
16620dff-d4e5-47ba-b11d-34c59fb01206 770f0d5f-5f87-4d8d-a4d7-7b16602f2408 running  4   BRM44220011
33e6d4a4-6cc7-440f-8a9f-152d94136191 31837146-7de7-4317-9072-cd448b95f56c running  4   BRM42220017
ad5a6c89-2845-4c2e-b247-8ca034e10597 92c59a48-f56e-48f0-9792-88a664072a26 running  4   BRM42220017
0ec2a7ac-fcc5-4bf4-9adc-f05c6823ee93 ca5d77ff-8c37-4740-86fd-89c855fb5e5b running  4   BRM42220017
09bbbb51-757c-4cfc-ac25-b619b8523981 4e65bb95-29ca-4063-b4a4-ef73521a8185 running  4   BRM42220051
1a0effb3-1338-409f-a8ad-99583203dd7e 5f521878-98f3-441a-9674-69987e29666a running  4   BRM42220051
9ecfb1d7-0526-47a9-8b58-2f7d639174ef 62366fdc-3ed6-41e4-8049-dd1b5be29fae running  4   BRM42220051
eb3a39f5-6657-487b-bfb6-5e540ff07b35 a8e83499-eebc-4e4f-a05e-4b24049304e7 running  4   BRM42220051
764977e8-5e6d-4e62-ad72-82bd55948289 dfaabeb6-a621-46f3-ad65-740d6732c045 running  4   BRM42220051
fa658925-3e4d-4e76-b839-53a62beb4f5d 446be12a-2cb9-4b16-aaa7-31720c054304 running  4   BRM44220010
ac595ce5-3a9a-4bb0-8daf-f64d09686525 d6f8970e-8b24-4c6b-8b49-6c8bcefe3d83 running  4   BRM44220010
6b252c74-548c-4f1b-a98a-6c0e2da02c09 0672e7ff-918c-4fcb-b8b7-04d66a3e10dd running  4   BRM42220014
cdffcf35-6ae3-488d-a03d-64cf45f88fb2 5c67e0e6-38b6-4fce-ab08-8e0d348b86ee running  4   BRM42220014
db4aca25-6828-4d7e-8744-83336110e82b bf8faa13-5c4a-4e21-86a8-e2fa6d198c3a running  4   BRM42220014
9e2451c0-6eec-4b90-b49a-6a9a2e02f35f 28dce81a-cf61-4001-a5da-78349926521f running  4   BRM42220031
2cae6a66-1e62-4403-98d0-a5384726cc91 80e80ef2-9948-45ce-9852-eda7d3fc7723 running  4   BRM42220031
18cdcbf1-41b3-4f8c-8395-d8191c916fb3 ceed88d0-6a3e-4ff9-abf7-ce1162631c34 running  4   BRM42220031
ea20be02-a257-467b-83e7-f16f4db68f0e 4fe59185-cb8f-498d-b17f-6b6bfdf92591 running  4   BRM42220006
1cd1945f-c86b-47eb-8b0e-5e50f7b1ea24 db150e5d-dd71-4ccc-a79d-d9188e6ab3ff running  4   BRM42220006
923477a5-1183-48aa-80cc-bac548521f31 ec1b6428-6f1d-400f-aba1-1a442ea55295 running  4   BRM42220006
e3795c9a-1009-4b4b-a706-a94987220a7e ee25dd9f-244b-4512-9417-16dc0aa114e6 running  4   BRM42220006
372b637b-a583-499d-8ac0-21dd6cc8e1ba 99393822-556c-4e09-b3a6-4f5c2750fcce running  4   BRM42220016
b30ae7ac-9092-4c01-8586-23ed388d2363 195138f1-6036-4a44-ad83-c2cfae871641 running  4   BRM44220005
18c8ee75-b51e-4d9e-bfc5-f9d31ee27184 2ac06d55-b2f9-4f66-9eb0-a76e5f77812d running  4   BRM44220005
f93288a6-acbb-4396-a4da-d5a33e189577 79aed6d7-4e47-436a-ad61-8caf86a163ba running  4   BRM44220005
1669ecf5-c3ef-430f-aabc-eb010d5fda39 92a82706-8bc9-48da-8af1-23f91104f8be running  4   BRM44220005
d07b0fcc-17d0-479c-9688-1bc5731f0d6a c44e9370-5af6-400d-b05f-eb038892c3dc running  4   BRM44220005
c9e210b9-037e-4433-b904-1a20ac7052a6 d6a159d4-d13c-4115-a340-3bf359548783 running  4   BRM44220005
b0db674c-de4f-49df-bdce-965632f61915 d6e699ad-bb6a-47b0-a772-864e94dd09b8 starting 3   BRM44220005
ece2768b-2820-463e-b92c-268eaa42c27e d8a57699-1be7-41bf-9baf-a590e626ac37 running  4   BRM44220005
```

</details>
<details open>
<summary>Listing VMMs by state:</details>

```console
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm ls 2>/dev/null --state starting
INSTANCE_ID                          ID                                   STATE    GEN SLED
e97e9fb9-62c3-4745-9d91-b0b6fa2baeba fa525216-9f81-4a8f-8fca-f7858994dce2 starting 3   BRM42220057
b0db674c-de4f-49df-bdce-965632f61915 d6e699ad-bb6a-47b0-a772-864e94dd09b8 starting 3   BRM44220005
```

</details>
<details>
<summary>Listing "failed" or "destroyed" VMMs suggests adding
`--include-deleted`, because VMMs in those states may be
soft-deleted:</summary>

```console
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm ls --state failed
note: database URL not specified.  Will search DNS.
note: (override with --db-url or OMDB_DB_URL)
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using database URL postgresql://root@[fd00:1122:3344:109::3]:32221,[fd00:1122:3344:105::3]:32221,[fd00:1122:3344:10b::3]:32221,[fd00:1122:3344:107::3]:32221,[fd00:1122:3344:108::3]:32221/omicron?sslmode=disable
WARN: found schema version 110.0.0, expected 111.0.0
It's possible the database is running a version that's different from what this
tool understands.  This may result in errors or incorrect output.
WARN: VMMs in the `Failed` state may have been deleted, but `--include-deleted` was not specified
INSTANCE_ID ID STATE GEN SLED
08T04:41:51.913Z 2024-10-08T04:41:51.913Z
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm ls --state failed --include-deleted --fetch-limit 10
note: database URL not specified.  Will search DNS.
note: (override with --db-url or OMDB_DB_URL)
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using database URL postgresql://root@[fd00:1122:3344:109::3]:32221,[fd00:1122:3344:105::3]:32221,[fd00:1122:3344:10b::3]:32221,[fd00:1122:3344:107::3]:32221,[fd00:1122:3344:108::3]:32221/omicron?sslmode=disable
WARN: found schema version 110.0.0, expected 111.0.0
It's possible the database is running a version that's different from what this
tool understands.  This may result in errors or incorrect output.
WARN: loading VMMs: found 10 items (the limit).  There may be more items that were ignored.  Consider overriding with --fetch-limit.
INSTANCE_ID                          ID                                   STATE  GEN SLED        TIME_DELETED             TIME_DELETED
740c1640-f393-4336-8b2f-94f6a4cb7739 009555fe-cbbc-4685-a3a0-a9421d9eca35 failed 5   BRM42220017 2024-10-11T00:15:55.583Z 2024-10-11T00:15:55.583Z
eb3a39f5-6657-487b-bfb6-5e540ff07b35 00aa5a57-91e5-4a1a-bd7e-677a2b53d272 failed 5   BRM42220017 2024-10-22T15:57:42.972Z 2024-10-22T15:57:42.972Z
53b5ddc3-31f2-4bd4-8fb3-5f3a6d895ee1 031bdbcb-d8ed-4eef-bbe8-717d97141162 failed 6   BRM42220051 2024-10-11T00:16:03.142Z 2024-10-11T00:16:03.142Z
49520f14-e4f9-49b7-ad41-cea1f3ac0deb 0330bafd-e27c-44ed-855b-155ab8803821 failed 6   BRM42220051 2024-10-02T18:19:07.577Z 2024-10-02T18:19:07.577Z
ca808ed8-01d9-49d4-9f5b-578af1720ab2 02720bfe-84a5-4185-a2b6-319fce430b6c failed 6   BRM44220010 2024-10-11T00:16:12.615Z 2024-10-11T00:16:12.615Z
060589b6-f22b-47f4-a27b-666c49990a5b 0342228b-e590-4bb9-a1de-98848494942a failed 5   BRM42220031 2024-10-08T04:41:50.745Z 2024-10-08T04:41:50.745Z
9fa2efa0-a2a5-4ce0-98ce-5de4dd7166ee 036740c5-88a1-4ed7-8340-cc972ce0e96b failed 6   BRM42220031 2024-10-09T04:32:35.365Z 2024-10-09T04:32:35.365Z
9e2451c0-6eec-4b90-b49a-6a9a2e02f35f 00211b27-b9a8-4771-b608-3f8fb117c4f7 failed 5   BRM42220006 2024-09-29T22:14:09.703Z 2024-09-29T22:14:09.703Z
a2dbe0a0-c9ea-43df-b08c-7c418da21fcf 045df015-3229-4454-87ea-f88b593c5746 failed 6   BRM42220006 2024-10-02T18:19:44.825Z 2024-10-02T18:19:44.825Z
0b492c84-c1b1-45a0-b8fc-bb5b4f79bac6 01e1d5c9-e564-4390-bc8e-cc45b9e98134 failed 5   BRM42220006 2024-10-07T05:41:46.991Z 2024-10-07T05:41:46.991Z
```

</details>
<details>
<summary>`--verbose` includes more contents in the table (for wide
terminal windows/small font sizes!):</summary>

```console
root@oxz_switch1:~# /var/tmp/omdb-eliza-vmms-6 db vmm ls --verbose --fetch-limit 5
note: database URL not specified.  Will search DNS.
note: (override with --db-url or OMDB_DB_URL)
note: using DNS server for subnet fd00:1122:3344::/48
note: (if this is not right, use --dns-server to specify an alternate DNS server)
note: using database URL postgresql://root@[fd00:1122:3344:109::3]:32221,[fd00:1122:3344:105::3]:32221,[fd00:1122:3344:10b::3]:32221,[fd00:1122:3344:107::3]:32221,[fd00:1122:3344:108::3]:32221/omicron?sslmode=disable
WARN: found schema version 110.0.0, expected 111.0.0
It's possible the database is running a version that's different from what this
tool understands.  This may result in errors or incorrect output.
WARN: loading VMMs: found 5 items (the limit).  There may be more items that were ignored.  Consider overriding with --fetch-limit.
INSTANCE_ID                          ID                                   STATE   GEN SLED        SLED_ID                              ADDRESS                           TIME_CREATED             TIME_UPDATED
8ee8bf04-9435-49bb-a8e7-b1368315359a 040678cf-2e31-4bc6-baa2-79b20f40ca83 running 4   BRM42220057 0c7011f7-a4bf-4daf-90cc-1c2410103300 [fd00:1122:3344:104::1:417]:12400 2024-10-23T20:02:34.980Z 2024-10-23T20:02:50.078Z
6dd05e98-7a36-4799-8a0b-97b7ab101a05 6ff33c2d-3291-4d61-a3b5-43e8203c1b41 running 4   BRM42220057 0c7011f7-a4bf-4daf-90cc-1c2410103300 [fd00:1122:3344:104::1:41b]:12400 2024-10-23T20:15:51.922Z 2024-10-23T20:16:05.275Z
6ead53ad-e997-4b85-86b8-342905a68ada ae6e42a2-37af-4bbe-80b2-e4515482018d running 4   BRM42220057 0c7011f7-a4bf-4daf-90cc-1c2410103300 [fd00:1122:3344:104::1:418]:12400 2024-10-23T20:02:35.211Z 2024-10-23T20:07:25.195Z
1c8803ac-d1a1-4e2f-a827-fca538d5cbb3 c24058bd-c747-44dd-906a-d2d1bc46755d running 4   BRM42220057 0c7011f7-a4bf-4daf-90cc-1c2410103300 [fd00:1122:3344:104::1:41a]:12400 2024-10-23T20:06:22.391Z 2024-10-23T20:07:24.170Z
228b79fd-b4ff-4f50-97a5-286c949e695d c70b5536-8bdc-4bdf-810b-4d3836ea2947 running 4   BRM42220057 0c7011f7-a4bf-4daf-90cc-1c2410103300 [fd00:1122:3344:104::1:419]:12400 2024-10-23T20:06:06.612Z 2024-10-23T20:07:35.934Z
root@oxz_switch1:~#
```

</details>
  • Loading branch information
hawkw authored Oct 28, 2024
1 parent e55a518 commit 2c51979
Show file tree
Hide file tree
Showing 4 changed files with 576 additions and 57 deletions.
Loading

0 comments on commit 2c51979

Please sign in to comment.