Volume plugin only - fails with MISSING_URI

[rck]

This happens on xenserver 7.6. After realizing that datapath plugins are more complicated than expected (#82), I tried to implement a Volume plugin only. Creating a VM eventually fails with "MISSING_URI - Please include a URI in the device config". At the time the block device exists and is accessible.

The volume plugin now returns a config with config['uri']=['file:///dev/drbd1000']. I also tried with raw+file://.
The SR was created via:
xe sr-create name-label=foo type=linstor device-config:uri=/existing/dir. I also tried device-config:uri="".

I also did not find any xe sr- command that would allow me to list the device-config. Is there one?

Here is a part of the output in the log.

Feb 11 15:06:46 localhost xapi: [debug|xenserver-210|900 UNIX /var/lib/xcp/xapi|session.slave_login D:9ad2fdc65f9c|mscgen] xapi=>xapi [label="pool.get_all"];
Feb 11 15:06:46 localhost xapi: [debug|xenserver-210|901 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:1f426cbed22b created by task D:9ad2fdc65f9c
Feb 11 15:06:46 localhost xapi: [error|xenserver-210|865 |Async.VM.start R:50261b4abb8b|xenops] Re-raising as MISSING_URI [ Please include a URI in the device-config ]
Feb 11 15:06:46 localhost xapi: [error|xenserver-210|228 |xapi events D:c1a19e961dc9|xenops] events_from_xapi: missing from the cache: [ 0f7738ca-4e1d-41f7-ac7f-1a7151fc93a8
]
Feb 11 15:06:46 localhost xapi: [debug|xenserver-210|865 |Async.VM.start R:50261b4abb8b|xapi] PIF type of e14d435e-99f2-5886-f815-c6e0ed56b7af is: Physical
Feb 11 15:06:46 localhost xapi: [debug|xenserver-210|902 ||mscgen] xapi=>xapi [label="event.from"];
Feb 11 15:06:46 localhost xapi: [error|xenserver-210|865 ||backtrace] Async.VM.start R:50261b4abb8b failed with exception Server_error(MISSING_URI, [ Please include a URI in
the device-config ])
Feb 11 15:06:46 localhost xapi: [error|xenserver-210|865 ||backtrace] Raised Server_error(MISSING_URI, [ Please include a URI in the device-config ])
Feb 11 15:06:46 localhost xapi: [error|xenserver-210|865 ||backtrace] 1/1 xapi @ xenserver-210 Raised at file (Thread 865 has no backtrace table. Was with_backtraces called?,
 line 0

Any hints? What am I doing wrong this time? :)

[gaborigloi]

Hi @rck ,
The scheme of each URI stored under the uri key indicates a specific datapath. It needs to match exactly the directory name of a datapath plugin installed on XenServer.
For example, qdisk://....
Therefore I think it is best to write your own datapath at the moment, because the datapath has to know how to interpret the rest of the URI you return from your volume plugin.
It would be great if there was a generic qdisk datapath that took the path to the qdisk file, that everyone could use, but I've briefly looked at the datapath plugins used in XenServer, and I think they are too tied to the XenServer storage code, and the URI they accept is more complex.
So I think it's best to write your own datapath plugin at the moment, but luckily there is open-source code available that you can use as a starting point. Or maybe you could just use the open-source datapath I linked in #82 as is, I haven't looked into that in detail.
Please let me know if your disagree with this @MarkSymsCtx @stefanopanella @edwintorok .

[rck]

Hm, thanks again for your feedback.

file:// is explicitly mentioned along other supported data paths (i.e, nfs, iscsi,...) in the api doc, so I assumed that would work out of the box.

This is just feedback, but the "promise" is that one can implement a volume plugin only for easy/basic cases. The most basic case I can imagine is a file/blockdevice. If there is no handler provided by the framework for this most basic case, then something is wrong. Handing over a block device is the base case. If a plugin writer then has to go down the rabbit whole of qemu disks, nbd, XenDisks, something is wrong with the framework. Sorry. That is not what I have expected by reading the API docs. Again, please see this as constructive feedback, but I worked on and/or know and/or maintain the integration plugins for OpenStack Cinder, Opennebula, Proxmox, "old" k8s, k8s CSI. Xen provides the worst developer experience so far. The time needed is now at list 3 times as high as for any of the other plugins. Which is a pity, because it on the other hands provides the best API documentation. But implementing it is not enough, you need to have a mega ton of internal Xen knowledge. Sorry, but the purpose of a plugin system is to avoid that :-/. When I as a plugin implementer can go that far to provide a block device, that really should be good enough. One very weird and unique point of the API is that the plugin implementer basically has to implement a KV store (Volume.set()).

So to some existing code: How does the ffs plugin do it? I don't see a datapath implementation, it just returns a "raw+file://". That and the API doc made me think that "file" would be "built in".
https://github.com/xapi-project/ffs/tree/master/volume/org.xen.xapi.storage.ffs

Best and thanks for you help, rck

[gaborigloi]

I think that list of URIs is given as an example. But I agree that it's a bit misleading, that section of the docs should be polished and updated.

Thank you for the feedback. The promise of the docs is true, as far as the framework itself is concerned. If there already exists a datapath plugin for your file (and URI) format, then it is sufficient to implement a volume plugin and the framework will handle the rest.

The problem is that I think currently there is no datapath plugin meant for use by third-parties, as far as I can tell. We are not currently working on one. But this is where the community could help. Maybe the plugin from xcp-ng is already suitable, or could be after some modifications.
The issue with the datapath plugins on XenServer is that they are not meant for use by third-parties, as far as I know, and the URI scheme they accept is not trivial and might change in the future. But I'll try to look into that, maybe you could still use it. But it's the best to use another datapath plugin, if possible.

[gaborigloi]

As @MarkSymsCtx suggested, I have created a new ticket for making a very simple sample datapath plugin that works on block devices and could be a good starting point for datapath plugin developers. It could be useful to you, if your SR exposes the VDIs as files/block devices, instead of qcow2 files.

[rck]

That is very good news and highly appreciated! We are talking about storage here, so not having an easy way to deal with local block devices just feels wrong.

Thanks, rck