uefi: Add safe protocol wrapper for EFI_EXT_SCSI_PASS_THRU_PROTOCOL

[seijikun]

Added a safe wrapper for the extended SCSI Pass Thru protocol.
I did my best to design the API in a way that avoids accidentally using it incorrectly, yet allowing it to operate efficiently.
The ScsiRequestBuilder API is designed in a way that should easily make it possible to use it for both EFI_EXT_SCSI_PASS_THRU_PROTOCOL and a possible future safe protocol wrapper of EFI_SCSI_IO_PROTOCOL.

Exemplary usage to probe all devices potentially connected to every SCSI channel in the system:

Easy variant (io/cmd buffer allocations per request):

// query handles with support for the protocol (one handle per SCSI controller)
let scsi_ctrl_handles = uefi::boot::find_handles::<ExtScsiPassThru>().unwrap();
// Iterate over scsi controllers with passthru support:
for handle in scsi_ctrl_handles {
    // open protocol for controller
    let scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
    // iterate (potential!) devices on the controller
    for device in scsi_pt.iter_devices() {
        // this is not an actual (guaranteed-to-exist) device, but a !!potential!! device.
        // We have to probe it to find out if there is something connected to this chan/target/lun

        // construct SCSI INQUIRY (0x12) request
        let request = ScsiRequestBuilder::read(scsi_pt.io_align())
            .with_timeout(Duration::from_millis(500))
            .with_command_data(&[0x12, 0x00, 0x00, 0x00, 0xFF, 0x00]).unwrap()
            .with_read_buffer(255).unwrap()
            .build();
        // send device through controller to potential device
        if let Ok(response) = device.execute_command(request) {
            println!(
                "SUCCESS HostAdapterStatus: {:?}, TargetStatus: {:?}\r",
                response.host_adapter_status(),
                response.target_status()
            );
            let inquiry_response = response.read_buffer().unwrap();
            println!("ResponseBfr: {:?}\r", inquiry_response);
        } else {
            println!("ERROR - probably not a device\r");
        }
    }
}

Buffer-reuse API variant:

[...]
    // open protocol for controller
    let scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
    // allocate buffers and reuse amongst drives on this SCSI controller
    // It's important this is not shared across SCSI controllers !! Alignment differs
    let mut cmd_bfr = scsi_pt.alloc_io_buffer(6).unwrap();
    cmd_bfr.copy_from(&[0x12, 0x00, 0x00, 0x00, 0xFF, 0x00]);
    let mut read_bfr = scsi_pt.alloc_io_buffer(255).unwrap();
    // iterate (potential!) devices on the controller
    for device in scsi_pt.iter_devices() {
        // this is not an actual devices, but a !!potential!! device.
        // We have to probe it to find out if there is something connected to this chan/target/lun

        // construct SCSI INQUIRY (0x12) request
        let request = ScsiRequestBuilder::read(scsi_pt.io_align())
            .with_timeout(Duration::from_millis(500))
            .use_command_buffer(&mut cmd_bfr).unwrap()
            .use_read_buffer(&mut read_bfr).unwrap()
            .build();
[...]

Checklist

• Sensible git history (for example, squash "typo" or "fix" commits). See the Rewriting History guide for help.
• Update the changelog (if necessary)

[seijikun]

[phip1611]

Thanks for your contribution! I left a few remarks. Further, can you please add an integration test for the new functionality?

[seijikun]

@phip1611 @nicholasbishop
I hope I addressed all opened review comments now.
CI on x86_64 is green now, including the AlignedBuffer unit-test and the integration-test.
For the integration test, I changed the second (FAT32 formatted) disk to SCSI.

Since that change, the qemu process of the aarch64 integration test runner is hard-crashing in the Disk I/O 2 test..
Do you have an idea of what this might be?

EDIT: Was able to fix it by leaving the second disk to the way it was before and instead adding a third disk, then located on a SCSI Controller.

[phip1611]

This is a good PR with a comprehensive improvement! Thanks! LGTM

As this PR is a little bigger than others, I'd like to ask @nicholasbishop to give a second approval

[nicholasbishop]

This one also feels "logicially mut" to me, can we make this mut and ditto for the corresponding uefi wrapper?

[seijikun]

@nicholasbishop @phip1611
I can make the raw API mut, but...:

The entity in the safe uefi wrapper that corresponds to this funcitonality is ScsiDevice<'a> in function ScsiDevice::reset().
And the only way to get an instance of ScsiDevice<'a> is through the device iterator: ExtScsiPassThru::iter_devices().

How would you suggest to make this mut? Have a second ExtScsiPassThru::iter_devices_mut()?

[nicholasbishop]

I think ScsiDevice::execute_command should also be &mut. So maybe there should only be iter_devices_mut, and drop iter_devices (since iter_devices indicates you will need to probe with execute_command anyway). Would that work, or do you see problems in API usage if we require exclusive access?

[nicholasbishop]

Oh, and a general note: even if we decide to make the uefi interface & instead of &mut, we can still use *mut in the raw interface. Making a mut pointer from a non-mut reference is allowed, and doesn't affect soundness in and of itself.

[seijikun]

I just rewrote the API to use mutable and I ran into some ugly problems:

When you call iter_devices_mut() to produce ScsiDevices, you mutably borrow the protocol (ExtScsiPassThru) itself for the lifetime of the generated ScsiDevice. Thus, while a device exists, you can't interact with the protocol itself. That produces all kinds of usability issues:

for mut device in scsi_pt.iter_devices_mut() {
            let request = ScsiRequestBuilder::read(scsi_pt.io_align())
                .with_timeout(Duration::from_millis(500))
                ...

[phip1611]

Thanks for experimenting and analyzing this. I don't have the capacity right not unfortunately to provide good guidance. @nicholasbishop can you help, please?

[nicholasbishop]

I'm looking through this in more detail now :) Mind moving the first commit ("uefi-raw: Add documentation to ScsiIoScsiRequestPacket") to a new PR? Can merge that separately from the rest of the changes.

[seijikun]

Done: #1593

[nicholasbishop]

For SCSI, does the alignment ever exceed 4096? I'm wondering if it would make more sense for callers to allocate memory with boot::allocate_pages, which are always 4K aligned. We might not need AlignedBuffer in that case. (It might still be a helpful interface either way, I just want to see if a simpler solution can work.)

[seijikun]

Main idea behind the AlignedBuffer struct was, that I don't have to think if dealloc-ing a buffer I allocated myself.
I disliked having the user copy over the io_align myself - but I'd be too scared to make assumptions as to what can be expected as possible values.

[seijikun]

Instead of letting a user construct the request builder freely, we could instead add something like:

pub enum ScsiRequestDirection { READ, WRITE, BIDIRECTIONAL }

And then add the following method to ExtScsiPassThru for starting new requests:

pub fn start_request(&self, direction: ScsiRequestDirection) -> ScsiRequestBuilder {}

That would then avoid the user ever coming into direct contact with io_align.

So something like this:

let scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
let request = ScsiRequestBuilder::read(scsi_pt.io_align())
    .with_timeout(Duration::from_millis(500))
    .build();

becomes:

let scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
let request = scsi_pt.start_request(ScsiRequestDirection::READ)
    .with_timeout(Duration::from_millis(500))
    .build();

Then we are more free w.r.t. changing the buffer logic. I would prefer to keep the AlignedBuffer struct for that, though.

[seijikun]

@nicholasbishop I think I might have a nice solution to the mutable / immutable problem. Please have a look at 5a44ef4

Usage looks like this:

let scsi_ctrl_handles = uefi::boot::find_handles::<ExtScsiPassThru>().unwrap();
for handle in scsi_ctrl_handles {
    //     v  does not require mutable
    let scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
    //  ScsiDevices returned from the device iterator are owned structs (no &ref, no &mut ref)
    //   v requires mutable            v does not require mutable
    for mut device in scsi_pt.iter_devices() {
        println!("- Disk: {:?} | {}\r", device.target(), device.lun());

        let request = ScsiRequestBuilder::read(scsi_pt.io_align())
            .with_timeout(Duration::from_millis(500))
            .use_command_data(&[0x06, 0x00, 0x00, 0x00, 0x00]).unwrap()
            .with_read_buffer(255).unwrap()
            .build();
        //             v requires device to be mutable
        let result = device.execute_command(request).unwrap();
    }
}

Internally, ScsiDevice only has a *const pointer (otherwise the Clone derive doesn't work).
And on actions that require a *mut pointer, I cast it.

What do you think about this?

[nicholasbishop]

I'm wondering if some of the lifetime complexity could be reduced by moving the methods of ScsiDevice to ExtScsiPassThru. So instead of ScsiDevice being a "smart" object that you can call execute_command (and reset) on, you would call execute_command on the protocol, and pass the ScsiTargetLun as an argument.

Would that work? If so, are there downsides I'm not thinking of?

[seijikun]

Yes, that would eliminate the lifetime problems. If you prefer that, I can refactor it.

[nicholasbishop]

Yes, as long as it doesn't cause other issues I think that would be a good refactor. Fewer lifetimes, and avoiding internal pointers and Phantom markers feels worth it to me, to make it less likely that the code is accidentally unsound.

[seijikun]

grr
I just did the refactoring, and that architecture produces the same lifetime problems.
I can not send commands within an iterator, because the device iterator borrows the protocol.

let mut scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
                // v borrows scsi_pt immutably
for device in scsi_pt.iter_devices() {
    let request = ...;
    let result = scsi_pt.execute_command(&device, request);
                // ^ attempts to borrow scsi_pt mutably
}

[nicholasbishop]

Ah right. What about collecting the iterator into a Vec first?

[seijikun]

Yes, that works (just tested it). Although it's an additional allocation. On the other hand, we require alloc for this anyway.
Your call.

let mut scsi_pt = uefi::boot::open_protocol_exclusive::<ExtScsiPassThru>(handle).unwrap();
let devices: Vec<_> = scsi_pt.iter_devices().collect();
for device in devices {
    let request = ...;
    let result = scsi_pt.execute_command(&device, request);
    // ...
    // profit
}

---

One minor caveat: ScsiDevices are then nolonger bound to a certain protocol. So you could accidentally use an ScsiDevice instance on any protocol instance.

let device = scsi_pt.iter_devices().next().unwrap();

and then run:

other_scsi_pt.execute_command(&device, request);

[seijikun]

@nicholasbishop
My personal favorite stays the smart object design I suggested. It has several advantages:

• By far the best usability (it is intuitive - you don't have to search what to do with the objects returned from the iterator)
• It avoids an unnecessary allocation (no iter_devices().collect())
• It actually makes sense, in that:
  • it does not allow you to borrow the protocol instance mutably while you have ScsiDevice instances alive. So you can e.g. not reset the entire channel with ScsiDevice's borrowing the protocol immutably
  • it's not possible to accidentally use a ScsiDevice instance with another ExtScsiPassThru protocol instance

I just pushed a version without internal pointer and without PhantomData usage - thus alleviating some of your concerns with it.