dxtbx.format.FormatNXmxDLS16M.FormatNXmxDLS16M no longer supports legacy data without VDS

[graeme-winter]

Or it partially supports it:

ws448 x :) $ dials.python fail.py 
 28%|██████████▊                            | 1000/3600 [01:37<04:14, 10.21it/s]
Traceback (most recent call last):
  File "/tmp/x/fail.py", line 11, in <module>
    i.get_raw_data(j)
  File "/dls_sw/apps/dials/dials-v3-17-0-2/modules/dxtbx/src/dxtbx/format/FormatMultiImage.py", line 47, in read
    return format_instance.get_raw_data(index)
  File "/dls_sw/apps/dials/dials-v3-17-0-2/modules/dxtbx/src/dxtbx/format/FormatNXmx.py", line 115, in get_raw_data
    raw_data = dxtbx.nexus.get_raw_data(
  File "/dls_sw/apps/dials/dials-v3-17-0-2/modules/dxtbx/src/dxtbx/nexus/__init__.py", line 578, in get_raw_data
    sliced_outer = data[index]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/dls_sw/apps/dials/dials-v3-17-0-2/conda_base/lib/python3.10/site-packages/h5py/_hl/dataset.py", line 768, in __getitem__
    return self._fast_reader.read(args)
  File "h5py/_selector.pyx", line 361, in h5py._selector.Reader.read
  File "h5py/_selector.pyx", line 151, in h5py._selector.Selector.apply_args
IndexError: Index (1000) out of range for (0-999)

where fail.py consists of

from tqdm import tqdm

from dxtbx.model.experiment_list import ExperimentList

el = ExperimentList.from_file("imported.expt")

e = el[0]
i = e.imageset

for j in tqdm(range(len(i))):
    i.get_raw_data(j)

[graeme-winter]

Annoying that I did not leave a note of where such data are 🤬

[graeme-winter]

OK, reproduced it today:

/dls/mx-scratch/gw56/data/i03-eiger-overload/

Much fail:

  File "/dls_sw/apps/dials/dials-v3-23-0/modules/cctbx_project/libtbx/scheduling/result.py", line 119, in __call__
    self.traceback( exception = self.exception() )
  File "/dls_sw/apps/dials/dials-v3-23-0/modules/cctbx_project/libtbx/scheduling/stacktrace.py", line 117, in __call__
    self.raise_handler( exception = exception )
  File "/dls_sw/apps/dials/dials-v3-23-0/modules/cctbx_project/libtbx/scheduling/stacktrace.py", line 147, in raise_with_stacktrace
    self.stacktrace( exception = exception )
  File "/dls_sw/apps/dials/dials-v3-23-0/modules/cctbx_project/libtbx/scheduling/stacktrace.py", line 88, in __call__
    raise exception
IndexError: Please report this error at https://github.com/dials/dials/issues: Index (1000) out of range for (0-999)

[graeme-winter]

OK, at first glance this looks to be a "feature" of how the nxmx library is implemented... that is not helpful

[graeme-winter]

OK, added a workaround to only understand() NXmx files if entry/data/data is a key - see commentary in #785