Speed up datastore hashing

[charles-turner-1]

Is your feature request related to a problem? Please describe.

The dataset hashing in the use_datastore/build-esm-datastore functionality is painfully slow & is going to negatively impact user experience. We need to speed it up as much as is reasonably possible.

Describe the feature you'd like

Make it go.

Describe alternatives you've considered

<Replace this text with a clear and concise description of any alternative solutions or features you've considered.>

Additional context