Failed installation in M1 Mac running OSX 13.4.1

[akhst7]

HI,

I am having an issue updating Snapatac2 to 2.3.1 in my M1 Mac. An excerpt of the output is as follows;

Running `rustc --crate-name hdf5_src --edition=2018 /Users/akihikohoji/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hdf5-src-0.8.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="libz-sys"' --cfg 'feature="threadsafe"' --cfg 'feature="zlib"' -C metadata=1964dd5dce0d7ab8 -C extra-filename=-1964dd5dce0d7ab8 --out-dir /private/var/folders/qp/qm7rgfps34v9p46d6mr3_80h0000gn/T/pip-install-odxcaqev/snapatac2_1fb74d9cfd6b40d3a58ef3815faee548/target/release/deps -L dependency=/private/var/folders/qp/qm7rgfps34v9p46d6mr3_80h0000gn/T/pip-install-odxcaqev/snapatac2_1fb74d9cfd6b40d3a58ef3815faee548/target/release/deps --extern libz_sys=/private/var/folders/qp/qm7rgfps34v9p46d6mr3_80h0000gn/T/pip-install-odxcaqev/snapatac2_1fb74d9cfd6b40d3a58ef3815faee548/target/release/deps/liblibz_sys-5686334d729a4fc4.rmeta --cap-lints allow -L native=/private/var/folders/qp/qm7rgfps34v9p46d6mr3_80h0000gn/T/pip-install-odxcaqev/snapatac2_1fb74d9cfd6b40d3a58ef3815faee548/target/release/build/libz-sys-b3b0146e1c1a1cf7/out/lib`
     error: failed to run custom build command for `hdf5-sys v0.8.1`

     Caused by:
       process didn't exit successfully: `/private/var/folders/qp/qm7rgfps34v9p46d6mr3_80h0000gn/T/pip-install-odxcaqev/snapatac2_1fb74d9cfd6b40d3a58ef3815faee548/target/release/build/hdf5-sys-39dac31c6c85c728/build-script-build` (exit status: 101)
       --- stdout
       Setting HDF5 root from environment variable:
           HDF5_DIR = "/opt/homebrew/opt/hdf5"
       Custom HDF5_DIR provided; rpath can be set via:
           RUSTFLAGS="-C link-args=-Wl,-rpath,/opt/homebrew/opt/hdf5/lib"
       On some OS X installations, you may also need to set:
           DYLD_FALLBACK_LIBRARY_PATH="/opt/homebrew/opt/hdf5/lib"
       Parsing HDF5 config from:
           "/opt/homebrew/opt/hdf5/include/H5pubconf.h"

       --- stderr
       thread 'main' panicked at 'Invalid H5_VERSION: "1.14.1-2"', /Users/akihikohoji/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hdf5-sys-0.8.1/build.rs:200:21
       note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
     warning: build failed, waiting for other jobs to finish...
     error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib -- -C 'link-args=-undefined dynamic_lookup -Wl,-install_name,@rpath/_snapatac2.cpython-310-darwin.so'` failed with code 101
     [end of output]

A version of a brew installed hdf5 seems to be invalid but does the installation require 1.14.1-2 ?

The brew installed version is just 1.14.1

brew info hdf5                                                                                                                                                                        master 
==> hdf5: stable 1.14.1 (bottled)
File format designed to store large amounts of data
https://www.hdfgroup.org/HDF5
Conflicts with:
  hdf5-mpi (because hdf5-mpi is a variant of hdf5, one can only use one or the other)
/opt/homebrew/Cellar/hdf5/1.14.1 (284 files, 21.0MB) *
  Poured from bottle using the formulae.brew.sh API on 2023-05-13 at 11:18:55
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/hdf5.rb
License: BSD-3-Clause

[akhst7]

The issue is a custom build command that builds a v.0.8.1 hdf5-sys crate which is not compatible with hdf5 v1.14.x. In order to use hdf5 v.1.14.x, hdf5-sys has to be build from the master.

[nleroy917]

@akhst7 A solution is to build snapatac2 from source and provide a patch in the Cargo.toml:

Steps:

1. Create a new virtual env:

python -m venv venv
source venv/bin/activate

2. Move somewhere convenient and clone this repository:

git clone https://github.com/kaizhang/SnapATAC2.git

3. Open the repo in your favorite editor, and add a crates.io patch to the the snapatac2-python bindings:

# Cargo.toml
[patch.crates-io]
hdf5 = { git = "https://github.com/aldanor/hdf5-rust.git" }
hdf5-sys = { git = "https://github.com/aldanor/hdf5-rust.git", features = ["static", "zlib", "threadsafe"] }

4. Install:

pip install /path/to/SnapATAC2/snapatac2-python

You can see a discussion about this over on the hdf5 repo

[kaizhang]

I believe this problem may be caused by a locally installed HDF5 library. Either uninstalling the HDF5 library installed by homebrew or unsetting the "HDF5_DIR" environmental variable may solve this problem.

The patch is unnecessary as I've successfully installed it on my M1 macbook without patch. Just try the following:

unset HDF5_DIR
pip install snapatac2

[nleroy917]

The patch is unnecessary as I've successfully installed it on my M1 macbook without patch. Just try the following:

Good tip, thank you for that.

[akhst7]

@kaizhang @nleroy917

So I somehow got around a hdf5 issue but now am running into another issue;

error: package bed-utils v0.1.0 (https://github.com/kaizhang/bed-utils.git?rev=d06f2b4ff189c64bc0d3f7054f4ee8d0797368ca#d06f2b4f)` cannot be built because it requires rustc 1.73 or newer`

This has nothing to do with snapATAC2 installation. It is just my rust version is 1.72.0-nightly and am having some issues upgrading to 1.73.

But I really appreciate suggestions to get around the hdf5 issue.

[kaizhang]

@akhst7 The following commands should allow you to upgrade rust to the latest version.

rustup default stable
rustup update

[akhst7]

@kaizhang , I forgot to do rustup update. Anyway, snapATAC2 got installed !!

Successfully installed Cython-0.29.36 MACS3-3.0.0b3 anndata-0.9.2 cykhash-2.0.1 hmmlearn-0.3.0 snapatac2-2.5.1

Thanks.