Skip to content

libpressio? #300

@mathomp4

Description

@mathomp4

As we explore tools to better compress GEOS NetCDF4 output, one interesting set of tools looks to be the pressio suite of @robertu94:

https://github.com/robertu94/libpressio
https://github.com/robertu94/pressio-tools

where the latter has a CLI:

pressio -i ~/git/datasets/hurricane/100x500x500/CLOUDf48.bin.f32 \
    -b compressor=sz3 -o abs=1e-4 -O all \
    -m time -m size -m error_stat -M all \
    -w /path/to/output.dec

I'm really interested in that if we use Spack, we can define a lot of variants to let us test:

Variants:
    arc [false]                   false, true
        build support for the ARC error correction tool
    bitgrooming [false]           false, true
        build support for the bitgrooming
    blosc [false]                 false, true
        support the blosc lossless compressors
    boost [false]                 false, true
        support older compilers using boost
    build_system [cmake]          cmake
        Build systems supported by the package
    bzip2 [false]                 false, true
        build support for the bzip2 compressor
    core [true]                   false, true
        build core builtin libraries
    cuda [false]                  false, true
        Build with CUDA
    digitrounding [false]         false, true
        build support for the digit rounding
    docs [false]                  false, true
        build and install manual pages
    fpzip [false]                 false, true
        support for the FPZIP lossy compressor
    ftk [false]                   false, true
        build support for the feature tracking toolkit
    hdf5 [false]                  false, true
        support reading and writing from hdf5 files
    json [false]                  false, true
        build the JSON support
    libdistributed [false]        false, true
        support for distributed multi-buffer support
    lua [false]                   false, true
        support for composite metrics using lua
    magick [false]                false, true
        support the imagemagick image compressors
    mgard [false]                 false, true
        support for the MAGARD error bounded lossy compressor
    mgardx [false]                false, true
        build support for the MGARDx compressor
    mpi [false]                   false, true
        support for launching processes using mpi
    ndzip [false]                 false, true
        build support for the NDZIP compressor
    netcdf [false]                false, true
        build support for the NDFCDF data format
    openmp [false]                false, true
        build plugins that use openmp
    petsc [false]                 false, true
        support IO using petsc format
    python [false]                false, true
        build the python wrappers
    qoz [false]                   false, true
        build support for the qoz compressor
    remote [false]                false, true
        build the remote launch plugin
    sz [false]                    false, true
        support for the SZ error bounded lossy compressor
    sz3 [false]                   false, true
        build support for the SZ3 compressor family
    szauto [false]                false, true
        build szauto support
    unix [false]                  false, true
        build support for unixisms like mmap and rusage
    zfp [false]                   false, true
        support for the ZFP error bounded lossy compressor

Thanks to @robertu94 for adding so many. Now the spack packages do seem a bit out of date (there is a pressio 1.0.4 and 0.5.3 currently but those aren't in spack), but I can add them when I test (and then submit up to spack-packages).

Metadata

Metadata

Assignees

Labels

SpackMore for Spack than ESMA-Baselibs

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions