Skip to content

Index error when creating refs for this AWS NetCDF3 dataset #551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
rsignell opened this issue Apr 9, 2025 · 0 comments · May be fixed by #554
Open

Index error when creating refs for this AWS NetCDF3 dataset #551

rsignell opened this issue Apr 9, 2025 · 0 comments · May be fixed by #554

Comments

@rsignell
Copy link

rsignell commented Apr 9, 2025 •
edited
Loading

I tried generating netcdf3 refs for a new AWS Public Dataset, and I'm getting a IndexError: too many indices for array; expected 0, got 1 error.

This is with kerchunk=0.2.7 and zarr=2.18.4 (the latest kerchunk-on-zarr-v2 environment)

You can reproduce this error with:

from kerchunk.netCDF3 import NetCDF3ToZarr
d0 = NetCDF3ToZarr("s3://umassd-fvcom/gom3/hindcast/gom3_197801.nc", 
                  storage_options={"anon": True},
                  inline_threshold=400, version=2).translate()

I don't know if this is relevant, but this is an unstructured grid (triangular mesh) model, so there isn't the usual time,depth,lat,lon dimensions, but time,depth,node instead:

$ncdump -k gom3_197801.nc
64-bit offset

$ncdump -h gom3_197801.nc

dimensions:
        nele = 90415 ;
        node = 48451 ;
        siglay = 45 ;
        siglev = 46 ;
        three = 3 ;
        time = UNLIMITED ; // (745 currently)
        DateStrLen = 26 ;
        maxnode = 11 ;
        maxelem = 9 ;
        four = 4 ;
variables:
        int nprocs ;
                nprocs:long_name = "number of processors" ;
        int partition(nele) ;
                partition:long_name = "partition" ;
        float x(node) ;
                x:long_name = "nodal x-coordinate" ;
                x:units = "meters" ;
        float y(node) ;
                y:long_name = "nodal y-coordinate" ;
                y:units = "meters" ;
        float lon(node) ;
                lon:long_name = "nodal longitude" ;
                lon:standard_name = "longitude" ;
                lon:units = "degrees_east" ;
        float lat(node) ;
                lat:long_name = "nodal latitude" ;
                lat:standard_name = "latitude" ;
                lat:units = "degrees_north" ;
        float xc(nele) ;
                xc:long_name = "zonal x-coordinate" ;
                xc:units = "meters" ;
        float yc(nele) ;
                yc:long_name = "zonal y-coordinate" ;
                yc:units = "meters" ;
        float lonc(nele) ;
                lonc:long_name = "zonal longitude" ;
                lonc:standard_name = "longitude" ;
                lonc:units = "degrees_east" ;
        float latc(nele) ;
                latc:long_name = "zonal latitude" ;
                latc:standard_name = "latitude" ;
                latc:units = "degrees_north" ;
        float siglay(siglay, node) ;
                siglay:long_name = "Sigma Layers" ;
                siglay:standard_name = "ocean_sigma/general_coordinate" ;
                siglay:positive = "up" ;
                siglay:valid_min = -1.f ;
                siglay:valid_max = 0.f ;
                siglay:formula_terms = "sigma: siglay eta: zeta depth: h" ;
        float siglev(siglev, node) ;
                siglev:long_name = "Sigma Levels" ;
                siglev:standard_name = "ocean_sigma/general_coordinate" ;
                siglev:positive = "up" ;
                siglev:valid_min = -1.f ;
                siglev:valid_max = 0.f ;
                siglev:formula_terms = "sigma:siglay eta: zeta depth: h" ;
        float h(node) ;
                h:long_name = "Bathymetry" ;
                h:standard_name = "sea_floor_depth_below_geoid" ;
                h:units = "m" ;
                h:positive = "down" ;
                h:grid = "Bathymetry_Mesh" ;
                h:coordinates = "x y" ;
                h:type = "data" ;
        int nv(three, nele) ;
                nv:long_name = "nodes surrounding element" ;
        int iint(time) ;
                iint:long_name = "internal mode iteration number" ;
        float time(time) ;
                time:long_name = "time" ;
                time:units = "days since 1858-11-17 00:00:00" ;
                time:format = "modified julian day (MJD)" ;
                time:time_zone = "UTC" ;
        int Itime(time) ;
                Itime:units = "days since 1858-11-17 00:00:00" ;
                Itime:format = "modified julian day (MJD)" ;
                Itime:time_zone = "UTC" ;
        int Itime2(time) ;
                Itime2:units = "msec since 00:00:00" ;
                Itime2:time_zone = "UTC" ;
        char Times(time, DateStrLen) ;
                Times:time_zone = "UTC" ;
        float zeta(time, node) ;
                zeta:long_name = "Water Surface Elevation" ;
                zeta:units = "meters" ;
                zeta:positive = "up" ;
                zeta:standard_name = "sea_surface_height_above_geoid" ;
                zeta:grid = "Bathymetry_Mesh" ;
                zeta:coordinates = "x y" ;
                zeta:type = "data" ;
        char file_date(time, DateStrLen) ;
                file_date:time_zone = "UTC" ;
        int nbe(three, nele) ;
                nbe:long_name = "elements surrounding each element" ;
        int ntsn(node) ;
                ntsn:long_name = "#nodes surrounding each node" ;
        int nbsn(maxnode, node) ;
                nbsn:long_name = "nodes surrounding each node" ;
        int ntve(node) ;
                ntve:long_name = "#elems surrounding each node" ;
        int nbve(maxelem, node) ;
                nbve:long_name = "elems surrounding each node" ;
        float a1u(four, nele) ;
                a1u:long_name = "a1u" ;
        float a2u(four, nele) ;
                a2u:long_name = "a2u" ;
        float aw0(three, nele) ;
                aw0:long_name = "aw0" ;
        float awx(three, nele) ;
                awx:long_name = "awx" ;
        float awy(three, nele) ;
                awy:long_name = "awy" ;
        float art2(node) ;
                art2:long_name = "Area of elements around a node" ;
        float art1(node) ;
                art1:long_name = "Area of Node-Base Control volume" ;
        float u(time, siglay, nele) ;
                u:long_name = "Eastward Water Velocity" ;
                u:units = "meters s-1" ;
                u:grid = "fvcom_grid" ;
                u:type = "data" ;
        float v(time, siglay, nele) ;
                v:long_name = "Northward Water Velocity" ;
                v:units = "meters s-1" ;
                v:grid = "fvcom_grid" ;
                v:type = "data" ;
        float omega(time, siglev, node) ;
                omega:long_name = "Vertical Sigma Coordinate Velocity" ;
                omega:units = "s-1" ;
                omega:grid = "fvcom_grid" ;
                omega:type = "data" ;
@rsignell rsignell changed the title Index error when creating refs for this NetCDF3 dataset Index error when creating refs for this AWS NetCDF3 dataset Apr 9, 2025
@martindurant martindurant linked a pull request Apr 16, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix zero-dim variables in netCDF3 martindurant/fsspec-reference-maker
1 participant
@rsignell