Code

pypromicev1.4.0#291

Code

Merged

BaptisteVandecrux merged 127 commits intomainGEUS-Glaciology-and-Climate/pypromice:mainfrom developGEUS-Glaciology-and-Climate/pypromice:developCopy head branch name to clipboardAug 20, 2024

+38,615-35,916

Commits

Commits on Jun 4, 2024

• Revert "revert"

  Show description for 64b6aa2
  BaptisteVandecrux

  committed
  64b6aa2

  Copy full SHA for 64b6aa2

• correcting typo

  BaptisteVandecrux

  committed
  df5a9c5

  Copy full SHA for df5a9c5

Commits on Jun 7, 2024

• aws process handling modified

  PennyHow

  committed
  8b4d430

  Copy full SHA for 8b4d430

• Level 2 processing capabilities added to get_l3

  PennyHow

  committed
  f8de75c

  Copy full SHA for f8de75c

• get_l3 renamed to l2_to_l3

  PennyHow

  committed
  17245f7

  Copy full SHA for 17245f7

• l2_to_l3 functionality from pypromice

  PennyHow

  committed
  cc6bfb2

  Copy full SHA for cc6bfb2

• minor de-bug for process test to run

  PennyHow

  committed
  09f12a0

  Copy full SHA for 09f12a0

Commits on Jun 10, 2024

• Update .gitignore

  BaptisteVandecrux

  authored
  17b647c

  Copy full SHA for 17b647c

• L2 split from L3 CLI processing

  PennyHow

  committed
  5927dbb

  Copy full SHA for 5927dbb

• unit tests moved to separate module

  PennyHow

  committed
  b9a2e70

  Copy full SHA for b9a2e70

• file writing functions moved to separate module

  PennyHow

  committed
  bfb1603

  Copy full SHA for bfb1603

• Loading functions moved to separate module

  PennyHow

  committed
  eb58371

  Copy full SHA for eb58371

• Handling and reformating functions moved

  PennyHow

  committed
  f536e74

  Copy full SHA for f536e74

• resampling functions moved

  PennyHow

  committed
  9b2d8f4

  Copy full SHA for 9b2d8f4

• aws module updated with structure changes

  PennyHow

  committed
  2e98bfd

  Copy full SHA for 2e98bfd

• Logging package typo

  PennyHow

  committed
  526f0f0

  Copy full SHA for 526f0f0

• de-bugging

  PennyHow

  committed
  de7e5d8

  Copy full SHA for de7e5d8

• get_l2 and l2_to_l3 process test added

  PennyHow

  committed
  e9bcc8c

  Copy full SHA for e9bcc8c

• data prep and write function moved out of AWS class

  PennyHow

  committed
  496ef03

  Copy full SHA for 496ef03

• actions de-bug

  PennyHow

  committed
  facf0ee

  Copy full SHA for facf0ee

• stations for testing changed

  PennyHow

  committed
  bd8d8a5

  Copy full SHA for bd8d8a5

• l2 action de-bug

  PennyHow

  committed
  679aa4f

  Copy full SHA for 679aa4f

• loop de-bug in l2 test action

  PennyHow

  committed
  e5db5d6

  Copy full SHA for e5db5d6

• out filepath changed

  PennyHow

  committed
  a6911f0

  Copy full SHA for a6911f0

• Path alteration 2

  PennyHow

  committed
  ba1a424

  Copy full SHA for ba1a424

• poutput directory alteration 3

  PennyHow

  committed
  0ae13cd

  Copy full SHA for 0ae13cd

• action stages merged

  PennyHow

  committed
  6a9b66b

  Copy full SHA for 6a9b66b

• get_l2tol3 action test skipped

  PennyHow

  committed
  eaa49fd

  Copy full SHA for eaa49fd

Commits on Jun 11, 2024

• typo fix in join_levels.py

  BaptisteVandecrux

  authored
  e0d93b5

  Copy full SHA for e0d93b5

• further debugging

  BaptisteVandecrux

  committed
  a6e8807

  Copy full SHA for a6e8807

• creating folder before writing files, writing hourly daily monthly files out in L2toL3, trying not to re-write sorted tx file if already sorted

  BaptisteVandecrux

  committed
  1a406f4

  Copy full SHA for 1a406f4

Commits on Jun 12, 2024

• typo fix

  BaptisteVandecrux

  committed
  697fa60

  Copy full SHA for 697fa60

• fixed daily output file name

  BaptisteVandecrux

  committed
  b9fb365

  Copy full SHA for b9fb365

• update get_l3 to add historical data

  BaptisteVandecrux

  committed
  0607d31

  Copy full SHA for 0607d31

• update get_l3 to add historical data

  BaptisteVandecrux

  committed
  fe7149c

  Copy full SHA for fe7149c

• resampling frequency specified

  PennyHow

  committed
  9d0cb14

  Copy full SHA for 9d0cb14

• Create get_l3_new.py

  BaptisteVandecrux

  committed
  8f9e69d

  Copy full SHA for 8f9e69d

• modified file filter added

  PennyHow

  committed
  59cfbf0

  Copy full SHA for 59cfbf0

• reverts changes on get_l2.py to write to file both 10min and hourly data

  BaptisteVandecrux

  committed
  fdb6b02

  Copy full SHA for fdb6b02

• renamed join_levels to join_l2 because join_l3 will have different merging function, attribute management and use site_id and list_station_id

  BaptisteVandecrux

  committed
  3bd0435

  Copy full SHA for 3bd0435

• skipping resample after join_l2, fixed setup.py for join_l2

  BaptisteVandecrux

  committed
  2cebff4

  Copy full SHA for 2cebff4

Commits on Jun 13, 2024

• fixing test

  BaptisteVandecrux

  committed
  cb1f6a3

  Copy full SHA for cb1f6a3

• fixed function names

  BaptisteVandecrux

  committed
  d1d46b7

  Copy full SHA for d1d46b7

• update get_l3 to add historical data

  BaptisteVandecrux

  committed
  e914403

  Copy full SHA for e914403

• update get_l3 to add historical data

  BaptisteVandecrux

  committed
  a8b37ec

  Copy full SHA for a8b37ec

• Create get_l3_new.py

  BaptisteVandecrux

  committed
  e276064

  Copy full SHA for e276064

• Merge branch 'add-historical-data-to-l3' of https://github.com/GEUS-Glaciology-and-Climate/pypromice into add-historical-data-to-l3

  BaptisteVandecrux

  committed
  50ef93e

  Copy full SHA for 50ef93e

• further work on join_l3, varible_aliases in ressource folder

  BaptisteVandecrux

  committed
  ef4ae44

  Copy full SHA for ef4ae44

• cleaning up debug code in join_l3

  BaptisteVandecrux

  committed
  b0ceb91

  Copy full SHA for b0ceb91

• small fix in join_l3

  BaptisteVandecrux

  committed
  a2fa829

  Copy full SHA for a2fa829

• working verion

  BaptisteVandecrux

  committed
  4533b7f

  Copy full SHA for 4533b7f

Commits on Jun 14, 2024

• delete encoding info after reading netcdf, debug of getColNames

  BaptisteVandecrux

  committed
  320ab52

  Copy full SHA for 320ab52

• delete get_l3.py

  BaptisteVandecrux

  committed
  1499e5d

  Copy full SHA for 1499e5d

• removing variables and output files metadata

  BaptisteVandecrux

  committed
  efbe35f

  Copy full SHA for efbe35f

• new variable description files

  BaptisteVandecrux

  committed
  28160e0

  Copy full SHA for 28160e0

• added back ressource files, use level attributes for output definition

  BaptisteVandecrux

  committed
  a847958

  Copy full SHA for a847958

Commits on Jun 15, 2024

• make list of sites from station_config, switched print to logger.info

  BaptisteVandecrux

  committed
  569b8bd

  Copy full SHA for 569b8bd

Commits on Jun 18, 2024

• removing get_l3, remove inst. values from averaged files, fixes on logging, attributes and tests,

  BaptisteVandecrux

  committed
  22b88dd

  Copy full SHA for 22b88dd

• Updates to numpy dependency version and pandas deprecation warnings (#258)

  Show description for a42c814
  

  PennyHow

  authored and

  BaptisteVandecrux

  committed
  a42c814

  Copy full SHA for a42c814

• small changes following review, restored variable.csv history

  Show description for 96e6674
  BaptisteVandecrux

  committed
  96e6674

  Copy full SHA for 96e6674

• buiding a station list instead of a station_dict

  BaptisteVandecrux

  committed
  34643df

  Copy full SHA for 34643df

• renamed l3m to l3_merged, reintroduced getVars and getMeta

  BaptisteVandecrux

  committed
  35ea174

  Copy full SHA for 35ea174

• moving gcnet_postprocessing as part of readNead

  BaptisteVandecrux

  committed
  ce802a7

  Copy full SHA for ce802a7

• sorting out the station_list in reverse chronological order

  BaptisteVandecrux

  committed
  5e7673b

  Copy full SHA for 5e7673b

• using tilde notation in setup.py

  BaptisteVandecrux

  committed
  1272a7b

  Copy full SHA for 1272a7b

• better initialisation of station_attributes attribute

  BaptisteVandecrux

  committed
  d445a45

  Copy full SHA for d445a45

Commits on Jun 19, 2024

• moved addMeta, addVars, roundValues, reformatTime, reformatLon to write.py

  BaptisteVandecrux

  committed
  7756f30

  Copy full SHA for 7756f30

• Inline comment describing enocding attribute removal when reading a netcdf

  BaptisteVandecrux

  committed
  48817e6

  Copy full SHA for 48817e6

• loading toml file as dictionary within join_l3

  Show description for a49de43
  BaptisteVandecrux

  committed
  a49de43

  Copy full SHA for a49de43

• ressources renamed to resources (#261)

  PennyHow

  authored
  48f6823

  Copy full SHA for 48f6823

• using project attribute of a station locate AWS file and specify whether it's a Nead file

  BaptisteVandecrux

  committed
  79a5972

  Copy full SHA for 79a5972

• Merge branch 'add-historical-data-to-l3' of https://github.com/GEUS-Glaciology-and-Climate/pypromice into add-historical-data-to-l3

  BaptisteVandecrux

  committed
  ba8cd10

  Copy full SHA for ba8cd10

• update test after moving addVars and addMeta

  BaptisteVandecrux

  committed
  a184277

  Copy full SHA for a184277

• fixed logger message in resample

  BaptisteVandecrux

  committed
  a2fe858

  Copy full SHA for a2fe858

• better definition of monthly sample rates in addMeta

  BaptisteVandecrux

  committed
  bd1983c

  Copy full SHA for bd1983c

• dummy datasaet built in unit test now has 'level' attribute

  BaptisteVandecrux

  committed
  036ba78

  Copy full SHA for 036ba78

• not storing timestamp_max for each station but pulling the info directly from the dataset when sorting

  BaptisteVandecrux

  committed
  9d56d84

  Copy full SHA for 9d56d84

• removing unecessary import of addMeta, roundValues

  BaptisteVandecrux

  committed
  1263d7f

  Copy full SHA for 1263d7f

Commits on Jun 24, 2024

• make CLI scripts usable within python

  BaptisteVandecrux

  committed
  8ca65a8

  Copy full SHA for 8ca65a8

• return result in join_l2 and join_l3

  BaptisteVandecrux

  committed
  d25d8cb

  Copy full SHA for d25d8cb

• removing args from join_l2 function

  BaptisteVandecrux

  committed
  b65d0cc

  Copy full SHA for b65d0cc

Commits on Jun 26, 2024

• Merge pull request #255 from GEUS-Glaciology-and-Climate/250-make-a-fast-track-l2-product-before-any-derived-variables-are-being-calculated

  Show description for c7ddb4e
  BaptisteVandecrux

  authored
  c7ddb4e

  Copy full SHA for c7ddb4e

• Merge pull request #252 from GEUS-Glaciology-and-Climate/add-historical-data-to-l3

  Show description for b82b586
  BaptisteVandecrux

  authored
  b82b586

  Copy full SHA for b82b586

Commits on Jun 27, 2024

• proper removal of encoding info when reading netcdf

  BaptisteVandecrux

  committed
  0dcac0c

  Copy full SHA for 0dcac0c

Commits on Jun 28, 2024

• Merge pull request #262 from GEUS-Glaciology-and-Climate/make-python-friendly-CLI-scripts

  Show description for f79edbf
  BaptisteVandecrux

  authored
  f79edbf

  Copy full SHA for f79edbf

Commits on Jul 5, 2024

• Refactored and Organized Test Modules

  Show description for cd42bb3
  ladsmund

  committed
  cd42bb3

  Copy full SHA for cd42bb3

• Limited the ci tests to only run e2e

  ladsmund

  committed
  76ffa55

  Copy full SHA for 76ffa55

• naming conventions changed

  

  PennyHow

  authored and

  ladsmund

  committed
  44f0d3c

  Copy full SHA for 44f0d3c

• Feature/smoothing and extrapolating gps coordinates (#268)

  Show description for 2eb7f79
  BaptisteVandecrux

  authored
  2eb7f79

  Copy full SHA for 2eb7f79

Commits on Aug 8, 2024

• Updated persistence.py to use explicit variable thresholds

  Show description for 63961be
  ladsmund

  committed
  63961be

  Copy full SHA for 63961be

• Fixed bug in persistence QC where initial repetitions were ignored

  Show description for 9db5b44
  ladsmund

  committed
  9db5b44

  Copy full SHA for 9db5b44

• Updated python version in unittest

  ladsmund

  committed
  37527b1

  Copy full SHA for 37527b1

Commits on Aug 12, 2024

• Fixed bug in get_bufr

  Show description for 37d671f
  ladsmund

  committed
  37d671f

  Copy full SHA for 37d671f

• Added __all__ to get_bufr.py

  ladsmund

  committed
  25d41db

  Copy full SHA for 25d41db

• Applied black code formatting

  ladsmund

  committed
  8952dd1

  Copy full SHA for 8952dd1

• Made bufr_to_csv as cli script in setup.py

  Show description for d93f185
  ladsmund

  committed
  d93f185

  Copy full SHA for d93f185

• Added script to recreate bufr files

  Show description for 8f371d0
  ladsmund

  committed
  8f371d0

  Copy full SHA for 8f371d0

• Updated tests parameters

  Show description for b6281b3
  ladsmund

  committed
  b6281b3

  Copy full SHA for b6281b3

• Added test for missing data in get_bufr

  Show description for 95c0d0c
  ladsmund

  committed
  95c0d0c

  Copy full SHA for 95c0d0c

• Updated get_bufr to support static GPS heights.

  Show description for e6c9023
  ladsmund

  committed
  e6c9023

  Copy full SHA for e6c9023

• Updated github/workflow to run unittests

  Show description for 46bf574
  ladsmund

  committed
  46bf574

  Copy full SHA for 46bf574

• Updated get_bufr to support station config files in folder

  Show description for 1bc04bb
  ladsmund

  committed
  1bc04bb

  Copy full SHA for 1bc04bb

• Updated BUFRVariables with scales and descriptions

  Show description for 485cfa8
  ladsmund

  committed
  485cfa8

  Copy full SHA for 485cfa8

• Increased the real_time_utilities rounding precisions

  ladsmund

  committed
  06ec9f3

  Copy full SHA for 06ec9f3

• Updated get_bufr to separate station position from bufr

  Show description for 35c0662
  ladsmund

  committed
  35c0662

  Copy full SHA for 35c0662

• Minor cleanup

  ladsmund

  committed
  18dde7f

  Copy full SHA for 18dde7f

• Updated StationConfiguration IO to handle unknown attributes from input

  ladsmund

  committed
  891a408

  Copy full SHA for 891a408

• Updated docstring in create_bufr_files.py

  ladsmund

  committed
  09950de

  Copy full SHA for 09950de

Commits on Aug 15, 2024

• Renamed e2e unittest methods

  Show description for cba8fd9
  ladsmund

  committed
  cba8fd9

  Copy full SHA for cba8fd9

• Feature/surface heights and thermistor depths (#278)

  Show description for 777c5e8
  BaptisteVandecrux

  authored
  777c5e8

  Copy full SHA for 777c5e8

Commits on Aug 19, 2024

• better adjustment of surface height in join_l3, also adjusting z_ice_surf (#289)

  BaptisteVandecrux

  authored
  232da0f

  Copy full SHA for 232da0f

• different decoding of GPS data if "L" is in GPS string (#288)

  BaptisteVandecrux

  authored
  17c5946

  Copy full SHA for 17c5946

Commits on Aug 20, 2024

• Updated pressure field for BUFR output files

  ladsmund

  committed
  6977336

  Copy full SHA for 6977336

• Updated get_l2 to use aws.vars and aws.meta

  Show description for f92f1cc
  ladsmund

  committed
  f92f1cc

  Copy full SHA for f92f1cc

• Removed static processing level attribute from file_attributes

  ladsmund

  committed
  eb0472e

  Copy full SHA for eb0472e

• Run black on write.py

  ladsmund

  committed
  41b7cc2

  Copy full SHA for 41b7cc2

• Implemented alternative helper functions for reading variables and metadata files

  Show description for c187642
  ladsmund

  committed
  c187642

  Copy full SHA for c187642

• Select format from multiple L0 input files

  Show description for 6473cb7
  ladsmund

  committed
  6473cb7

  Copy full SHA for 6473cb7

• Updated attribute metadata

  Show description for d0959e9
  ladsmund

  committed
  d0959e9

  Copy full SHA for d0959e9

• Updated test_process with full pipeline test

  Show description for 5b9601a
  ladsmund

  committed
  5b9601a

  Copy full SHA for 5b9601a

• Removed station configuration generation

  ladsmund

  committed
  314d63b

  Copy full SHA for 314d63b

• Renamed folder name in temporaty test directory

  ladsmund

  committed
  314a902

  Copy full SHA for 314a902

• Added data issues repository path as an explicit parameter to AWS

  Show description for 3824c67
  ladsmund

  committed
  3824c67

  Copy full SHA for 3824c67

• Applied black on join_l3

  ladsmund

  committed
  1e19ace

  Copy full SHA for 1e19ace

• Updated join_l3 to generate source attribute for sites

  Show description for 11a9704
  ladsmund

  committed
  11a9704

  Copy full SHA for 11a9704

• job name changed

  

  PennyHow

  authored and

  ladsmund

  committed
  3357e62

  Copy full SHA for 3357e62

• Bugfix/passing adj dir to l3 processing plus attribute fix (#292)

  Show description for b29958a
  BaptisteVandecrux

  authored
  b29958a

  Copy full SHA for b29958a