further nudge from main to release (#104)

* generalized rate-cell a bit

* touched up rate-cell further

* minor mod to lif

* updated lif-cell to use units/tags and minor cleanup and edits

* Monitor plot (#66)

* Update base_monitor.py

* added plotting viewed compartments

* added meta-data to rate-cell, input encoders, adex

* fixed minor saving/loading in rate-cell w/ vectorized compartments

* Added auto resolving for monitors (#67)

* fixed surr arg in lif-cell

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* Patched synapses added (#68)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Fix typo in pcn_discrim.md (#69)

* model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

* Fix/reorganize feature library (#74)

* Update ode_utils.py

* Update ode_solver.py

rk4 revised and __main__ added

* Delete ngclearn/utils/diffeq/ode_functions.py

* Create odes.py

odes name and structure changed

* Update __init__.py

* Create feature_library.py

* Create __init__.py

* Create base.py

* Delete docs/museum/pc-sindy.md

* Create m.md

* Add files via upload

* Delete docs/images/museum/sindy/m.md

* Add files via upload

* Create sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* Update sindy.md

* fix: correct feature library path and directory name

* Delete ngclearn/utils/dymbolic_dictionary directory

* Update model_utils.py (#78)

* Additions for inhibition stuff

* add sindy documentation for exhibits (#81)

* Add files via upload

* Add files via upload

* Update ode_utils.py (#79)

refactor: delete @partial(jit, static_argnums=(2, )) lines

Co-authored-by: Will Gebhardt <[email protected]>

* Add patched synapse (#80)

* Update __init__.py

Add point to patched components

* Add patched in __init__.py

Add patched synapses importing

* Add patched synaptic components

* Delete ngclearn/components/synapses/patched/__pycache__ directory

* Update __init__.py

new line characters added

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

new line characters added

* Update staticPatchedSynapse.py

New line characters + comments for describing each input vars

* Update patchedSynapse.py

Removed a comment line

* Update hebbianPatchedSynapse.py

remove unused arguments

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

add description for w_mask

* Update hebbianPatchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update patchedSynapse.py

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update __init__.py (#83)

* Update __init__.py

typo fixed

* Update staticPatchedSynapse.py

a typo fixed

* Update hebbianPatchedSynapse.py

typo foxed

* Add l1 decay term to update calculation (#84)

* Update hebbianSynapse.py

* update main

update main at the end

* Update hebbianSynapse.py

add regularization argument and w_decay is deprecated.

* Update hebbianSynapse.py

add elastic_net

* Update hebbianSynapse.py

* Update hebbianSynapse.py

* feat NGC module regression (#86)

* feat npc module regression

* Update __init__.py

* Update __init__.py

* Update elastic_net.py

* Update lasso.py

* Update ridge.py

* Update elastic_net.py

* Update ridge.py

* Update lasso.py

* Update odes.py

removed @partial(jit, static_argnums=(0,))

* Update odes.py (#87)

removed @partial(jit, static_argnums=(0,))

* Update odes.py

typo fixed in __main__

* Update __init__.py

add dot

* Update __init__.py

add dot

* Add attribute 'lr' (#90)

* Update elastic_net.py

* add lr as attribute to lasso.py

* add lr as attribute to ridge.py

* refactor w_bound=0. for weights elastic_net.py

deactivated w_bound for weights elastic_net.py

* Update lasso.py

* deactivated w_bound for weights ridge.py

* commit probes/mods to utils to analysis_tools branch

* commit probes/mods to utils to analysis_tools branch

* update documentation

* cleaned up probes/docs for probes

* change heads_dim to attn_dim, and modify the mlp to be as similar as possible to the attentive probing pattern

* in layer normalization or any other Gaussian, standardeviation can never be zero. Additionally, if the subtraction inside the square root goes to zero, the gradient will become NaN. Therefore, adding a clipping is necessary.

* update attentive probe code

* minor tweak to attentive prob code comments

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* cleaned up probe parent fit routine

* minor edits to attn probe

* update attentive probe with input layer norm

* update input layer normalization

* update code to fix nan bug

* minor tweak to attn probe

* cleaned up probes

* cleaned up probes

* cleaned up probes

* cleaned up probes

* generalized dropout in terms of shape

* tweak to atten probe

* tweak to atten probe

* added silu/swish/elu to model_utils

* cleaned up model_utils

* fix bug in attention probe dropout, fix bug in None noise_key passed in the probing jit function, add the spliting of noise_keys to two dropout in two cross attention

* hyperparameter tunning arguments added

* Merging over Dynamics feature branch to main (#92)

* modded bernoulli-cell to include max-frequency constraint

* added warning check to bernoulli, some cleanup

* integrated if-cell, cleaned up lif and inits

* mod to latency-cell

* updated the poissonCell to be a true poisson

* fixed minor bug in deprecation for poiss/bern

* fixed minor bug in deprecation for poiss/bern

* fixed validation fun in bern/poiss

* moved back and cleaned up bernoulli and poisson cells

* added threshold-clipping to latency cell

* updates to if/lif

* added batch-size arg to slif

* fixed minor load bug in lif-cell

* fixed a blocking jit-partial call in lif update_theta method; when loading

* minor edit to dim-reduce

* updated monitor plot code

* update to dim-reduce

* integrated phasor-cell, minor cleanup of latency

* tweak to adex thr arg

* tweak to adex thr arg

* integrated resonate-and-fire neuronal cell

* mod to raf-cell

* cleaned up raf

* cleaned up raf

* cleaned up raf-cell

* cleaned up raf-cell

* cleaned up raf-cell

* minor tweak to dim-reduce in utils

* Additions for inhibition stuff

* update to API modeling docs to reflect RAF neuronal cell

---------

Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Will Gebhardt <[email protected]>

* remove unused local variables

* update note

* update model utils

* remove notes

* Update ode utils (#94)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* minor fix to header in diffeq

* Update files with ode_solver (#95)

* Update ode_utils.py

merge ode_solver into ide_utils

* Delete ngclearn/utils/diffeq/ode_solver.py

* Update ode_utils.py

refactor doc-string

* Update odes.py

* Update sindy.md

ode_solver to ode_utils

* revised/cleaned up sindy tutorial doc/imgs

* add prior for hebbian patched synapse (#96)

* prior replaced w_decay hebbianPatchedSynapse.py

remove w_decay 
add prior_type and prior_lmbda

* revised typo hebbianSynapse.py

dWweight was typo

* cleaned up doc-strings in odes.py to comply w/ ngc-learn format

* minor tweak to sig-figs printing in probe utils

* add-sigma-to-gaussianErrorCell (#97)

* add-sigma-to-gaussianErrorCell

add not updating scalar variance for gaussian errors

* Update gaussianErrorCell.py

* cleaned up ode_utils, cleaned up gaussian/laplacian cell

* Update gaussianErrorCell.py (#98)

added `and not isinstance(sigma, int)`

* cleaned up gauss/laplace error cells

* integrated bernoulli err-cell

* Major release update merge to main (in prep for 2.0.0 release on release branch/pip) (#99)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* update test cases

---------

Co-authored-by: Viet Dung Nguyen <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Will Gebhardt <[email protected]>

* Major release update (to 2.0.0) (#100)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Will Gebhardt <[email protected]>

* Major release update merge to main (sync up) (#101)

* add initial patch mask features

* minor edit to bern-cell

* fixed bernoulli error cell

* example rate cell test

* made some corrections to bern err-cell and heb syn

* made some corrections to bern err-cell and heb syn

* cleaned up bern-cell, hebb-syn

* minor mod to model-utils

* attempted rewrite of bernoulli-cell

* got bernoulli-cell rewritten and unit-tested

* edit to bern-cell

* bernoulli and poisson cells revised, unit-tested

* latency-cell refactored and unit-tested

* refactored Rate Cell

* minor revisions to input-encoders, revised phasor-cell w/ unit-test

* revised and add unit-test for varTrace

* revised and added unit-test for exp-kernel

* revised and added unit-test for exp-kernel

* revised slif cell w/ unit-test; needed mod to diffeq

* revised slif-cell w/ unit-test; cleaned up ode_utils to play nicer w/ new sim-lib

* revised lif-cell w/ unit-test

* revised unit-tests to pass globally; some minor patches to phasor-cell and lif

* minor cleanup of unit-test for phasor

* revised if-cell w/ unit-test

* revised if-cell w/ unit-test

* revised quad-lif w/ unit-test

* revised adex-cell w/ unit test, minor cleanup of quad-lif

* minor edit to adex unit-test

* refactor bernoulli, laplacian, and rewarderror cells

* revised raf-cell w/ unit test; fixed typos/mistakes in all spiking cells

* revised wtas-cell w/ unit test

* revised fh-cell w/ unit test

* revised izh-cell w/ unit test

* patched ode_utils backend wrt jax, cleaned up unit-tests, added disable flag for phasor-cell

* update rate cell

* fix test rate cell

* update test for bernoulli cell

* update refactoring for gaussian error cell

* update unit testing for all graded neurons

* wrote+unit-test of hodgkin-huxley spike cell, minor tweaks/clean-up elsewhere

* added rk2 support for H-H cell

* update rate cell and fix bug of passing a tuple of (jax Array -- not hashable)  to jax jit functions. Basically, simplify the codebase by using a hashmap of functions

* update test rate cell

* refactored dense and trace-stdp syn w/ unit-test

* refactored exp-stdp syn w/ unit-test

* refactored event-stdp w/ unit-test

* cleanup of stdp-syn

* refactored bcm syn w/ unit-test

* refactored stp-syn with unit-test

* cleaned up modulated

* refactored mstdp-et syn w/ unit-test

* refactored lava components to new sim-lib

* refactored conv/hebb-conv syn w/ unit-test

* refactored/revised hebb-deconv syn w/ unit-test

* revised/refactored hebb/stdp conv/deconv syn w/ unit-tests

* updated modeling doc to point to hodgkin-huxley cell

* updated modeling docs

* fixed typo in adex-cell tutorial doc

* revised tutorials to reflect new sim-lib config/syntax

* revised tutorials to reflect new sim-lib config/syntax

* patched docs to reflect revisions/refactor

* tweaked requirements in prep for major release

* cleaned up a few unit tests to use deterministic syn init vals

* mod to requirements

* nudge toml to upcoming 2.0.0

* update to support docs in prep for 2.0.0

* update patched synapses and their test cases

* cleaned up syn modeling doc

* push hebbian synapse

* push reinforce synapse

* push np seed

* patched minor prior None arg issue in hebb-syn

* moved reinforce-syn to right spot

* update reinforce synapse and testing

* tweaked trace-stdp and mstdpet

* patched mstdpet unit-test

* update reinforce synapse and test cases

* add reinforce synapse fix

* minor mod to mstdpet

* update test code for more than 1 steps

* Updated monitors

* patched tests to use process naming

* Added wrapper for reset and advance_state

* Added a JaxProcess

Added Jax Process to allow for scanning over the process.

* update the old rate cell

* update old hebbian synapse

* minor edit to if-cell

* ported over adex tutorial to new ngclearn format

* hh-cell supports rk4 integration

* clean up and integrated hodgkin-huxley mini lesson in neurocog tutorials

* Update jaxProcess.py

Updated the jax process to allow for more configurations of inputs.

* update working reinforce synapse

* update correct reinforce and testing

* update documentation

* update features, documentation, and testing

* update testing for REINFORCE cell

* update code and test

* update code

* add clipping gradient to model utils

* update reinforce cell to the new model utils clip

* major cleanup in prep for merge over to main/prep for major release

* update test cases

* update to require file in docs

---------

Co-authored-by: Viet Dung Nguyen <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Will Gebhardt <[email protected]>

* update test cases

* added hh-plot for hh tutorial

* tweak to img folder for sindy

---------

Co-authored-by: Will Gebhardt <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Faezeh Habibi <[email protected]>
Co-authored-by: Sonny George <[email protected]>
Co-authored-by: Viet Dung Nguyen <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>
Co-authored-by: Viet Dung Nguyen <[email protected]>
Co-authored-by: Viet Nguyen <[email protected]>
Co-authored-by: Alexander Ororbia <[email protected]>

Author: 8 people

docs/museum/sindy.md

@@ -8,8 +8,7 @@ Flow diagrams lack clear directional indicators
 Inconsistent color schemes across visualizations
 -->
 
-
-# Sparse Identification of Non-linear Dynamical Systems (SINDy)[1]
+# Sparse Identification of Non-linear Dynamical Systems (SINDy) 
 
 In this section, we will study, create, simulate, and visualize a model known as the sparse identification of non-linear dynamical systems (SINDy) [1], implementing it in NGC-Learn and JAX. After going through this demonstration, you will:
 

docs/tutorials/neurocog/hodgkin_huxley_cell.md

@@ -83,9 +83,9 @@ Formally, the core dynamics of the H-H cell can be written out as follows:
 
 $$
 \tau_v \frac{\partial \mathbf{v}_t}{\partial t} &= \mathbf{j}_t - g_Na * \mathbf{m}^3_t * \mathbf{h}_t * (\mathbf{v}_t - v_Na) - g_K * \mathbf{n}^4_t * (\mathbf{v}_t - v_K) - g_L * (\mathbf{v}_t - v_L) \\
-\frac{\partial \mathbf{n}_t}{\partial t} &= alpha_n(\mathbf{v}_t) * (1 - \mathbf{n}_t) - beta_n(\mathbf{v}_t) * \mathbf{n}_t \\
-\frac{\partial \mathbf{m}_t}{\partial t} &= alpha_m(\mathbf{v}_t) * (1 - \mathbf{m}_t) - beta_m(\mathbf{v}_t) * \mathbf{m}_t \\
-\frac{\partial \mathbf{h}_t}{\partial t} &= alpha_h(\mathbf{v}_t) * (1 - \mathbf{h}_t) - beta_h(\mathbf{v}_t) * \mathbf{h}_t
+\frac{\partial \mathbf{n}_t}{\partial t} &= \alpha_n(\mathbf{v}_t) * (1 - \mathbf{n}_t) - \beta_n(\mathbf{v}_t) * \mathbf{n}_t \\
+\frac{\partial \mathbf{m}_t}{\partial t} &= \alpha_m(\mathbf{v}_t) * (1 - \mathbf{m}_t) - \beta_m(\mathbf{v}_t) * \mathbf{m}_t \\
+\frac{\partial \mathbf{h}_t}{\partial t} &= \alpha_h(\mathbf{v}_t) * (1 - \mathbf{h}_t) - \beta_h(\mathbf{v}_t) * \mathbf{h}_t
 $$
 
 where we observe that the above four-dimensional set of dynamics is composed of nonlinear ODEs. Notice that, in each gate or channel probability ODE, there are two generator functions (each of which is a function of the membrane potential $\mathbf{v}_t$) that produces the necessary dynamic coefficients at time $t$; $\alpha_x(\mathbf{v}_t)$ and $\beta_x(\mathbf{v}_t)$ produce different biopphysical weighting values depending on which channel $x = \{n, m, h\}$ they are related to.