Rules for nonlinearities?

[phreeza]

Are the nonlinearities assumed to be some relatively benign pure function? In the current state of passing an arbitrary python function that could be used to do some rather sophisticated computation which is probably not intended here? Maybe it should just be a fixed set to choose from?

[rougier]

I agree and this one of the reason there is a single common activation function for all the neurons. Question is, can you abuse it such as to do something very specific. As you suggest, a fixed set of common functions might be better and we coudl extend it when necessary. Want to do a PR?

[phreeza]

Sure, will send an PR.

[phreeza]

#12