Adding functions that work with @contractor

[MartinDAndersson]

I am wondering if there is a way to add new functions (functions that are not composed of the regular ones already in src) that the @constraint macro can handle? For instance, I am interested in neural networks, and would like to have constraints using the ReLU function (g(x) = x ? x \geq 0 : 0), but the @function macro does not seem to handle conditionals. It would be nice if I can somehow add ReLU, providing forward and backwards code for the contraction step. Is there some way to do this?

[dpsanders]

Sorry I missed this.

I think you can write ReLU in terms of abs, floor etc. that are already implemented?

Conditionals are not possible with interval methods, but we should be able to simulate them with piecewise functions.

[MartinDAndersson]

Yes, thank you! I also realized I could just use f(x) = max(0,x) in this particular case.

[dpsanders]

(Piecewise functions have been added to IntervalArithmetic, but we have not yet implemented reverse functions for them.)

[dpsanders]

Moving forward this will need to be done via ReversePropagation.jl.