Ability to access residuals and Jacobians of algebraic systems in ME FMUs for Optimization

[adwait-gm]

Is there any plan to introduce support for Differential-Algebraic Equation (DAE) models in future FMI 3.0 updates for Model Exchange FMUs? Specifically, would future versions expose residuals of nonlinear systems of equations for algebraic variables from an FMU instead of solving them internally? We found some issues talking about this, but the approaches are mainly interested in model simulation and not optimization.

Handling Constraints in ModelExchange #144
Handling ODE with constraints (drift elimination for index reduced DAE) #1651

Why this matters?

• Many complex models contain internal algebraic loops, which require solving nonlinear systems at each simulation step.
• Currently, FMUs often solve these equations internally, which can increase execution time when used in optimization loops.
• If residuals of these nonlinear equations could be exposed instead of solved inside the FMU, external solvers (such as IPOPT/SNOPT) could handle them as constraints, potentially leading to better convergence and faster execution in optimization workflows.
• Since these residuals will be solved externally, the FMU should be able to differentiate them and provide the necessary Jacobians required by the external solver

Are there any ongoing discussions or plans for enabling residuals in FMI for DAE models? If not, would this be considered as a potential feature request? The model in the following issue can serve as a test example for this feature

ME-FMU - correct calling sequence to repeatedly set states, inputs and obtain derivatives #2030

[chrbertsch]

FMI Design Meeting:
Adweit explained the problem.
Klaus: there was a working group on DAE support before the release of FMI 3.0
Pierre: there was not so much interest, and volunteers to work out the details. It is not clear if this will work on many tools.
Klaus: (re-)initialization of DAEs was one of the problems.
Adweit: I do not see the problems in initialization ...
Christian: The latest summary is from https://github.com/modelica/fmi-design/blob/master/WorkingGroups/02_DAE_PartialDerivatives/Meetings/2018-04-04_05-FMI-Design-Meeting-Dresden/DAE.pdf
Srikanth: we would only need to access the residuals.
Christian: There were also some prototypes out ... It could be done in a layered standard
Pierre: If we have only one FMU in optimization, this is much simpler than for many connected FMUs ...

[hubertus65]

Clarification (I think). The request by GM is not about the old "FMI-for-DAE" effort requiring fully implicit, sparse solvers, , but for exposing algebraic loops in an FMU to an external solver for optimization. This is the "normal" way high-index DAEs are transformed into an explicit, sorted, index 1 DAE that can use an ODE solver, if combined with non-linear solvers for algebraic loops / embedded non-linear systems. Note that this is orthogonal to the choice of an implicit or explicit ODE-solver. That feature request is a straightforward extension to ME-FMUs as defined today, and would therefore be a good candidate for a layered standard. It needs to be written carefully though. Tome it seems that this request is coming indirectly from the CasADi folks: it could be good to invite Joel Andersson into this discussion.

[chrbertsch]

Clarification (I think). The request by GM is not about the old "FMI-for-DAE" effort requiring fully implicit, sparse solvers, , but for exposing algebraic loops in an FMU to an external solver for optimization. This is the "normal" way high-index DAEs are transformed into an explicit, sorted, index 1 DAE that can use an ODE solver, if combined with non-linear solvers for algebraic loops / embedded non-linear systems. Note that this is orthogonal to the choice of an implicit or explicit ODE-solver. That feature request is a straightforward extension to ME-FMUs as defined today, and would therefore be a good candidate for a layered standard. It needs to be written carefully though. Tome it seems that this request is coming indirectly from the CasADi folks: it could be good to invite Joel Andersson into this discussion.

@jaeandersson : Could you join one of the coming FMI-Design Webmeetings (8.4.2025 15-17:00 CEST and then every two weeks to discuss this with us?)

[adwait-gm]

Clarification (I think). The request by GM is not about the old "FMI-for-DAE" effort requiring fully implicit, sparse solvers, , but for exposing algebraic loops in an FMU to an external solver for optimization. This is the "normal" way high-index DAEs are transformed into an explicit, sorted, index 1 DAE that can use an ODE solver, if combined with non-linear solvers for algebraic loops / embedded non-linear systems. Note that this is orthogonal to the choice of an implicit or explicit ODE-solver. That feature request is a straightforward extension to ME-FMUs as defined today, and would therefore be a good candidate for a layered standard. It needs to be written carefully though. Tome it seems that this request is coming indirectly from the CasADi folks: it could be good to invite Joel Andersson into this discussion.

@hubertus65 Yes, the request is to enable efficient solution of optimization problems where the model comes from an FMU. The residuals of the algebraic loops are currently solved internally which may be causing some issues with accuracy and computational efficiency. Exposing these algebraic loops outside the FMU like an input-output pair could lead to more efficient problem formulations.

We are working with @jaeandersson on this issue. This would also be useful for other applications like steady state solutions (quasi-static simulations) using NLP solvers.

[jaeandersson]

@jaeandersson : Could you join one of the coming FMI-Design Webmeetings (8.4.2025 15-17:00 CEST and then every two weeks to discuss this with us?)

Sure, feel free to send an invite for the series (I was forwarded an invitation for today's meeting)

[chrbertsch]

FMI Design Webmeeting:
Adweit: we would propose to expose the residuals and solve them simultaneously
Joel: You would like to jump between time points and start from them again.
Newton type methods have a certain accuracy
Srikanth: we are in contact with DS, and they have a prototype
Joel: exposing the API is a way. Modelon Impact have an OCT solution for steady state. They have a naming convention with inputs, outputs.
Klaus: if the importer does not know about the naming convention, it calculates nonsense. It is better in FMI 3.1 than in a layered standard.
Adweit: Is it necessary that any FMI 3.0 supporting tool can simulate this FMU if it does not know about the layered standard
Andreas: Layered standards provide additions. But the FMUs should still be valid FMUs.
Klaus: what you could do is to expose a parameter that you do not set ... this could be ok
Andreas: like a structural parameter
Joel: you could create a parameter that allows for external solving of algebraic loops
Klaus: It should be an "opt-in"

Peter: We do similar things internally. We use FMUs as a container for our proprietary things. This would not be a "normal" FMU any more.
Andreas: I am not happy with this. These should not called FMUs any more if the contract is broken. We should have a warning in the implementers' guide.
Klaus: it is an "FMI-inspired artefact"

[hubertus65]

Interesting discussion :-). I think there are 2 cases:

1. Exposing internal algebraic loops of a normal dynamic FMU for external solving. I think this can be done as a layered standard.
2. Using a variant of this for overall steady-state modeling, as Modelon does for their steady-state models. That can not be done with a layered standard.
   For case one, one can have a "switching parameter" and an optional API that allows to solve these non-linear equations outside of the ME-FMU instead of inside.