Support roadmap for ixmp4

[khaeru]

This post gives an outline/roadmap for MESSAGEix users and developers about future changes in message_ix and ixmp related to a new package in development named ixmp4.

The goal of the MESSAGE team is to provide stable and reliable research tools for use by ourselves and the MESSAGE community. This means ensuring a smooth migration path even as major underlying changes are made (in order to unlock future improvements). In particular, we aim for no breakage of researchers' modeling workflows and codes, and allowing time and providing adequate support for each user/developer to migrate at a pace that matches their capabilities.

Background

The message_ix Python package depends on the ixmp ("ix modeling platform") package. The choice of multiple packages follows the software design principle known as “separation of concerns”. In brief:

• ixmp handles local and remote storage of any scenario and its contents (sets, parameters, variables, etc.), and interfacing with (any) GAMS code that implements optimization models. It also handles storage of data in the IAM Consortium (IAMC) structure.
• message_ix provides a specific set of GAMS files that implement the MESSAGE optimization model, and tools for working with the specific sets, parameters, etc. in the MESSAGE formulation.

Separation of concerns allows each package to have a smaller scope and well-defined and -tested functionality. This in turn has made it easier for the MESSAGE team to maintain both packages.

Since its first release (June 2019), the ixmp package has implemented data storage using the JPype1 package, the Java Runtime Environment (JRE), and underlying Java code. Starting with ixmp v2.0.0 (released January 2020), these features have been collected in the JDBCBackend class.

In 2023, colleagues in the IIASA Scenario Services & Scientific Software team began work on a new package named ixmp4, currently at version 0.9.8. One motivation for the new package was to implement storage in Python code that is more easily maintained and supported by IIASA colleagues (compared to the Java code underlying the JDBCBackend).

• ixmp4 development initially focused on storage of IAMC-structured data, particularly as used to support the “Scenario Explorer” web application.
• More recently, development has extended to scenario data of the kinds used by MESSAGEix.

Future releases of message_ix and ixmp

Users can expect the following milestones.

Note that releases from version 3.12 are not yet scheduled for specific dates. They may occur faster or slower than the usual 6-month frequency. Exact version numbers ‘A’, ‘B’, and ‘C’ are not yet set, and points for these milestones may be combined in fewer releases or spread across additional releases.

message_ix and ixmp versions 3.11 (June 2025)

ixmp gains a new IXMP4Backend class. This class uses ixmp4 internally, but allows ixmp.Platform, ixmp.Scenario, message_ix.Scenario, and related code to present exactly the same interface and behaviour as in ixmp and message_ix versions 3.10 and earlier.

• IXMP4Backend has substantial yet incomplete support for message_ix features. In particular:
  • All the MESSAGE tutorials that are used in the annual MESSAGE Training Workshop can be run with this back end. All features used in these tutorials are supported.
  • Some more advanced or lesser-used features are not yet supported.
  • Only local ixmp4 databases are supported—not remote.
  • The set of unsupported features is clearly documented.
• ixmp4 is a new, optional dependency of ixmp and thus of message_ix. Specifically, the dependency is on ixmp4 version 0.10.
• JDBCBackend remains the default back end, as in v3.10 and earlier. Users can override this in their local configuration.

message_ix and ixmp versions 3.12

• IXMP4Backend has feature parity with JDBCBackend.
  • All code—particularly, all code in the message-ix-models package—that works with JDBCBackend also works with IXMP4Backend.
  • Shared and remote databases are supported.
  • Any discrepancy in behaviour or usage versus JDBCBackend is small and clearly documented.
  • Performance regressions with IXMP4Backend compared to JDBCBackend are clearly documented.
• ixmp4 remains an optional dependency of ixmp and thus of message_ix.
• JDBCBackend remains the default.

message_ix and ixmp versions ‘A’

• IXMP4Backend becomes the default back end for ixmp and thus message_ix.
• ixmp4 and its dependencies become mandatory.
• Dependencies of JDBCBackend—JPype1 and other packages—remain mandatory.
• IXMP4Backend is at least as performant on key metrics and for key operations as JDBCBackend.

message_ix and ixmp versions ‘B’

• JPype1 and other packages become optional dependencies, no longer required or installed by default.
• Users wishing to use JDBCBackend must install these packages

message_ix and ixmp versions ‘C’

• JDBCBackend code and support is entirely removed from ixmp.
• Direct tests of message_ix against JDBCBackend are removed.

Notes

• ixmp4 also establishes an entirely new interface to interact with Scenario (renamed Run) objects and their contents. This new interface has no backwards compatibility with the interface of ixmp and message_ix v3.10 and earlier.
  • In parallel with the roadmap steps above, features may be added to message_ix to, inter alia, allow users to access the ixmp4.Run object corresponding to a message_ix.Scenario object, so that they may write code using the ixmp4.Run interface.
  • As of ixmp4 v0.9.8, the Run interface is not documented, and so no concrete design or timeline exists for these features.
• As of 2025-06-01, ixmp4 has not yet reached a 1.0 release. This means that the public API should not be considered stable. The dependency of ixmp v3.11 and subsequent versions on ixmp4 will be adjusted to ensure ixmp4 API changes do not cause breakage in ixmp or message_ix.
• No decisions have yet been taken on:
  • If, when, or how methods of ixmp.Platform or message_ix.Scenario will be removed, renamed, or changed in backwards-incompatible ways.
  • If, when, or how message_ix_models code (in whole or in part) will be adapted to use ixmp4 interfaces directly.

See also

“Versions and releases” in the message_ix documentation.