Containers of @constraints with the same indexes

[felipemarkson]

Hi! I open this issue to discuss the support of collections of constraints with the same indexes in the macro @constraints.
Let me know if there's a better way to implement this in the current version. Thank's!

The current implementation

In version 0.23.0, you can implement collections of constraints with the same indexes in @constraints with the following syntax:

@constraints(model, begin
          cons1[i=set_I, j=set_J], expression1[i,j] <= 0
          cons2[i=set_I, j=set_J], expression2[i,j] <= 0
          cons3[i=set_I, j=set_J], expression3[i,j] <= 0
end)

The problem

In very large models, there are many constraints with many indexes that use the same indexes. These constraints often belong to the same context in the model and can be grouped to better conciseness.

However, If you want to group these constraints in the current version, the readability of the code is harmed. Here is a example:

@constraints(model, begin
          cons1[i=set_I, j=set_J,  k=set_K,  l=set_L,  m=set_M,  n=set_N, o=set_O], expression1[i,j,k,l,m,n,o] <= 0
          cons2[i=set_I, j=set_J,  k=set_K,  l=set_L,  m=set_M,  n=set_N, o=set_O], expression2[i,j,k,l,m,n,o] <= 0
          cons3[i=set_I, j=set_J,  k=set_K,  l=set_L,  m=set_M,  n=set_N, o=set_O], expression3[i,j,k,l,m,n,o] <= 0
end)

The suggestion

As a suggestion, I think that a collection of containers constraints is a good way to keep the code and model conciseness. Here is a example:

@constraints(model, context[i=set_I, j=set_J] = begin
          cons1[i,j], expression1[i,j] <= 0
          cons2[i,j], expression2[i,j] <= 0
          cons3[i,j], expression3[i,j] <= 0
end)

or

@constraints(model, context[i=set_I, j=set_J] = begin
          expression1[i,j] <= 0
          expression2[i,j] <= 0
          expression3[i,j] <= 0
end)

In very large models:

@constraints(model, context[i=set_I, j=set_J,  k=set_K,  l=set_L,  m=set_M,  n=set_N, o=set_O] = begin
          cons1[i,j,k,l,m,n,o], expression1[i,j,k,l,m,n,o] <= 0
          cons2[i,j,k,l,m,n,o], expression2[i,j,k,l,m,n,o] <= 0
          cons3[i,j,k,l,m,n,o], expression3[i,j,k,l,m,n,o] <= 0
end)

Of course, there is always the possibility of declaring the constraint separated. But this syntax sounds like a good alternative if you have very large models.

Let me know what you guys think about it.

[odow]

Assuming you don't need the cons1 containers stored somewhere, I would write:

for i=set_I, j=set_J,  k=set_K,  l=set_L,  m=set_M,  n=set_N, o=set_O
    @constraints(model, begin
        expression1[i,j,k,l,m,n,o] <= 0
        expression2[i,j,k,l,m,n,o] <= 0
        expression3[i,j,k,l,m,n,o] <= 0
    end)
end

Is that sufficient?

[felipemarkson]

Yeah, I think that's work in most cases.

But, using this, isn't there some performance issue?

I guess that I read in JuMP documentation somewhere to avoid for-loops, but I'm not sure about it.

[odow]

No, there is no performance issue. If you can point to any confusing parts of the documentation, we can edit them to reduce the confusion.

[felipemarkson]

Ok! Thanks!

I saw the same issue using the @expressions macro. But, I think it is impossible to use for-loops in this case as cited by @odow in #2907 (comment).

The Current implementation

@expressions(model, begin
          expression1[i=set_I,j=set_J], other_expr1[i,j] + 1
          expression2[i=set_I,j=set_J], other_expr2[i,j] + 2
          expression3[i=set_I,j=set_J], other_expr3[i,j] + 3
end)

The suggestion

@expressions(model, context[i=set_I, j=set_J] = begin
          expression1[i,j], other_expr1[i,j] + 1
          expression2[i,j], other_expr2[i,j] + 2
          expression3[i,j], other_expr3[i,j] + 3
end)

or

@expressions(model, context[i=set_I, j=set_J] = begin
          other_expr1[i,j] + 1
          other_expr2[i,j] + 2
          other_expr3[i,j] + 3
end)

Should I create a new issue for that?

[odow]

No need for a new issue. All the macro code uses the same underlying functionality, so if it works for one it will work for the rest.

Does something like this work?

@expression(model, expression1[i=set_I,j=set_J], other_expr1[i,j] + 1)
expression2 = similar(expression1)
expression3 = similar(expression2)
for i in set_I, j in set_J
    expression2[i, j] = @expression(model, other_expr2[i, j] + 2)
    expression3[i, j] = @expression(model, other_expr3[i, j] + 3)
end

I guess my feeling is that the macro syntax is already complicated and convoluted already, and adding another syntax only makes things more complicated. But, this is likely to be a common request, so we should have some way to document it.

[felipemarkson]

Sorry, I think that I mess up with #2907 (comment). I have already fixed it.

Does your #2907 (comment) still work for the problem in #2907 (comment) now?

[felipemarkson]

I guess my feeling is that the macro syntax is already complicated and convoluted already, and adding another syntax only makes things more complicated. But, this is likely to be a common request, so we should have some way to document it.

Ok! I think that your first solution is a good way to solve this problem for constraints.

[odow]

Does your #2907 (comment) still work for the problem in #2907 (comment) now?

Yes. I noticed the typos.

I think that your first #2907 (comment) is a good way to solve this problem for constraints.

Yeah. I see the point for @expressions, but I think on balance it introduces complicated syntax. Sometimes, more verbose code is more readable. It's probably a sign that you could refactor some of your code in a different way.

For example, you might want:

S = [(i, j) for (i, j) in zip(set_I, set_I)]
@expressions(model, begin
    expression1[(i, j)=S], other_expr1[i,j] + 1
    expression2[(i, j)=S], other_expr2[i,j] + 2
    expression3[(i, j)=S], other_expr3[i,j] + 3
end)

[felipemarkson]

Ok! Thanks! #2907 (comment) solves the both problems!

But, this is likely to be a common request, so we should have some way to document it.

I agreed with that

[felipemarkson]

I've never contribute to a Julia Project, but if you guys agreed I can put this on documentation.

[joaquimg]

Sure! Open a PR.

[odow]

I've never contribute to a Julia Project, but if you guys agreed I can put this on documentation.

Sure! Let me know if you get stuck somewhere.

The easiest way is to find the page you want to edit, and then click "Edit on GitHub" in the top-right corner:
https://jump.dev/JuMP.jl/dev/manual/expressions/

Other helpful links:
https://jump.dev/JuMP.jl/stable/developers/contributing/#Improve-the-documentation

[odow]

Closed by #2969