Enable basket export for kek

[domi4484]

Based on #182

[ponceta]

#181 will be rebased after this.

[sjib]

======================================================================
ERROR: test_case_b_export_complete_qgep_to_xtf (qgepqwat2ili.tests.test_qgep.TestQGEPUseCases.test_case_b_export_complete_qgep_to_xtf)
# B. export the whole QGEP model to INTERLIS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1256, in _execute_context
    self.dialect.do_executemany(
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 912, in do_executemany
    cursor.executemany(statement, parameters)
psycopg2.errors.NotNullViolation: null value in column "t_basket" violates not-null constraint
DETAIL:  Failing row contains (0, null, organisation, ch13p7mzOG000002).

@domi4484 Can you fix this also?

[sjib]

@domi4484 It looks like organisation also needs a t_basket column. Could you finish this asap - there are other developments waiting for this to be finished.

[domi4484]

@sjib the basket column get created. But for some reason sometimes the value does not get populated.

[sjib]

@domi4484 Here you are not writing the current_basket if you have class organisation - is this the problem? The tests fail on organisation data.

[domi4484]

Because it was giving an error. But it is set later here https://github.com/QGEP/qgepqwat2ili/pull/183/files#diff-eafeae0f9c5218fb04f5c903a4a2f22919fb6233a3cbb0e0746f611ad6906afcR287

[domi4484]

On INTERLIS import 2015 this works already, see example above where based on the text values VSA and SBU two new organisations are created.

In the import I don't see any code handling those values. Where is this implemented?

This is handeled here: def create_or_update_organisation(name):

qgepqwat2ili/qgepqwat2ili/qgepsia405/import_.py

Line 90 in 27ba967

def create_or_update_organisation(name):

Ok but there is no special handling of the values "VSA" and "SBU" right? 2 "normals" organisation get created with the default prefix

[sjib]

On INTERLIS import 2015 this works already, see example above where based on the text values VSA and SBU two new organisations are created.

In the import I don't see any code handling those values. Where is this implemented?

This is handeled here: def create_or_update_organisation(name):

qgepqwat2ili/qgepqwat2ili/qgepsia405/import_.py

Line 90 in 27ba967

def create_or_update_organisation(name):

Ok but there is no special handling of the values "VSA" and "SBU" right? 2 "normals" organisation get created with the default prefix

Yes, we have nothing hardcoded. Why should we?
But we can handle whether an organisation comes with its identifier such as "VSA" and "SBU" or with its obj_id such as ch080qwzPR000012

[domi4484]

On INTERLIS import 2015 this works already, see example above where based on the text values VSA and SBU two new organisations are created.

In the import I don't see any code handling those values. Where is this implemented?

This is handeled here: def create_or_update_organisation(name):

qgepqwat2ili/qgepqwat2ili/qgepsia405/import_.py

Line 90 in 27ba967

def create_or_update_organisation(name):

Ok but there is no special handling of the values "VSA" and "SBU" right? 2 "normals" organisation get created with the default prefix

Yes, we have nothing hardcoded. Why should we? But we can handle whether an organisation comes with its identifier such as "VSA" and "SBU" or with its obj_id such as ch080qwzPR000012

I am just trying to understand why those ID are making problems in this PR that in fact doesn't change anything in the import. And I think I found out now:

The test test_case_f_export_selection_sia405 which is failing with the organisation ID error doesn't explicitly set up the database as other tests do.

• This means it uses the setup of another test.
• But test are not guaranteed to run in a special order -> this test is probably in this case not using the demodata as a base and for this reason VSA and SBU get created with an invalid ID

[sjib]

I tried to test the current status - No output in KEK file:

<?xml version="1.0" encoding="UTF-8"?><TRANSFER xmlns="http://www.interlis.ch/INTERLIS2.3">
<HEADERSECTION SENDER="ili2pg-5.2.0-375168b6abf225834531d7aa07a1fbb159937361" VERSION="2.3"><MODELS><MODEL NAME="VSA_KEK_2019_LV95" VERSION="20.01.2021" URI="http://www.vsa.ch/models"></MODEL></MODELS></HEADERSECTION>
<DATASECTION>
<VSA_KEK_2019_LV95.KEK BID="5">
</VSA_KEK_2019_LV95.KEK>
</DATASECTION>
</TRANSFER>

[sjib]

@domi4484 Here some testdata for importing VSA-KEK 2019 data:
testdata_vsa_kek_2019.zip

[domi4484]

I tried to test the current status - No output in KEK file:

Yes in the tests output it looks empty as well.

Thanks for the additional testdata. I think I found the issue. The objects are put into the wrong basket and so ili2db can't export them. Ill try to fix that

[domi4484]

Now the .xtf file is filled with data for me locally. On the pipeline the file is still empty

[sjib]

@domi4484 Can you finish this today so we can make the release of 1.6.6 tomorrow?

[ponceta]

As this is working on local installations, we will merge it, test it and fix the CI in 2025.