Color node forgets color values on saving

[iNLyze]

Orange version

3.19.0

Expected behavior

Color node saves all RGB values for colors once set with the entire workflow.

Actual behavior

After closing orange and reloading an existing workflow containing a color node all RGB values for all colors are replaced by default value.

Steps to reproduce the behavior

Load a table
Select columns
Change default colors in color node
Save workflow
Close Orange
Open Orange
Load workflow
Open color node and find default colors

Additional info (worksheets, data, screenshots, ...)

[TomFreudenberg]

I am working with 3.20.1 and it has still the same anoying behavior.

Everytime after pushing "Reload" on my larger (320.000 records) dataset
all color settings are reset to their defaults.

Does not make fun to work with colors currently

Cheers
Tom

[TomFreudenberg]

Just found a solution to go with for me.

Open the Color Widget
Disable "Automatic Reply"
Leave Color Widget dialog opened
Reload Dataset
Switch back to Color Widget dialog
The "Apply" button is now enabled but the colors still remain
After pressing "Apply" Button coloring is as before

[iNLyze]

I also noticed, that the color widget keeps generating new instances of the color profile on reload, which seems to be the root cause of the problem.

[janezd]

I can't replicate this. I'm using 3.21. Also looking at the code, there's nothing suspicious there, either.

Could you please check whether the problem persist in 3.21? It contains some changes in handling of stored setting, which may be related to the bug you encountered.

[TomFreudenberg]

Hi, I have updated on 3.21 and loaded my working file.

Unhappy but still loose the previous settings.

Just to know what happens on my side:

1. I open my project.ows file which is about 145 MB
2. After loading the ows file the data "File" element has a warning sign "The file is too large to load automatically. Please reload"
3. Opening the "Color" dialog all is empty
4. After opening "File" object and selecting the underlaying data.tsv file, I press "Reload"
5. When checking the "Color" dialog all settings are resetted :-(

Hope that helps to figure out whats happen

[janezd]

The .ows shouldn't be so large. But that's another matter.

Is your data secret or could you upload the first ten lines of data.tsv?

Does your data have any duplicate column names? I don't think Orange checks for it -- not yet, but it should and will, because they can cause problems.

[TomFreudenberg]

Hi Janez @janezd

the data are secret but not the table layout. So this what we use as data:

create TABLE values_kpi (
  `FLD_BARCODE` varchar(64) DEFAULT NULL,
  `FLD_RXMC` varchar(64) DEFAULT NULL,
  `FLD_TYPE` INT DEFAULT NULL,
  `FLD_FSL_VERSION` INT DEFAULT NULL,
  `FLD_FSL_ITEMS` varchar(32) DEFAULT NULL,
  `FLD_CARE_TYPE` varchar(32) DEFAULT NULL,
  `FLD_M16` INT DEFAULT NULL,
  `FLD_KT_NAME_CLUSTER` varchar(64) DEFAULT NULL,
  `FLD_KT_NAME` varchar(256) DEFAULT NULL,
  `FLD_KT_IK` INT DEFAULT NULL,
  `FLD_BSNR` varchar(64) DEFAULT NULL,
  `FLD_CUST_ID` INT DEFAULT NULL,
  `FLD_CUST_TYPE` varchar(32) DEFAULT NULL,
  `FLD_KVNR` varchar(32) DEFAULT NULL,
  `FLD_CUST_YOB` INT DEFAULT NULL,
  `FLD_CUST_AGE_VO` INT DEFAULT NULL,
  `CAT_CUST_AGE_VO` varchar(32) DEFAULT NULL,
  `FLD_CUST_PRC_ERRORS` INT DEFAULT NULL,
  `CAT_CUST_PRC_ERRORS` varchar(32) DEFAULT NULL,
  `FLD_TS_VO` DATE DEFAULT NULL,
  `FLD_TS_PE` DATE DEFAULT NULL,
  `FLD_PE_YEAR` INT DEFAULT NULL,
  `FLD_PE_MONTH` INT DEFAULT NULL,
  `FLD_PE_YEAR_MONTH` varchar(10) DEFAULT NULL,
  `FLD_TS_SC` DATE DEFAULT NULL,
  `FLD_TS_SC_TIME` DATETIME DEFAULT NULL,
  `FLD_TS_SB` DATE DEFAULT NULL,
  `FLD_TS_VO_START` DATE DEFAULT NULL,
  `FLD_TS_VO_END` DATE DEFAULT NULL,
  `FLD_TS_VVO_START` DATE DEFAULT NULL,
  `FLD_TS_VVO_END` DATE DEFAULT NULL,
  `FLD_TS_FVO_START` DATE DEFAULT NULL,
  `FLD_TS_FVO_END` DATE DEFAULT NULL,
  `DIFF_VO_PE` INT DEFAULT NULL,
  `DIFF_PE_SC` INT DEFAULT NULL,
  `DIFF_PE_SB` INT DEFAULT NULL,
  `DIFF_VO_SB` INT DEFAULT NULL,
  `DIFF_VO_VVO` INT DEFAULT NULL,
  `DIFF_PE_VVO` INT DEFAULT NULL,
  `DIFF_VO_FVO` INT DEFAULT NULL,
  `DIFF_PE_FVO` INT DEFAULT NULL,
  `FLD_GAP_VO` INT DEFAULT NULL,
  `FLD_RECEIPT_SENDBACK_CLASS` varchar(512) DEFAULT NULL,
  `FLD_RECEIPT_SENDBACK_CLASSES` varchar(512) DEFAULT NULL,
  `FLD_RECEIPT_ERROR_CLASS` varchar(512) DEFAULT NULL,
  `FLD_RECEIPT_ATTRIBUTES` varchar(512) DEFAULT NULL,
  `CAT_RECEIPT_CLUSTER_CLASS` varchar(512) DEFAULT NULL,
  `CAT_DIFF_VO_PE` varchar(32) DEFAULT NULL,
  `CAT_DIFF_PE_SB` varchar(32) DEFAULT NULL,
  `CAT_DIFF_VO_SB` varchar(32) DEFAULT NULL,
  `CAT_DIFF_VO_VVO` varchar(32) DEFAULT NULL,
  `CAT_DIFF_PE_VVO` varchar(32) DEFAULT NULL,
  `CAT_DIFF_VO_FVO` varchar(32) DEFAULT NULL,
  `CAT_DIFF_PE_FVO` varchar(32) DEFAULT NULL,
  `CAT_GAP_VO` varchar(32) DEFAULT NULL,
  `CAT_RECEIPT_STATUS` varchar(32) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

create TABLE values_raw (
	barcode varchar(255) not null,
	rxmc varchar(255) null,
	typeoforder int null,
	ts_posteingang date null,
	ts_scan datetime null,
	ts_clearing datetime null,
	ts_wait_for_account datetime null,
	ts_wait_for_account_last datetime null,
	ts_wait_for_unit datetime null,
	ts_wait_for_unit_last datetime null,
	ts_print_ghost_account_rezept datetime null,
	ts_print_kostenerstattung datetime null,
	ts_print_vo_fehler datetime null,
	ts_print_nicht_muster16 datetime null,
	ts_print_kostenvoranschlag datetime null,
	ts_print_ghost_account_erinnerung datetime null,
	ts_print_call_customer_service datetime null,
	ts_print_return_offline_form datetime null,
	ts_print_cp_transformed datetime null,
	ts_print_ghost_account_cancel datetime null,
	ts_print_vo_to_old_if_needed datetime null,
	ts_print_offline_account_order_send datetime null,
	ts_print_vo_to_old_no_account datetime null,
	ts_sendback datetime null,
	ts_waitforcs datetime null,
	ts_csanswered datetime null,
	ts_webshop1 datetime null,
	ts_ekvankvabridge datetime null,
	statuskvabridge varchar(255) null,
	ts_ausgangkvabridge datetime null,
	ts_ktentscheidung datetime null,
	kt_entscheidung_last datetime null,
	vorgangsstatusekv varchar(255) null,
	ts_finalwebshop datetime null,
	ts_inital_invoice datetime null,
	ts_initial_pod datetime null,
	ts_tp5_generated datetime null,
	ts_abrechnungmarkiert datetime null,
	ts_abrechnungversendet datetime null,
	kt_ik int null,
	kt_name varchar(255) null,
	icd10 varchar(255) null,
	kvnr varchar(10) null,
	versorgzeitraum_von date null,
	versorgzeitraum_bis date null,
	vodatum date null,
	channel varchar(255) null,
	decissionstate varchar(255) null,
	wiedereroeffnung varchar(255) null,
	automaticapproval varchar(255) null,
	m16 varchar(255) null,
	bsnr varchar(255) null,
	antrag_hmv1 varchar(255) null,
	antrag_hmv2 varchar(255) null,
	gen_hmv1 varchar(255) null,
	gen_hmv2 varchar(255) null,
	antrag_sku1 char(10) null,
	antrag_sku2 char(10) null,
	gen_sku1 char(10) null,
	gen_sku2 char(10) null,
	customer_id int null,
	geburtsdatum date null
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

and then a view to dump data for work with orange

CREATE VIEW values_orange
  AS
    SELECT
      kpi.`FLD_BARCODE` as `mS#FLD_BARCODE`,
      kpi.`FLD_RXMC` as `mS#FLD_RXMC`,
      kpi.`FLD_TYPE` as `D#FLD_TYPE`,
      kpi.`FLD_FSL_VERSION` as `D#FLD_FSL_VERSION`,
      kpi.`FLD_FSL_ITEMS` as `D#FLD_FSL_ITEMS`,
      kpi.`FLD_CARE_TYPE` as `D#FLD_CARE_TYPE`,
      kpi.`FLD_M16` as `D#FLD_M16`,
      kpi.`FLD_KT_NAME_CLUSTER` as `c#FLD_KT_NAME_CLUSTER`,
      kpi.`FLD_KT_NAME` as `mS#FLD_KT_NAME`,
      kpi.`FLD_KT_IK` as `D#FLD_KT_IK`,
      kpi.`FLD_BSNR` as `mS#FLD_BSNR`,
      kpi.`FLD_CUST_ID` as `mS#FLD_CUST_ID`,
      kpi.`FLD_CUST_TYPE` as `D#FLD_CUST_TYPE`,
      kpi.`FLD_KVNR` as `mS#FLD_KVNR`,
      kpi.`FLD_CUST_YOB` as `mS#FLD_CUST_YOB`,
      kpi.`FLD_CUST_AGE_VO` as `D#FLD_CUST_AGE_VO`,
      kpi.`CAT_CUST_AGE_VO` as `D#CAT_CUST_AGE_VO`,
      kpi.`FLD_CUST_PRC_ERRORS` as `D#FLD_CUST_PRC_ERRORS`,
      kpi.`CAT_CUST_PRC_ERRORS` as `D#CAT_CUST_PRC_ERRORS`,
      kpi.`FLD_TS_VO` as `mS#FLD_TS_VO`,
      kpi.`FLD_TS_PE` as `mS#FLD_TS_PE`,
      kpi.`FLD_PE_YEAR` as `D#FLD_PE_YEAR`,
      kpi.`FLD_PE_MONTH` as `i#FLD_PE_MONTH`,
      kpi.`FLD_PE_YEAR_MONTH` as `D#FLD_PE_YEAR_MONTH`,
      kpi.`FLD_TS_SC` as `mS#FLD_TS_SC`,
      kpi.`FLD_TS_SC_TIME` as `i#FLD_TS_SC_TIME`,
      kpi.`FLD_TS_SB` as `mS#FLD_TS_SB`,
      kpi.`FLD_TS_VO_START` as `mS#FLD_TS_VO_START`,
      kpi.`FLD_TS_VO_END` as `mS#FLD_TS_VO_END`,
      kpi.`FLD_TS_VVO_START` as `mS#FLD_TS_VVO_START`,
      kpi.`FLD_TS_VVO_END` as `mS#FLD_TS_VVO_END`,
      kpi.`FLD_TS_FVO_START` as `mS#FLD_TS_FVO_START`,
      kpi.`FLD_TS_FVO_END` as `mS#FLD_TS_FVO_END`,
      kpi.`DIFF_VO_PE` as `D#DIFF_VO_PE`,
      kpi.`DIFF_PE_SC` as `D#DIFF_PE_SC`,
      kpi.`DIFF_PE_SB` as `D#DIFF_PE_SB`,
      kpi.`DIFF_VO_SB` as `D#DIFF_VO_SB`,
      kpi.`DIFF_VO_VVO` as `D#DIFF_VO_VVO`,
      kpi.`DIFF_PE_VVO` as `D#DIFF_PE_VVO`,
      kpi.`DIFF_VO_FVO` as `D#DIFF_VO_FVO`,
      kpi.`DIFF_PE_FVO` as `D#DIFF_PE_FVO`,
      kpi.`FLD_GAP_VO` as `D#FLD_GAP_VO`,
      kpi.`FLD_RECEIPT_SENDBACK_CLASS` as `c#FLD_RECEIPT_SENDBACK_CLASS`,
      kpi.`FLD_RECEIPT_SENDBACK_CLASSES` as `mS#FLD_RECEIPT_SENDBACK_CLASSES`,
      kpi.`FLD_RECEIPT_ERROR_CLASS` as `c#FLD_RECEIPT_ERROR_CLASS`,
      kpi.`FLD_RECEIPT_ATTRIBUTES` as `mS#FLD_RECEIPT_ATTRIBUTES`,
      kpi.`CAT_RECEIPT_CLUSTER_CLASS` as `c#CAT_RECEIPT_CLUSTER_CLASS`,
      kpi.`CAT_DIFF_VO_PE` as `D#CAT_DIFF_VO_PE`,
      kpi.`CAT_DIFF_PE_SB` as `D#CAT_DIFF_PE_SB`,
      kpi.`CAT_DIFF_VO_SB` as `D#CAT_DIFF_VO_SB`,
      kpi.`CAT_DIFF_VO_VVO` as `D#CAT_DIFF_VO_VVO`,
      kpi.`CAT_DIFF_PE_VVO` as `D#CAT_DIFF_PE_VVO`,
      kpi.`CAT_DIFF_VO_FVO` as `D#CAT_DIFF_VO_FVO`,
      kpi.`CAT_DIFF_PE_FVO` as `D#CAT_DIFF_PE_FVO`,
      kpi.`CAT_GAP_VO` as `D#CAT_GAP_VO`,
      kpi.`CAT_RECEIPT_STATUS` as `c#CAT_RECEIPT_STATUS`,
      raw.`ts_clearing` as `mS#RAW_ts_clearing`,
      raw.`ts_wait_for_account` as `mS#RAW_ts_wait_for_account`,
      raw.`ts_wait_for_account_last` as `mS#RAW_ts_wait_for_account_last`,
      raw.`ts_wait_for_unit` as `mS#RAW_ts_wait_for_unit`,
      raw.`ts_wait_for_unit_last` as `mS#RAW_ts_wait_for_unit_last`,
      raw.`ts_print_ghost_account_rezept` as `mS#RAW_ts_print_ghost_account_rezept`,
      raw.`ts_print_kostenerstattung` as `mS#RAW_ts_print_kostenerstattung`,
      raw.`ts_print_vo_fehler` as `mS#RAW_ts_print_vo_fehler`,
      raw.`ts_print_nicht_muster16` as `mS#RAW_ts_print_nicht_muster16`,
      raw.`ts_print_kostenvoranschlag` as `mS#RAW_ts_print_kostenvoranschlag`,
      raw.`ts_print_ghost_account_erinnerung` as `mS#RAW_ts_print_ghost_account_erinnerung`,
      raw.`ts_print_call_customer_service` as `mS#RAW_ts_print_call_customer_service`,
      raw.`ts_print_return_offline_form` as `mS#RAW_ts_print_return_offline_form`,
      raw.`ts_print_cp_transformed` as `mS#RAW_ts_print_cp_transformed`,
      raw.`ts_print_ghost_account_cancel` as `mS#RAW_ts_print_ghost_account_cancel`,
      raw.`ts_print_vo_to_old_if_needed` as `mS#RAW_ts_print_vo_to_old_if_needed`,
      raw.`ts_print_offline_account_order_send` as `mS#RAW_ts_print_offline_account_order_send`,
      raw.`ts_print_vo_to_old_no_account` as `mS#RAW_ts_print_vo_to_old_no_account`,
      raw.`ts_waitforcs` as `mS#RAW_ts_waitforcs`,
      raw.`ts_csanswered` as `mS#RAW_ts_csanswered`,
      raw.`ts_webshop1` as `mS#RAW_ts_webshop1`,
      raw.`ts_ekvankvabridge` as `mS#RAW_ts_ekvankvabridge`,
      raw.`statuskvabridge` as `mS#RAW_statuskvabridge`,
      raw.`ts_ausgangkvabridge` as `mS#RAW_ts_ausgangkvabridge`,
      raw.`ts_ktentscheidung` as `mS#RAW_ts_ktentscheidung`,
      raw.`kt_entscheidung_last` as `mS#RAW_kt_entscheidung_last`,
      raw.`vorgangsstatusekv` as `mS#RAW_vorgangsstatusekv`,
      raw.`ts_finalwebshop` as `mS#RAW_ts_finalwebshop`,
      raw.`ts_inital_invoice` as `mS#RAW_ts_inital_invoice`,
      raw.`ts_initial_pod` as `mS#RAW_ts_initial_pod`,
      raw.`ts_tp5_generated` as `mS#RAW_ts_tp5_generated`,
      raw.`ts_abrechnungmarkiert` as `mS#RAW_ts_abrechnungmarkiert`,
      raw.`ts_abrechnungversendet` as `mS#RAW_ts_abrechnungversendet`,
      raw.`icd10` as `mS#RAW_icd10`,
      raw.`channel` as `mS#RAW_channel`,
      raw.`decissionstate` as `mS#RAW_decissionstate`,
      raw.`wiedereroeffnung` as `mS#RAW_wiedereroeffnung`,
      raw.`automaticapproval` as `mS#RAW_automaticapproval`,
      raw.`antrag_hmv1` as `mS#RAW_antrag_hmv1`,
      raw.`antrag_hmv2` as `mS#RAW_antrag_hmv2`,
      raw.`gen_hmv1` as `mS#RAW_gen_hmv1`,
      raw.`gen_hmv2` as `mS#RAW_gen_hmv2`,
      raw.`antrag_sku1` as `mS#RAW_antrag_sku1`,
      raw.`antrag_sku2` as `mS#RAW_antrag_sku2`,
      raw.`gen_sku1` as `mS#RAW_gen_sku1`,
      raw.`gen_sku2` as `mS#RAW_gen_sku2`
FROM values_kpi kpi, values_raw raw
WHERE (raw.barcode = kpi.FLD_BARCODE);

That view will be exported to TSV file which we import in orange.

[janezd]

I created random data with such variables, but still cannot replicate the problem. Can anybody else? I'm attaching the data.

color-data.tsv.zip

[janezd]

We discussed this at the developers' meeting. Nobody can replicate it, but somebody had an idea that this could be related to #3398.

If you're still willing to investigate, could you:

• go to Options / Reset Widget Settings (it will reset all widget settings to defaults - I hope you don't mind).
• Close Orange and reopen it.
• Don't reload the schema you have, but create a new schema and redefine colors.
• Close Orange, open it, reload schema.

Does it remember colors now? If so, it's probably #3398. If not, could you do the following?

• Open it as a text file (it's an XML, actually).
• In "nodes" find the id of the Color widget (sorry, I could add it to the scripr below but I'm lazy :)
• In "node_properties" find the base64 encoded settings for the widget; copy it to clipboard
• Run the following Python script

import pickle
import base64
s = pickle.loads(base64.b64decode(b"""<<<paste the content of the "property" element for the widget here>>>"""))
print(s["context_settings"][0].__dict__)

and send us the result?

While at it, if your schema file is still huge, would you be so kind and check which widget stores so much data in this XML? If resetting the settings would decrease the schema size to normal, you can also do this on an earlier schema.

[TomFreudenberg]

Hi Janez @janezd

I will continue my support for that issue. Will check next week what you have proposed and send feedback afterwards.

Tom

[janezd]

You can wait for a while.

Color widget used to modify variable definitions in-place. This used to be ugly, but today it stopped working due to changes in other parts of Orange. The fix will require fundamentally changing the widget's code. I'm working on it and I'll keep you posted.

[janezd]

Cannot replicate. Probably fixed in #4041. Reopened if needed.