Change in BLEV in UM header causing mule-cumf to give incorrect differences in the data field

[ccharltonp]

In the past, BLEV in the UM header for our SURF files has been set to the soil layer thickness and not the centre of the layer. This is not right and so we have fixed it. But then after changing BLEV from a certain set of positive real numbers to a slightly different set of positive real numbers, the data in the fields no longer compares.

Files DO NOT compare
  * 0 differences in fixed_length_header (with 7 ignored indices)
  * 13 field differences, of which 4 are in data

Compared 17/29 fields, with 4 matches
6 fields found in file 1 were not in file 2
6 fields found in file 2 were not in file 1

Maximum RMS diff as % of data in file 1: 10724406067200     (field 12)
Maximum RMS diff as % of data in file 2: 18443219763200     (field 12)

%%%%%%%%%%%%%%%
* Field 3/17  *
%%%%%%%%%%%%%%%
Lookup compares, data DOES NOT compare
Compared 64/64 lookup values.
File_1 lookup info:
  t1(2018/01/14 06:00:00)  lblev(9999)/blev(0.0)  lbproc(0)
Data differences:
  Number of point differences  : 203329/307200
  Maximum absolute difference  : 1073741824        
  RMS difference               : 873552320         
  RMS diff as % of file_1 data : 530611814400      
  RMS diff as % of file_2 data : 912515686400   

When I use the old/wrong BLEV, the same data in the actual fields compares nicely.

[hdyson]

Doesn't mule-cumf use the lookup headers to work out which field to compare to which other field? I'm suspicious of this section:

Compared 17/29 fields, with 4 matches
6 fields found in file 1 were not in file 2
6 fields found in file 2 were not in file 1

I suspect what's happened is that you have 4 fields where the new BLEV values match the BLEV values for 4 different fields in old version? So for a simple example, if the old BLEV values were [8, 9, 10] and the new values were [10, 11, 12], mule-cumf would be comparing the data for the last old field with the first new field - and so you'd be expecting a difference.

[ccharltonp]

Hi Harold, I see what you are saying, but that would mean that all the other header information would also have to be the same right? We have 4 fields where the old LBLEV are 0.1, 0.25, 0.65 and 2.0. The new LBLEV are 0.05, 0.225, 0.675 and 2.0. So the last LBLEV just happens not to change.

[jmancell]

Are the fields with data differences actually soil levels? The one you posted:

File_1 lookup info:
t1(2018/01/14 06:00:00) lblev(9999)/blev(0.0) lbproc(0)

Looks like a surface field (lblev 9999) and also none of your soil levels should have a BLEV of 0.0. I wonder if something has changed with the field ordering and cumf is accidentally comparing the wrong fields. Also the maximum difference is the missing data indicator RMDI 107374E+10 (I don't know if that's a clue as to what is going on)