Updating opacity with multiple volumes

[hsellik]

Having multiple volumes loaded with Niivue React and trying to toggle colormaps. When toggling, the color seems to change only when the last element is modified.

The change is not properly detected in the NVRVolume object for some reason.

The issue can be reproduced with the example from this repo when changing draft.vetricle to draft.brain in setOpacity , so the opacity of the first element is changed.

import {NiivueCanvas, NVROptions, NVRVolume} from "niivue-react";

import {useImmer} from "use-immer";
import React from "react";

const ReadmeExample = () => {
  const [volumes, setVolumes] = useImmer<{[key: string]: NVRVolume}>({
    brain: {
      url: 'https://fetalmri-hosting-of-medical-image-analysis-platform-dcb83b.apps.shift.nerc.mghpcc.org/api/v1/files/23/template.nii.gz',
    },
    ventricle: {
      url: 'https://fetalmri-hosting-of-medical-image-analysis-platform-dcb83b.apps.shift.nerc.mghpcc.org/api/v1/files/49/ventricles.nii.gz',
      opacity: 0.5,
      colormap: 'red'
    }
  });
  const [options, setOptions] = useImmer<NVROptions>({
    isOrientCube: true
  });

  const setOpacity = (value: number) => {
    setVolumes((draft) => {
      draft.brain.opacity = value;
    });
  };

  const setOrientCube = (value: boolean) => {
    setOptions((draft) => {
      draft.isOrientCube = !value;
    });
  };
  
  const volumesList = React.useMemo(() => Object.values(volumes), [volumes]);

  return (<>
    <div>
      <label>
        Show Orient Cube
        <input
          type="checkbox"
          onChange={(e) => setOrientCube(!e.target.checked)}
          checked={options.isOrientCube}
        />
      </label>
      <label>
        Brain Opacity
        <input
          type="range" min="0.0" max="1.0" step="0.1"
          onChange={(e) => setOpacity(e.target.value)}
          value={volumes.brain.opacity}
        />
      </label>
    </div>
    <div>
      <NiivueCanvas
        options={options}
        volumes={React.useMemo(() => Object.values(volumes), [volumes])}
      />
    </div>
  </>);
}

[jennydaman]

Sorry, I am not understanding the issue.

trying to toggle colormaps. When toggling, the color seems to change only when the last element is modified.

The code in the example does not have any colormap toggle. Could you rephrase?

I tried running the example and things are working as-expected on the latest commit, 6bd0a10

5.mp4

[hsellik]

Initially, I was going to provide colormap toggle example, but found out that this applies to other properties, such as changing the opacity (via slider in the example).

To be more specific, when you change the ventricle opacity, the volume render behaves correctly, but when you switch to brain, the opacity does not affect volume render anymore. This is also visible in the video example.

[hsellik]

@jennydaman , do you need any more information?

[jennydaman]

oooohhh, i finally understand. You’re talking about the 3D render pane, not the 2D slice views (my mind unconsciously ignores the 3D render pane most of the times).

This is indeed a bug but I think it’s an upstream bug. I will investigate further and let the right people know.

[neurolabusc]

@jennydaman and @hsellik this is a feature not a bug of NiiVue. Slow voxel-based computations are deferred until you call updateGLVolume(). Therefore, setting a property like opacity does not take full effect until you call for updating the data (or some other change requires NiiVue to process deferred changes). This allows you to change lots of properties at once without the display bogging down, and then issue a refresh once you have everything ready.

The call updateGLVolume() refreshes everything, calling refreshLayers() for all of the current volumes. If you only want to update a single volume, you can call refreshLayers() directly, e.g. to refresh the background volume call refreshLayers(nv1.volumes[0]).

Be aware that updateGLVolume() is the most computationally expensive call in the entire NiiVue API, resynchronizing data on both the CPU and GPU. Due to this reason, for pure html I would call this on slider.onchange() but not on slider.oninput():

oninput → Fires continuously as the slider is being moved.
onchange → Fires only when the user stops moving the slider.

I do not use react, so unsure what the equivalence is.

[hsellik]

@neurolabusc , thank you yet again for a very detailed explanation. It was confusing that it did change the opacity when changing one of the volumes, but didn't change when changing the other volume.

There is some additional logic in the React wrapper that looks at the diff and calls updateGLVolume, but I have not looked very closely into that yet.

For now, I'll try to acquire the NiiVue instance with onStart callback, whenever I make changes, I debounce and call updateGLVolume or refreshLayers.

[jennydaman]

Bug is confirmed and reported upstream. niivue/niivue#1174

[jennydaman]

Note, I was unable to reproduce this bug with colormap but I was able to reproduce it with opacity (Niivue.setOpacity)