gv.Points raises an error with color dim in the presence of null coordinates

[maximlt]

It looks like it's a Geo/HoloViews bug. First report in holoviz/hvplot#1202.

import geoviews as gv
gv.extension('bokeh')

data = {"x": [10, None], "y": [10, None], "color": [1, 2]}
gv.Points(data, ["x", "y"], ["color"]).opts(color=gv.dim('color'))

failed to validate Scatter(id='p1074', ...).fill_color: expected an element of either String, Nullable(Color), Instance(Value), Instance(Field), Instance(Expr), Struct(value=Nullable(Color), transform=Instance(Transform)), Struct(field=String, transform=Instance(Transform)) or Struct(expr=Instance(Expression), transform=Instance(Transform)), got dim('color')

Full Traceback

Traceback (most recent call last):

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/holoviews/plotting/bokeh/element.py", line 2055, in _init_glyphs
    renderer, glyph = self._init_glyph(plot, mapping, properties)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/holoviews/plotting/bokeh/element.py", line 2886, in _init_glyph
    ret = super()._init_glyph(plot, mapping, properties)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/holoviews/plotting/bokeh/element.py", line 1744, in _init_glyph
    renderer = getattr(plot, plot_method)(**dict(properties, **mapping))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/plotting/glyph_api.py", line 1134, in scatter
    return self._scatter(*args, marker=marker_type, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/plotting/_decorators.py", line 89, in wrapped
    return create_renderer(glyphclass, self.plot, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/plotting/_renderer.py", line 113, in create_renderer
    glyph = make_glyph(glyphclass, kwargs, glyph_visuals)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/plotting/_renderer.py", line 142, in make_glyph
    return glyphclass(**kws)
           ^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyphs.py", line 1408, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyphs.py", line 169, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyph.py", line 80, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyph.py", line 101, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyph.py", line 111, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyph.py", line 131, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/models/glyph.py", line 58, in __init__
    super().__init__(*args, **kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/model/model.py", line 119, in __init__
    super().__init__(**kwargs)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/core/has_props.py", line 304, in __init__
    setattr(self, name, value)

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/core/has_props.py", line 336, in __setattr__
    return super().__setattr__(name, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/core/property/descriptors.py", line 330, in __set__
    value = self.property.prepare_value(obj, self.name, value)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/core/property/dataspec.py", line 599, in prepare_value
    return super().prepare_value(cls, name, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/Users/mliquet/dev/geoviews/.pixi/envs/default/lib/python3.12/site-packages/bokeh/core/property/bases.py", line 363, in prepare_value
    raise ValueError(f"failed to validate {obj_repr}.{name}: {error}")

ValueError: failed to validate Scatter(id='p1074', ...).fill_color: expected an element of either String, Nullable(Color), Instance(Value), Instance(Field), Instance(Expr), Struct(value=Nullable(Color), transform=Instance(Transform)), Struct(field=String, transform=Instance(Transform)) or Struct(expr=Instance(Expression), transform=Instance(Transform)), got dim('color')

HoloViews in _init_glyphs first calls get_data and then _apply_transforms.

https://github.com/holoviz/holoviews/blob/f9c7f05c4078cc7ebb571c6b5aa10c1cc2662962/holoviews/plotting/bokeh/element.py#L2080-L2093

Calling get_data will project the element first in GeoViews:

geoviews/geoviews/plotting/bokeh/plot.py

Lines 174 to 177 in 5f02c71

	def get_data(self, element, ranges, style):
	if self._project_operation and self.geographic:
	element = self._project_operation(element, projection=self.projection)
	return super().get_data(element, ranges, style)

The project_points operation drops rows with null coordinates. This was implemented a while back to fix another bug #169.

geoviews/geoviews/operation/projection.py

Lines 187 to 191 in 5f02c71

	mask = np.isfinite(coordinates[:, 0])
	dims = [d for d in element.dimensions() if d not in (xdim, ydim)]
	new_data = {k: v[mask] for k, v in element.columns(dims).items()}
	new_data[xdim.name] = coordinates[mask, 0]
	new_data[ydim.name] = coordinates[mask, 1]

So the data returned by the get_data call doesn't have the NaN rows. However, the element in this context and that is passed to _apply_transforms still does. In this method, the value for color=hv.dim('color') is obtained from:

https://github.com/holoviz/holoviews/blob/f9c7f05c4078cc7ebb571c6b5aa10c1cc2662962/holoviews/plotting/bokeh/element.py#L1830

which returns np.array([1, 2]) that has one more value than data.

If makes this condition True and then hits continue. This explains why hv.dim('color') is passed as is to Bokeh properties, it's not transformed at all.

https://github.com/holoviz/holoviews/blob/f9c7f05c4078cc7ebb571c6b5aa10c1cc2662962/holoviews/plotting/bokeh/element.py#L1845