Slow SVG updates for interactive images with PIL source

[denniswittich]

First Check

• I added a very descriptive title here.
• This is not a Q&A. I am sure something is wrong with NiceGUI or its documentation.
• I used the GitHub search to find a similar issue and came up empty.

Example Code

import time
from io import BytesIO

import numpy as np
import PIL.Image
from fastapi import Response
from nicegui import app, ui


def startup():
    @ui.page('/')
    def index_page():
        ui.label('Choose a test page:')
        ui.link('Fast Images', '/fast')
        ui.link('Slow Images', '/slow')

    @ui.page('/fast')
    def fast_images_page():
        ui.label('Showing image from resource endpoint')
        timestamp = time.time()
        ii = ui.interactive_image(source=f'/image/{timestamp}').classes('w-1/2')
        ui.button('Click me!', on_click=lambda: ii.set_content(_random_svg()))

    @ui.page('/slow')
    def slow_images_page():
        ui.label('Showing image from generated PIL image')
        pil_image = _gen_pil_image()
        ii = ui.interactive_image(source=pil_image).classes('w-1/2')
        ui.button('Click me!', on_click=lambda: ii.set_content(_random_svg()))

    app.add_api_route('/image/{timestamp}', _get_image)


def _gen_pil_image() -> PIL.Image.Image:
    random_pixels = np.random.randint(0, 255, (2000, 2000, 3), dtype=np.uint8)
    pil_image = PIL.Image.fromarray(random_pixels)
    return pil_image


def _get_image(timestamp: str) -> Response:  # pylint: disable=unused-argument
    pil_image = _gen_pil_image()
    buffer = BytesIO()
    pil_image.save(buffer, format='JPEG', quality=85)
    jpeg_bytes = buffer.getvalue()
    return Response(content=jpeg_bytes, media_type='image/jpeg')


def _random_svg() -> str:
    colors = ['red', 'green', 'blue', 'yellow', 'purple', 'orange']
    svg_parts = ['<svg xmlns="http://www.w3.org/2000/svg" width="2000" height="2000">']
    for _ in range(10):
        x = np.random.randint(0, 2000)
        y = np.random.randint(0, 2000)
        radius = np.random.randint(10, 50)
        color = np.random.choice(colors)
        svg_parts.append(f'<circle cx="{x}" cy="{y}" r="{radius}" fill="{color}" />')
    svg_parts.append('</svg>')
    return ''.join(svg_parts)


if __name__ in {"__main__", "__mp_main__"}:
    app.on_startup(startup)
    ui.run(title='Slow Images Test', port=8080)

Description

The MRP provides two pages that do the exact same thing, which is to generate a random svg overlay for an random image, both using ui.interactive_image. Both pages have an button to regenerate a random SVG.

When providing the image source via an endpoint, everything works as expected. However, when setting the image source as PIL.Image the updates are super slow.

I expected that later variant would internally do the same and, thus, not be slower.

NiceGUI Version

3.3.1

Python Version

3.12.0

Browser

Chrome

Operating System

Linux

Additional Context

No response

[evnchn]

Hello @denniswittich

I have successfully debugged why.

nicegui/nicegui/elements/image.py

Lines 47 to 58 in dcfda54

	def pil_to_base64(pil_image: 'PIL_Image', image_format: str) -> str:
	"""Convert a PIL image to a base64 string which can be used as image source.
	:param pil_image: the PIL image
	:param image_format: the image format
	:return: the base64 string
	"""
	buffer = io.BytesIO()
	pil_image.save(buffer, image_format)
	base64_encoded = base64.b64encode(buffer.getvalue())
	base64_string = base64_encoded.decode('utf-8')
	return f'data:image/{image_format.lower()};base64,{base64_string}'

Shoving base64 into source has 2 side effects if the image is large:

• Page becomes large, so slow initial page load
• Due to the fact that for element update we (so far) send the entire props dict over, it also sends to base64 string, so updates are also slow

To fix this, we need deep engineering to use a source path rather than base64.

I would recommend to just host the image over an endpoint like what you have done for now, as the fix won't be fast, likely...

[falkoschindler]

Just for reference, a more compact reproduction:

@ui.page('/slow')
def slow():
    ui.button('Update', on_click=lambda: ii.set_content(f'<text x="100" y="100">{time.time()}</text>'))
    ii = ui.interactive_image(PIL.Image.fromarray(np.random.randint(0, 255, (2000, 2000, 3), dtype=np.uint8)))

@ui.page('/fast')
def fast():
    ui.button('Update', on_click=lambda: ii.set_content(f'<text x="100" y="100">{time.time()}</text>'))
    ii = ui.interactive_image(f'/image/{time.time()}')

@app.get('/image/{timestamp}')
def image() -> Response:
    buffer = BytesIO()
    PIL.Image.fromarray(np.random.randint(0, 255, (2000, 2000, 3), dtype=np.uint8)).save(buffer, format='JPEG')
    return Response(buffer.getvalue(), media_type='image/jpeg')