Qt render canvas resizes before viewport rect is updated

[kushalkolar]

On Qt it seems like the resize event is sent only after the canvas has already resized, with pygfx this causes the renderer to try and draw the scene to a canvas that is smaller than expected. This does not happen on glfw or jupyter_rfb.

For example if we use this example and resize the window to make it smaller we get a GPUValidationError:

import imageio.v3 as iio
import PyQt6
from rendercanvas.qt import RenderCanvas, loop
import pygfx as gfx


canvas = RenderCanvas()
renderer = gfx.renderers.WgpuRenderer(canvas)
viewport = gfx.Viewport(renderer)
scene = gfx.Scene()

im = iio.imread("imageio:astronaut.png")[:, :, 1]

image = gfx.Image(
    gfx.Geometry(grid=gfx.Texture(im, dim=2)),
    gfx.ImageBasicMaterial(clim=(0, 255)),
)
scene.add(image)

camera = gfx.OrthographicCamera(512, 512)
camera.local.position = (256, 256, 0)
camera.local.scale_y = -1


def update_rect(*ev):
    viewport.rect = 0, 0, *canvas.get_logical_size()


if __name__ == "__main__":
    viewport.rect = 0, 0, *canvas.get_logical_size()
    canvas.add_event_handler(update_rect, "resize")

    canvas.request_draw(lambda: viewport.render(scene, camera, flush=True))
    loop.run()

Draw error
Traceback (most recent call last):
  File "/home/kushal/repos/rendercanvas/rendercanvas/_coreutils.py", line 41, in log_exception
    yield
  File "/home/kushal/repos/rendercanvas/rendercanvas/base.py", line 409, in _draw_frame_and_present
    self._draw_frame()
  File "/home/kushal/repos/pygfx/examples/feature_demo/geometry_image.py", line 43, in <lambda>
    canvas.request_draw(lambda: viewport.render(scene, camera, flush=True))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kushal/repos/pygfx/pygfx/utils/viewport.py", line 78, in render
    self.renderer.render(scene, camera, rect=self.rect, flush=flush)
  File "/home/kushal/repos/pygfx/pygfx/renderers/wgpu/engine/renderer.py", line 592, in render
    command_buffers += self._render_recording(
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kushal/repos/pygfx/pygfx/renderers/wgpu/engine/renderer.py", line 722, in _render_recording
    render_pass.end()
  File "/home/kushal/repos/wgpu-py/wgpu/backends/wgpu_native/_api.py", line 3289, in end
    libf.wgpuRenderPassEncoderEnd(self._internal)
  File "/home/kushal/repos/wgpu-py/wgpu/backends/wgpu_native/_helpers.py", line 394, in proxy_func
    raise wgpu_error  # the frame above is more interesting ↑↑
    ^^^^^^^^^^^^^^^^
wgpu._classes.GPUValidationError: Validation Error

Caused by:
  In wgpuRenderPassEncoderEnd
    In a set_viewport command
      Viewport has invalid rect Rect { x: 0.0, y: 0.0, w: 1280.0, h: 960.0 }; origin and/or size is less than or equal to 0, and/or is not contained in the render target (1278, 960, 1)

[almarklein]

I can indeed reproduce this. This is because the scheduler does basically this:

• wait
• process events, then ask backend to make a draw
• draw

I the canvas is resized between asking for a draw and the actual draw taking place, the size will mismatch. This delay is generally really small, and should not affect the experience (similar to how events can happen while the draw takes place). But resize events are special, because it can cause the canvas size to mismatch.

Perhaps the "resize" event should be special, possibly firing more often, and not allowing a sync handler (otherwise we cannot call it during the draw). This fits into the idea to maybe turn the draw into an event as well.

---

In the mean time, I'd recommend simply updating the rect on each draw, right before calling .render().

[almarklein]

I ran into a similar problem with imgui on glfw, see pygfx/wgpu-py#706

In short, the current system can sometimes do a draw before while not all resize events are emitted, causing a discrepancy in the size. We should solve this, but since this relates to the scheduling and async stuff, I don't want to do that right now.

The approach should be that stuff that requires the physical size should query it from the canvas, so it has the up-to-date version. Code that need the logical size is probably fine, maybe things scale a bit funny during resizing a window, but I think we can live with that until we fix it properly.