screencopy: Fix transform for damage rect advertised to client

This fixes two issues:
- The `area` passed to `to_buffer()` should match the dimensions of the
  output/etc. being captured, rather than coming from the damage rect
  size.
- The transform needs to be inverted.

Previously, rotated outputs could cause a crash
`xdg-desktop-portal-cosmic`, since the compositor was passing negative
coordinates in `damage`, and the client used the same in
`damage_buffer`. This was causing
https://github.com/pop-os/xdg-desktop-portal-cosmic/issues/165.

The portal crash no longer occurs, and logging in
xdg-desktop-portal-cosmic shows damage rects that match expectation
while moving the cursor over different corners of a workspace.
This commit is contained in:
Ian Douglas Scott 2025-08-26 16:47:52 -07:00 committed by Victoria Brekenfeld
parent 321894728a
commit 9cc68dcbcc

View file

@ -118,9 +118,10 @@ where
};
let buffer = frame.buffer();
let buffer_size = buffer_dimensions(&buffer).unwrap();
if let Some(fb) = offscreen {
assert!(matches!(buffer_type(&buffer), Some(BufferType::Shm)));
let buffer_size = buffer_dimensions(&buffer).unwrap();
if let Err(err) = with_buffer_contents_mut(&buffer, |ptr, len, data| {
let offset = data.offset;
let width = data.width;
@ -175,7 +176,7 @@ where
.into_iter()
.map(|rect| {
let logical = rect.to_logical(1);
logical.to_buffer(1, transform, &logical.size)
logical.to_buffer(1, transform.invert(), &buffer_size.to_logical(1, transform))
})
.collect(),
sync,