Camera background blur and new city backgrounds

[richiemcilroy]

Camera background blur (on-device segmentation, Light/Heavy), a cities wallpaper pack, smaller stock backgrounds, plus sleep/wake recovery and window-position persistence.

Greptile Summary

This PR introduces on-device camera background blur (Light/Heavy) using an ONNX selfie-segmentation model + GPU Gaussian blur pipeline (cap-camera-effects crate), a cities wallpaper pack, and several reliability improvements — sleep/wake recovery via power_observer, cancellable FakeWindowListeners, and debounced window-position persistence.

• P1 – stride ignored in WsBlurState::process (camera_legacy.rs): the _stride parameter is suppressed and bytes_per_row is hard-coded to width * 4; if the FFmpeg scaler outputs a padded row stride the GPU texture upload will be corrupted.
• P2 – panic in CoreML/DirectML fallback (segmentation.rs): if Session::builder() fails in the EP-registration error branch the app panics instead of propagating an error.
• P2 – blur always reset to Off on record start (recording.rs): exported recordings will never carry blur even when the user had it enabled; intent should be clarified with a comment.

Confidence Score: 4/5

Safe to merge after addressing the stride mismatch in WsBlurState::process; all other issues are non-blocking.

One P1 finding (stride ignored in WsBlurState::process) that can corrupt the blurred WebSocket preview output when the FFmpeg scaler produces a padded stride. The remaining findings are P2 — a panic risk in an unlikely fallback path and an undocumented design choice around blur being disabled during recording.

apps/desktop/src-tauri/src/camera_legacy.rs (stride bug), crates/camera-effects/src/segmentation.rs (panic fallback)

Important Files Changed

Filename	Overview
crates/camera-effects/src/lib.rs	New crate implementing on-device background segmentation (ONNX) + two-pass Gaussian blur + composite pipeline; async GPU readback pattern is correct; EMA mask smoothing and output_generation counter look sound.
crates/camera-effects/src/segmentation.rs	Loads selfie_segmentation ONNX model; CoreML/DirectML EP fallback uses Session::builder().unwrap_or_else(
apps/desktop/src-tauri/src/camera_legacy.rs	Adds double-buffered GPU readback blur path for WebSocket camera preview; stride parameter is suppressed with _ and bytes_per_row is hardcoded to width*4, risking corrupted texture upload if FFmpeg adds row padding.
apps/desktop/src-tauri/src/camera.rs	Integrates BlurProcessor into native camera preview renderer; acquire_surface_texture refactor with Lost/Outdated recovery is a solid improvement; blur path using copy_texture_to_texture + render_no_upload is correct.
apps/desktop/src-tauri/src/power_observer.rs	New module; macOS uses NSWorkspace notification center with objc2, Windows uses PowerRegisterSuspendResumeNotification; both store handle behind OnceLock+Mutex and uninstall cleanly.
apps/desktop/src-tauri/src/fake_window.rs	FakeWindowListeners with cancellation tokens replaces the unbounded spin loop; sleep-guard and dead-window detection via consecutive error threshold are well-handled.
apps/desktop/src-tauri/src/recording.rs	Background blur is hard-coded to Off on recording start, so the exported project config will never carry a blur mode even if the user had blur enabled; may be intentional but undocumented.
crates/rendering/src/layers/camera.rs	Adds blur_active flag, blur_bind_group, and blur cache keyed on (recording_time, mode, texture_idx, output_generation); blur_active is reset to false at start of both prepare functions so attach_shared_blur must always be called after prepare.
apps/desktop/src/routes/camera.tsx	Adds blur cycle button and refactors resize handles into CameraResizeHandles component; backgroundBlur typed as BackgroundBlurMode
crates/project/src/configuration.rs	Adds BackgroundBlurMode enum (Copy+Default=Off) and BackgroundBlurConfig struct with serde defaults; backward-compatible with existing project files.
apps/desktop/src-tauri/src/window_position_persistence.rs	New debounced writer (350 ms debounce, 150 ms min interval) for main-window and camera-window position persistence; uses Notify+spawn_blocking cleanly.
apps/desktop/src/routes/editor/ConfigSidebar.tsx	Adds city wallpapers to WALLPAPER_NAMES list and a Background Blur KSelect dropdown in the camera config section; mirrors existing shape/position selects.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Camera Frame - FFmpeg RGBA] --> B{blur enabled?}
    B -- No --> C[Raw frame to WSFrame / PreparedTexture.render]
    B -- Yes --> D[write_texture to blur_source_texture]
    D --> E[BlurProcessor.process]
    E --> F1[Downsample 256x256]
    F1 --> F2[GPU readback async]
    F2 --> F3[ONNX selfie segmentation inference]
    F3 --> F4[EMA smoothed mask upload]
    F4 --> F5[Two-pass Gaussian blur]
    F5 --> F6[Composite: sharp + blurred + mask]
    F6 --> G{Output path}
    G -- Native preview --> H[copy_texture_to_texture + render_no_upload + present]
    G -- WS legacy preview --> I[Double-buffered GPU readback to WSFrame]
    G -- Rendering export --> J[attach_shared_blur + blur_bind_group in CameraLayer]

Loading

Comments Outside Diff (2)

1. apps/desktop/src-tauri/src/camera_legacy.rs, line 581-594 (link)

   Stride ignored when uploading frame to GPU texture

   _stride is accepted but never used — bytes_per_row is hardcoded to width * 4 instead. If the FFmpeg RGBA frame has row padding (stride > width × 4, which can happen when the scaler outputs an alignment-padded buffer), wgpu will misinterpret the row boundaries and the blurred output will be corrupted. The _ prefix only suppresses the unused-variable warning but does not make the mismatch safe.

   res.queue.write_texture(
       // ...
       wgpu::TexelCopyBufferLayout {
           offset: 0,
           bytes_per_row: Some(_stride),  // use the actual stride, not width * 4
           rows_per_image: Some(height),
       },

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: apps/desktop/src-tauri/src/camera_legacy.rs
   Line: 581-594
   
   Comment:
   **Stride ignored when uploading frame to GPU texture**
   
   `_stride` is accepted but never used — `bytes_per_row` is hardcoded to `width * 4` instead. If the FFmpeg RGBA frame has row padding (stride > width × 4, which can happen when the scaler outputs an alignment-padded buffer), wgpu will misinterpret the row boundaries and the blurred output will be corrupted. The `_` prefix only suppresses the unused-variable warning but does not make the mismatch safe.
   
   ```rust
   res.queue.write_texture(
       // ...
       wgpu::TexelCopyBufferLayout {
           offset: 0,
           bytes_per_row: Some(_stride),  // use the actual stride, not width * 4
           rows_per_image: Some(height),
       },
   ```
   
   How can I resolve this? If you propose a fix, please make it concise.

2. crates/camera-effects/src/segmentation.rs, line 1107-1113 (link)

   Panic in non-fatal CoreML/DirectML fallback path

   When CoreML (or DirectML on Windows) EP registration fails, the error branch discards the original builder and calls Session::builder().unwrap_or_else(|_| panic!(...)). If Session::builder() fails in this fallback, the application panics — but that failure is independent of the CoreML/DirectML error and could happen on resource-constrained systems. Returning the error via anyhow::Result would let the caller decide how to handle it, rather than crashing.

   The same pattern exists in try_register_directml.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: crates/camera-effects/src/segmentation.rs
   Line: 1107-1113
   
   Comment:
   **Panic in non-fatal CoreML/DirectML fallback path**
   
   When CoreML (or DirectML on Windows) EP registration fails, the error branch discards the original `builder` and calls `Session::builder().unwrap_or_else(|_| panic!(...))`. If `Session::builder()` fails in this fallback, the application panics — but that failure is independent of the CoreML/DirectML error and could happen on resource-constrained systems. Returning the error via `anyhow::Result` would let the caller decide how to handle it, rather than crashing.
   
   The same pattern exists in `try_register_directml`.
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: apps/desktop/src-tauri/src/camera_legacy.rs
Line: 581-594

Comment:
**Stride ignored when uploading frame to GPU texture**

`_stride` is accepted but never used — `bytes_per_row` is hardcoded to `width * 4` instead. If the FFmpeg RGBA frame has row padding (stride > width × 4, which can happen when the scaler outputs an alignment-padded buffer), wgpu will misinterpret the row boundaries and the blurred output will be corrupted. The `_` prefix only suppresses the unused-variable warning but does not make the mismatch safe.

```rust
res.queue.write_texture(
    // ...
    wgpu::TexelCopyBufferLayout {
        offset: 0,
        bytes_per_row: Some(_stride),  // use the actual stride, not width * 4
        rows_per_image: Some(height),
    },
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: crates/camera-effects/src/segmentation.rs
Line: 1107-1113

Comment:
**Panic in non-fatal CoreML/DirectML fallback path**

When CoreML (or DirectML on Windows) EP registration fails, the error branch discards the original `builder` and calls `Session::builder().unwrap_or_else(|_| panic!(...))`. If `Session::builder()` fails in this fallback, the application panics — but that failure is independent of the CoreML/DirectML error and could happen on resource-constrained systems. Returning the error via `anyhow::Result` would let the caller decide how to handle it, rather than crashing.

The same pattern exists in `try_register_directml`.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: apps/desktop/src-tauri/src/recording.rs
Line: 607-610

Comment:
**Background blur is always reset to `Off` when recording starts**

`background_blur` is hard-coded to `BackgroundBlurMode::Off` in the camera state that `start_recording` sends to the preview. `project_config_from_recording` then propagates this value into the project config, so exported/rendered recordings will never have blur applied even if the user had blur enabled before clicking Record. If this is intentional (blur is a live-preview-only feature), a comment would help clarify the design decision.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "chore(desktop): re-encode stock blue and..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[paragon-review]

Paragon Review Skipped

Hi @richiemcilroy! Your Polarity credit balance is insufficient to complete this review.

Please visit https://app.paragon.run to finish your review.

[tembo]

The cache key is so specific that this will rebuild the blur bind group basically every frame (since current_texture / recording_time change). Since the bind group just points at the processor’s output view, you can usually cache on (mode, output_generation).

Suggested change

	let cache_hit = matches!(
	let cache_hit = matches!(
	self.blur_cache,
	Some(entry) if entry.mode == mode && entry.output_generation == processor_generation
	) && self.blur_bind_group.is_some();

[tembo]

rgba_256x256 is assumed to be the right size; if the readback ever returns a shorter buffer this will panic on indexing. A quick length check keeps it safe.

Suggested change

	let channel_size = MODEL_INPUT_SIZE * MODEL_INPUT_SIZE;
	let channel_size = MODEL_INPUT_SIZE * MODEL_INPUT_SIZE;
	let expected_len = channel_size * 4;
	if rgba_256x256.len() < expected_len {
	return Err(anyhow::anyhow!(
	"Expected {expected_len} RGBA bytes, got {}",
	rgba_256x256.len()
	));
	}
	let mut flat = vec![0.0f32; 3 * channel_size];

[tembo]

The panic fallback here seems a bit harsh for something that’s essentially an optional acceleration path. Might be worth making the provider registration helpers return anyhow::Result<SessionBuilder> and just fall back to CPU (or propagate) without panicking if builder creation fails.

[tembo]

No need for let _ = here; the return value isn’t #[must_use].

Suggested change

	let _ = processor.process(device, queue, source_texture, mode);
	processor.process(device, queue, source_texture, mode);

[greptile-apps]

Background blur is always reset to Off when recording starts

background_blur is hard-coded to BackgroundBlurMode::Off in the camera state that start_recording sends to the preview. project_config_from_recording then propagates this value into the project config, so exported/rendered recordings will never have blur applied even if the user had blur enabled before clicking Record. If this is intentional (blur is a live-preview-only feature), a comment would help clarify the design decision.

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/desktop/src-tauri/src/recording.rs
Line: 607-610

Comment:
**Background blur is always reset to `Off` when recording starts**

`background_blur` is hard-coded to `BackgroundBlurMode::Off` in the camera state that `start_recording` sends to the preview. `project_config_from_recording` then propagates this value into the project config, so exported/rendered recordings will never have blur applied even if the user had blur enabled before clicking Record. If this is intentional (blur is a live-preview-only feature), a comment would help clarify the design decision.

How can I resolve this? If you propose a fix, please make it concise.