feat(pip): Add Picture-in-Picture support for Electron #16704

hristoterezov · 2025-11-25T05:05:55Z

Summary

Implements Picture-in-Picture functionality for the Electron wrapper to maintain video engagement when users are not actively focused on the conference window. The feature automatically manages PiP mode based on user interaction, displays either the large video participant's video stream or their avatar, and provides MediaSession API integration for audio/video mute controls directly from the PiP window.

Key Features

Automatic PiP management: Enters and exits PiP based on user interaction patterns
Smart video/avatar switching: Displays participant video when available, falls back to rendered canvas avatar when video is unavailable, muted, or not streaming
MediaSession controls: Provides audio and video mute/unmute buttons in the PiP window using the MediaSession API
Electron integration: Adds _pip-requested API event for proper user gesture handling in Electron wrapper

Implementation Details

New Feature Module (`react/features/pip/`)

Redux architecture with actions, reducer, and middleware
PiPVideoElement component manages hidden video element for PiP
Comprehensive avatar rendering on canvas with custom backgrounds support
MediaSession handlers for mute controls
Logger for debugging PiP operations

API Integration

Added notifyPictureInPictureRequested() method to internal API
Added _pip-requested event to external API for Electron communication
Allows Electron wrapper to handle PiP with proper user gesture context

Video Source Management

Automatically switches between real video track and canvas-rendered avatar
Canvas avatar includes participant initials or image with display name
Supports custom avatar backgrounds from dynamic branding
Maintains 16:9 aspect ratio for PiP window

State Management

Tracks PiP active state in Redux
Synchronizes with browser PiP events (enter/leave)
Integrates with existing audio/video mute logic from toolbar
Respects GUM pending states for optimistic UI updates

Dependencies

Depends on: Replace always-on-top with native picture-in-picture jitsi-meet-electron-sdk#479 for Electron PiP support
Uses existing Jitsi avatar rendering functions
Integrates with large video participant selection
Leverages track streaming status from connection indicator

Test Plan

saghul

Some early feedback. Nothing stands out as a big problem, good work!

saghul · 2025-11-25T13:27:18Z

react/features/pip/components/PiPVideoElement.tsx

+        if (!defaultIconRef.current) {
+            let svgText = IconUserSVG;
+
+            if (!svgText.includes('fill=')) {


Why is this all of this needed?

Which exactly?

We are generally retrieve the default icon svg content in order to draw it in the canvas (by rendering it and then using drawImage in the canvas).

If you mean the part with the fill, currently the svg doesn't have any color, so we specify it.

saghul · 2025-11-25T13:29:02Z

react/features/pip/components/PiPVideoElement.tsx

+        } else if (videoTrack?.jitsiTrack) {
+            // Attach real video track.
+            videoTrack.jitsiTrack.attach(videoElement)
+                .then(() => videoElement.play())


Is the play necessary here? When would it not play?

Removed this one and also the previous .play()

saghul · 2025-11-25T13:32:38Z

react/features/pip/functions.ts

+ * @param {IReduxState} state - Redux state.
+ * @returns {boolean} Whether audio should be shown as muted.
+ */
+export function isAudioMutedForPiP(state: IReduxState): boolean {


Rather than duplicate it, can we factor it out? I worry of both going out of sync.

well I've tried to mimic the audio button state. Basically the core logic is in isLocalTrackMuted(state['features/base/tracks'], MEDIA_TYPE.AUDIO) so they kind of share the implementation trough this function already. The additional part is about the GUM check but this is used only in the web toolbar button.

Do you want me to include somehow the GUM check in a function and use it here for the PiP and also for the toolbar button? The downside is that the native button don't need the GUM check so it won't use the function or I can also add react-native check.

The whole thing sounds like over-engineering but I'm not against it. Let me know if this is what you have in mind?

Do we really need the gUM check?

saghul · 2025-11-25T13:33:50Z

react/features/pip/functions.ts

+    ctx.fill();
+
+    ctx.fillStyle = '#FFFFFF';
+    ctx.font = 'bold 80px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif';


We probnably don't want to set the font here and let the default one?

saghul · 2025-11-25T13:34:21Z

react/features/pip/functions.ts

+        canvasWidth: number
+) {
+    ctx.fillStyle = '#FFFFFF';
+    ctx.font = '24px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif';


saghul · 2025-11-25T13:34:39Z

react/features/pip/functions.ts

+    const textY = centerY + avatarRadius + spacing;
+
+    // Clear and fill background.
+    ctx.fillStyle = '#474747';


Use a constant?

react/features/pip/functions.ts

saghul · 2025-11-25T13:38:12Z

react/features/pip/functions.ts

+        // This bypasses the transient activation requirement by executing
+        // requestPictureInPicture with userGesture: true in the main process.
+        if (browser.isElectron()) {
+            exposeRequestPiPForElectron();


It feels weird to be exposing the function on every click. We should do it only once, likely elsewhere?

Well inside exposeRequestPiPForElectron I kind of check if we set it. Although on second thought probably this is even less efficient than just setting it.

I agree with you here. Do you have any suggestions where to put it? Maybe here in the global scope of the file instead of in the function? WDYT?

I've put it in the subscriber. When PiP is enabled we attach it and when it is disabled we delete it. It will normally happens once but in theory you can update the config to disable/enable PiP...

custom.d.ts

webpack.config.js

filmstrip/actions.web was imported in TileView native component. filmstrip/actions.web was imported in config middleware.any.

Implements Picture-in-Picture functionality for the Electron wrapper to maintain video engagement when users are not actively focused on the conference window. This feature addresses the need to keep users visually connected to the conference even when multitasking. Key features: - Automatic PiP mode activation and deactivation based on user interaction - Displays large video participant's stream or renders their avatar on canvas when video unavailable - Provides audio/video mute controls via MediaSession API directly in PiP window - Adds API events (_pip-requested) for Electron wrapper integration Implementation includes new pip feature module with Redux architecture, canvas-based avatar rendering with custom backgrounds support, and integration with existing mute/unmute logic. Depends on jitsi-meet-electron-sdk#479 for proper user gesture handling in Electron.

saghul · 2025-12-04T09:29:22Z

react/features/base/config/middleware.web.ts

+ * @private
+ * @returns {*} The return value of {@code next(action)}.
+ */
+function _setDynamicBrandingData({ dispatch }: IStore, next: Function, action: AnyAction) {


Isn't this already handled in the shared middleware?

saghul · 2025-12-04T09:30:27Z

react/features/base/config/middleware.web.ts

@@ -0,0 +1,251 @@
+import { AnyAction } from 'redux';


Not sure I understand the motivation behind this change, since there is no pip code involved.

saghul · 2025-12-04T09:32:21Z

react/features/pip/actions.ts

+export function toggleAudioFromPiP() {
+    return (dispatch: IStore['dispatch'], getState: IStore['getState']) => {
+        const state = getState();
+        const audioMuted = isAudioMutedForPiP(state);


WHy do we need specific "for pip" functions?

saghul · 2025-12-04T09:32:55Z

react/features/pip/actions.ts

+            document.exitPictureInPicture().catch((err: Error) => {
+                logger.error(`Error while exiting PiP: ${err.message}`);
+            });
+            logger.log('Exited Picture-in-Picture mode');


You should log this in a then() otherwise it will be printed always.

saghul · 2025-12-04T09:35:29Z

react/features/pip/functions.ts

+ *
+ * @param {IReduxState} state - Redux state.
+ * @param {IParticipant | undefined} participant - Participant to get track for.
+ * @returns {any} The video track or undefined.


We have types for track don't we?

saghul · 2025-12-04T09:36:27Z

react/features/pip/functions.ts

+ * @param {IReduxState} state - Redux state.
+ * @returns {boolean} Whether audio should be shown as muted.
+ */
+export function isAudioMutedForPiP(state: IReduxState): boolean {


Do we really need the gUM check?

react/features/pip/functions.ts

saghul · 2025-12-04T09:40:25Z

modules/API/external/external_api.js


+                // Since this is internal event we don't need to emit it to the consumer of the API.
+                return true;
+            case 'config-overwrite': {


Won't we need the same logic when a user changes the setting? When we add it, that is. In that case, doesn't it make more sense to emit an event when the PiP setting changes? Overriding the config should effectively change the setting.

saghul · 2025-12-04T09:41:49Z

react/features/pip/external-api.shared.ts

+ * It's bundled into external_api.min.js and we want to keep that bundle slim.
+ * Only import lightweight modules here.
+ */
+import BrowserDetection from '@jitsi/js-utils/browser-detection/BrowserDetection';


THis will pull the entire UA parser library, won't it? Can't we just inline the Electron check?

saghul reviewed Nov 25, 2025

View reviewed changes

hristoterezov force-pushed the add-video-pip branch from 7b86902 to 15a7ba0 Compare November 25, 2025 14:15

hristoterezov force-pushed the add-video-pip branch 5 times, most recently from 166c2d4 to f3b8d24 Compare December 4, 2025 00:05

hristoterezov marked this pull request as ready for review December 4, 2025 00:05

hristoterezov added 2 commits December 3, 2025 22:11

fix(RN): Remove web files from build.

f5efe5b

filmstrip/actions.web was imported in TileView native component. filmstrip/actions.web was imported in config middleware.any.

hristoterezov force-pushed the add-video-pip branch from f3b8d24 to 815ae7d Compare December 4, 2025 04:17

saghul reviewed Dec 4, 2025

View reviewed changes

feat(pip): Add Picture-in-Picture support for Electron #16704

Are you sure you want to change the base?

feat(pip): Add Picture-in-Picture support for Electron #16704

Conversation

hristoterezov commented Nov 25, 2025

Summary

Key Features

Implementation Details

New Feature Module (react/features/pip/)

API Integration

Video Source Management

State Management

Dependencies

Test Plan

Uh oh!

saghul left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

New Feature Module (`react/features/pip/`)