GUACAMOLE-1415: RDP Camera Redirection (RDPECAM) - Client Implementation

[tanonl]

Note: This PR works in conjunction with the corresponding guacamole-server PR for complete camera redirection functionality.

Overview

This PR implements client-side support for RDP Camera Redirection using the MS-RDPECAM protocol. This enables web browsers to stream local camera video (H.264-encoded) through Apache Guacamole to remote Windows sessions, with features including per-camera selection, persistent preferences, and audio/video synchronization controls.

---

Problem Statement

Users connecting to remote Windows desktops need seamless access to their local cameras for video conferencing, authentication, and other applications. This client-side implementation provides the browser-based camera capture, H.264 encoding, and user interface necessary to stream camera data to the guacamole-server RDPECAM implementation.

---

Technical Implementation

1. Camera Recording Foundation (07db985b4)

Generic, reusable camera recording infrastructure in guacamole-common-js/src/main/webapp/modules/:

CameraRecorder.js - Abstract Camera Recorder

Core recording engine with H.264 encoding:

Key Features:

• MediaRecorder API integration with H.264 codec support
• Device enumeration via navigator.mediaDevices.enumerateDevices()
• Configurable video constraints:
  • Resolution (width, height)
  • Frame rate (15-60 fps)
  • Bit rate (500kbps - 8Mbps)
• Camera switching without stopping recording
• Capability caching for performance

Design Philosophy:

• Protocol-agnostic (not specific to RDPECAM)
• Reusable for any Guacamole protocol requiring camera access
• Clean separation of concerns (capture vs. protocol handling)

H264AnnexBUtil.js - H.264 Processing Utilities

Low-level H.264 Annex-B format utilities:

Capabilities:

• NAL unit extraction from byte streams
• Start code detection (0x00000001, 0x000001)
• SPS (Sequence Parameter Set) parsing
• PPS (Picture Parameter Set) parsing
• Frame boundary detection
• Annex-B format validation

Use Case:
Essential for processing MediaRecorder output into proper H.264 frames for transmission via Guacamole protocol.

2. RDPECAM Service (6efbab318)

Complete RDPECAM protocol implementation in guacamole/src/main/frontend/src/app/client/services/:

guacRDPECAM.js - RDPECAM Service

The centerpiece of the client implementation, providing:

Core Protocol Features:

Capability Negotiation:

• Parse device capabilities from RDPECAM channel
• Match browser MediaRecorder capabilities with server requirements
• Generate optimal video constraints (resolution, frame rate, format)

Camera Lifecycle Management:

• Initialize CameraRecorder with negotiated constraints
• Start/stop camera streams
• Handle camera switching
• Device enumeration with capability prefetching
• Error recovery and retry logic

Stream Management:

• H.264 frame packetization
• Guacamole stream creation and management
• Frame transmission via Guacamole protocol
• Proper stream termination

Video Delay Synchronization Feature:

Addresses audio/video synchronization issues in remote video calls:

Problem: Video frames may arrive faster than audio, causing lip-sync issues in video conferencing applications.

Solution: Configurable video delay (0-1000ms) that buffers outgoing video frames.

Features:

• Delay range: 0-1000ms (adjustable in 50ms increments)
• Slider visible in left menu when camera in use
• Visual toast notifications for delay changes
• localStorage persistence (per-connection)
• Real-time adjustment during active streaming

3. UI Integration (07966fc90, d68ad85f8)

User interface for camera control in guacamole/src/main/frontend/src/app/client/:

clientController.js (165 new lines)

AngularJS controller integration:

Camera State Management:

$scope.cameraState = {
    available: false,        // RDPECAM supported
    active: false,           // Camera streaming
    devices: [],             // Available cameras
    selectedDevice: null,    // Current camera
    videoDelay: 0           // Current A/V delay
};

client.html (58 lines modified)

Template updates for camera controls:

Camera Selection Dropdown:

<select ng-model="cameraState.selectedDevice"
        ng-change="selectCamera(cameraState.selectedDevice)"
        ng-options="device.deviceId as device.label for device in cameraState.devices">
</select>

Status Indicators:

<span class="camera-status" ng-show="cameraState.active">
    <i class="icon-camera"></i> In Use
</span>

Video Delay Display:

<div class="video-delay-notification" ng-show="videoDelayNotification.visible">
    Video Delay: {{cameraState.videoDelay}}ms
</div>

ManagedClient.js (69 lines modified)

Service Integration:

• Wire RDPECAM service into ManagedClient
• Expose camera state to controllers
• Event propagation for camera state changes

4. Configuration Support (c59f1b9a5)

extensions/guacamole-auth-json/protocols/rdp.json

Protocol configuration additions
Purpose:

• Declares H.264 video mimetype support
• Enables video instruction handling
• Configures RDPECAM feature availability

guacamole-common-js/src/main/webapp/modules/Client.js

Core client updates for video streaming:

Video Instruction Handler:

Guacamole.Client.prototype.handleVideoInstruction = function(stream, mimetype, duration) {
    // Route to appropriate protocol handler
    if (this.onvideo)
        this.onvideo(stream, mimetype, duration);
};

Stream Management:

• Create video streams for camera data
• Handle mimetype negotiation
• Manage stream lifecycle

5. Preference Persistence (preferenceService.js modifications)

Camera selection persistence:
Benefits:

• Remember user's preferred cameras
• Persist video delay settings
• Reduce setup friction for returning users

6. Polish and Refinements

Error Handling (6839eadcb)

• Promise catch handlers for all async operations:
  • enumerateDevices() failures
  • getUserMedia() denials
  • Stream creation errors
• Consistent error logging
• User-friendly error messages

---

Key Features

For End Users

✅ Zero installation - Works entirely in browser, no plugins
✅ Camera selection - Choose from multiple cameras if available
✅ Persistent preferences - Remember camera choice per connection
✅ Visual feedback - Clear indicators for camera status
✅ A/V sync control - Adjust video delay for lip-sync (0-1000ms)
✅ Real-time switching - Change cameras without reconnecting

Technical Capabilities

• H.264 encoding via browser MediaRecorder API
• Automatic capability negotiation with server
• Adaptive constraints based on camera capabilities
• Frame packetization for Guacamole protocol
• Protocol-agnostic foundation (CameraRecorder reusable)
• Unit tested H.264 utilities

---

Browser Compatibility

Requirements

• MediaRecorder API support
• H.264 encoding support
• getUserMedia support
• WebRTC capabilities

Tested Browsers

• ✅ Chrome 90+ (full support)
• ✅ Edge 90+ (full support)
• ✅ Firefox 100+ (requires H.264 configuration)
• ⚠️ Safari 14+ (limited H.264 support)

---

Known Limitations

1. Browser support - Requires MediaRecorder API with H.264 encoding
2. H.264 only - No fallback codecs currently supported
3. Single streaming camera - One streaming camera per connection
4. No audio - Video-only streaming (by design, audio handled separately in RDP)
5. Manual sync - Video delay adjustment is manual, not automatic

---

Security Considerations

• Camera permissions - Browser prompts for user consent
• HTTPS required - getUserMedia requires secure context
• Privacy controls - Clear visual indicators when camera active
• No persistent access - Camera stops on disconnect/tab close

---

Ready for review and merge.

[tanonl]

As you will realize, this PR have been mostly using AI Coding. While exploring agentic AI programming, I wanted to do something actually useful and It can out pretty good.
Special thanks to Oleg for his guidance over LinkedIn. It's actually him who implemented it the support of RDPECAM in FreeRDP

[necouchman]

As you will realize, this PR have been mostly using AI Coding. While exploring agentic AI programming, I wanted to do something actually useful and It can out pretty good. Special thanks to Oleg for his guidance over LinkedIn. It's actually him who implemented it the support of RDPECAM in FreeRDP

Thanks for letting us know this - the Apache Software Foundation has some governance around using AI-generated code, so I'm going to rely on @mike-jumper to provide guidance on that.

Also, the PR title needs to be changed to match the format of the commit messages: GUACAMOLE-1415: RDP....