feat: add on-demand live transcoding for browser-incompatible video formats

[cat101]

Hi. I love that pigallery2 works over the filesystem and does not need to create a parallel database/index. I found that one exception was the need to pre-transcode all the incompatible videos. In may case, I have over 100k files which most of them will never be accessed. In my case I also run on a beefy server so I can transcode on near realtime.

Since some folks may run on smaller hardware I have made this feature as opt-in. Like you did with transcoding I have built a segment cache so that work is done only once. The feature also plays nice with existing transcoded files

I know this is a more complex feature so let me know if you need any changes to merge it. This feature was executed using AI. It is running in my home server and I'm using it/testing it/enjoying it daily :)

--------------------------- AI Generated summary --------------------

Adds an opt-in HLS live transcoding fallback for video formats that browsers cannot decode natively (RealMedia, AVI, MKV/non-H.264, MOV, WMV, etc.).

How it works

When a video fails to load natively (onSourceError), and liveVideoTranscodingEnabled is true in Settings → Media → Video, the browser lazy-loads hls.js v1.x and requests an HLS playlist from the new /api/gallery/hls/:path/playlist.m3u8 endpoint.

The backend spawns FFmpeg on-demand:

• Transmux mode (-c copy) for H.264+AAC sources (MKV, etc.) — near-instant
• Transcode mode (libx264 + aac) for incompatible codecs (RealMedia, AVI, WMV, etc.)

fMP4 segments (.m4s) are served progressively as FFmpeg writes them. The playlist is returned as soon as the first segment is ready (~6 seconds), using -hls_playlist_type event so hls.js treats it as a live/growing stream. Playback starts immediately while remaining segments are transcoded in the background. When FFmpeg finishes it appends #EXT-X-ENDLIST, converting the playlist to a standard VOD manifest — hls.js detects this automatically and enables full seeking.

FFmpeg flags for early-start streaming:

• -hls_playlist_type event — playlist grows dynamically; #EXT-X-ENDLIST written at end
• -hls_init_time 0 — write first segment at first keyframe boundary (minimal delay)
• -hls_flags independent_segments+discont_start — decoder-independent segments; discont_start handles PTS discontinuities common in RealMedia/AVI sources
• -force_key_frames expr:gte(t,n_forced*6) (transcode mode only) — inserts an IDR keyframe every 6s so FFmpeg can always cut at the target boundary. Without this, RealMedia sources produce 12–15s segments (TARGETDURATION: 12) with 22s gaps between segments. With forced keyframes: TARGETDURATION: 6, exact 6s EXTINF, and playback resumes in ~6s after each segment is encoded.

Playlist serving strategy (server-side long-poll): The playlist endpoint holds the HTTP connection open until FFmpeg writes a new segment (or ENDLIST), then responds immediately. This eliminates hls.js's fixed targetDuration/2 poll interval lag — the player receives the updated manifest the moment a segment is produced (~100ms after FFmpeg flushes it), rather than waiting up to 3s for its scheduled poll. Cache-Control: no-cache, no-store ensures the response is never cached.

Interrupted transcode recovery: on startup, if a cached playlist.m3u8 exists but lacks #EXT-X-ENDLIST, the cache dir is deleted and FFmpeg is re-spawned from scratch.

What's unchanged

• All native mp4/webm playback
• Pre-transcoded files served via /bestFit (no HLS overhead)
• <video> element, <source>, all player controls (seek, volume, loop, fullscreen)
• Auth — HLS routes use same auth chain as gallery

New & modified files

File	Change
src/backend/middlewares/HLSMWs.ts	New — FFmpeg job lifecycle, playlist, segment serving
src/backend/routes/HLSRouter.ts	New — Express routes: playlist.m3u8, init.mp4, segment_*.m4s
src/frontend/.../hls.d.ts	New — Ambient type stub so tsc compiles before npm install completes
src/common/config/public/ClientConfig.ts	liveVideoTranscodingEnabled: boolean = false in ClientVideoConfig
src/backend/routes/Router.ts	Register HLSRouter
src/backend/model/jobs/jobs/TempFolderCleaningJob.ts	Kill active HLS jobs + rm -rf ./tmp/hls/ on cleanup
src/frontend/.../MediaIcon.ts	getHLSPlaylistPath() method
src/frontend/.../media.lightbox.gallery.component.ts	onSourceError() fallback + initHLSPlayer() / destroyHLS()
package.json	"hls.js": "1.5.20" dependency

New config

Media.Video.liveVideoTranscodingEnabled (boolean, default: false) — appears in the standard Media → Video settings panel, tagged as experimental.

Dependencies

• hls.js v1.x (frontend, lazy-loaded — only bundled when feature is used)
• No new backend dependencies (uses existing fluent-ffmpeg + ffmpeg-static)

Segment cache layout

tmp/
  hls/
    <sha256(fullPath + mtime)>/
      playlist.m3u8       ← FFmpeg-generated (accurate EXTINF durations)
      init.mp4            ← fMP4 init segment
      segment_000.m4s     ← fMP4 media segments
      segment_001.m4s
      ...

Cache key includes mtime — invalidates automatically when source file changes. Cleaned by TempFolderCleaningJob (Settings → Jobs → Temp Folder Cleaning).

[cat101]

Mitigated by the route middleware chain: AuthenticationMWs.normalizePathParam('mediaPath') (path.normalize + strips leading ../) and AuthenticationMWs.authoriseMedia('mediaPath') (403 unless the user has gallery ACL on that media) both run before servePlaylist. Same protection as every existing /api/gallery/photo/:mediaPath/... and /api/gallery/video/:mediaPath/bestFit route in GalleryRouter. Treating as false positive — CodeQL can't reason about Express middleware chains.

[cat101]

Consistent with the rest of the codebase — pigallery2 has no rate-limiting on any existing route (no express-rate-limit dep, no usage in src/). /api/gallery/photo, /api/gallery/video/bestFit, /api/gallery/zip, and login all do authorization + FS access without a limiter. Adding one only to HLS routes would be inconsistent; if desired, it belongs in a separate repo-wide PR layered at Router.ts.

[cat101]

Consistent with the rest of the codebase — pigallery2 has no rate-limiting on any existing route (no express-rate-limit dep, no usage in src/). /api/gallery/photo, /api/gallery/video/bestFit, /api/gallery/zip, and login all do authorization + FS access without a limiter. Adding one only to HLS routes would be inconsistent; if desired, it belongs in a separate repo-wide PR layered at Router.ts.

[cat101]

Consistent with the rest of the codebase — pigallery2 has no rate-limiting on any existing route (no express-rate-limit dep, no usage in src/). /api/gallery/photo, /api/gallery/video/bestFit, /api/gallery/zip, and login all do authorization + FS access without a limiter. Adding one only to HLS routes would be inconsistent; if desired, it belongs in a separate repo-wide PR layered at Router.ts.

[cat101]

Consistent with the rest of the codebase — pigallery2 has no rate-limiting on any existing route (no express-rate-limit dep, no usage in src/). /api/gallery/photo, /api/gallery/video/bestFit, /api/gallery/zip, and login all do authorization + FS access without a limiter. Adding one only to HLS routes would be inconsistent; if desired, it belongs in a separate repo-wide PR layered at Router.ts.