Support orientation changes without restarting the RTMP encoder

[fdcgilmore]

Hi,

I wanted to ask whether it might be possible to support orientation changes during a live session without needing to restart the encoder.
At the moment, it seems that switching orientation requires a full encoder restart, which can interrupt the stream and affect the user experience. If feasible, it would be great to handle orientation updates dynamically while keeping the encoder running.

Main goals:

• Preserve the live session during portrait/landscape changes
• Avoid stream interruption or reconnects
• Keep UI/video orientation in sync without restarting the pipeline

[pedroSG94]

Hello,

That is already supported. Rotation example do exactly that:
https://github.com/pedroSG94/RootEncoder/tree/master/app/src/main/java/com/pedro/streamer/rotation

[fdcgilmore]

Hi Pedro,

Thanks for the quick response! I understand that autoHandleOrientation = true rotates the video content seamlessly without restarting the encoder and the stream — that works great.

My concern is the encoded output resolution. With fixed dimensions (e.g. 640×480) and rotation=0, rotating to portrait causes a letterbox effect on the stream since the encoder output stays landscape. To get true portrait output (480×640), it seems a full release() + prepareVideo(rotation=90) is needed, which interrupts the stream.

Is there a way to swap encoder dimensions dynamically without stopping the stream?

Thanks again for your work on the library!

[pedroSG94]

Hello,

No, it is not supported. In SRT and WHIP the protocol itself supports resolution changes mid-stream, so it could be a feature to add in the future.

In RTMP and RTSP the situation is different. In RTSP the resolution is negotiated at connection time (in the SDP), so changing it mid-stream is not officially supported. In RTMP the protocol technically allows re-sending the video config (SPS/PPS) mid-stream, but most platforms (YouTube, Twitch, etc.) don't handle it. The stream usually breaks or the player shows corrupted video. Since the result depends on the server and can't be guaranteed, this feature is not supported in the library.

[pedroSG94]

In most of cases the resolution is changed only in server side. Like YouTube or Twitch that offer multiple resolutions to reproduce stream. That is because the server receive the ingest from the encoder (Rootencoder, OBS, etc) and create multiple streams scaling the original resolution.

[gilz688]

Hi Pedro,

(Note: responding from my personal GitHub account — I'm the same person who opened the issue.)
Thank you for the detailed explanation! That makes total sense — especially the RTSP/SDP constraint and the server-side transcoding approach.

For my use case (Wowza Streaming Engine), I found that Wowza does handle mid-stream SPS/PPS changes correctly, so I went ahead and implemented a workaround using reflection. Sharing it in case it's useful to anyone else targeting Wowza or a similarly flexible server.

The idea: update VideoEncoder's width/height/rotation fields via reflection, call the existing reset() method to restart just the MediaCodec session (leaving the RTMP connection alive), then rewire the new inputSurface back into GlStreamInterface. The existing onVideoInfoImp callback then naturally sends the new SPS/PPS as a Video Sequence Header to the server.

object VideoEncoderResizer {

    fun resize(stream: StreamBase, newWidth: Int, newHeight: Int, newRotation: Int): Boolean {
        return try {
            val videoEncoder = getVideoEncoder(stream) ?: return false
            if (!videoEncoder.isRunning) return false

            // Update stored fields so reset() re-prepares with new values
            setField(videoEncoder, "width", newWidth)
            setField(videoEncoder, "height", newHeight)
            setField(videoEncoder, "rotation", newRotation)

            // reset() = stop + prepareVideoEncoder(stored fields) + restart()
            if (!videoEncoder.reset()) return false

            // Rewire the new Surface into the GL pipeline
            val newSurface = videoEncoder.inputSurface ?: return false
            val glInterface = stream.getGlInterface() as? GlStreamInterface ?: return false

            // Match prepareVideo's swap logic for rotation=90/270
            val (glW, glH) = if (newRotation == 90 || newRotation == 270)
                Pair(newHeight, newWidth) else Pair(newWidth, newHeight)

            glInterface.setEncoderSize(glW, glH)
            glInterface.addMediaCodecSurface(newSurface)
            true
        } catch (e: Exception) {
            false
        }
    }

    private fun getVideoEncoder(stream: StreamBase): VideoEncoder? {
        val delegateField = StreamBase::class.java
            .getDeclaredField("videoEncoder\$delegate")
            .apply { isAccessible = true }
        @Suppress("UNCHECKED_CAST")
        return (delegateField.get(stream) as? Lazy<*>)?.value as? VideoEncoder
    }

    private fun setField(target: Any, fieldName: String, value: Int) {
        target.javaClass.getDeclaredField(fieldName).apply { isAccessible = true }.setInt(target, value)
    }
}