How can I decrease the latency below the 2second to 400ms or 500ms

[vishaldybot]

Hi @razorRun, first of all, I want to thank you for your efforts and hard work in creating this library; it's very helpful.

Actually, my problem involves sending a video stream from Gstreamer using rtspserver.py and receiving the same in your react-native-vlc-media-player library. I was able to receive the stream successfully, but the latency is too high, around 2 seconds, which is the same as what I was receiving on the VLC desktop. I made some changes, such as {:network-caching=0 :0latency}, and I was able to reduce the latency to 400ms. However, when I attempt to do the same in this library, the latency doesn't decrease.

Could you please help me with this?

Thank you in advance!

rtspserver,py
import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GObject

Gst.init(None)

mainloop = GObject.MainLoop()

Create RTSP server

server = GstRtspServer.RTSPServer.new()
server.set_address("192.168.1.72") # Set server's IP
server.set_service("8554") # Set the port where to serve

mounts = server.get_mount_points()
factory = GstRtspServer.RTSPMediaFactory.new()

Set pipeline

#factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=faster ! rtph264pay name=pay0 pt=96 )')

factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=faster ! rtph264pay name=pay0 pt=96 )')

#factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! omxh264enc control-rate=variable target-bitrate=1000000 ! video/x-h264,profile=high ! h264parse ! rtph264pay name=pay0 pt=96 )')

mounts.add_factory("/test", factory)

server.attach(None)

print(f"RTSP server ready at rtsp://192.168.1.72:8554/test")
mainloop.run()

import React from 'react';
import { VLCPlayer } from 'react-native-vlc-media-player';

react code:-
function Camera() {
return (
<VLCPlayer
style={{ height: 480, width: 640 }}
videoAspectRatio="16:9"
source={{
uri: 'rtsp://192.168.70.50:8554/test',
autoplay: true,
// initOptions: [/"--no-audio", "--no-stats", "--rtsp-tcp",/ "--0latency", "--no-hw-dec"] // I have commented it out as it was not working
}}
/>
);
}

export default Camera;

[razorRun]

Hey @vishaldybot
Can you please try on the test app.. from the memory we managed to get the latency to around 500-1000ms over the internet (Around 400s local) when i was working on this. However this will depend on the stream quality as well. Have a play reducing that. Also hermes helped a bit as well

[vishaldybot]

Hi @razorRun

As you mentioned, I tried my best to achieve the desired outcome, but I'm unable to reduce the latency below 2 seconds regardless of my efforts. The pipeline I'm using to transmit the video feed has a latency of only 10-20ms. However, when it comes to this library, I'm struggling to achieve the same results. I might be overlooking something.

I'd appreciate your assistance. React Native doesn't have another library offering this functionality. I've implemented all the other features but this one has me stuck.

rtspserver.py:

import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstRtspServer', '1.0')
from gi.repository import Gst, GstRtspServer, GObject

Gst.init(None)

mainloop = GObject.MainLoop()

server = GstRtspServer.RTSPServer.new()
server.set_address("192.168.22.227") # Set server's IP
server.set_service("8554") # Set the port to serve

mounts = server.get_mount_points()
factory = GstRtspServer.RTSPMediaFactory.new()

factory.set_launch('( v4l2src do-timestamp=true io-mode=mmap device=/dev/video0 ! videoconvert ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency speed-preset=ultrafast key-int-max=30 bframes=0 threads=1 ! rtph264pay name=pay0 pt=96 )')

mounts.add_factory("/test", factory)

server.attach(None)

print(f"RTSP server ready at rtsp://192.168.22.227:8554/test")
mainloop.run()

app.tsx:

import React from 'react';
import { VLCPlayer } from 'react-native-vlc-media-player';
import { View } from 'react-native';

function App() {
return (
<View style={{ flex: 1 }}>
<VLCPlayer
autoplay={true}
autoAspectRatio={true}
style={{
height: 480, width: 640
}}
source={{
initType: 2,
hwDecoderEnabled: 1,
hwDecoderForced: 1,
uri: 'rtsp://192.168.22.227:8554/test',
initOptions: [
"--rtsp-tcp",
"--network-caching=150",
"--rtsp-caching=150",
"--no-stats",
"--tcp-caching=150",
"--realrtsp-caching=150",
],
}}
isLive={true}
autoReloadLive={true}
onPlaying={(e) => {
console.log("aaa", e);
}}
/>

);
}

export default App;

it`s a request that please give it a try and see whether it is working or not and if not please help me out with this as the react native does not have any other way to do the same.

[Wolfleader101]

@vishaldybot any update on this?

[vishaldybot]

Yes, The issue has been resolved

[Wolfleader101]

Yes, The issue has been resolved

How did you get it working with low latency?

[vishaldybot]

By modifying the
--network-caching=0',
'--rtsp-caching=0',

If this doesn`t work please do let me know I will try to resolve the issue

[Wolfleader101]

By modifying the --network-caching=0', '--rtsp-caching=0',

If this doesn`t work please do let me know I will try to resolve the issue

I have tried with this already and it didn't work, latency was still upwards of 1000ms

[vishaldybot]

also you need to disable hardware decoder to 0

hwDecoderEnabled: 0,
hwDecoderForced: 0,

[jwee1369]

Hi, i'm having similar issue, my VLC player on mac plays the rtsp stream fine. But on the app, there is a huge delay, it shows up but only like 1 frame every minute. I followed the example App, any ideas?

[vishaldybot]

May I know the motive that what do you want to do ??
so that I would be able to suggest some other alternatives too else if you are interested in VLC than I will try to help you out with this only

[vishaldybot]

@Wolfleader101 Please share your standalone code I will debug it and resolve and share

but I will do in android only

[jwee1369]

Hi @vishaldybot i have a Linux computer as the rtsp server with IP camera. Streaming from my Mac VLC player is ok with slight delay. But using this package on react native there's a lot of delay like 1 frame per minute. Playing mp4 uri is fine on RN. I used the sample test code and still same problem. Tried the different options above as well.

ios17. Was looking at Flutter library seems people are having the same problem with iOS 17 solid-software/flutter_vlc_player#444 - something to do with com.apple.developer.networking.multicast

I'm going to try Android later

Edit: I tested on Android and it's much better, with the 2sec delay or so, any solution for iOS?

[jwee1369]

Just circling back - after adding the Multicast Networking to xcode, works now with the bearable sub 1 second delay

[vishaldybot]

how latency you are expecting??

[Gabriellsp]

@jwee1369 Is VLC working normally for you on iOS version 17?

[roshansinghg]

By modifying the --network-caching=0', '--rtsp-caching=0',

If this doesn`t work please do let me know I will try to resolve the issue

@razorRun
I am doing live streaming using rtmp server, but streaming video is taking time to load and frequently showing white screen before loading.
and also getting stucked while playing, this issue is mainly related to iOS.
bitrate is 1500 kbps.

    <VLCPlayer
      ref={playerRef}
      source={{
        uri: liveStreamingUrl ? liveStreamingUrl : "",
      }}
      style={styles.videoPlayer}
      videoAspectRatio={isFullscreen ? "16:9" : "3:2"}
      resizeMode="contain"
      paused={!isPlaying}
      onError={(e) => console.log("onError", e)}
      onPlaying={onPlaying}
      onPaused={onPaused}
      mediaOptions={{
        ":network-caching": 150,
        ":live-caching": 300,
        ":file-caching": 0,
        ":network-caching": 150,
        ":clock-jitter": 0,
        ":clock-synchro": 0,
      }}
    /> 

[ItxAltaf]

@vishaldybot Screen stuck sometime

`import React from 'react';
import { VLCPlayer } from 'react-native-vlc-media-player';
import { View, StyleSheet } from 'react-native';

const App = () => {
return (

<VLCPlayer
source={{
uri: 'rtsp://192.168.0.105:8554/stream',
initType: 2,
hwDecoderEnabled: 0,
hwDecoderForced: 0,
initOptions: [
"--rtsp-tcp",
"--network-caching=0",
"--rtsp-caching=0",
"--no-stats",
"--tcp-caching=150",
"--realrtsp-caching=150",
]
}}
style={styles.video}
autoAspectRatio={true}
resizeMode="contain"

  />
</View>

);
};

const styles = StyleSheet.create({
container: {
flex: 1,
justifyContent: 'center',
alignItems: 'center',
backgroundColor: '#000',
},
video: {
width: '100%',
height: '100%',
},
});

export default App;
`

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[vishaldybot]

@ItxAltaf like how??

[SandhiyalalEIC]

--rtsp-caching=0

I'm facing the same issue, I am doing live streaming using rtmp server, it is frequently stalling & going to buffering state.
once it's buffered 99%, it's running at 2x, 3x ..What can be the reason

media.addOption(":network-caching=0")
media.addOption(":rtsp-caching=0")

    media.addOption(":live-caching=3000")        // Set live caching for smoother playback
    media.addOption(":input-repeat=-1")         // Loop the media if required
  
    media.addOption(":buffer-size=200000")     // Set a larger buffer size (1MB)

    media.addOption(":imem-data")               // Ensure custom input module (IMEM) is properly utilized
    media.addOption(":demux=h264")               // Allow VLC to auto-detect the demuxer
    media.addOption(":codec=h264")
    media.addOption(":verbose=2")

    media.addOption(":file-caching=3000")
    media.addOption(":sout-mux-caching=1500")

    media.addOption(":drop-late-frames=0")
    media.addOption(":skip-frames=0")

    media.addOption(":verbose=2")
    media.addOption(":bitrate=200000")
    media.addOption(":no-audio")
    media.addOption(":codec=videotoolbox")

[cornejobarraza]

Version 1.0.75 has been just released and includes a fix for the initOptions prop when creating the VLC Player.

Consider the initType defines if the player makes use of default options or those provided in the prop. See the added types for more.

Android seemed to be working just fine during testing, please upgrade and try again