You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hang on for a moment! I know that after slipping through the tons of the documentation you are really eager to start coding, but let's think for a moment before taking any actions. How do we want our application to look like?
11
+
Hang on for a moment! I know that after slipping through the tons of the documentation you are really eager to start coding, but let's think for a moment before taking any actions. What do we want our application to look like?
12
12
Can we somehow decompose our application?
13
13
14
14
Sure we can - as in each web application we have two independent subsystems:
15
15
+ server (backend) - written in Elixir, one and the only for the whole system. It will spawn a `Room` process for each of the rooms created by the users, which will handle
16
-
signaling and relay media among the peers in the room.
17
-
+ client application (frontend) - the one written in form of JS code and executed on each client's machine (to be precise - by client's web browser). It will be responsible for fetching user's media stream as well as displaying the stream from the peers.
16
+
signaling and relay media among the peers in the room.
17
+
+ client application (frontend) - the one written in form of JS code and executed on each client's machine (to be precise - by client's web browser). It will be responsible for fetching the user's media stream as well as displaying the stream from the peers.
18
18
19
19
## We might need something else than the plain Elixir standard library...
20
-
Ugh...I am sure till now on you have already found out that media streaming is not that easy. It covers many topic which originates to the nature of the reality.
20
+
Ugh...I am sure till now on you have already found out that media streaming is not that easy. It covers many topics which originate from the nature of reality.
21
21
We need to deal with some limitations brought to us by the physics of the surrounding universe, we want to compress the data being sent with the great tools
22
-
mathematics has equipped us with, we are taking an advantage of imperfections of our perception system...
22
+
mathematics has equipped us with, we are taking advantage of imperfections of our perception system...
23
23
All this stuff is both complex and complicated - and that is why we don't want to design it from very scratch. Fortunately, we have access to the protocols
24
-
and codecs - ICE, RTP, H264, VP9, VP8, Opus etc. - which already solves the aforementioned problems. But that's not enough -
24
+
and codecs - ICE, RTP, H264, VP9, VP8, Opus, etc. - which already solves the aforementioned problems. But that's not enough -
25
25
those protocols are also complicated and implementing or even using them requires digging into their fundamentals.
26
26
That is why we will be using the framework that provides some level of abstraction on top of these protocols. Ladies and gents - let me introduce to you - the Membrane framework.
27
27
## What does Membrane framework do?
@@ -33,48 +33,49 @@ It would be good for you to know that the Membrane Framework consists of the fol
33
33
34
34
We will be using one of its plugins - [RTC Engine plugin](https://github.com/membraneframework/membrane_rtc_engine), which has both the server part (written in Elixir)
35
35
and the client's library (written in Javascript). This plugin provides both the implementation of the
36
-
[Selective Forwarding Unit (RTC)](https://github.com/membraneframework/membrane_rtc_engine) and the signaling server logic (with the usage of ICE protocol).
36
+
[Selective Forwarding Unit (SFU)](https://github.com/membraneframework/membrane_rtc_engine) and the signaling server logic (with the usage of ICE protocol).
37
37
38
38
## System scheme
39
39
The diagram below describes the desired architecture of the events passing system which is the part of the system we need to provide on our own: <br>
And here is how the **RTC Engine** will relay multimedia streams:<br>
43
-
<br>
42
+
And here is how the **SFU Engine** will relay multimedia streams:<br>
43
+
<br>
44
44
45
-
In terms of media streaming, our server will be a Selective Forwarding Unit (RTC).
45
+
In terms of media streaming, our server will be a Selective Forwarding Unit (SFU).
46
46
Why do we want our server to be a Selective Forwarding Unit? The reason is that such a model of streaming data
47
-
among peers allows us to balance between server's and client's bandwidth and limit CPU usage of the server.
48
-
RTC is receiving stream from each of the peers and passes each of these streams to each of the other peers. <br>
47
+
among peers allows us to balance between the server's and client's bandwidth and limit CPU usage of the server.
48
+
RTC is receiving streams from each of the peers and passes each of these streams to each of the other peers. <br>
49
49
50
50
## Server
51
-
As pointed previously, the server will have two responsibilities - the first one is that it will work as a signalling server, broadcasting event messages among the peers.
51
+
As pointed out previously, the server will have two responsibilities - the first one is that it will work as a signaling server, broadcasting event messages among the peers.
52
52
The second one is that it will act as a streaming server.
53
53
A Selective Forwarding Unit implementation in the Membrane Framework can be achieved with `RTC Engine` plugin, which is capable of both the signaling and streaming media.
54
-
In the tutorial we will wrap the `RTC Engine` and provide business logic in order to add video room functionalities.
54
+
In the tutorial, we will wrap the `RTC Engine` and provide business logic in order to add video room functionalities.
55
55
56
56
The server will consist of two components holding the logic and two components needed for communication.
57
57
The communication will be done with the use of Phoenix sockets and that is why we will need to define the `socket` itself and a `channel` for each of the rooms.
58
58
59
-
The "heart" of the server will be `RTC Engine` - it will deal with all the dirty stuff connected with both the signaling and streaming. We will also have a separate `Room` process (one per each of the video rooms) which responsibility will be to aggregate information about peers in the particular room.
59
+
The "heart" of the server will be `RTC Engine` - it will deal with all the dirty stuff connected with both the signaling and streaming. We will also have a separate `Room` process (one per each of the video rooms) whose responsibility will be to aggregate information about peers in the particular room.
60
60
`RTC Engine` will send event messages (e.g. `:new_peer` or `:sfu_media_event` messages) on which obtainment the `Room` process will react, for instance, by dispatching them to the appropriate peer's `channel`. `Channel` will then send those messages to the client via the `socket`.
61
-
Messages coming on the `socket` will be dispatched to the appropriate `channel`. Then the `channel` will send them to the `Room`'s process, which finally will pass them to the `RTC Engine`. RTC Engine will receive them inside its endpoints, since each peer will have a corresponding endpoint in the RTC Engine.
61
+
Messages coming on the `socket` will be dispatched to the appropriate `channel`. Then the `channel` will send them to the `Room`'s process, which finally will pass them to the `RTC Engine`. RTC Engine will receive them inside its endpoints since each peer will have a corresponding endpoint in the RTC Engine, as presented on the diagram below:
62
62
63
63
64
64
65
-
If you want to find out more about the inner architecture of the RTC Engine, please refere [here](https://blog.swmansion.com/modular-rtc-engine-is-our-little-big-revolution-in-video-conferencing-cfde806c5beb).
65
+
Note that the scheme is simplified and does not show elements (i.e. channels) that are in between the RTC Engine and the peers' browsers
66
+
If you want to find out more about the inner architecture of the RTC Engine, please refer [here](https://blog.swmansion.com/modular-rtc-engine-is-our-little-big-revolution-in-video-conferencing-cfde806c5beb).
66
67
67
-
Media transmission will be done with the use of streaming protocols. The way in which this will be performed is out the scope of this tutorial. The only thing you need to know is that RTC Engine will also take care of it.
68
+
Media transmission will be done with the use of streaming protocols. How this will be performed is out of the scope of this tutorial. The only thing you need to know is that RTC Engine will also take care of it.
68
69
69
70
## Client
70
71
Each client's application will have a structure reassembling the structure of the server.
71
72
72
73
In the `Room` instance, the client will receive messages sent from the server on the associated `channel`. The `Room` will call the appropriate methods of `MembraneWebRTC` object.
73
-
At the same time, `MembraneWebRTC` object will be able to change the `Room`'s state by invoking the callbacks provided during construction of this object. These callbacks as well as the `Room` object itself will be able to update the user's interface.
74
+
At the same time, `MembraneWebRTC` object will be able to change the `Room`'s state by invoking the callbacks provided during the construction of this object. These callbacks as well as the `Room` object itself will be able to update the user's interface.
74
75
75
76
Be aware that `MembraneWebRTC` object will also take care of the incoming media stream.
76
77
<br><br>
77
78
[NEXT - Server's communication channels](4_CreatingServersCommunicationChannels.md)<br>
# I know you have been waiting for that moment - let's start coding!
12
12
## Let's prepare the server's endpoint
13
13
Do you still remember about Phoenix's sockets? Hopefully, since we will make use of them in a moment! We want to provide a communication channel between our client's application and our server.
14
-
Sockets fit just in a place - but be aware, that it is not the only possible option. Neither WebRTC nor Membrane Framework expects you to use any particular mean of communication between
14
+
Sockets fit just in a place - but be aware, that it is not the only possible option. Neither WebRTC nor Membrane Framework expects you to use any particular means of communication between
15
15
the server and the client - they just want you to communicate.
16
16
17
17
### Socket's declaration
@@ -22,11 +22,11 @@ You will find the following code there:
22
22
#FILE: lib/videoroom_web/user_socket.ex
23
23
24
24
defmoduleVideoRoomWeb.UserSocketdo
25
-
usePhoenix.Socket
25
+
usePhoenix.Socket
26
26
27
-
channel("room:*", VideoRoomWeb.PeerChannel)
27
+
channel("room:*", VideoRoomWeb.PeerChannel)
28
28
29
-
...
29
+
...
30
30
end
31
31
```
32
32
@@ -42,12 +42,12 @@ That's quite easy - we defined the usage of our socket in `lib/videoroom_web/end
42
42
#FILE: lib/videoroom_web/endpoint.ex
43
43
44
44
defmoduleVideoRoomWeb.Endpointdo
45
-
...
46
-
socket("/socket", VideoRoomWeb.UserSocket,
47
-
websocket:true,
48
-
longpoll:false
49
-
)
50
-
...
45
+
...
46
+
socket("/socket", VideoRoomWeb.UserSocket,
47
+
websocket:true,
48
+
longpoll:false
49
+
)
50
+
...
51
51
end
52
52
```
53
53
In this piece of code we are simply saying, that we are defining socket-type endpoint with path ```"/socket"```, which behavior will be described by
@@ -59,9 +59,9 @@ It is in `lib/videoroom_web/peer_channel.ex` file! However, for now on, this fil
59
59
#FILE: lib/videoroom_web/peer_channel.ex
60
60
61
61
defmoduleVideoRoomWeb.PeerChanneldo
62
-
usePhoenix.Channel
62
+
usePhoenix.Channel
63
63
64
-
requireLogger
64
+
requireLogger
65
65
66
66
end
67
67
```
@@ -74,52 +74,52 @@ Let's implement our first callback!
Just the beginning - note how do we fetch the room's name by using pattern matching in the argument list of `join/3`. ([pattern matching in Elixir](https://elixir-lang.org/getting-started/pattern-matching.html#pattern-matching)). <br>
109
109
110
110
What happens here?
111
111
`join/3` is called when the client joins the channel. First, we are looking for a `Videoroom.Room` process saved in the `:global` registry under the `room_id` key.
112
112
(`Videoroom.Room` module will hold the whole business logic of the video room - we will implement this module in the next chapter).
113
-
If videoroom process is already registered, we are simply returning its PID. Otherwise, we are trying to create
113
+
If the videoroom process is already registered, we are simply returning its PID. Otherwise, we are trying to create
114
114
a new `Videoroom.Room` process on the fly (and we register it with `room_id` key in the global registry).
115
115
If we are successful we return the PID of the newly created room's process.
116
116
At the entrance point of the following step, we already have a `Videoroom.Room` process's pid or an `:error` notification.
117
117
Errors can occur due to multiple reasons. One of them is a situation in which a race condition between peers trying to create a room takes place.
118
-
Imagine a situation, that two users are trying to join a non-existent room in the exactly same moment. Since they are working asynchroniously, there is a probability, that both of them will
119
-
get an answer from the `:global.whereis_name(room_id)` saying that the room with given name does not exists. Both them will then try to create such a room. The request from one of these users will come to the `:global` registry first, the room will be
118
+
Imagine a situation, that two users are trying to join a non-existent room at the same moment. Since they are working asynchronously, there is a probability, that both of them will
119
+
get an answer from the `:global.whereis_name(room_id)` saying that the room with the given name does not exist. Both of them will then try to create such a room. The request from one of these users will come to the `:global` registry first, the room will be
120
120
registered - and the second user will receive an `:already_started` error, along with the PID of that room process, since the process already exists. Handling of that error is quite straightforward - the user can safely join the room with the provided PID.
121
-
Ofcourse, some other errors might also occur, but we do not distinguish between them and we simply log the fact that there was a problem with the room creation.
122
-
In case we retrieve a PID of the room process, we call the `do_join/3` support function.
121
+
Of course, some other errors might also occur, but we do not distinguish between them and we simply log the fact that there was a problem with the room creation.
122
+
In case we retrieve a PID of the room process, we call the `do_join/3` support function.
123
123
`do_join/3` holds some repeatable parts of code concerning the joining process.
124
124
Inside that function, we start to monitor the room process (so that we will receive ```:DOWN``` message in case of the room's process crash/failure). Then we notify the room's process that
125
125
it should take us (peer channel) under consideration - we send our peer_id (generated as unique id with UUID module) along with the peer channel's PID to
@@ -134,18 +134,18 @@ The first one is done by implementing `handle_info/2` callback as shown below:
134
134
135
135
@impltrue
136
136
defhandle_info({:media_event, event}, socket) do
137
-
push(socket, "mediaEvent", %{data: event})
138
-
{:noreply, socket}
137
+
push(socket, "mediaEvent", %{data: event})
138
+
{:noreply, socket}
139
139
end
140
140
```
141
-
The second one is done by providing following implementation of `handle_in/3`:
141
+
The second one is done by providing the following implementation of `handle_in/3`:
142
142
```elixir
143
143
#FILE: lib/videoroom_web/peer_channel.ex
144
144
145
145
@impltrue
146
146
defhandle_in("mediaEvent", %{"data"=> event}, socket) do
0 commit comments