You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hang on for a moment! I know that after slipping through the tons of the documentation you are really eager to start coding, but let's think for a moment before taking any actions. How do we want our application to look like?
11
+
Hang on for a moment! I know that after slipping through the tons of the documentation you are really eager to start coding, but let's think for a moment before taking any actions. What do we want our application to look like?
12
12
Can we somehow decompose our application?
13
13
14
14
Sure we can - as in each web application we have two independent subsystems:
15
15
+ server (backend) - written in Elixir, one and the only for the whole system. It will spawn a `Room` process for each of the rooms created by the users, which will handle
16
-
signaling and relay media among the peers in the room.
17
-
+ client application (frontend) - the one written in form of JS code and executed on each client's machine (to be precise - by client's web browser). It will be responsible for fetching user's media stream as well as displaying the stream from the peers.
16
+
signaling and relay media among the peers in the room.
17
+
+ client application (frontend) - the one written in form of JS code and executed on each client's machine (to be precise - by client's web browser). It will be responsible for fetching the user's media stream as well as displaying the stream from the peers.
18
18
19
19
## We might need something else than the plain Elixir standard library...
20
-
Ugh...I am sure till now on you have already found out that media streaming is not that easy. It covers many topic which originates to the nature of the reality.
20
+
Ugh...I am sure till now on you have already found out that media streaming is not that easy. It covers many topics which originate from the nature of reality.
21
21
We need to deal with some limitations brought to us by the physics of the surrounding universe, we want to compress the data being sent with the great tools
22
-
mathematics has equipped us with, we are taking an advantage of imperfections of our perception system...
22
+
mathematics has equipped us with, we are taking advantage of imperfections of our perception system...
23
23
All this stuff is both complex and complicated - and that is why we don't want to design it from very scratch. Fortunately, we have access to the protocols
24
-
and codecs - ICE, RTP, H264, VP9, VP8, Opus etc. - which already solves the aforementioned problems. But that's not enough -
24
+
and codecs - ICE, RTP, H264, VP9, VP8, Opus, etc. - which already solves the aforementioned problems. But that's not enough -
25
25
those protocols are also complicated and implementing or even using them requires digging into their fundamentals.
26
26
That is why we will be using the framework that provides some level of abstraction on top of these protocols. Ladies and gents - let me introduce to you - the Membrane framework.
27
27
## What does Membrane framework do?
@@ -44,32 +44,38 @@ And here is how the **SFU Engine** will relay multimedia streams:<br>
44
44
45
45
In terms of media streaming, our server will be a Selective Forwarding Unit (SFU).
46
46
Why do we want our server to be a Selective Forwarding Unit? The reason is that such a model of streaming data
47
-
among peers allows us to balance between server's and client's bandwidth and limit CPU usage of the server.
48
-
SFU is receiving stream from each of the peers and passes each of these streams to each of the other peers. <br>
47
+
among peers allows us to balance between the server's and client's bandwidth and limit CPU usage of the server.
48
+
RTC is receiving streams from each of the peers and passes each of these streams to each of the other peers. <br>
49
49
50
50
## Server
51
-
As pointed previously, the server will have two responsibilities - the first one is that it will work as a signalling server, broadcasting event messages among the peers.
51
+
As pointed out previously, the server will have two responsibilities - the first one is that it will work as a signaling server, broadcasting event messages among the peers.
52
52
The second one is that it will act as a streaming server.
53
-
The Membrane Framework provides Selective Forwarding Unit implementation called `SFU Engine`, which handles both the signaling and streaming.
54
-
In the tutorial we will wrap the `SFU Engine` and provide business logic in order to add video room functionalities.
53
+
A Selective Forwarding Unit implementation in the Membrane Framework can be achieved with `RTC Engine` plugin, which is capable of both the signaling and streaming media.
54
+
In the tutorial, we will wrap the `RTC Engine` and provide business logic in order to add video room functionalities.
55
55
56
56
The server will consist of two components holding the logic and two components needed for communication.
57
57
The communication will be done with the use of Phoenix sockets and that is why we will need to define the `socket` itself and a `channel` for each of the rooms.
58
58
59
-
The "heart" of the server will be `SFU Engine` - it will deal with all the dirty stuff connected with both the signaling and streaming. We will also have a separate `Room` process (one per each of the video rooms) whose responsibility will be to aggregate information about peers in the particular room.
60
-
`SFU Engine` will send event messages (e.g. `:new_peer` or `:sfu_media_event` messages) on which obtainment the `Room` process will react, for instance, by dispatching them to the appropriate peer's `channel`. `Channel` will then send those messages to the client via the `socket`.
61
-
Messages coming on the `socket` will be dispatched to the appropriate `channel`. Then the `channel` will send them to the `Room`'s process, which finally will pass them to the `SFU Engine`.
62
-
Media transmission will be done with the use of streaming protocols. The way in which this will be performed is out the scope of this tutorial. The only thing you need to know is that SFU Engine will also take care of it.
59
+
The "heart" of the server will be `RTC Engine` - it will deal with all the dirty stuff connected with both the signaling and streaming. We will also have a separate `Room` process (one per each of the video rooms) whose responsibility will be to aggregate information about peers in the particular room.
60
+
`RTC Engine` will send event messages (e.g. `:new_peer` or `:sfu_media_event` messages) on which reception the `Room` process will react, for instance, by dispatching them to the appropriate peer's `channel`. `Channel` will then send those messages to the client via the `socket`.
61
+
Messages coming on the `socket` will be dispatched to the appropriate `channel`. Then the `channel` will send them to the `Room`'s process, which finally will pass them to the `RTC Engine`. RTC Engine will receive them inside its endpoints since each peer will have a corresponding endpoint in the RTC Engine, as presented on the diagram below:
62
+
63
+
64
+
65
+
Note that the scheme is simplified and does not show elements (i.e. channels) that are in between the RTC Engine and the peers' browsers
66
+
If you want to find out more about the inner architecture of the RTC Engine, please refer [here](https://blog.swmansion.com/modular-rtc-engine-is-our-little-big-revolution-in-video-conferencing-cfde806c5beb).
67
+
68
+
Media transmission will be done with the use of streaming protocols. How this will be performed is out of the scope of this tutorial. The only thing you need to know is that RTC Engine will also take care of it.
63
69
64
70
## Client
65
71
Each client's application will have a structure reassembling the structure of the server.
66
72
67
73
In the `Room` instance, the client will receive messages sent from the server on the associated `channel`. The `Room` will call the appropriate methods of `MembraneWebRTC` object.
68
-
At the same time, `MembraneWebRTC` object will be able to change the `Room`'s state by invoking the callbacks provided during construction of this object. These callbacks as well as the `Room` object itself will be able to update the user's interface.
74
+
At the same time, `MembraneWebRTC` object will be able to change the `Room`'s state by invoking the callbacks provided during the construction of this object. These callbacks as well as the `Room` object itself will be able to update the user's interface.
69
75
70
76
Be aware that `MembraneWebRTC` object will also take care of the incoming media stream.
71
77
<br><br>
72
78
[NEXT - Server's communication channels](4_CreatingServersCommunicationChannels.md)<br>
# I know you have been waiting for that moment - let's start coding!
12
12
## Let's prepare the server's endpoint
13
13
Do you still remember about Phoenix's sockets? Hopefully, since we will make use of them in a moment! We want to provide a communication channel between our client's application and our server.
14
-
Sockets fit just in a place - but be aware, that it is not the only possible option. Neither WebRTC nor Membrane Framework expects you to use any particular mean of communication between
14
+
Sockets fit just in a place - but be aware, that it is not the only possible option. Neither WebRTC nor Membrane Framework expects you to use any particular means of communication between
15
15
the server and the client - they just want you to communicate.
16
16
17
17
### Socket's declaration
@@ -22,11 +22,11 @@ You will find the following code there:
22
22
#FILE: lib/videoroom_web/user_socket.ex
23
23
24
24
defmoduleVideoRoomWeb.UserSocketdo
25
-
usePhoenix.Socket
25
+
usePhoenix.Socket
26
26
27
-
channel("room:*", VideoRoomWeb.PeerChannel)
27
+
channel("room:*", VideoRoomWeb.PeerChannel)
28
28
29
-
...
29
+
...
30
30
end
31
31
```
32
32
@@ -42,12 +42,12 @@ That's quite easy - we defined the usage of our socket in `lib/videoroom_web/end
42
42
#FILE: lib/videoroom_web/endpoint.ex
43
43
44
44
defmoduleVideoRoomWeb.Endpointdo
45
-
...
46
-
socket("/socket", VideoRoomWeb.UserSocket,
47
-
websocket:true,
48
-
longpoll:false
49
-
)
50
-
...
45
+
...
46
+
socket("/socket", VideoRoomWeb.UserSocket,
47
+
websocket:true,
48
+
longpoll:false
49
+
)
50
+
...
51
51
end
52
52
```
53
53
In this piece of code we are simply saying, that we are defining socket-type endpoint with path ```"/socket"```, which behavior will be described by
@@ -59,9 +59,9 @@ It is in `lib/videoroom_web/peer_channel.ex` file! However, for now on, this fil
59
59
#FILE: lib/videoroom_web/peer_channel.ex
60
60
61
61
defmoduleVideoRoomWeb.PeerChanneldo
62
-
usePhoenix.Channel
62
+
usePhoenix.Channel
63
63
64
-
requireLogger
64
+
requireLogger
65
65
66
66
end
67
67
```
@@ -74,39 +74,54 @@ Let's implement our first callback!
Just the beginning - note how do we fetch the room's name by using pattern matching in the argument list of `join/3`. ([pattern matching in Elixir](https://elixir-lang.org/getting-started/pattern-matching.html#pattern-matching)). <br>
100
109
101
110
What happens here?
102
-
`join/3` is called when the client joins the channel. First, we are looking for a `Videoroom.Room` process saved in the global registry under the `room_id` key.
111
+
`join/3` is called when the client joins the channel. First, we are looking for a `Videoroom.Room` process saved in the `:global` registry under the `room_id` key.
103
112
(`Videoroom.Room` module will hold the whole business logic of the video room - we will implement this module in the next chapter).
104
-
If videoroom process is already registered, we are simply returning its PID. Otherwise, we are trying to create
113
+
If the videoroom process is already registered, we are simply returning its PID. Otherwise, we are trying to create
105
114
a new `Videoroom.Room` process on the fly (and we register it with `room_id` key in the global registry).
106
115
If we are successful we return the PID of the newly created room's process.
107
116
At the entrance point of the following step, we already have a `Videoroom.Room` process's pid or an `:error` notification.
108
-
In case of an error occurring we have a simple error handler that logs the fact, that the room has failed to start. Otherwise, we can make use of the room's process.
109
-
First, we start to monitor it (so that we will receive ```:DOWN``` message in case of the room's process crash/failure). Then we notify the room's process that
117
+
Errors can occur due to multiple reasons. One of them is a situation in which a race condition between peers trying to create a room takes place.
118
+
Imagine a situation, that two users are trying to join a non-existent room at the same moment. Since they are working asynchronously, there is a probability, that both of them will
119
+
get an answer from the `:global.whereis_name(room_id)` saying that the room with the given name does not exist. Both of them will then try to create such a room. The request from one of these users will come to the `:global` registry first, the room will be
120
+
registered - and the second user will receive an `:already_started` error, along with the PID of that room process, since the process already exists. Handling of that error is quite straightforward - the user can safely join the room with the provided PID.
121
+
Of course, some other errors might also occur, but we do not distinguish between them and we simply log the fact that there was a problem with the room creation.
122
+
In case we retrieve a PID of the room process, we call the `do_join/3` support function.
123
+
`do_join/3` holds some repeatable parts of code concerning the joining process.
124
+
Inside that function, we start to monitor the room process (so that we will receive ```:DOWN``` message in case of the room's process crash/failure). Then we notify the room's process that
110
125
it should take us (peer channel) under consideration - we send our peer_id (generated as unique id with UUID module) along with the peer channel's PID to
111
126
the room process in the `:add_peer_channel` message so that the room will have a way to identify our process. The last thing we do is that we are adding information about the association between
112
127
room's identifier, room's PID, and peer's identifier to the map of socket's assigns. We will refer to this information later so we need to store it somehow.
@@ -119,18 +134,18 @@ The first one is done by implementing `handle_info/2` callback as shown below:
119
134
120
135
@impltrue
121
136
defhandle_info({:media_event, event}, socket) do
122
-
push(socket, "mediaEvent", %{data: event})
123
-
{:noreply, socket}
137
+
push(socket, "mediaEvent", %{data: event})
138
+
{:noreply, socket}
124
139
end
125
140
```
126
-
The second one is done by providing following implementation of `handle_in/3`:
141
+
The second one is done by providing the following implementation of `handle_in/3`:
127
142
```elixir
128
143
#FILE: lib/videoroom_web/peer_channel.ex
129
144
130
145
@impltrue
131
146
defhandle_in("mediaEvent", %{"data"=> event}, socket) do
0 commit comments