Skip to content

Commit 0274c1a

Browse files
author
kidp330
committed
Update 11_Pipeline.md
1 parent 1c20407 commit 0274c1a

File tree

1 file changed

+37
-90
lines changed

1 file changed

+37
-90
lines changed

basic_pipeline/11_Pipeline.md

Lines changed: 37 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -6,30 +6,30 @@ In many real-life scenarios, this part would be the only thing you would need to
66

77
## Defining the pipeline
88

9-
The pipeline is another behavior introduced by the Membrane Framework. To make the module a pipeline, we need to make it `use Membrane.Pipeline`. That is how we will start our implementation of the pipeline module, in the `lib/Pipeline.ex` file:
9+
The pipeline is another behavior introduced by the Membrane Framework. To make the module a pipeline, we need to make it `use Membrane.Pipeline`. That is how we will start our implementation of the pipeline module, in the `lib/pipeline.ex` file:
1010

11-
**_`lib/Pipeline.ex`_**
11+
**_`lib/pipeline.ex`_**
1212

1313
```elixir
1414

1515
defmodule Basic.Pipeline do
1616

17-
use Membrane.Pipeline
17+
use Membrane.Pipeline
1818
...
1919
end
2020
```
2121

22-
You could have guessed - all we need to do now is to describe the desired behavior by implementing the callbacks! In fact, the only callback we want to implement if the pipeline is the `handle_init/1` callback, called once the pipeline is initialized - of course, there are plenty of other callbacks which you might find useful while dealing with a more complex task. You can read about them [here](https://hexdocs.pm/membrane_core/Membrane.Pipeline.html#callbacks).
22+
You could have guessed - all we need to do now is to describe the desired behavior by implementing the callbacks! In fact, the only callback we want to implement if the pipeline is the `handle_init/2` callback, called once the pipeline is initialized - of course, there are plenty of other callbacks which you might find useful while dealing with a more complex task. You can read about them [here](https://hexdocs.pm/membrane_core/Membrane.Pipeline.html#callbacks).
2323
Please add the following callback signature to our `Basic.Pipeline` module:
2424

25-
**_`lib/Pipeline.ex`_**
25+
**_`lib/pipeline.ex`_**
2626

2727
```elixir
2828

2929
defmodule Basic.Pipeline do
3030
...
3131
@impl true
32-
def handle_init(_opts) do
32+
def handle_init(_context, _options) do
3333

3434
end
3535
end
@@ -45,110 +45,57 @@ With such a concept in mind, it's possible to create reliable and fault-tolerant
4545
[Here](https://blog.appsignal.com/2021/08/23/using-supervisors-to-organize-your-elixir-application.html) there is a really nice article describing that concept and providing an example of the actor system. Feel free to stop here and read about the supervision mechanism in Elixir if you have never met with that concept before.
4646
Our pipeline will also be an actor system - with the `Basic.Pipeline` module being the supervisor of all its elements.
4747
As you have heard before - it is the supervisor's responsibility to launch its children's processes.
48-
In the Membrane Framework's pipeline there is a special action designed for that purpose - [`:spec`](https://hexdocs.pm/membrane_core/Membrane.Pipeline.Action.html#t:spec_t/0).
49-
As you can see, you need to specify the `Membrane.ParentSpec` for that purpose.
50-
It consists of:
48+
In the Membrane Framework's pipeline there is a special action designed for that purpose - [`:spec`](https://hexdocs.pm/membrane_core/Membrane.Pipeline.Action.html#t:spec/0).
49+
As you can see, you need to specify the `Membrane.ChildrenSpec` for that purpose.
5150

52-
- `:children` - map of the pipeline's (or a bin) elements
53-
- `:links` - list consisting of links between the elements
54-
55-
Please stop for a moment and read about the [`Membrane.ParentSpec`](https://hexdocs.pm/membrane_core/Membrane.ParentSpec.html).
51+
Please stop for a moment and read about the [`Membrane.ChildrenSpec`](https://hexdocs.pm/membrane_core/Membrane.ChildrenSpec.html).
5652
We will wait for you and once you are ready, we will define our own children and links ;)
5753

58-
Let's start with defining what children we need inside the `handle_init/1` callback! If you have forgotten what structure we want to achieve please refer to the [2nd chapter](02_SystemArchitecture.md) and recall what elements we need inside of our pipeline.
54+
Let's start with defining what children we need inside the `handle_init/2` callback! If you have forgotten what structure we want to achieve please refer to the [2nd chapter](02_SystemArchitecture.md) and recall what elements we need inside of our pipeline.
5955

60-
**_`lib/Pipeline.ex`_**
56+
**_`lib/pipeline.ex`_**
6157

6258
```elixir
63-
64-
@impl true
65-
def handle_init(_opts) do
66-
children = %{
67-
input1: %Basic.Elements.Source{location: "input.A.txt"},
68-
ordering_buffer1: Basic.Elements.OrderingBuffer,
69-
depayloader1: %Basic.Elements.Depayloader{packets_per_frame: 4},
70-
71-
input2: %Basic.Elements.Source{location: "input.B.txt"},
72-
ordering_buffer2: Basic.Elements.OrderingBuffer,
73-
depayloader2: %Basic.Elements.Depayloader{packets_per_frame: 4},
74-
75-
mixer: Basic.Elements.Mixer,
76-
output: %Basic.Elements.Sink{location: "output.txt"}
77-
}
78-
end
79-
```
80-
81-
Remember to pass the desired file paths in the `:location` option!
82-
We have just created the map, which keys are in the form of atoms that describe each of the children and the values are particular module built-in structures (with all the required fields passed).
83-
84-
Now we have a `children` map which we will use to launch the processes. But the Membrane needs to know how those children elements are connected (and, in fact, how the pipeline is defined!). Therefore let's create a `links` list with the description of the links between the elements:
85-
86-
**_`lib/Pipeline.ex`_**
87-
88-
```elixir
89-
90-
def handle_init(_opts) do
91-
...
92-
links = [
93-
link(:input1) |> to(:ordering_buffer1) |> to(:depayloader1),
94-
link(:input2) |> to(:ordering_buffer2) |> to(:depayloader2),
95-
link(:depayloader1) |> via_in(:first_input) |> to(:mixer),
96-
link(:depayloader2) |> via_in(:second_input) |> to(:mixer),
97-
link(:mixer) |> to(:output)
98-
]
99-
...
100-
end
101-
```
102-
103-
We hope the syntax is visually descriptive enough to show what is the desired result of that definition. Just to be sure let us go through it bit-by-bit.
104-
Each pipeline's "branch" starts with the `link/1` which takes as an argument an atom corresponding to a given element. All the further elements in the branch can be accessed with the use of the `to/1` function, expecting an atom that identifies that element to be passed as an argument. Note, that the mentioned atoms must be the same as the ones you have used as keys in the `children` map!
105-
`|>` operator allows "linking" of the elements accessed in the way described above, via their [pads](../glossary/glossary.md#pad). By default, the elements' link will be using an `:input` pad as the input pad and an `:output` pad as the output pad.
106-
`via_in/1` allows specifying an input pad with a given name. Since in a [mixer](../glossary/glossary.md#mixer) there are two input pads (`:first_input` and `:second_input`, defined in `Basic.Elements.Mixer` module with `def_input_pad` and `def_output_pad` macros), we need to distinguish between them while linking the elements.
107-
Of course, there is also a `via_out/1` function, which is used to point the output pad with the given identifier, but there was no need to use it.
108-
In the case of other elements we do not need to explicitly point the desired pads since we are taking advantage of the default pads name - `:input` for the input pads and `:output` for the output ones (see what names we have given to our pads in the elements other than the mixer!). However, we could rewrite the following `links` definitions and explicitly specify the pad names:
109-
110-
**_`lib/Pipeline.ex`_**
111-
112-
```elixir
113-
114-
def handle_init(_opts) do
59+
defmodule Basic.Pipeline do
11560
...
116-
links = [
117-
link(:input1) |> via_out(:output) |> via_in(:input) |> to(:ordering_buffer1) |> via_out(:output) |> via_in(:input) |> to(:depayloader1),
118-
link(:input2) |> via_out(:output) |> via_in(:input) |> to(:ordering_buffer2) |> via_out(:output) |> via_in(:input) |>to(:depayloader2),
119-
link(:depayloader1) |> via_out(:output) |> via_in(:first_input) |> to(:mixer),
120-
link(:depayloader2) |> via_out(:output) |> via_in(:second_input) |> to(:mixer),
121-
link(:mixer) |> via_out(:output) |> via_in(:input) |> to(:output)
122-
]
61+
@impl true
62+
def handle_init(_context, _options) do
63+
spec = [
64+
child(:input1, %Basic.Elements.Source{location: "input.A.txt"})
65+
|> child(:ordering_buffer1, Basic.Elements.OrderingBuffer)
66+
|> child(:depayloader1, %Basic.Elements.Depayloader{packets_per_frame: 4})
67+
|> via_in(:first_input)
68+
|> child(:mixer, Basic.Elements.Mixer)
69+
|> child(:output, %Basic.Elements.Sink{location: "output.txt"}),
70+
child(:input2, %Basic.Elements.Source{location: "input.B.txt"})
71+
|> child(:ordering_buffer2, Basic.Elements.OrderingBuffer)
72+
|> child(:depayloader2, %Basic.Elements.Depayloader{packets_per_frame: 4})
73+
|> via_in(:second_input)
74+
|> get_child(:mixer)
75+
]
76+
77+
{[spec: spec], %{}}
78+
end
12379
...
124-
end
12580
```
12681

127-
That's almost it! All we need to do is to return a proper tuple from the `handle_init/1` callback, with the `:spec` action defined:
128-
129-
**_`lib/Pipeline.ex`_**
130-
131-
```elixir
82+
Remember to pass the desired file paths in the `:location` option!
13283

133-
def handle_init(_opts) do
134-
...
135-
spec = %ParentSpec{children: children, links: links}
136-
{ {:ok, spec: spec}, %{} }
137-
end
138-
```
84+
Now... that's it! :)
85+
The spec list using Membrane's DSL is enough to describe our pipeline's topology. The child keywords spawn components of the specified type and we can use the `|>` operator to link them together. When the pads that should be linked are unamibiguous this is straightforward but for links like those with `Mixer` we can specify the pads using `via_in/1`. There also exists a `via_out/1` keyword which works in a similar way.
86+
As you can see the first argument to `child/2` is a component identifier, but it's also possible to have anonymous children using `child/1`, which just has Membrane generate a unique id under the hood.
13987

14088
Our pipeline is ready! Let's try to launch it.
14189
We will do so by starting the pipeline, and then playing it. For the ease of use we will do it in a script.
14290

143-
**_`lib/start.exs`_**
91+
**_`start.exs`_**
14492

14593
```elixir
146-
{:ok, pid} = Basic.Pipeline.start()
147-
Basic.Pipeline.play(pid)
94+
{:ok, _sup, _pipeline} = Membrane.Pipeline.start_link(Basic.Pipeline)
14895
Process.sleep(500)
14996
```
15097

151-
You can execute it by running `mix run lib/start.exs` in the terminal.
98+
You can execute it by running `mix run start.exs` in the terminal.
15299

153100
In the output file (the one specified in the `handle_init/1` callback of the pipeline) you should see the recovered conversation.
154101

0 commit comments

Comments
 (0)