You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Mar 17, 2025. It is now read-only.
A fault-tolerant, multi-worker, multi-stage job pipeline built on Firebase.
7
+
A fault-tolerant, multi-worker, multi-stage job pipeline built on the Firebase realtime database.
8
8
9
9
10
10
## Purpose of a Queue
11
11
12
-
Queues in Firebase can be used to organize workers or perform background work on data stored in a Firebase like generating thumbnails of images, filtering message contents and censoring data, or fanning data out to other locations in your Firebase. First, let's define a few terms we'll use when talking about a queue:
12
+
Queues can be used in your Firebase app to organize workers or perform background work like generating thumbnails of images, filtering message contents and censoring data, or fanning data out to multiple locations in your Firebase database. First, let's define a few terms we'll use when talking about a queue:
13
13
-`task` - a unit of work that a queue worker can process
14
14
-`spec` - a definition of an operation that the queue will perform on matching tasks
15
15
-`job` - one of more `spec`'s that specify a series of ordered operations to be performed
@@ -23,9 +23,9 @@ Since chat message sanitization can't happen purely on the client side, as that
23
23
24
24
Using Firebase Queue, you can create specs for each of these tasks, and then use workers to process the individual tasks to complete the job. We'll explore the queue, adding tasks, assigning workers, and creating custom specs to create full jobs, then [revisit the example](#message-sanitization-revisited) above.
25
25
26
-
## The Queue in Firebase
26
+
## The Queue in Your Firebase Database
27
27
28
-
The queue relies on having a Firebase reference to coordinate workers e.g. `https://<your-firebase>.firebaseio.com/queue`. This queue can be stored at any path in your Firebase, and you can have multiple queues as well. The queue will respond to tasks pushed onto the `tasks` subtree and optionally read specifications from a `specs` subtree.
28
+
The queue relies on having a Firebase database reference to coordinate workers e.g. `https://<your-firebase>.firebaseio.com/queue`. This queue can be stored at any path in your Firebase database, and you can have multiple queues as well. The queue will respond to tasks pushed onto the `tasks` subtree and optionally read specifications from a `specs` subtree.
29
29
```
30
30
queue
31
31
- specs
@@ -37,7 +37,7 @@ queue
37
37
38
38
The basic unit of the queue is the queue worker: the process that claims a task, performs the appropriate processing on the data, and either returns the transformed data, or an appropriate error.
39
39
40
-
You can start a worker process by passing in a Firebase [`ref`](https://www.firebase.com/docs/web/guide/understanding-data.html#section-creating-references) along with a processing function ([described below](#the-processing-function)), as follows:
40
+
You can start a worker process by passing in a Firebase database [`ref`](https://www.firebase.com/docs/web/guide/understanding-data.html#section-creating-references) along with a processing function ([described below](#the-processing-function)), as follows:
41
41
42
42
```js
43
43
// my_queue_worker.js
@@ -356,7 +356,7 @@ tasksRef.push({
356
356
});
357
357
```
358
358
359
-
Your Firebase should now look like this:
359
+
Your Firebase database should now look like this:
360
360
361
361
```
362
362
root
@@ -429,7 +429,7 @@ root
429
429
- name: "Chris"
430
430
```
431
431
432
-
Now, you want to fan the data out to the `messages` subtree of your firebase, using the spec, `fanout_message`, so you can set up a second processing function to find tasks whose `_state` is `sanitize_message_finished`:
432
+
Now, you want to fan the data out to the `messages` subtree of your Firebase database, using the spec, `fanout_message`, so you can set up a second processing function to find tasks whose `_state` is `sanitize_message_finished`:
433
433
434
434
```js
435
435
...
@@ -439,7 +439,7 @@ var options = {
439
439
'numWorkers':5
440
440
};
441
441
var fanoutQueue =newQueue(tasksRef, options, function(data, progress, resolve, reject) {
442
-
// fan data out to /messages, ensure that Firebase errors are caught and cause the task to fail
442
+
// fan data out to /messages; ensure that errors are caught and cause the task to fail
443
443
messagesRef.push(data, function(error){
444
444
if (error) {
445
445
reject(error.message);
@@ -456,4 +456,4 @@ While this example is a little contrived since you could perform the sanitizatio
456
456
457
457
## Wrap Up
458
458
459
-
As you can see, Firebase Queue is a powerful tool that allows you to securely and robustly perform background work on your Firebase, from sanitization to data fanout and more. We'd love to hear about how you're using Firebase-Queue in your project! Let us know on [Twitter](https://twitter.com/firebase), [Facebook](https://www.facebook.com/Firebase), or [G+](https://plus.google.com/115330003035930967645). If you have any questions, please direct them to our [Google Group](https://groups.google.com/forum/#!forum/firebase-talk) or [[email protected]](mailto:[email protected]).
459
+
As you can see, Firebase Queue is a powerful tool that allows you to securely and robustly perform background work on your Firebase data, from sanitization to data fanout and more. We'd love to hear about how you're using Firebase-Queue in your project! Let us know on [Twitter](https://twitter.com/firebase), [Facebook](https://www.facebook.com/Firebase), or [G+](https://plus.google.com/115330003035930967645). If you have any questions, please direct them to our [Google Group](https://groups.google.com/forum/#!forum/firebase-talk) or [[email protected]](mailto:[email protected]).
0 commit comments