Skip to content
This repository was archived by the owner on Oct 24, 2024. It is now read-only.

Option to reset tries in ping() #40

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
efa4f38
👷 Set up GitHub Actions
alecgibson Jun 25, 2021
8d42e8d
👷 Add Mongo to GH Actions
alecgibson Jun 25, 2021
e97567c
👷 Add Publish action
alecgibson Jun 25, 2021
b0d9700
🙈 Ignore `package-lock.json`
alecgibson Jun 25, 2021
f415bea
Option to reset tries in `ping()`
alecgibson Jun 25, 2021
a32799a
Merge pull request #1 from reedsy/reset-tries
alecgibson Jun 25, 2021
182ca32
⬆️ Bump `mongodb`
alecgibson Jun 29, 2021
5f0d9ab
Merge pull request #2 from reedsy/mongodb
alecgibson Jun 29, 2021
804fa22
👽 Add option to use `returnDocument`
alecgibson Jun 29, 2021
0e11c97
Merge pull request #3 from reedsy/return-document
alecgibson Jun 29, 2021
341d8e2
🔖 v4.0.0-reedsy-1.1.0
alecgibson Jun 29, 2021
39ed065
💥 Require `mongodb` v4
alecgibson Jul 23, 2021
e9d7fe0
Merge pull request #4 from reedsy/mongodb4
alecgibson Jul 23, 2021
0b162cb
⚡ Add deleted index
dawidreedsy Nov 24, 2022
9c7e02f
Merge pull request #5 from reedsy/add-deleted-index
dawidreedsy Nov 24, 2022
f5c4758
💥 Promisify and add `mongodb@5` support
alecgibson Mar 1, 2023
05d4a6b
Merge pull request #6 from reedsy/promisify
alecgibson Mar 2, 2023
3e0b48c
👷‍♀️ Bump GitHub Actions versions
alecgibson Mar 2, 2023
ff780d9
💥 Move to Typescript
alecgibson Mar 2, 2023
836b208
Merge pull request #8 from reedsy/bump-actions
alecgibson Mar 2, 2023
3b4e5ba
Merge pull request #7 from reedsy/typescript
alecgibson Mar 2, 2023
d2e76c3
👷‍♀️ Fix release
alecgibson Mar 2, 2023
e2045fb
🚨 Add linting
alecgibson Mar 2, 2023
dfdc3a5
Merge pull request #9 from reedsy/lint
alecgibson Mar 2, 2023
1760551
👷‍♀️ Add build matrix
alecgibson Aug 29, 2023
b69ffee
⬆️ Add support for `mongodb@6`
alecgibson Aug 29, 2023
6591bf7
Merge pull request #10 from reedsy/mongodb6
alecgibson Aug 30, 2023
9a5c893
👷‍♀️ Build against MongoDB v5.0
alecgibson Nov 6, 2023
772580d
🐛 Remove undefined props
dawidreedsy Jan 2, 2024
bf640e1
Merge pull request #11 from reedsy/remove-undefined-prop
dawidreedsy Jan 2, 2024
b0df39e
👷‍♀️ Move publishing inside single workflow
alecgibson Jan 18, 2024
4e7cf41
👷‍♀️ Use `GITHUB_TOKEN` for install
alecgibson Jan 18, 2024
860daf8
👷‍♀️ Run against MongoDB v6
alecgibson May 7, 2024
f58f61f
👷‍♀️ Add `success` job
alecgibson May 7, 2024
24d10d5
💥 Remove default exports
alecgibson May 7, 2024
29681c1
Merge pull request #12 from reedsy/no-default-export
alecgibson May 8, 2024
c44f974
⚡️ Sort `get()` by `visible`
alecgibson Oct 23, 2024
8d04c21
Merge pull request #13 from reedsy/sort-visible
alecgibson Oct 23, 2024
a2a51bd
🗃 Change `ack` to (string) `ObjectId`
alecgibson Dec 4, 2024
edc03cb
Merge pull request #14 from reedsy/object-id-ack
alecgibson Dec 4, 2024
2822f74
🗃 Remove `visible` when acking jobs
alecgibson Feb 6, 2025
3b121da
Merge pull request #16 from reedsy/remove-visible
alecgibson Feb 7, 2025
a67c70a
💥 Change `deleted` from `string` to `Date`
alecgibson Feb 10, 2025
d4da1aa
Merge pull request #17 from reedsy/date-deleted
alecgibson Feb 10, 2025
795a8d9
💥 Improve database performance
alecgibson Feb 6, 2025
8393845
Merge pull request #15 from reedsy/performance-improvement
alecgibson Feb 10, 2025
c26194d
✨ Allow optional TTL on `deleted` field
alecgibson Feb 11, 2025
6133fc9
Merge pull request #18 from reedsy/ttl-option
alecgibson Feb 14, 2025
5830ed9
✨ Allow resetting `ack` when calling `ping()`
alecgibson Feb 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.json
*.sh
*.yml
*.md
*.d.ts
*.js.map
*.js
!test/*.js
16 changes: 16 additions & 0 deletions .eslintrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
extends:
- 'plugin:@reedsy/recommended'
parserOptions:
project: './tsconfig.json'
env:
node: true
rules:
# default exports have bad CJS/ESM interoperability
no-restricted-exports:
- error
- restrictDefaultExports:
direct: true
named: true
defaultFrom: true
namedFrom: true
namespaceFrom: true
61 changes: 61 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: CI

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
mongodb:
- '6.0'
mongo_driver:
- mongodb6
services:
mongodb:
image: mongo:${{ matrix.mongodb }}
ports:
- 27017:27017
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20.x'
registry-url: 'https://npm.pkg.github.com'
- name: Install
# Skip post-install to avoid malicious scripts stealing PAT
run: npm install --ignore-script
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Post-install
run: npm rebuild && npm run prepare --if-present
- name: Lint
run: npm run lint
- name: Test
run: npm test
env:
MONGO_DRIVER: ${{ matrix.mongo_driver }}
- name: Release
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
run: ./release.sh
env:
NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# A silly job just so we can mark this as the required check for PRs
# instead of maintaining the full list of matrix jobs
success:
needs:
- build
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Success
run: echo "Success"
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
node_modules/*
*.log
*~
package-lock.json
*.d.ts
*.js.map
*.js
!test/*.js
3 changes: 3 additions & 0 deletions .npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
**/*
!*.d.ts
!*.js.map
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@reedsy:registry=https://npm.pkg.github.com
6 changes: 0 additions & 6 deletions .travis.yml

This file was deleted.

43 changes: 33 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const client = new mongodb.MongoClient(url, { useNewUrlParser: true })

client.connect(err => {
const db = client.db('test')
const queue = mongoDbQueue(db, 'my-queue')
const queue = new MongoDbQueue(db, 'my-queue')

// ...

Expand Down Expand Up @@ -102,9 +102,9 @@ passed in:
var mongoDbQueue = require('mongodb-queue')

// an instance of a queue
var queue1 = mongoDbQueue(db, 'a-queue')
var queue1 = new MongoDbQueue(db, 'a-queue')
// another queue which uses the same collection as above
var queue2 = mongoDbQueue(db, 'a-queue')
var queue2 = new MongoDbQueue(db, 'a-queue')
```

Using `queue1` and `queue2` here won't interfere with each other and will play along nicely, but that's not
Expand All @@ -116,7 +116,7 @@ it's not something you should do.
To pass in options for the queue:

```
var resizeQueue = mongoDbQueue(db, 'resize-queue', { visibility : 30, delay : 15 })
var resizeQueue = new MongoDbQueue(db, 'resize-queue', { visibility : 30, delay : 15 })
```

This example shows a queue with a message visibility of 30s and a delay to each message of 15s.
Expand All @@ -131,8 +131,8 @@ Each queue you create will be it's own collection.
e.g.

```
var resizeImageQueue = mongoDbQueue(db, 'resize-image-queue')
var notifyOwnerQueue = mongoDbQueue(db, 'notify-owner-queue')
var resizeImageQueue = new MongoDbQueue(db, 'resize-image-queue')
var notifyOwnerQueue = new MongoDbQueue(db, 'notify-owner-queue')
```

This will create two collections in MongoDB called `resize-image-queue` and `notify-owner-queue`.
Expand All @@ -149,7 +149,7 @@ You may set this visibility window on a per queue basis. For example, to set the
visibility to 15 seconds:

```
var queue = mongoDbQueue(db, 'queue', { visibility : 15 })
var queue = new MongoDbQueue(db, 'queue', { visibility : 15 })
```

All messages in this queue now have a visibility window of 15s, instead of the
Expand All @@ -167,7 +167,7 @@ retrieval 10s after being added.
To delay all messages by 10 seconds, try this:

```
var queue = mongoDbQueue(db, 'queue', { delay : 10 })
var queue = new MongoDbQueue(db, 'queue', { delay : 10 })
```

This is now the default for every message added to the queue.
Expand All @@ -182,8 +182,8 @@ automatically see problem messages.
Pass in a queue (that you created) onto which these messages will be pushed:

```js
var deadQueue = mongoDbQueue(db, 'dead-queue')
var queue = mongoDbQueue(db, 'queue', { deadQueue : deadQueue })
var deadQueue = new MongoDbQueue(db, 'dead-queue')
var queue = new MongoDbQueue(db, 'queue', { deadQueue : deadQueue })
```

If you pop a message off the `queue` over `maxRetries` times and still have not acked it,
Expand Down Expand Up @@ -237,6 +237,18 @@ msg = {
Notice that the payload from the `deadQueue` is exactly the same as the original message
when it was on the original queue (except with the number of tries set to 5).

### returnDocument ###

The `mongodb` Node.js driver [deprecated](https://github.com/mongodb/node-mongodb-native/pull/2808)
use of `returnOriginal` in favor of `returnDocument` when using `findOneAndUpdate()`.

If you want to opt in to using the newer `returnDocument`, set the `returnDocument` option
to `true`:

```
var queue = new MongoDbQueue(db, 'queue', { returnDocument : true })
```

## Operations ##

### .add() ###
Expand Down Expand Up @@ -345,6 +357,17 @@ queue.get((err, msg) => {
})
```

You can also reset the job tries, effectively creating an atomic ack + add for the
same job using `resetTries`:

```js
queue.get((err, msg) => {
queue.ping(msg.ack, { resetTries: true }, (err, id) => {
// This message now has 0 tries
})
})
```

### .total() ###

Returns the total number of messages that has ever been in the queue, including
Expand Down
Loading