Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: cursor pagination docs #42

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

docs: cursor pagination docs #42

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 127 additions & 61 deletions docs/4.x/schemas/pagination.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,9 @@ using the [JSON:API `page` query parameter.](https://jsonapi.org/format/#fetchin
This package supports two approaches to pagination:

- **Page-based**: Laravel's `paginate()` and `simplePaginate()` pagination
implementations, that use a page number and size query parameters.
- **Cursor-based**: cursor pagination inspired by Stripe's implementation.
This implementation pre-dates Laravel's `cursorPaginate()` feature, and
requires the installation of the `laravel-json-api/cursor-pagination` package.
implementations, that use a page number and size query parameters.
- **Cursor-based**: Laravel's `cursorPaginate()` pagination that uses an
opaque before or after cursor and limit query parameters.

You can choose which approach to use for each resource type, so your API
can use different approaches for different resource types if needed. If you
Expand Down Expand Up @@ -138,13 +137,10 @@ details of the last page and total resources available.

## Cursor-Based Pagination

The cursor-based pagination provided by this package is inspired by
[Stripe's pagination implementation](https://stripe.com/docs/api#pagination).
Install via Composer:
The Cursor-based pagination provided by this package uses Laravel's `cursorPaginate()` implementation.

```bash
composer require laravel-json-api/cursor-pagination
```
:::warning
If you are currently using the `laravel-json-api/pagination` package please see the [upgrade guide](https://github.com/laravel-json-api/cursor-pagination/blob/develop/UPGRADE.md) for information on migrating to the new cursor pagination implementation.

Cursor-based pagination is based on the paginator being given a context as to
what results to return next. So rather than an API client saying it wants
Expand All @@ -153,33 +149,36 @@ last item it received. This is ideal for infinite scroll implementations, or
for resources where rows are regularly inserted (which would affect page
numbers if you used paged-based pagination).

Cursor-based pagination works by keeping the list in a fixed order. This means
that if you use cursor-based pagination for a resource type, you should not
support sort parameters as this can have adverse effects on the cursor
pagination.
Cursor-based pagination requires keeping the list in a consistent order between requests,
therefore it requires that the ordering is based on at least one unique column or a combination
of columns that are unique. Columns with null values are not supported.
By default, this order is my model key descending, such that in the case of auto-incrementing
or other monotonic keys like UUIDv7 it will be in chronological order (i.e. most recent first,
oldest last).

Our implementation utilizes cursor-based pagination via the `"after"` and
`"before"` page parameters. Both parameters take an existing resource ID
value (see below) and return resources in a fixed order. By default this
fixed order is reverse chronological order (i.e. most recent first,
oldest last). The `"before"` parameter returns resources listed before the
named resource. The `"after"` parameter returns resources listed after the
named resource. If both parameters are provided, only `"before"`is used.
If neither parameter is provided, the first page of results will be returned.
`"before"` page parameters. Both parameters accept a **cursor**, an opaque encoded string containing
the location that the next paginated query should start paginating and the direction that it should paginate.
You can think of a cursor as a bookmark that tells the server where to start and in which direction.

| Parameter | Description |
| :--- | :--- |
| `after` | A cursor for use in pagination. `after` is a resource ID that defines your place in the list. For instance, if you make a paged request and receive 100 resources, ending with resource with id `foo`, your subsequent call can include `page[after]=foo` in order to fetch the next page of the list. |
| `before` | A cursor for use in pagination. `before` is a resource ID that defines your place in the list. For instance, if you make a paged request and receive 100 resources, starting with resource with id `bar` your subsequent call can include `page[before]=bar` in order to fetch the previous page of the list. |
| `limit` | A limit on the number of resources to be returned, i.e. the per-page amount. |
A `"before"` cursor returns resources listed before the
results of the previous request. An `"after"` cursor returns resources listed after the results
of the previous request. If both parameters are provided, only `"before"`is used.
If neither parameter is provided, the first page of results will be returned along with an `after` cursor for you to retrieve the next page.

| Parameter | Description |
|:----------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `after` | A cursor for use in pagination. `after` is an opaque cursor from a previous request |
| `before` | A cursor for use in pagination. `before` is an opaque cursor from a previous request |
| `limit` | A limit on the number of resources to be returned, i.e. the per-page amount. |

To use cursor-based pagination, return our `CursorPagination` class from your
schema's `pagination` method. For example:

```php
namespace App\JsonApi\V1\Posts;

use LaravelJsonApi\CursorPagination\CursorPagination;
use LaravelJsonApi\Eloquent\Pagination\CursorPagination;
use LaravelJsonApi\Eloquent\Schema;

class PostSchema extends Schema
Expand All @@ -193,15 +192,15 @@ class PostSchema extends Schema
*/
public function pagination(): CursorPagination
{
return CursorPagination::make();
return CursorPagination::make(ID::make());
}
}
```

This means the following request:

```http
GET /api/v1/posts?page[limit]=10&page[after]=03ea3065-fe1f-476a-ade1-f16b40c19140 HTTP/1.1
GET /api/v1/posts?page[limit]=15&page[after]=eyJ1dWlkIjoiMDE5MTZmNDgtNzM4ZS03MWUxLWFlYjktZTEzYWFiMjAyZTM3IiwiX3BvaW50c1RvTmV4dEl0ZW1zIjp0cnVlfQ HTTP/1.1
Accept: application/vnd.api+json
```

Expand All @@ -214,16 +213,16 @@ Content-Type: application/vnd.api+json
{
"meta": {
"page": {
"from": "bfdaa836-68a3-4427-8ea3-2108dd48d4d3",
"from": "eyJ1dWlkIjoiMDE5MTZmNDgtNzM4MS03MzRhLTgxZDctNjk0MWY3ZjAyOTlkIiwiX3BvaW50c1RvTmV4dEl0ZW1zIjpmYWxzZX0",
"hasMore": true,
"perPage": 10,
"to": "df093f2d-f042-49b0-af77-195625119773"
"perPage": 15,
"to": "eyJ1dWlkIjoiMDE5MTZmNDgtNmNkOS03MGQ4LTk0NDctYTY4MGEzNGZlNGQ3IiwiX3BvaW50c1RvTmV4dEl0ZW1zIjp0cnVlfQ"
}
},
"links": {
"first": "http://localhost/api/v1/posts?page[limit]=10",
"prev": "http://localhost/api/v1/posts?page[limit]=10&page[before]=bfdaa836-68a3-4427-8ea3-2108dd48d4d3",
"next": "http://localhost/api/v1/posts?page[limit]=10&page[after]=df093f2d-f042-49b0-af77-195625119773"
"prev": "http://localhost/api/v1/posts?page[limit]=10&page[before]=eyJ1dWlkIjoiMDE5MTZmNDgtNzM4MS03MzRhLTgxZDctNjk0MWY3ZjAyOTlkIiwiX3BvaW50c1RvTmV4dEl0ZW1zIjpmYWxzZX0",
"next": "http://localhost/api/v1/posts?page[limit]=10&page[after]=eyJ1dWlkIjoiMDE5MTZmNDgtNmNkOS03MGQ4LTk0NDctYTY4MGEzNGZlNGQ3IiwiX3BvaW50c1RvTmV4dEl0ZW1zIjp0cnVlfQ"
},
"data": [...]
}
Expand All @@ -244,7 +243,7 @@ For example:
```php
public function pagination(): CursorPagination
{
return CursorPagination::make()
return CursorPagination::make(ID::make())
->withLimitKey('size')
->withAfterKey('starting-after')
->withBeforeKey('ending-before');
Expand All @@ -254,46 +253,52 @@ public function pagination(): CursorPagination
The client would need to send the following request:

```http
GET /api/v1/posts?page[size]=25&page[starting-after]=df093f2d-f042-49b0-af77-195625119773 HTTP/1.1
GET /api/v1/posts?page[size]=25&page[starting-after]=eyJ1dWlkIjoiMDE5MTZmNDgtNzM4ZS03MWUxLWFlYjktZTEzYWFiMjAyZTM3IiwiX3BvaW50c1RvTmV4dEl0ZW1zIjp0cnVlfQ HTTP/1.1
Accept: application/vnd.api+json
```

### Customising the Cursor Column

By default the cursor-based approach uses a model's created at column in
descending order for the list order. This means the most recently created
model is the first in the list, and the oldest is last. As the created at
column is not unique (there could be multiple rows created at the same time),
it uses the resource's route key column as a secondary sort order, as this
column must always be unique.
### Customising the Sort Order

To change the column that is used for the list order use the `withCursorColumn`
method. If you prefer your list to be in ascending order, use the
`withAscending` method. For example:
By default, the sort order is my model key descending, such that in the case of auto-incrementing
or other monotonic keys like UUIDv7 it will be in chronological order (i.e. most recent first,
oldest last).
You can reverse this order (i.e. oldest first, newest last) by using the `withAscending` method. For example:

```php
public function pagination(): CursorPagination
{
return CursorPagination::make()
->withCursorColumn('published_at')
return CursorPagination::make(ID::make())
->withAscending();
}
```

To modify this default sort you can specify a default sort on your Schema. [See Sorting](../schemas/sorting.md)

```php
protected $defaultSort = '-createdAt';
````

When cursor paginating a model key sort will be added to your sort order to ensure a consistent order. If your sort order already includes the model key, it will not be added again.
If your sort order is already deterministic you can disable teh additional key sort with the `withoutKeySort` method. For example:

```php
protected $defaultSort = ['-createdAt', 'uuid'];

public function pagination(): CursorPagination
{
return CursorPagination::make(ID::make())
->withoutKeySort();
}
```

### Validating Cursor Parameters

You should always validate page parameters that sent by an API client.
You should always validate page parameters sent by an API client.
This is described in the [query parameters chapter.](../requests/query-parameters.md)

For the cursor-based approach, you **must** validate that the identifier
provided by the client for the `"after"` and `"before"` parameters are valid
identifiers, because invalid identifiers cause an error in the cursor.
It is also recommended that you validate the `limit` so that it is within an
It is recommended that you validate the `limit` so that it is within an
acceptable range.

As the cursor relies on the list being in a fixed order (that it controls),
you **must** also disable sort parameters.

For example:

```php
Expand All @@ -310,24 +315,85 @@ class PostCollectionQuery extends ResourceQuery
return [
// ...other rules

'sort' => JsonApiRule::notSupported(),
'sort' => JsonApiRule::sort(),

'page' => [
'nullable',
'array',
JsonApiRule::page(),
],

'page.limit' => ['filled', 'numeric', 'between:1,100'],
'page.limit' => ['filled', 'integer', 'between:1,100'],

'page.after' => ['filled', 'string', 'exists:posts,id'],
'page.after' => ['filled', 'string'],

'page.before' => ['filled', 'string', 'exists:posts,id'],
'page.before' => ['filled', 'string'],
];
}
}
```

### Counting results

Cursor-based pagination does not return the total number of resources available by default.

However to add `total` value to the page meta, you can use the `withTotal` method:

```php
public function pagination(): CursorPagination
{
return CursorPagination::make(ID::make())
->withTotal();
}
```
This will return a `total` in the page meta as follows:

```http
HTTP/1.1 200 OK
Content-Type: application/vnd.api+json

{
"meta": {
"page": {
"from": "eyJ1dWlkIjoiMDE5MTZmNDgtNzM4MS03MzRhLTgxZDctNjk0MWY3ZjAyOTlkIiwiX3BvaW50c1RvTmV4dEl0ZW1zIjpmYWxzZX0",
"hasMore": true,
"perPage": 15,
"to": "eyJ1dWlkIjoiMDE5MTZmNDgtNmNkOS03MGQ4LTk0NDctYTY4MGEzNGZlNGQ3IiwiX3BvaW50c1RvTmV4dEl0ZW1zIjp0cnVlfQ"
"total": 54
}
},
"links": {...},
"data": [...]
}
```

This option results in an additional query to count the results on every page request.
This is not recommended since the count could change between successive requests when paginating.

Instead you can opt to calculate this only on a paginated request without a cursor, i.e. the first page, by using the `withTotalOnFirstPage` method:

```php
public function pagination(): CursorPagination
{
return CursorPagination::make(ID::make())
->withTotalOnFirstPage();
}
```

With this option the following request would return the total count:

```http
GET /api/v1/posts?page[limit]=15 HTTP/1.1
Accept: application/vnd.api+json
```

But subsequent requests with a `before` or `after` cursor would not:

```http
GET /api/v1/posts?page[limit]=15&page[after]=eyJ1dWlkIjoiMDE5MTZmNDgtNzM4MS03MzRhLTgxZDctNjk0MWY3ZjAyOTlkIiwiX3BvaW50c1RvTmV4dEl0ZW1zIjpmYWxzZX0 HTTP/1.1
Accept: application/vnd.api+json
```

## Page Size

Both the page-based and cursor-based pagination approaches have a page
Expand Down