Skip to content
This repository was archived by the owner on Oct 15, 2022. It is now read-only.
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions packages/dynamodb-cursor-based-pagination/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.log
.DS_Store
node_modules
dist
coverage
scripts
162 changes: 162 additions & 0 deletions packages/dynamodb-cursor-based-pagination/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# DynamoDB Cursor-Based Pagination

## Introduction

### DynamoDB

From Amazon DynamoDB [page](https://aws.amazon.com/dynamodb):

> Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It's a fully managed, multiregion, multimaster, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications. DynamoDB can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second.

To achieve this hyper performance and scalability , it lacks common RDBMS and some NoSQL databases features. For that reason, DynamoDB provides only features that are scalable and its query is one of them.

[Querying on DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html) is not powerful compared to other databases. It must satisfy some requirements:

1. The table or the secondary index must have a [composite primary key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey) (a partition/hash key and a sort/range key).
2. The partition key must be defined and the sort key is not required. If defined, it's used to [sort the items](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions).

For instance, you cannot query items whose hash key is different. Because of that you have to [design your table and indexes](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html) to support all queries you need to perform.

### Cursor-Based Pagination

If you're familiarized with GraphQL, you may have seen from [its documentation](https://graphql.org/learn/pagination/) that cursor-based pagination is more powerful than others pagination designs. Also, if we do some search, we can find comparisons among pagination designs ([here](https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32)) and cursor-based pagination is the winner.

Considering the advantages of cursor-based pagination, this package proposes a design to allow us perform cursor-based pagination in a DBB table.

## Table Design

There aren't much requirements to achieve to be able to perform cursor-based pagination. In resume, we need:

1. A table or a secondary index with a [composite primary key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey).
1. Items must be saved in such way that the range key ordination must represent the ordination of the pagination.

Why do we need the second requirement? First we need to understand what cursor is. Cursor is like an edge identifier, with cursor we must be able to retrieve and locate that edge on your backend. In a DBB table with composite primary key, the sort key is a good choice to be our cursor.

## Installation

```
npm install -S @ttoss/dynamodb-cursor-based-pagination
```

or

```
yarn add @ttoss/dynamodb-cursor-based-pagination
```

You also need install `aws-sdk` in your project because it is a peer dependency of this project.

## How to Use

```
import { paginate } from 'dynamodb-cursor-based-pagination';
```

`paginate` is a method whose signature is:

```ts
type paginate<T = any> = ({
dynamoDBClient,
tableName,
hashKey,
hashKeyValue,
rangeKey,
index,
projectionExpression,
filterExpression,
filterAttributeNames,
filterAttributeValues,
beginsWith = '',
sort = 'DESC',
after,
first,
before,
last,
}: {
dynamoDBClient: DynamoDBClient;
tableName: string;
hashKey: string;
hashKeyValue: string;
rangeKey: string;
beginsWith?: string;
index?: string;
projectionExpression?: string;
filterExpression?: string;
filterAttributeNames?: { [key: string]: string };
filterAttributeValues?: { [key: string]: any };
sort?: Sort;
after?: string;
before?: string;
first?: number;
last?: number;
}) => Promise<{
edges: {
cursor: string;
node: T;
}[];
pageInfo: {
hasPreviousPage: boolean;
hasNextPage: boolean;
startCursor?: string | undefined;
endCursor?: string | undefined;
};
consumedCapacity: number | undefined;
count: number | undefined;
scannedCount: number | undefined;
lastEvaluatedKey: string | undefined;
}>;
```

### DynamoDB Table Parameters

The parameters `region`, `tableName`, `hashKeyValue` are used to identify your DynamoDB table and the partition.

### Cursor Parameters

- `first` and `after`: [forward pagination arguments.](https://relay.dev/graphql/connections.htm#sec-Forward-pagination-arguments)
- `last` and `before`: [backward pagination arguments.](https://relay.dev/graphql/connections.htm#sec-Backward-pagination-arguments)

### Sorting

- `sort: 'ASC' | 'DESC' (default 'DESC')`

Querying on DynamoBD is related to the sorting of the items in function of their sort key value. Because of this, the parameter `sort` defines the items order before perform pagination. `ASC` is for ascending sorting (`a`, `b`, ..., `z`) and `DESC`, for descending (`z`, `y`, ..., `a`).

### Begins With

- `beginsWith: string | undefined`

Your DynamoDB table may have an architecture that made the items have a [`beginsWith` property](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions). If you want to paginate over items that have such property, just add `beginsWith` to `paginate` method.

### Projection Expression

- `projectionExpression: string | undefined`

[DynamoDB projection expression reference](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ProjectionExpressions.html).

### Filtering

- `filterExpression?: string | undefined`
- `filterAttributeNames: { [key: string]: string; } | undefined`
- `filterAttributeValues: { [key: string]: string; } | undefined`

[DynamoDB filtering reference](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.FilterExpression).

```ts
// Example

paginate({
... // oher params,
filterExpression: '#parity = :parity',
filterAttributeNames: {
'#parity': 'parity',
},
filterAttributeValues: {
':parity': 'EVEN',
},
});
```

## Examples

Let's check [some examples](examples/README.md) to understand better [these requirements and the design](#table-design).
Loading