This repository was archived by the owner on Oct 15, 2022. It is now read-only.
generated from ttoss/monorepo
-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add ddb module #73
Draft
arthurlolico
wants to merge
8
commits into
main
Choose a base branch
from
feat-add-ddb-cursor-based-pag-module
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
44d37f5
feat: add ddb module
arthurlolico a2e37af
Merge branch 'feat-add-ddb-cursor-based-pag-module' of https://github…
arthurlolico 1c698a1
feat: changes in Testing and minor changes in Docs
arthurlolico 4e01af0
Merge branch 'main' into feat-add-ddb-cursor-based-pag-module
arthurlolico 08ac39b
fix: delete several files and fix Docs
arthurlolico 03dca11
Merge branch 'feat-add-ddb-cursor-based-pag-module' of https://github…
arthurlolico dc20e8e
chore: minor changes in some files
arthurlolico 5c10512
chore: delete LICENSE file
arthurlolico File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| *.log | ||
| .DS_Store | ||
| node_modules | ||
| dist | ||
| coverage | ||
| scripts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| # DynamoDB Cursor-Based Pagination | ||
|
|
||
| ## Introduction | ||
|
|
||
| ### DynamoDB | ||
|
|
||
| From Amazon DynamoDB [page](https://aws.amazon.com/dynamodb): | ||
|
|
||
| > Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It's a fully managed, multiregion, multimaster, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications. DynamoDB can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second. | ||
|
|
||
| To achieve this hyper performance and scalability , it lacks common RDBMS and some NoSQL databases features. For that reason, DynamoDB provides only features that are scalable and its query is one of them. | ||
|
|
||
| [Querying on DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html) is not powerful compared to other databases. It must satisfy some requirements: | ||
|
|
||
| 1. The table or the secondary index must have a [composite primary key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey) (a partition/hash key and a sort/range key). | ||
| 2. The partition key must be defined and the sort key is not required. If defined, it's used to [sort the items](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions). | ||
|
|
||
| For instance, you cannot query items whose hash key is different. Because of that you have to [design your table and indexes](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html) to support all queries you need to perform. | ||
|
|
||
| ### Cursor-Based Pagination | ||
|
|
||
| If you're familiarized with GraphQL, you may have seen from [its documentation](https://graphql.org/learn/pagination/) that cursor-based pagination is more powerful than others pagination designs. Also, if we do some search, we can find comparisons among pagination designs ([here](https://medium.com/swlh/how-to-implement-cursor-pagination-like-a-pro-513140b65f32)) and cursor-based pagination is the winner. | ||
|
|
||
| Considering the advantages of cursor-based pagination, this package proposes a design to allow us perform cursor-based pagination in a DBB table. | ||
|
|
||
| ## Table Design | ||
|
|
||
| There aren't much requirements to achieve to be able to perform cursor-based pagination. In resume, we need: | ||
|
|
||
| 1. A table or a secondary index with a [composite primary key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey). | ||
| 1. Items must be saved in such way that the range key ordination must represent the ordination of the pagination. | ||
|
|
||
| Why do we need the second requirement? First we need to understand what cursor is. Cursor is like an edge identifier, with cursor we must be able to retrieve and locate that edge on your backend. In a DBB table with composite primary key, the sort key is a good choice to be our cursor. | ||
|
|
||
| ## Installation | ||
|
|
||
| ``` | ||
| npm install -S @ttoss/dynamodb-cursor-based-pagination | ||
| ``` | ||
|
|
||
| or | ||
|
|
||
| ``` | ||
| yarn add @ttoss/dynamodb-cursor-based-pagination | ||
| ``` | ||
|
|
||
| You also need install `aws-sdk` in your project because it is a peer dependency of this project. | ||
|
|
||
| ## How to Use | ||
|
|
||
| ``` | ||
| import { paginate } from 'dynamodb-cursor-based-pagination'; | ||
| ``` | ||
|
|
||
| `paginate` is a method whose signature is: | ||
|
|
||
| ```ts | ||
| type paginate<T = any> = ({ | ||
| dynamoDBClient, | ||
| tableName, | ||
| hashKey, | ||
| hashKeyValue, | ||
| rangeKey, | ||
| index, | ||
| projectionExpression, | ||
| filterExpression, | ||
| filterAttributeNames, | ||
| filterAttributeValues, | ||
| beginsWith = '', | ||
| sort = 'DESC', | ||
| after, | ||
| first, | ||
| before, | ||
| last, | ||
| }: { | ||
| dynamoDBClient: DynamoDBClient; | ||
| tableName: string; | ||
| hashKey: string; | ||
| hashKeyValue: string; | ||
| rangeKey: string; | ||
| beginsWith?: string; | ||
| index?: string; | ||
| projectionExpression?: string; | ||
| filterExpression?: string; | ||
| filterAttributeNames?: { [key: string]: string }; | ||
| filterAttributeValues?: { [key: string]: any }; | ||
| sort?: Sort; | ||
| after?: string; | ||
| before?: string; | ||
| first?: number; | ||
| last?: number; | ||
| }) => Promise<{ | ||
| edges: { | ||
| cursor: string; | ||
| node: T; | ||
| }[]; | ||
| pageInfo: { | ||
| hasPreviousPage: boolean; | ||
| hasNextPage: boolean; | ||
| startCursor?: string | undefined; | ||
| endCursor?: string | undefined; | ||
| }; | ||
| consumedCapacity: number | undefined; | ||
| count: number | undefined; | ||
| scannedCount: number | undefined; | ||
| lastEvaluatedKey: string | undefined; | ||
| }>; | ||
| ``` | ||
|
|
||
| ### DynamoDB Table Parameters | ||
|
|
||
| The parameters `region`, `tableName`, `hashKeyValue` are used to identify your DynamoDB table and the partition. | ||
|
|
||
| ### Cursor Parameters | ||
|
|
||
| - `first` and `after`: [forward pagination arguments.](https://relay.dev/graphql/connections.htm#sec-Forward-pagination-arguments) | ||
| - `last` and `before`: [backward pagination arguments.](https://relay.dev/graphql/connections.htm#sec-Backward-pagination-arguments) | ||
|
|
||
| ### Sorting | ||
|
|
||
| - `sort: 'ASC' | 'DESC' (default 'DESC')` | ||
|
|
||
| Querying on DynamoBD is related to the sorting of the items in function of their sort key value. Because of this, the parameter `sort` defines the items order before perform pagination. `ASC` is for ascending sorting (`a`, `b`, ..., `z`) and `DESC`, for descending (`z`, `y`, ..., `a`). | ||
|
|
||
| ### Begins With | ||
|
|
||
| - `beginsWith: string | undefined` | ||
|
|
||
| Your DynamoDB table may have an architecture that made the items have a [`beginsWith` property](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.KeyConditionExpressions). If you want to paginate over items that have such property, just add `beginsWith` to `paginate` method. | ||
|
|
||
| ### Projection Expression | ||
|
|
||
| - `projectionExpression: string | undefined` | ||
|
|
||
| [DynamoDB projection expression reference](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ProjectionExpressions.html). | ||
|
|
||
| ### Filtering | ||
|
|
||
| - `filterExpression?: string | undefined` | ||
| - `filterAttributeNames: { [key: string]: string; } | undefined` | ||
| - `filterAttributeValues: { [key: string]: string; } | undefined` | ||
|
|
||
| [DynamoDB filtering reference](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.FilterExpression). | ||
|
|
||
| ```ts | ||
| // Example | ||
|
|
||
| paginate({ | ||
| ... // oher params, | ||
| filterExpression: '#parity = :parity', | ||
| filterAttributeNames: { | ||
| '#parity': 'parity', | ||
| }, | ||
| filterAttributeValues: { | ||
| ':parity': 'EVEN', | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| Let's check [some examples](examples/README.md) to understand better [these requirements and the design](#table-design). | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.