A storr driver using TileDB Embedded storage engine.
What is storr?
A key-value store with content addressable storage for R objects with optional in-memory caching layer.
The interface is provided by storr
package and defines a set of common operations (set, get, del methods) across a range of different
storage drivers (DBI, LMDB, Redis, RDS, environment). The storr package is
written by Rich FitzJohn.
What is storr.tiledb?
The storr.tiledb contributes a new storr compliant driver using TileDB Engine for backend.
It defines a new storr R6 class that leverages the advantages of TileDB and provides new features such as adding notes and expiration time-stamps along with key-namespace pairs.
Warning
The package is in experimental status. Currently, the driver is complete and fully functional in that it passes the auto test specification defined by storr package. More testing is needed and additional features to be implemented before it is moved to mature state. Feedback is welcomed.
-
Key-value operations use driver's methods for speed and efficiency
-
Key-value metadata such as notes and timestamps for data expiration (time-to-live, TTL)
-
Asynchronous key-value operations or in parallel through mirai framework
-
Hash tables (hashtab) for in memory caching layers instead of environments
-
Cloud storage (S3, Azure, GSC)
-
Data versioning (time-travelling)
-
Encryption
Development version from GitHub:
# pak
pak::pkg_install("cgiachalis/storr.tiledb")
# remotes
remotes::install_github("cgiachalis/storr.tiledb")library(storr.tiledb)
# Temp URI path
uri <- tempfile()
# Set up storr
sto <- storr_tiledb(uri, init = TRUE)
# Set
sto$set("mykey1", list(a = 1))
sto$set("mykey2", "abc")
# Get
sto$get("mykey2")
[1] "abc"
# List all keys
sto$list()
[1] "mykey1" "mykey2"
# Del
sto$del("mykey1")
- storr_enviroment() in memory storage using R enviroment
- storr_rds() on disk storage using RDS file format
- storr_dbi() using DBI interface
- storr_redis_api() using Redis through redux
- storr_thor() - using LMDB Lightning Memory-Mapped Database through thor