-
Notifications
You must be signed in to change notification settings - Fork 0
Storage Medium
The storage medium is a storage entity that will be used to hold and store the QueueMessages. There are specific performance benefits when using some over others which will be formalised in future.
The MessageQueue's storage medium is controlled by one main environment variable message-queue.storage-medium. Prior to version 0.3.0 please use MULTI_QUEUE_TYPE.
Depending on which storage medium is selected, more environmental properties will be required. If no value is provided for this property, the default storage medium is IN_MEMORY which stores messages in the application memory meaning all messages will be lost when the MessageQueue is stopped.
Below is a list of the different storage mediums and sample configuration.
Please reach out if there are any new database technologies that this should support.
The In-Memory configuration is the default and requires no further configuration.
environment:
- message-queue.storage-medium=IN_MEMORYTo set the application into SQL mode you need to provide the following environment variable with the appropriate value: message-queue.storage-medium=SQL.
The following are the officially supported databases:
- PostgreSQL
- MySQL
- Microsoft SQL (as of 3.3.3)
- SQLite (as of version 3.4.0)
Use of other databases may be unstable.
Below are the required properties for SQL configuration.
This property is optional.
Depending on your setup this may be required. Please refer to https://docs.spring.io/spring-boot/docs/1.1.0.M1/reference/html/howto-database-initialization.html#howto-initialize-a-database-using-hibernate around how this property is used and how Hibernate is used to create and generate the database structure.
Assuming you are not manually creating and maintaining the database message structure, I recommend setting this property to create (E.g. spring.jpa.hibernate.ddl-auto=create).
This property is required.
Just providing spring.autoconfigure.exclude= (with an empty value) as one of the environment variables is required to force JPA to initialise correctly.
By default, it is suppressed to allow of more stream lined configuration of the other mediums.
This property is required.
This defines the database connection string that the application should connect to. E.g: jdbc:mysql://localhost:3306/message-queue
This property is required.
This is the username/account name used to access the database at the configured endpoint.
This property is required.
This is the password used to access the database at the configured endpoint.
environment:
- message-queue.storage-medium=SQL
- spring.jpa.hibernate.ddl-auto=create
- spring.autoconfigure.exclude=
- spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/postgres
- spring.datasource.username=postgres
- spring.datasource.password=5up3r5tR0nG!To set the application into Redis mode you need to provide the following environment variable with the appropriate value: message-queue.storage-medium=REDIS.
Once this is set you will need to provide further environment variable in order to correctly configure the redis standalone or sentinel configuration (depending on your setup).
The MessageQueue supports both standalone and sentinel Redis configurations.
Below are the optional and required properties for the Redis configuration.
This property is optional.
*Prior to version 0.3.0 please use REDIS_USE_SENTINELS.
By default, this property is set to false (defaulting to direct connection to a single-instance/standalone Redis instance).
This flag indicates whether the MultiQueue should connect directly to the redis instance or connect via one or more sentinel instances.
If set to true the MultiQueue will create a sentinel pool connection instead of a direct connection to a single redis node.
This property is optional.
*Prior to version 0.3.0 please use REDIS_PREFIX.
By default, this property is set to "" the application will apply no prefix.
For each defined "sub-queue" or "queue type", the application will create a single set entry into redis. This prefix can be used to remove/reduce the likelihood of any collisions if this is being used in an existing redis instance.
The defined value will be used as a prefix used for all redis entry keys that the application will create.
E.g. if the initial value for the redis entry key is my-key and no prefix is defined the entries would be stored under my-key.
Using the same scenario if the prefix is prefix then the prefix and entry key will be concatenated and the resultant key would be prefixmy-key.
This property is required.
*Prior to version 0.3.0 please use REDIS_ENDPOINT.
By default, this property is set to 127.0.0.1 (the application will auto append the default redis port if it is not defined).
The endpoint string which is used for both standalone and the sentinel redis configurations.
This supports a comma separated list or single definition of a redis endpoint in the following formats:
<endpoint>:<port>,<endpoint2>:<port2>,<endpoint3>.
If you are using standalone and provide multiple endpoints, only the first endpoint in the comma separated list will be used.
This property is optional.
*Prior to version 0.3.0 please use REDIS_MASTER_NAME.
By default, this property is set to mymaster.
This is REQUIRED when message-queue.redis.sentinel is set to true. Is used to indicate the name of the redis master instance.
An example of Redis configuration environment variables is below:
environment:
- message-queue.storage-medium=REDIS
- message-queue.redis.sentinel=true
- message-queue.redis.prefix=my-prefix
- message-queue.redis.endpoint=sentinel1.com:5545,sentinel2.org:9980
- message-queue.redis.master-name=not-my-masterThe application can be set into MONGO mode to interface with a NoSQL database. Similarly to the others you can set the type with message-queue.storage-medium=MONGO.
You can either specify all properties individually via, spring.data.mongodb.host, spring.data.mongodb.port, spring.data.mongodb.database, spring.data.mongodb.username, spring.data.mongodb.password.
Or you can provide all together in a single property: spring.data.mongodb.uri.
This property is required, unless spring.data.mongodb.uri is provided.
This is the host that the mongo DB is accessible from.
This property is required, unless spring.data.mongodb.uri is provided.
This is the database that should be connected to and where the related documents will be created.
This property is required, unless spring.data.mongodb.uri is provided.
This is the username/account name used to access the database at the configured endpoint.
This property is required, unless spring.data.mongodb.uri is provided.
This is the password used to access the database at the configured endpoint.
This property is required, unless spring.data.mongodb.uri is provided.
The port that the mongo db has exposed.
This property is required, unless the above properties are already provided.
The whole url can be provided in the following format: mongodb://<username>:<password>@<host>:<port>/<database> for example: mongodb://root:password@localhost:27107/messagequeue.
Note: the use of ?authSource=admin is to allow you to get up and running quickly, properly secured credentials and a non-admin account should always be used.
environment:
- message-queue.storage-medium=MONGO
- spring.data.mongodb.uri=mongodb://root:password@mongo:27017/messagequeue?authSource=admin