StreamSim is a real-time data streaming simulator tool that allows researchers and developers to simulate real-time streaming behavior for tabular data. It provides a convenient way to test real-time applications when the actual real-time data source is not yet available or when simulating different scenarios for testing purposes.
For detailed instructions on how to use the functions and modules provided by StreamSim, please refer to the StreamSim Documentation.
- Simulates real-time streaming behavior for tabular data.
- Supports stream simulation from a database table or a CSV file.
- Customizable data processing and simulation logic.
- Easy configuration.
Before using StreamSim, make sure you have the following prerequisites installed:
- Python 3.x
- Required dependencies (see Installation section)
-
Clone the repository:
git clone https://github.com/NIH-W4H/StreamSim.git
-
Install the required dependencies:
pip install -r requirements.txt
-
Configure the application settings in the
conf.pyfile. Specify the database connection details, dataset path, timeout interval, and other relevant parameters. -
Implement the desired data processing logic in the
process_stream()function in thestream_sim.pyfile. Customize it to suit your specific requirements. -
Start the Flask server to enable stream simulation:
python stream_sim.py
-
In a separate terminal, run the ingest_stream.py script to periodically fetch streams from the Flask server:
python ingest_stream.py
-
Observe the simulated stream data and verify that it meets your expectations.
Contributions are welcome! If you have any suggestions, bug reports, or feature requests, please open an issue or submit a pull request.
This project is licensed under the MIT License. Feel free to use and modify this code for your own purposes.