eved-explore

An exploration of the Extended Vehicle Energy Dataset (EVED).

Download the paper here.

Overview

This repository contains scripts and Jupyter notebooks for analyzing and processing the Extended Vehicle Energy Dataset (EVED). The project focuses on trajectory querying, map-matching, and energy consumption analysis using Python and SQLite.

Stack & Requirements

• Language: Python 3.x
• Database: SQLite
• Package Manager: pip with pip-tools (via makefile)
• Key Frameworks/Libraries:
  • pandas, numpy, matplotlib (Data analysis)
  • Cython (Performance optimizations for geometric calculations)
  • pyquadkey2 (Quadkey operations)
  • tqdm (Progress bars)
  • valhalla (Optional: Map-matching and routing)
• Infrastructure: pyenv, docker (for Valhalla)

Setup

1. Install Dependencies

The project uses a makefile to manage the environment.

# Install Python version via pyenv, create venv, and sync dependencies
make install

# Compile Cython extensions
make cython

2. Valhalla (Optional)

If you plan to use map-matching features:

make install-valhalla
make run-valhalla

Data Preparation

Follow these steps in order to prepare the database (db/eved.db):

1. Download Data: Run 01-download-data.ipynb to download and extract the EVED dataset.
2. Import Data: Run 02-import-data.ipynb to import CSV files into the SQLite database.
3. Calculate Bearings & Quadkeys: Run the calculate-bearings.py script. This calculates auxiliary columns (bearings and quadkeys) in the signal table.

   ./venv/bin/python calculate-bearings.py

4. Calculate Trajectories: Run the calculate-trajectories.py script. This creates and populates tables for trajectory querying.

   ./venv/bin/python calculate-trajectories.py

   Note: Both scripts will take quite a long time to run.

Usage

Once the data is prepared, you can run the following analysis notebooks:

• 05-trip-estimation.ipynb: Travel time estimation using quadkeys.
• 06-trajectory-querying.ipynb: Trajectory queries using space partitioning.
• 07-edge-matching.ipynb: Map-matching analysis.
• 08-road-network-analysis.ipynb: Road network exploration.

Scripts

• calculate-bearings.py: Calculates bearings and quadkeys for signal points.
• calculate-trajectories.py: Builds trajectory, link, and link_qk tables for efficient spatial querying.
• calculate-matches.py: TODO: Describe purpose.
• calculate-utm.py: TODO: Describe purpose.
• match-trips.py: TODO: Describe purpose.
• path-to-h3.py: TODO: Describe purpose.

Project Structure

• 01-11-*.ipynb: Analysis and data processing notebooks.
• db/: Database schema definitions (JSON/SQL) and Python API (api.py).
• geo/: Geometric and spatial calculation modules (includes Cython code).
• common/: Shared utility functions.
• raster/: Drawing and rasterization utilities.
• valhalla/: Configuration and scripts for running Valhalla in Docker.
• makefile: Environment management and build automation.

Environment Variables

• TODO: List any required environment variables if applicable.

Tests

• TODO: Add information about running tests once they are implemented.

Medium Articles

This project is associated with the following articles:

• Travel Time Estimation Using Quadkeys
• Trajectory Queries Using Space Partitioning
• Map-Matching for Trajectory Prediction
• Map-Matching for Speed Prediction

License

See the LICENSE file for details.