A small ETL that pulls your MyFitnessPal food, exercise, and weight entries and writes them to Postgres. Designed to run on a schedule so you end up with a local historical record of everything you logged, queryable with SQL.
You bring the Postgres. This project doesn't host or provision a database for
you. It connects over TCP using psycopg2, so any reachable Postgres works:
a local install, a Docker container, a homelab VM, or a managed service like
RDS, Supabase, or Neon. All it needs is a valid connection string in .env.
Once the data is in Postgres, you can point any BI tool at it. The dashboards above are built in Metabase. They're not part of this repo, but if you want the full self-hosted analytics stack (Postgres, Metabase, and related tools in one docker-compose), see mumtazm1/self-hosted-analytics.
Authentication piggybacks on an existing browser session. You log in to
MyFitnessPal in Firefox (or Chrome/Edge), and sync.py reads the session
cookie from the browser's cookie store. If no valid cookie is found, it opens
the browser to the login page and waits for you to sign in, then continues.
No credentials are stored anywhere.
Data is written with idempotent upserts, so re-running a sync for a date range already in the database is safe. Existing rows are replaced, new rows are inserted, nothing is duplicated. Weight measurements are append-only.
The schema has three tables: food_entries (with full macro and
micronutrient breakdown), exercises, and weight_measurements. See
docs/database_schema.md for column-level detail.
Requires Python 3.10+ and a running Postgres instance you can write to.
git clone https://github.com/mumtazm1/mfp-postgres-sync.git
cd mfp-postgres-sync
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# edit .env with your Postgres connection details
# create the schema (replace user/db with your own)
psql -U postgres -d your_database -f sql/create_tables.sqlpython sync.pyOn first run it opens Firefox, waits for you to log in to MyFitnessPal, and syncs the last 90 days (configurable). On subsequent runs it reuses the browser's stored cookie. A sync typically takes a few minutes depending on how many days and entries are involved; there's a configurable rate limit between requests to stay well under MyFitnessPal's limits.
config.py (gitignored, copy config.example.py to start) controls the
sync date range and rate limit. Date ranges can be fixed or dynamic:
from datetime import date, timedelta
START_DATE = lambda: date.today() - timedelta(days=90)
END_DATE = lambda: date.today()
RATE_LIMIT_SECONDS = 2Database credentials come from .env (also gitignored).
Authentication uses Firefox cookies by default. Chrome and Edge are not
currently wired up, though browser-cookie3 supports them if you want to
extend auth.py.
A cron-based runner lives in task-scheduler/linux/. setup_cron.sh install
registers a job that runs sync.py every 2 hours and appends output to
logs/sync.log. See task-scheduler/README.md for details.
prefect_flow.py exposes the sync as a Prefect flow if you'd rather run it
under a Prefect server than cron. Prefect isn't in the default
requirements.txt because it's a heavy dependency most users won't need:
pip install -r requirements-prefect.txtThe flow reads credentials from a Prefect Variable and a Secret block so it
never touches .env:
- Variable
postgres_config— JSON withhost,port,database,user - Secret
postgres-password— the database password
setup_blocks.py reads the same environment variables sync.py uses and
creates both. Run it once against your Prefect server:
export POSTGRES_HOST=... POSTGRES_PORT=... POSTGRES_DB=...
export POSTGRES_USER=... POSTGRES_PASSWORD=...
python setup_blocks.pyFrom there, register a deployment with prefect deploy or call the flow
directly with python prefect_flow.py.
docs/TROUBLESHOOTING.md covers the common failure modes: stale cookies,
missing tables, permission errors, browser detection issues.
MIT