Conversation
| ├── docker-compose.yml | ||
| ├── .env.example |
There was a problem hiding this comment.
Yes - these files make total sense
| Create /demo inside kwwhat with the following layout: | ||
|
|
||
| ``` | ||
| demo/ |
There was a problem hiding this comment.
@daria-sukhareva is demo folder name what you saw in other projects?
There was a problem hiding this comment.
I just saw one but there demo and dbt (a folder with a pipeline) were on the same level. I will give this a try and maybe change it later. I can not decide for now.
| Phase 2 — Service 1: duckdb-init (init container) | ||
|
|
||
| Goal: Populate /data/raw.duckdb with seed CSV data so dbt can read from RAW.SEED.*. | ||
|
|
||
| - Base image: python:3.12-slim | ||
| - Installs: duckdb (Python package) | ||
| - Mounts: ../seeds:/seeds:ro (kwwhat's CSV seed files), duckdb-data:/data | ||
| - init.py creates raw.duckdb, attaches it as catalog RAW, creates schema SEED, loads: | ||
| - RAW.SEED.ocpp_1_6_synthetic_logs_14d from ocpp_1_6_synthetic_logs_14d.csv | ||
| - RAW.SEED.ports from ports.csv | ||
| - Exits 0 when done (docker-compose service_completed_successfully condition) |
| Phase 3 — Service 2: dbt (dbt-core + dbt-mcp) | ||
|
|
||
| Goal: Run dbt build against DuckDB, write marts to ANALYTICS, then serve as MCP endpoint. | ||
|
|
||
| - Base image: python:3.12-slim | ||
| - Installs: dbt-duckdb, dbt-mcp (via pip/uvx) | ||
| - Mounts: ../:/kwwhat:ro, duckdb-data:/data | ||
| - profiles.yml configured with: | ||
| - Main path: /data/analytics.duckdb (target database ANALYTICS) | ||
| - attach: /data/raw.duckdb as alias RAW | ||
| - entrypoint.sh: | ||
| a. Wait for /data/raw.duckdb to exist | ||
| b. dbt deps | ||
| c. dbt build (seed is skipped — seeds already loaded by Service 1 into RAW) | ||
| d. dbt-mcp in HTTP/SSE mode on :8080, local mode (no dbt Cloud) | ||
| - Healthcheck: HTTP GET :8080 responds | ||
| - Exposes port 8080 |
|
|
||
| Key config for dbt-mcp local mode: | ||
| DISABLE_REMOTE=true | ||
| DISABLE_SEMANTIC_LAYER=true (no SL in local DuckDB) |
There was a problem hiding this comment.
I don't know what is this flag. @daria-sukhareva you should check if it makes sense to you
There was a problem hiding this comment.
@eugene-mobile ok thank you for catching this
| Phase 4 — Service 3: chat-bi (nao-core) | ||
|
|
||
| Goal: Interactive chat BI connected to mart models in DuckDB, using dbt-core as MCP. | ||
|
|
||
| - Base image: python:3.12-slim | ||
| - Installs: nao-core==0.0.59 | ||
| - Mounts: duckdb-data:/data | ||
| - nao_config.yaml: DuckDB connection to /data/analytics.duckdb, schema main, include fact_* and dim_* — | ||
| requires verifying nao supports DuckDB (currently only Snowflake in kwwhat-chat-BI — this is the biggest | ||
| open question) | ||
| - mcp.json: connects to Service 2 MCP via HTTP/SSE http://dbt:8080/sse | ||
| - entrypoint.sh: wait for Service 2 healthcheck → nao chat | ||
| - stdin_open: true, tty: true (interactive terminal) |
| Phase 5 — docker-compose.yml | ||
|
|
||
| services: | ||
| duckdb-init: | ||
| build: ./duckdb-init | ||
| volumes: [duckdb-data:/data, ../seeds:/seeds:ro] | ||
|
|
||
| dbt: | ||
| build: ./dbt | ||
| depends_on: | ||
| duckdb-init: { condition: service_completed_successfully } | ||
| volumes: [duckdb-data:/data, ../:/kwwhat:ro] | ||
| ports: ["8080:8080"] | ||
| healthcheck: { test: curl http://localhost:8080 } | ||
|
|
||
| chat-bi: | ||
| build: ./chat-bi | ||
| depends_on: | ||
| dbt: { condition: service_healthy } | ||
| volumes: [duckdb-data:/data] | ||
| stdin_open: true | ||
| tty: true | ||
|
|
||
| volumes: | ||
| duckdb-data: |
There was a problem hiding this comment.
This will orchestrate all of the docker images. Totally makes sense
| Phase 6 — run-demo.sh | ||
|
|
||
| Orchestration script: | ||
| 1. docker compose build | ||
| 2. docker compose up duckdb-init (wait for completion) | ||
| 3. docker compose up dbt -d (background, wait for healthy) | ||
| 4. docker compose run --rm chat-bi (interactive session) | ||
| 5. Teardown instructions |
eugene-mobile
left a comment
There was a problem hiding this comment.
I think that's a solid plan
Uh oh!
There was an error while loading. Please reload this page.