Skip to content

feat: bundled MC2MQTT broker presets (waev, letsmesh) + format family#227

Open
dmduran12 wants to merge 1 commit intorightup:devfrom
dmduran12:feat/generalized-mqtt
Open

feat: bundled MC2MQTT broker presets (waev, letsmesh) + format family#227
dmduran12 wants to merge 1 commit intorightup:devfrom
dmduran12:feat/generalized-mqtt

Conversation

@dmduran12
Copy link
Copy Markdown
Contributor

What this does

Adds bundled MeshCoreToMQTT (MC2MQTT) broker presets so operators can subscribe
to a known mesh network with one line of config instead of copy-pasting
multi-line broker definitions. Endpoints ship with the package, so URL/audience
updates arrive via pip install -U.

Why

Today, anyone publishing to LetsMesh, Waev, or any future MC2MQTT-compatible
platform has to copy/paste the full broker definition into config.yaml. When
the operator changes endpoints upstream, every node operator has to edit
manually. Hard-coded LETSMESH_BROKERS constant in mqtt_handler.py was the
only "registry" — Python code, not data.

How to use

Minimal — set and forget

mqtt_brokers:
  iata_code: "LAX"
  brokers:
    - preset: waev

Multiple consumers

mqtt_brokers:
  brokers:
  iata_code: "LAX"
    - preset: waev
    - preset: letsmesh

Override a preset broker: Place override entries after the preset entry. Later wins on name collision.

mqtt_brokers:
  brokers:
    - preset: waev
    - name: waev-b
      enabled: false

Mix presets with a custom broker:

mqtt_brokers:
  brokers:
    - preset: waev
    - name: my-lan-mqtt
      enabled: true
      host: mqtt.lan
      port: 1883
      transport: tcp
      format: mqtt
      username: repeater
      password: secret

API Changes
POST /api/update_mqtt_config now accepts preset references inside brokers
A {preset: } entry is passed through unchanged; the MQTT handler
expands it on next start. All other broker entries are validated as before
(name/host/port/format required).

{
  "brokers": [
    {"preset": "waev"},
    {"name": "waev-b", "enabled": false}
  ]
}

Behavior contract

• Empty brokers: → empty (current behavior).
• Only ordinary entries → unchanged from today.
• [{preset: waev}] → both Waev brokers enabled.
• [{preset: waev}, {name: waev-b, enabled: false}] → both, waev-b disabled.
• [{preset: bogus}] → empty + warning log; daemon does not crash.
• Legacy letsmesh: block → produces the same broker set as before for every
prior broker_index value (-1 / 0 / 1).

What's deleted

• Hard-coded LETSMESH_BROKERS constant (~24 lines) — its data is now
repeater/presets/letsmesh.yaml.
• convert_letsmesh_to_broker_config() collapsed from ~70 lines to ~25 lines.
The legacy migrator now emits {preset: letsmesh} plus disable overrides
and lets the same expansion path handle the rest.

Net code change is negative outside the new loader.

Tests

tests/test_presets.py — 9 tests covering:

• Preset loader (list_presets, get_preset, unknown returns {}).
• Pass 1 expansion (inlines bundled brokers, drops unknown with warning).
• Pass 2 merge (override AFTER preset wins).
• Documented order rule (override BEFORE preset is silently overwritten — locks
the published rule so a future refactor can't quietly flip it).
• MC2MQTT topic-family parity vs legacy mqtt topic.
• Parametrized legacy broker_index in (-1, 0, 1) migration.

All 9 pass. Pre-existing test failures on dev are unrelated to this PR
(verified by re-running them on dev before and after).

note:
The frontend dropdown for the new meshcoretomqtt format value lands in a
follow-up frontend PR — that bundle lives in a separate repo. The backend
already accepts both shapes, so YAML-edit operators get the full feature today.

Introduces a 'set format and forget' workflow for MQTT brokers. Users
reference a bundled preset by name inside the existing brokers: list,
and the package supplies the endpoints, audiences, and TLS settings.
Endpoint changes ship via 'pip install -U' instead of manual edits.

What changes
- New repeater/presets/ package with a tiny lazy YAML loader and two
  bundled presets: waev (mqtt-{a,b}.waev.app) and letsmesh (EU + US).
- New format-family constant MC2MQTT_FORMATS = ('meshcoretomqtt',
  'letsmesh', 'waev') replaces the inline tuple in topic resolution.
  The legacy 'mqtt' format keeps its custom-topic semantics unchanged.
- Two-pass broker assembly in mqtt_handler.py: pass 1 expands every
  {preset: <name>} entry inline; pass 2 collapses duplicates by name
  with later-wins semantics. Place override entries AFTER preset
  entries.
- Hard-coded LETSMESH_BROKERS constant deleted; its data now lives in
  repeater/presets/letsmesh.yaml.
- convert_letsmesh_to_broker_config() collapsed from ~70 to ~25 lines
  by emitting {preset: letsmesh} plus disable overrides for unwanted
  brokers. Honors broker_index in (-1, 0, 1), additional_brokers, and
  enabled flag exactly as before.
- update_mqtt_config API endpoint accepts {preset: <name>} entries and
  passes them through unchanged so the web UI can author them when the
  frontend is updated.
- config.yaml.example documents the preset entry shape, the override
  rule, and the format family hierarchy.
- pyproject.toml ships presets/*.yaml as package data.

How to use
  mqtt_brokers:
    iata_code: "LAX"
    brokers:
      - preset: waev

  # Override a single preset broker:
  brokers:
    - preset: waev
    - name: waev-b
      enabled: false

Tests
- tests/test_presets.py: 9 tests covering loader, expand/merge,
  MC2MQTT topic-family parity, and parametrized legacy migration.

Co-Authored-By: Oz <oz-agent@warp.dev>
@rightup
Copy link
Copy Markdown
Owner

rightup commented May 5, 2026

@Rigear this touches your work do you mind taking a look. thanks

@Rigear
Copy link
Copy Markdown
Contributor

Rigear commented May 5, 2026

@rightup Changes look good to me.

Nice work @dmduran12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants