Skip to content

Add AWS IoT MQTT to resolve 429 Rate Limit issues and provde real-time device shadow updates - #21

Merged
benjycov merged 3 commits into
benjycov:mainfrom
abrainwood:upstream-pr
Apr 27, 2026
Merged

benjycov merged 3 commits into
benjycov:mainfrom
abrainwood:upstream-pr

Conversation

@abrainwood

@abrainwood abrainwood commented Apr 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR replaces REST polling with MQTT push as the primary data channel, using the same AWS IoT Device Shadow protocol that the official iAqualink iOS app uses. The result is sub-second state sync, zero 429 rate limit errors, and instant writes.

No breaking changes. No additional setup required - the integration uses AWS credentials already returned by the Zodiac login API (previously discarded). Existing config entries work unchanged. REST polling is kept as a 1-hour fallback.

What changed

  • New mqtt_client.py: wraps awsiotsdk to connect to AWS IoT via MQTT over WebSocket with SigV4 auth. Handles subscribe, publish, reconnect, heartbeat (15-min shadow/get), and reconnect failure recovery.
  • api.py: login/refresh now extracts AWS credentials from the response. Coordinator fed by MQTT push via async_set_updated_data(). Writes route through MQTT when connected, REST as fallback. Credential refresh scheduled before expiry. Proper cleanup on unload.
  • manifest.json: added awsiotsdk dependency, iot_class changed to cloud_push.
  • number.py: fixed _attr_step -> _attr_native_step (fixes pH set point validation error in HA frontend).
  • __init__.py: calls cleanup_entry on unload (MQTT disconnect, task cancellation).
  • Dev tooling: docker-compose.dev.yml, Makefile, scripts/dev-setup.py for contributor onboarding.
  • 37 tests: unit tests for MQTT client + integration tests verifying shadow data contract against real device data.

How it works

  1. Login via REST (existing flow) - extract AWS credentials from response
  2. Connect MQTT to AWS IoT endpoint
  3. Subscribe to shadow topics (update/documents, get/accepted, etc.)
  4. Shadow push events -> coordinator.async_set_updated_data() -> entities update instantly
  5. Writes publish to shadow/update via MQTT (REST fallback if disconnected)
  6. 15-minute heartbeat keeps data fresh during quiet periods
  7. Credential refresh ~55 min before expiry, with reconnect failure recovery

Testing

  • 37 pytest tests (unit + integration), all passing
  • Tested on real hardware: Exo phIQ with pH, chlorinator, aux switches, schedules
  • 48+ hours production soak: zero 429s, clean credential refresh cycling, real-time sensor updates
  • Multiple rapid HA restarts: clean MQTT reconnect every time

Test plan

  • 37 pytest tests pass
  • Sub-second state sync between iAqualink app and HA dashboard
  • Writes from HA go via MQTT (pH setpoint, aux switches, chlorinator %)
  • Zero 429 rate limit errors over 48+ hour soak
  • Credential refresh cycles cleanly every ~55 min
  • HA restart reconnects MQTT without 429s
  • REST fallback works when MQTT is unavailable
  • pH set point validation error fixed

🤖 Generated with Claude Code with guardrail rules for strict TDD and clean code.

@abrainwood

Copy link
Copy Markdown
Contributor Author

@benjycov ack this is a pretty large PR; let me know if you'd prefer me to split it up into smaller chunks.

@abrainwood abrainwood changed the title Add AWS IoT MQTT for real-time device shadow updates Add AWS IoT MQTT to resolve 429 Rate Limit issues and provde real-time device shadow updates Apr 19, 2026
abrainwood and others added 3 commits April 20, 2026 00:38
Replace REST polling with MQTT push as the primary data channel for eXO
devices. The Zodiac login response includes AWS temporary credentials
(previously discarded) which allow connecting to the AWS IoT shadow
endpoint via MQTT over WebSocket with SigV4 auth.

New module mqtt_client.py:
- ExoMqttClient wrapping awsiotsdk with CRT-to-asyncio thread bridge
- Connect, subscribe to shadow topics, publish desired state
- Automatic reconnection with re-subscribe on resume
- 15-minute heartbeat (shadow/get) to keep data fresh during quiet periods
- Reconnect failure detection triggers credential refresh

Changes to api.py:
- _full_login and _refresh_token extract AWS credentials from response
- Coordinator fed by MQTT push via async_set_updated_data()
- MQTT-first startup: initial data loaded via MQTT shadow/get, avoiding
  REST data fetch and the 429 rate limits it triggers
- REST polling relaxed to 1-hour fallback (resets on each MQTT push)
- Writes route through MQTT publish when connected, REST POST as fallback
- Credential refresh scheduled as background task before expiry
- Reconnect-failure handler refreshes credentials and reconnects
- Proper cleanup in async_unload_entry (MQTT disconnect, task cancellation)

Other changes:
- manifest.json: add awsiotsdk dependency, iot_class changed to cloud_push
- number.py: fix _attr_step -> _attr_native_step (fixes pH validation bug)
- __init__.py: call cleanup_entry on unload

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- scripts/dev-setup.py: auto-onboards HA, configures debug logging,
  adds exo_pool integration via config flow using .env credentials
- docker-compose.dev.yml: isolated dev HA instance on port 8125
- Makefile: dev, stop, logs, restart, test targets
- README: add What's New entry for MQTT, update Features/Limitations,
  add Development section with quick-start guide
- .gitignore: exclude .env and .dev-token

Usage: create .env with EXO_EMAIL/EXO_PASSWORD, then make dev

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
37 tests covering:
- ExoMqttClient: connect, disconnect, subscribe, publish, reconnect,
  heartbeat lifecycle, reconnect failure callback, credential refresh
- Shadow data contract: real device data verified against all entity
  read paths (sensor, binary_sensor, switch, number, climate)
- Write payload shapes: pool, heating, schedule writes
- Inverse validation: update/accepted and update/delta do NOT feed
  coordinator (only update/documents and get/accepted do)
- Thread bridge: CRT callbacks are bridged to HA event loop

Tests are isolated from Home Assistant via conftest.py module stubbing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@benjycov

Copy link
Copy Markdown
Owner

Wow amazing work!!! Thank you so much... I'm going to test against my own Exo before merging in, will do this now!

@abrainwood

Copy link
Copy Markdown
Contributor Author

No probs! Thanks for all the work you've put into the integration - it's excellent!

@benjycov
benjycov merged commit 45ed0f2 into benjycov:main Apr 27, 2026
@benjycov

Copy link
Copy Markdown
Owner

Hi Andrew, this is all live now in v0.2.0 - thank you so much for your work on this. I can't believe it never occurred to me to use the MQTT feed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants