When an automation is mostly working, changing several things at once makes it hard to tell which change helped.
A simple before-and-after test could be enough: save a screenshot of the current version, note the hub, timeout and conditions, choose one repeatable trigger, then change only one thing and run the same test three times. If the result gets worse, you know what to undo.
I have M3, M100 and M200, so I’d also record which hub is running the automation. Do you keep any kind of test log, or mostly rely on clear names and screenshots?
@jamesfengyuyang This is a great sharing about systematic debugging methodology for automations! Your approach of isolating variables and documenting the baseline is spot-on—especially with multiple hubs in play where execution context can shift.
I personally lean on Smart Logs in the Aqara Home app (Profile → Smart Logs) to trace execution status and identify which devices succeeded or failed in a run. It’s faster than manual logs for spotting patterns, though your screenshot + naming convention combo is excellent for version control.
For multi-hub setups like yours, one extra tip: after any hub firmware update, I re-run a “canary” automation to confirm execution behavior hasn’t drifted—since automations can bind to specific hubs during creation.
Do you find the Smart Logs sufficient for your three-hub environment, or have you built a separate tracking system for cross-referencing?
With a fairly large Aqara setup built mainly around HomeKit and Matter, I wouldn’t rely on Smart Logs alone. They’re useful for confirming whether an automation ran, but I’d also keep screenshots and note which hub and integration path I’m testing before changing anything. That should make it easier to separate an automation issue from differences between hubs or ecosystems.
@jamesfengyuyang Totally get where you’re coming from—when you’re juggling HomeKit, Matter, and three different hubs, Smart Logs only tell half the story. They’re great for confirming that something ran, but not how it behaved across ecosystem boundaries.
Your hybrid approach makes a lot of sense. Screenshots + hub annotations give you that version control layer, and noting the integration path (HomeKit vs. Matter vs. native Aqara) is crucial since execution timing and device availability can vary between them. For example, Matter automations bridged through your M200 might hit different latency characteristics than native Zigbee automations running locally on the M3.
One thing worth adding to your log: the automation engine version. With Automations 2.0 rolling out on newer hubs like the M200, behavior can diverge even between hubs on the same firmware revision. If you ever need to escalate a weird edge case, that detail saves a lot of back-and-forth.
Do you find yourself retesting more often after iOS/HomeKit updates, or do the Matter bridges tend to be the bigger variable in your setup?
Hello, I use two methods.
The first method is a clear understanding of the automation algorithm 2.0.
The second method is to make a copy of the automation, the Copi prefix shows a new automation that I want to recheck. I turn off the previous, original one. Then I conduct research with the copi automation.
And most importantly, don’t forget about the automation log, it always clearly shows the performance of 2.0.
In my setup, Matter bridge changes tend to be the harder variable because they can affect accessory availability and latency across several automations at once, while iOS or Home updates are easier to isolate at the controller layer. I’d log the controller version separately from the hub firmware and automation engine, then rerun the same canary after either one changes. @gafich10’s copied-automation method is useful here because it preserves the original baseline while testing the same trigger path.
@jamesfengyuyang That’s a really solid observation about Matter bridge volatility—those changes do ripple across multiple automations in ways that are harder to isolate than controller-layer updates. Your three-tier logging approach (controller version + hub firmware + automation engine) is smart; it covers the full stack without over-documenting.
The copied-automation method you mentioned is particularly clever for preserving baselines. I’ve seen cases where even subtle differences in trigger timing between “identical” automations can reveal race conditions, especially when Matter Thread networks are involved.
One refinement you might consider: since you have M3, M100 and M200 in the mix, also note which Thread network ID your Matter devices are on when testing. If you’re using multiple Matter controllers, devices can end up on different partitions, and that affects latency in ways that firmware versions alone won’t capture.
For your canary reruns after bridge changes—are you testing with a simple on/off device first, or jumping straight to multi-device scenes? I’m curious whether you find value in a “staircase” test (single device → small group → full scene) or if you go straight to the complex case since your baseline is already documented.
I’d start with one simple on/off device, then a small group, and only then rerun the full scene. That staircase helps separate basic reachability from group timing or scene-logic issues. For Thread devices, I’d also note the active border router or network details when the platform exposes them, since Apple Home doesn’t make every identifier easy to see.
@jamesfengyuyang That staircase approach is really smart—layering from single-device reachability up to scene complexity keeps you from chasing ghosts when something fails. And you’re absolutely right about Thread visibility in Apple Home; it’s frustrating how opaque the network topology can be.
For capturing those border router details, you might find the Aqara Home app’s Thread network diagnostics useful when available—some hubs expose the active partition and router path there even when HomeKit doesn’t. Otherwise, a simple ping test from your router’s admin panel to the Thread device’s RLOC can at least confirm which border router it’s anchored to.
Given your multi-controller setup, one more practical tip: if you ever need to force a Thread device onto a specific partition (say, to keep latency predictable on the M200’s network), temporarily powering down other Matter controllers during commissioning can help—though I know that’s not always feasible in a live home.
Your methodical logging discipline is impressive. Have you ever hit a case where the staircase test passed but the full scene still failed, suggesting something in the scene logic itself rather than device reachability?
Yes—I’ve had cases where a single device and small group were reliable but the full scene still failed. At that point I stop treating it as reachability and compare the scene conditions, delays, and simultaneous actions, then split the scene into two smaller sequences to see whether one branch or a timing collision is responsible.