Nymea drops magic when deleting device

Development
1

Whenever I delete a device (a ZigBee device, in my case), nymea deletes all magic which refers to that device. I certainly understand the reasoning behind that design decision. However, deleting devices and adding the same devices again to the network has become something of a daily routine for me, alas. Other activities normal to the maintenance will also provoke the same behavior, such as re-flashing the ZigBee coordinator.

Rewriting all that magic whenever I had to renew the binding of a device is becoming quite a chore. Also, it requires me to do keep book about which magic mentions which devices, and I have to keep screenshots of each magic so that I can rewrite it from scratch whenever I lost it.

I think that instead of simply and silently discarding all magic that mentions a deleted device, there should be provisions to conserve the magic with enough information to restore it once the devices needed are back in operation.

There are several ways to do that.

One very simple way would be to just let the magic stand and to handle missing devices at runtime for each magic. That’s not very elegant and might be a bit error prone, but it’s simple.

Another way would be to replace each mention of a deleted device with an object of a class which represents deleted devices. The proxy object presumably would store at least the unique ID of the deleted object, preferably also its type and the properties and values associated with the operation.

The system then could search all magic or all instances of proxy devices whenever a device is freshly bound and restore the magic to its state prior to deleting the device.

A bit more involved: the GUI which supports the building of the magic could prompt the user for a replacement for the dropped device; but this is certainly much more complex and the specifications need some clarification before implementation. The first variant certainly would cover most cases when I have to delete devices and bind them again.

While we’re at it: I would much appreciate if there was another way of documenting my magic other than taking a screenshot of the builder GUI. A text representation of the same content would already be much more useful. It also would be handy for discussing magic here in the forum. I’r rather not use an external OCR software to decode what’s shown in the GUI.

2

This is a very valid point.

Now, at this moment, I would not question the general behavior of nymea, that removing a thing also removes the associated configuration (such as rules, tags etc). Normally, removing a thing is not something that happens accidentally. Having to clean up rules etc manually afterwards would IMO make for a worse user experience in the case where a thing is intentionally removed.

In earlier days, nymea had a mode where it would not remove a rule if a thing went missing, but instead update a rule and only remove conditions/actions for that particular thing. This, however, did not prove a useful thing in the end as it would nearly always leave broken rules in the system, which could even cause very bad system behavior. So somewhen last year that behavior was removed. I still believe that was the right decision in general.

However, as you’re rightly pointing out, this very much becomes an issue in the context of ZigBee, where things are automatically created and removed, reflecting the actual ZigBee network nodes. While in theory a node should never leave the network unless it’s removed for good, turns out that in practice it very much is an actual use case, especially when dealing with ZigBee network instabilities.

So, my proposed solution for this problem would be to not automatically remove “Things” when a ZigBee node disappears, but instead leave the Thing in the system, keeping it marked as disconnected. This kinda matches your proposal about the proxy object, given the “Thing” in nymea basically just is a proxy object for a ZigBee node, so…
That means, adding a ZigBee device will behave as it currently does. Removing a ZigBee node (either via ZigBee settings or triggering it from the ZigBee device itself) will not remove the thing any more but mark it as disconnected. In order to remove the Thing, it needs to be manually removed from the Thing settings now. If a Node comes back and an existing thing for it is already in the system, it will be associated again. Removing a thing from the thing configuration, will still also remove the node from the ZigBee network in one go.

I have altered the code to behave like this in this pull request: Don't destroy things when removing ZigBee nodes from the network by mzanetti · Pull Request #72 · nymea/nymea-plugins-zigbee · GitHub
As this changes the assumption that a Thing will always have a valid associated ZigBee node, the change, while not overly complicated, also isn’t exactly trivial so we’ll have to do some testing of the patch now to see how well it “flies”. The change is currently building in the experimental repository and should be available for testing there within a couple of hours.

3

I’m not sure it’s really useful that way.
The ‘vanished’ thing will appear in one of three functions in a magic:

  1. as the recipient of an action
  2. as the value for an action
  3. as the value for a condition or event
    Case 1 is trivial. If the thing does not exist any more it does not matter when the action command is not sent.
    Cases 2 and 3 are not trivial, because we can’t say beforehand whether ‘none’ (or other notation for an undefined value) will be acceptable as a parameter for a condition or action, and if the magic should fire or not.
    After some more deliberation I think the affected magic should be marked as ‘inactive’ as well. Perhaps we need a third state besides ‘active’ and ‘inactive’, such as ‘disabled’ ? That state could be changed to ‘active’ only if all things mentioned in the magic actually exist.
    This would also reduce the risk of any side effects which might occur by firing a magic with missing parameters.