At present, we have “sensors” that can “listen” and store the data that is sent from them into FarmOS.
I’m interested in also being able to send data the other way, for example clicking on a button to change the state of a valve and activate crop watering for a given field.
With LoRaWAN, this would be relatively simple as all it requires is a call to the appropriate API with the relevant payload, however with “home-brew” or devices connected via WiFi/Bluetooth it could be a bit more difficult.
I think for the second category of devices we probably have a couple of options:
There is a “hub” (raspberry pi or similar?) that polls the FarmOS API every n seconds/minutes and looks for state change. If it finds a state change then it does some analysis on the state and the “sensor”, then sends the appropriate message to activate the end-device
FarmOS publishes state changes to a messaging bus (I’d suggest MQTT here as it seems to be the most used in the IoT space). At this point “something” (either a hub as in (1) or the devices themselves) would connect to the message bus and act based on the messages they receive
Devices are coded so that they connect directly to the FarmOS API and monitor their own state. If the state changes then they act accordingly.
Given the work that we (Mockingbird Consulting) have done so far on IoT, I’d probably recommend (2) over the others as it allows for integration with AWS/Azure/Google Cloud IoT as well if required in future, but I’m open to suggestions on this.
Great summary! I like the idea of allowing different users/sensors to take different approaches depending on their needs. I helped implement a proof of concept using #3 to trigger a relay a while back with some friends.
Another thing we talked about, worth mentioning here: for resiliency, it’s best for devices to be able to operate independently of farmOS, in case they are unable to connect. So for example: if your device is opening and closing vents on a greenhouse in response to temperature changes, all of that logic should live in the device itself. It would be dangerous to rely on logic inside farmOS to make those kinds of decisions, in case network connection is lost. But at the same time, being able to manually override or trigger things from farmOS would be great. I hope we can see some reusable libraries develop to make some of these things standardized on various device platforms.
IFTTT would definitely be useful for a significant number of things, especially automatically exporting the data from FarmOS to other platforms in near real-time.
I’d be careful of calling it “maximum future interoperability”, as IFTTT is a commercial service with a “freemium” tier, and the platforms that they support are generally ones that they deem to be important. That doesn’t mean this is a bad idea, just that we need to be aware that if this was the primary option to connect other services and IFTTT went bust/changed their business model it could have reasonably serious consequences.
As a first step, it might be worth looking at WebHook integration (see https://www.chargebee.com/blog/what-are-webhooks-explained/ for a good guide on what WebHooks are and how they work), as webhooks work with all manner of services including IFTTT, Zapier (an IFTTT Competitor), GitHub, and many others.
Just read this thread the other day… these sorts of things really excite me!! IFTTT Webhook integration might be possible. Looks like it is rather complicated, however, and might be hard to setup with farmOS due to the distributed nature.
An interesting project I came across: Hugin - an opensource version of IFTTT. Seems like the project has become less active, but still seems to have a good following. https://github.com/huginn/huginn
Hey folks, apologies, I’ve taken my eye off the ball on this one, but it’s definitely something that I’m still interested in.
We’ve been busy over the past few months helping to organise The Things Network UK, which was delivered incredibly successfully last week.
Now that the conference is out of the way, I’m coming back to focus on our “demo farm” which will have FarmOS as it’s primary management solution, so I’ll be more active on here in the coming weeks!
@MBConsultingUK That sounds like some cool work you’re doing!! Would love to hear more about your ideas for the demo farm. I’ve been thinking about that & this (bi-directional IoT in farmOS) a lot more lately…
FarmOS publishes state changes to a messaging bus (I’d suggest MQTT here as it seems to be the most used in the IoT space). At this point “something” (either a hub as in (1) or the devices themselves) would connect to the message bus and act based on the messages they receive
Just re-reading your first post, I definitely agree that Option 2 makes the most sense. I’m quite interested in MQTT as a means of passing this data around. If farmOS could push state changes out over MQTT (when a log is created, a harvest is completed) and listen to data from sensors coming over MQTT (temperature, gate open/closed, etc), that would allow for a very robust system!!!
In Schedule a log based on a quantity value@mstenta mentioned the Drupal Rules module. This is an interesting concept that would allow some of this functionality internally within farmOS (create a log once another log is finished, etc) BUT I think this could be done externally, too.
I came here to mention node-red… this could be a neat IoT tool outside of farmOS. Have you heard of it @MBConsultingUK? It has some great ways of creating rules with “Nodes” - various inputs and outputs. The community can submit additional “Nodes”, and can also publish rules as “flows” and share them in a JSON format.
This is WAY cool - in 5 minutes I was able to download node-red and create a webpage that displays subway stations in San Francisco via this flow someone shared In another 5 minutes I created a flow that re-routed the results of a farmOS API request… You could easily create a flow that listens to an MQTT topic and sends the data via HTTP POST to a sensor in farmOS
Yup, I’m well aware of NodeRed, and it’s great for prototyping and setting stuff up if you’re not experienced in writing code, but I’d be reluctant to base a full application around it.
At the moment, I’ve written (and released!) code to do exactly what you describe over at https://github.com/mockingbirdconsulting/FarmOSMQTT - at the moment this is specifically for working with Chirpstack or thethingsnetwork.org, however the base of the code could be easily modified to take data from elsewhere if needed.
The idea is that this code runs on the same server as the database (or at least with access to the database) and then matches the device name to the sensor name in FarmOS and updates as appropriate.
What I’d love to do is to extract the sensor code from FarmOS completely so that we can use a proper time-series database such as InfluxDB or Elasticsearch and just draw the graphs based off the data stored in there - MySQL and Postgresql are great for relational data, but processing thousands of rows of time-series data isn’t really something they’re very good at.
Oh super cool!!! That’s a great example. I’ve been wondering how paho.mqtt might work…looks pretty easy to use.
Yeah, that makes sense on using a time-series database. Make me curious what the best way would be to view that data in farmOS if it is stored elsewhere… and in the Field Kit mobile app… I suppose a farmOS API for the sensor could just pull data from that database.
I’m curious why you’d be hesitant using Node-Red in a larger application? I definitely see the disadvantages in the abstraction from writing code, but with a slightly improved interface, I think it could be useful for and end user, maybe? I guess I’m kind of thinking at the level of IFTTT for farmOS haha. I agree sensor data wouldn’t be best-managed this way, but what about defining events “if sensor reading is > 10 -> do this” ?
@dornawcox has done a bunch of experimenting with Node Red! He and I spent a day at his house hacking on things. I made the farmOS sensor endpoint capable of receiving GET requests, and he used Node Red to pipe that through a series of logic on his Raspberry Pi. It was remarkably easy! One idea we had was to share “nodes” (which can be exported as JSON) in an open library to make it easier for DIY folks to hack things together.
Of course, there is also a need for a more complete pre-made solution like @MBConsultingUK describes. We can have both!
I’ve nothing against NodeRed as such, I just have concerns about it’s scalability.
If you’re dealing with a few dozen messages an hour, then it’s perfect for drag-and-drop “glue” to link components together, but once you start to get above a few hundred messages a minute then I’d be concerned about the performance.
Admittedly, I’ve not tried it in a while and it may have improved, but both the IBM-hosted version and a self-hosted version felt a lot slower compared with a 100-line python script when I used them.
As @mstenta says though, there’s no reason we can’t have both, and if we were to “standardise” on something like MQTT as the protocol, then it makes it easy to swap either option in as necessary.
Good points! It seems like it would have to be slowing things down since it relays messages, but maybe it would still be acceptable.
Yea I agree on both! That would open the door to many options. Can we start brainstorming more about how to get an MQTT client inside farmOS? I just created a separate topic for node-red, could you start a topic for MQTT @MBConsultingUK with some initial ideas? You seem to be the most experienced working with it
