farmOS, NodeRed, Home Assistant etc

Using farmOS
1

I’ve setup farmOS and Home Assistant on 2 Raspberry Pi’s, I had initially considered using the same for both, but I think for performance they are better off separated. I’ve been using Home Assistant for over 2 years and I’m building up a Sensor network spread over my fragmented farm, 4 separate farm blocks separated by a few kilometres using LoRa sensor nodes IOT sims in Lora Gateways.

It is only in the last few months that I’ve come across farmOS, and only in the last few weeks that I’ve decided to use it as the software package for the administration of the farm, the more I tinker with setting up farmOS the more I’m finding it a good fit with Home Assistant and wonder how much more I can use Home Assistant along with some of its Addons Like NodeRed in conjunction with extensive farmOS API’s.

Already HA and farmOS are sharing SSL certs for HTTPS,
I’ve also setup HA and Node Red to be able to start and stop the farmOS Docker containers,
NodeRed passes Sensor Data from HA to farmOS,
I can also access farmOS in HA via an iFrame,

Untitled
Untitled1437×757 61.7 KB

So I have full visibility of the farm from a single location, with every sensor, security cameras, calving cameras and now the administration and mapping also. (which is not bad for a little farm)

I’ve been dabbling with farmOS.py for the last few hours and trying to get my head around the farmOS API, but I just had the thought that NodeRed could really simplify some scripting and I’m wondering if many others have used it or are considering using it in this way?

4 Likes
2

I just noticed that @paul121 has done some experimenting with NodeRed in the past Node-RED farmOS library, I presume this is for farmOS 1.x?

I was thinking last night that it would be great to pull calving due dates from service dates within farmOS and display the due dates for cows in a pen with the camera feed.

1 Like
3

hey, yes! That NodeRed experimenting was with farmOS 1.x. The most complicated piece was getting the authentication going, but I got it working with OAuth, and since we have OAuth in 2.0 too, it should all still be possible.

Yes I think this could help! It would still require some knowledge of the API, but instead of writing code you could use the drag + drop UI to connect the inputs + outputs. I forget the details of how the NodeRed “libraries” or “nodes” all work… I think it would make sense to create “nodes” that mirror the different API layers of farmOS.py. For example, an authorize node for OAuth, and then a generalized resource node for making requests to get/create logs, assets, etc. That might help avoid the need to learn the farmOS API - for better or worse :slight_smile:

2 Likes
4

Thanks @paul121, I’m just going through some of it now. the OAuth bit I’ve modified to work for 2.0 and it is returning the access_token and refresh_token, guess I just need to figure out the API now.

I’m not entirely trying to avoid learning the API, but some reusable drag and drop blocks sound appealing too. My IOT setup is already a mix of code and NodeRed, some times code is unavoidable or just better for performance.

2 Likes
5

Oh great!! You just need to pass the access token in an Authorization header and start making requests then! Authorization: Bearer {access_token}

I would start with hard-coding some blocks to fetch logs and assets and later look into abstracting that out to a reusable/configurable block. I don’t think I ever got that far with my code. Would love to take a loot at this on the dev call if you’re able to join!

2 Likes
6

I will try to make that call.

Yes, that works fine once I sorted a few formatting issues. Nothing interesting yet just retrieved the server name and API version. Will delve a little deeper later.

Here is the flow so far FarmOS-Node-Red-Flows/farmos-auth-flow.json at main · Farmer-Eds-Shed/FarmOS-Node-Red-Flows · GitHub

2 Likes
7

Definitely some potential with Node-Red for me, nothing 2 complex yet, but I’ve updated the flow to produce a drop down list with animal names and their sex. Now to figure out some projects to use the data in.

image
image1438×756 19.6 KB

3 Likes
8

@paul121 I won’t make the dev call today, but I’ll try and join one soon.

1 Like
9

Excellent projects! I have been running Home Assistant for some years, initially on Rpi Model 2. However, even though I used USB drives the database (mySql) became corrupted on many occasions due to power cuts etc. Primarily I use HA for monitoring a ground source heat pump via a special interface connected to the Rpi GPIO. I now have HA in a Docker container on a Ubuntu laptop and the Rpi is sending the data via MQTT. This is much more reliable than my previous implementation. I use Farmier for our smallholding’s farmOS implementation but I am considering setting it up on a local system to have a go at development. I just feel guilty and inadequate when I see what work and ideas have gone into it! (And well done to all concerned!)

I look forward to hear how you progress. Looking at your blog you are also far ahead of me on the agricultural engineering front. I have an old MF 362 and really hope that nothing goes wrong with the gearbox! (Sorry this is off topic somewhat!)

3 Likes
10

I had similar issues with running a pi 2 but since moving to the pi 4 for HA it has been rock solid using a Sandisk USB thumb drive, I’m running farmOS on a Pi 3 with a USB attached SSD drive. I must look into a UPS solution for them as power cuts can be a concern alright. I like the low power nature of the Raspberry for these types of server but I guess I need to be careful because if I add many more a small PC may make more sense although some of the Pi’s are being used as IOT gateways on remote farms are solar powered.

So far in my projects I’m just implementing some of the amazing hard work by the developers of opensource projects like farmOS, HA and mySensors.org, I find it some what interesting how well they can potentially work together. My own programming skills are quite limited compared to some around here and I don’t just mean the development team, but I usually get by taking code similar to what I want to achieve and cutting an pasting until it does the job, I usually try to keep things basic enough but sometimes they can grow legs. I’ve only started using Node-Red very recently too, but it seems perfect for passing messages between the various different API’s with limited code.

I could probably discuss tractors/machines or engineering in general all day too but…
Anyone is welcome to strike up those conversations with the comments section of my blog or the linked Facebook account.

1 Like
11

So, I’ve been experimenting with reading and searching logs with Node Red and so far it’s going well, I created a flow that searches observations for the term “Due Date” and updates a Google calendar with the dates. Some thing simple to start with.

The flow is here: FarmOS-Node-Red-Flows/Calving-Calendar-Flow.json at main · Farmer-Eds-Shed/FarmOS-Node-Red-Flows · GitHub

This works OK but will create duplicate entries if the flow is re-run, so I’ve added a flag to the logs and use the flag to decide if entry should be added to Google calendar. So the next step for the flow would be to remove the flag after updating the calendar. (probably make more sense the other way round in the end but for now it will do).

The documentation on updating a log using the API is lacking at the moment, the document for 1.x says to use the PUT method but I’ve come across this Drupal document which says to use PATCH method.
Updating existing resources (PATCH) | JSON:API module | Drupal Wiki guide on Drupal.org.

Would I be right in saying that PATCH method and some thing like this would clear all flags:

{
“data”:{
“type”: “log–observation”,
“id”: “{{LOG-UUID}}”,
“attributes”: {
“flag”: []
}
}
}

2 Likes
12

That’s awesome @Farmer-Ed!

And yes, it looks like you found the correct documentation for v2.

One idea you could try (as an alternative to using flags) is to use the hidden data field that is available on logs. This is a simple field that you can save anything you want (text, JSON, XML, anything) - and it will be stored with the log, but not shown in the UI. For your purposes, perhaps you can store some JSON with a custom property.

Removing all flags might be a bit dangerous - especially if you start using flags for other things in the future and forget that you have NodeRed logic that clears them! :slight_smile:

1 Like
13

Thanks @mstenta.

Point taken on removing all flags, was planning ultimately to add a flag rather than remove, just the flow is setup the opposite way at the moment, but an unused data field is even better. I had considered just appending the notes field with “added to calendar”.

1 Like
14

I’ve updated the flow and it works with no more duplicate calendar entries.
It can be compressed down to fewer nodes, but I’ll leave it as it is on GitHub as a reference that can be easily followed for future flows.

image
image942×150 16 KB

It is dependant on a separate Authentication Flow.

image
image734×130 10.2 KB

Authentication, and button to run flow inside Home Assistant added to a tab with between farmOS and the Google Calendar.

image
image1436×755 25.1 KB

I’ve only added 3 cows calving dates to the database so far, the next step will be to automatically enter the calving dates as service/insemination logs are created (or in batches soon after).

image
image1439×739 23.7 KB

2 Likes
15

Wow so impressive what you can do with Home Assistant and Node Red @Farmer-Ed !! I need to try this sometime! :smile:

1 Like
16

Must admit it’s all fitting together better than I had expected. I’m sure I still have a fair bit to learn about the farmOS API but the visual nature of Node Red along with it’s Debug nodes really help to see what’s going on. And it makes a nice flow diagram to follow if I decide to port a flow to a python script later.

1 Like
17

@mstenta, What is the best practice with regards to access/refresh tokens?

I’m considering a flow to monitor Google Calendar, so If I enter an event in the calendar with the phrase “Cow Serviced {{Cow Name}}” in it will create a " Cow Serviced {{Cow Name}}" Observation and a “Due Calving {{Cow Name}}” Observation.

As this flow will run unattended, should the flow use the Refresh Token with every request regardless of expiry? In practice with this use case I know It shouldn’t matter as the bull is only likely to be observed with a few cows per week, but for higher frequency automated tasks would this be the process?

And I notice that farmOS seems to hold all of the tokens generated, is this indefinitely or should they be pruned in some way after a period?

1 Like
18

Yes, getting a new refresh token with each request is fine.

And I notice that farmOS seems to hold all of the tokens generated, is this indefinitely or should they be pruned in some way after a period?

Oh yes I’m just realizing we need to document how to configure farmOS’s cron to run in the installation instructions…

Basically you need to set up your system’s cron to run farmOS’s (Drupal’s) cron routine. This will handle the automatic cleanup of the tokens in the database.

Here is the upstream Drupal documentation: https://www.drupal.org/node/23714

We need to add a simple bit to our documentation to mention this too…

1 Like
19

So its just run the URL provided in /admin/config/system/cron periodically?
Easy :cowboy_hat_face: image

Which brings up one more question, the link provided starts with http:// instead of https://, it is the same elsewhere like the sensor listeners. I am using https and the links with http:// won’t work until I add the ‘s’ which is fine but I am wondering if I’m missing something, perhaps in settings.php?

2 Likes
20

@Farmer-Ed are you using Nginx as a reverse proxy, like what’s described here: Local HTTPS | farmOS ?

There are two important bits that affect how Drupal automatically determines whether or not to use http or https for links:

In Nginx config:

      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

In settings.php:

$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = [$_SERVER['REMOTE_ADDR']];
$settings['reverse_proxy_trusted_headers'] = \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_ALL;

Check those and see if they help!

1 Like