farmOS, NodeRed, Home Assistant etc

I will try to make that call.

Yes, that works fine once I sorted a few formatting issues. Nothing interesting yet just retrieved the server name and API version. Will delve a little deeper later.

Here is the flow so far FarmOS-Node-Red-Flows/farmos-auth-flow.json at main · Farmer-Eds-Shed/FarmOS-Node-Red-Flows · GitHub

2 Likes

Definitely some potential with Node-Red for me, nothing 2 complex yet, but I’ve updated the flow to produce a drop down list with animal names and their sex. Now to figure out some projects to use the data in.

3 Likes

@paul121 I won’t make the dev call today, but I’ll try and join one soon.

1 Like

Excellent projects! I have been running Home Assistant for some years, initially on Rpi Model 2. However, even though I used USB drives the database (mySql) became corrupted on many occasions due to power cuts etc. Primarily I use HA for monitoring a ground source heat pump via a special interface connected to the Rpi GPIO. I now have HA in a Docker container on a Ubuntu laptop and the Rpi is sending the data via MQTT. This is much more reliable than my previous implementation. I use Farmier for our smallholding’s farmOS implementation but I am considering setting it up on a local system to have a go at development. I just feel guilty and inadequate when I see what work and ideas have gone into it! (And well done to all concerned!)

I look forward to hear how you progress. Looking at your blog you are also far ahead of me on the agricultural engineering front. I have an old MF 362 and really hope that nothing goes wrong with the gearbox! (Sorry this is off topic somewhat!)

3 Likes

I had similar issues with running a pi 2 but since moving to the pi 4 for HA it has been rock solid using a Sandisk USB thumb drive, I’m running farmOS on a Pi 3 with a USB attached SSD drive. I must look into a UPS solution for them as power cuts can be a concern alright. I like the low power nature of the Raspberry for these types of server but I guess I need to be careful because if I add many more a small PC may make more sense although some of the Pi’s are being used as IOT gateways on remote farms are solar powered.

So far in my projects I’m just implementing some of the amazing hard work by the developers of opensource projects like farmOS, HA and mySensors.org, I find it some what interesting how well they can potentially work together. My own programming skills are quite limited compared to some around here and I don’t just mean the development team, but I usually get by taking code similar to what I want to achieve and cutting an pasting until it does the job, I usually try to keep things basic enough but sometimes they can grow legs. I’ve only started using Node-Red very recently too, but it seems perfect for passing messages between the various different API’s with limited code.

I could probably discuss tractors/machines or engineering in general all day too but…
Anyone is welcome to strike up those conversations with the comments section of my blog or the linked Facebook account.

1 Like

So, I’ve been experimenting with reading and searching logs with Node Red and so far it’s going well, I created a flow that searches observations for the term “Due Date” and updates a Google calendar with the dates. Some thing simple to start with.

The flow is here: FarmOS-Node-Red-Flows/Calving-Calendar-Flow.json at main · Farmer-Eds-Shed/FarmOS-Node-Red-Flows · GitHub

This works OK but will create duplicate entries if the flow is re-run, so I’ve added a flag to the logs and use the flag to decide if entry should be added to Google calendar. So the next step for the flow would be to remove the flag after updating the calendar. (probably make more sense the other way round in the end but for now it will do).

The documentation on updating a log using the API is lacking at the moment, the document for 1.x says to use the PUT method but I’ve come across this Drupal document which says to use PATCH method.
Updating existing resources (PATCH) | JSON:API module | Drupal Wiki guide on Drupal.org.

Would I be right in saying that PATCH method and some thing like this would clear all flags:

{
“data”:{
“type”: “log–observation”,
“id”: “{{LOG-UUID}}”,
“attributes”: {
“flag”: []
}
}
}

2 Likes

That’s awesome @Farmer-Ed!

And yes, it looks like you found the correct documentation for v2.

One idea you could try (as an alternative to using flags) is to use the hidden data field that is available on logs. This is a simple field that you can save anything you want (text, JSON, XML, anything) - and it will be stored with the log, but not shown in the UI. For your purposes, perhaps you can store some JSON with a custom property.

Removing all flags might be a bit dangerous - especially if you start using flags for other things in the future and forget that you have NodeRed logic that clears them! :slight_smile:

1 Like

Thanks @mstenta.

Point taken on removing all flags, was planning ultimately to add a flag rather than remove, just the flow is setup the opposite way at the moment, but an unused data field is even better. I had considered just appending the notes field with “added to calendar”.

1 Like

I’ve updated the flow and it works with no more duplicate calendar entries.
It can be compressed down to fewer nodes, but I’ll leave it as it is on GitHub as a reference that can be easily followed for future flows.

It is dependant on a separate Authentication Flow.

Authentication, and button to run flow inside Home Assistant added to a tab with between farmOS and the Google Calendar.

I’ve only added 3 cows calving dates to the database so far, the next step will be to automatically enter the calving dates as service/insemination logs are created (or in batches soon after).

2 Likes

Wow so impressive what you can do with Home Assistant and Node Red @Farmer-Ed !! I need to try this sometime! :smile:

1 Like

Must admit it’s all fitting together better than I had expected. I’m sure I still have a fair bit to learn about the farmOS API but the visual nature of Node Red along with it’s Debug nodes really help to see what’s going on. And it makes a nice flow diagram to follow if I decide to port a flow to a python script later.

1 Like

@mstenta, What is the best practice with regards to access/refresh tokens?

I’m considering a flow to monitor Google Calendar, so If I enter an event in the calendar with the phrase “Cow Serviced {{Cow Name}}” in it will create a " Cow Serviced {{Cow Name}}" Observation and a “Due Calving {{Cow Name}}” Observation.

As this flow will run unattended, should the flow use the Refresh Token with every request regardless of expiry? In practice with this use case I know It shouldn’t matter as the bull is only likely to be observed with a few cows per week, but for higher frequency automated tasks would this be the process?

And I notice that farmOS seems to hold all of the tokens generated, is this indefinitely or should they be pruned in some way after a period?

1 Like

Yes, getting a new refresh token with each request is fine.

And I notice that farmOS seems to hold all of the tokens generated, is this indefinitely or should they be pruned in some way after a period?

Oh yes I’m just realizing we need to document how to configure farmOS’s cron to run in the installation instructions

Basically you need to set up your system’s cron to run farmOS’s (Drupal’s) cron routine. This will handle the automatic cleanup of the tokens in the database.

Here is the upstream Drupal documentation: Configuring cron jobs using the cron command | Setting up cron | Drupal Wiki guide on Drupal.org

We need to add a simple bit to our documentation to mention this too…

1 Like

So its just run the URL provided in /admin/config/system/cron periodically?
Easy :cowboy_hat_face: image

Which brings up one more question, the link provided starts with http:// instead of https://, it is the same elsewhere like the sensor listeners. I am using https and the links with http:// won’t work until I add the ‘s’ which is fine but I am wondering if I’m missing something, perhaps in settings.php?

2 Likes

@Farmer-Ed are you using Nginx as a reverse proxy, like what’s described here: Local HTTPS | farmOS ?

There are two important bits that affect how Drupal automatically determines whether or not to use http or https for links:

In Nginx config:

      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-Host $http_host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;

In settings.php:

$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = [$_SERVER['REMOTE_ADDR']];
$settings['reverse_proxy_trusted_headers'] = \Symfony\Component\HttpFoundation\Request::HEADER_X_FORWARDED_ALL;

Check those and see if they help!

1 Like

Hmm… As far as I can see I have those options configured.
It’s a fairly minor issue as Https requests work and the http requests are refused. I’ll take another look at my settings later, maybe some other option in Nginx is overriding.

I’m working on a new Node Red flow. FarmOS-Node-Red-Flows/Cow-Service-Calendar.json at main · Farmer-Eds-Shed/FarmOS-Node-Red-Flows · GitHub

It Reads Google Calendar events starting with 'Cow Serviced {{Cow Name}}'

Then create 2 new farmOS logs 'Cow Serviced {{Cow Name}}' and 'Due Calving {{Cow Name}}'
Logs have the 'Animal Asset' attached and are marked with a 'Review Flag'
It also adds a 'Due Calving {{Cow Name}}' back to Google Calendar.
Notes added to calendar description are added to log notes.

It’s a bit rough, I’m sure it needs some refinement and some additional checks.

3 Likes

@Farmer-Ed Maybe you need to configure the trusted host settings? Trusted Host settings | Installing Drupal | Drupal Wiki guide on Drupal.org

Really cool to see these node-red scripts coming together!

2 Likes

Hi

@Farmer-Ed , I’m trying to follow your footsteps…

I’m working on this flow of yours: (Calving calendar)

I’ve cut the flow after the GET-node, just to see if I get that far, and to get familiar with the nodes.
I have authenticated, and the FarmOS url is sent to to the node.

But the GET-node produces these 2 errors:
Error: getaddrinfo EAI_AGAIN get get:80

Warning: msg properties can no longer override set node properties. See bit.ly/nr-override-msg-props

Object sent to GET
{"_msgid":"a975ad5e.8e4c3","payload":1644440749801,"topic":"","headers":{"Authorization":"Bearer __Stripped the token___ ..."},"url":"https://__farm__.farmos.net/api/log/observation","method":"GET"}

Did you encounter anything like this?

1 Like

Yes, it looks like you have sent the config for the http node in the message sent to the node and also set it in the node itself. It has to be one or the other but not both

For example, if you send the URL and/or method in the payload then don’t set it in the http node text box too.

1 Like

You were right @Farmer-Ed .
The errors disappeared. Got a bunch of HTML-code.
I’ll investigate further…

2 Likes