External notifications, webhooks, MQTT etc

[Farmer-Ed]

@mstenta

Those may be a bug with the Webhook module itself. It seems to be assuming that a certain route is defined by Log, which isn’t… ? Not sure…

Quite possibly, after a good bit of searching yesterday, there does not seem to be many options for this sort of function. I did however discover that Drupal Core includes a http client based on Guzzle, so its probably time I brush up on a little PHP anyway. Guzzle HTTP client library added to replace drupal_http_request() | Drupal.org

[mstenta]

@Farmer-Ed

its probably time I brush up on a little PHP anyway.

Yea!!

You don’t need any special modules to trigger simple code to run when entities are added/updated/deleted… You can do it easily in a simple custom module. Drupal’s hook system makes it super easy!

Check out these hooks:

hook_ENTITY_TYPE_insert()

hook_ENTITY_TYPE_update()

hook_ENTITY_TYPE_delete()

The way hooks work is… you just create a file called [mymodule].module (alongside [mymodule].info.yml) and add functions to it. These functions are named in a very specific way so that Drupal will automatically run them when necessary.

For example, if I wanted to run some code whenever a log is updated, in a module called mymodule, it would look like this:

/**
 * Implements hook_ENTITY_TYPE_update().
 */
function mymodule_log_update(EntityInterface $entity) {
  // My PHP code...
  // This will run whenever a log is updated.
  // $entity is the log.
}

The naming of the function is critical for this to work. In this case, you take the name of the hook (eg: hook_ENTITY_TYPE_update()), replace hook with your module name (mymodule), and ENTITY_TYPE with the entity type (log). Result: mymodule_log_update().

Whenever a log entity is updated, Drupal basically says “hey all you modules - anyone got a function named [modulename]_log_update()? yea? alright let’s go!” and it runs it.

That’s the quick intro - but hopefully that’s enough to get you going!

[mstenta]

Worth noting: hooks are the historical way to do these kinds of things in Drupal… but they are not the only way. With D8+, a lot of things are moving towards using event subscribers instead of hooks. Event subscribers are OO (object oriented), whereas hooks are more traditional procedural functions.

FWIW farmOS core tries to use event subscribers wherever possible because everyone is trying to move towards more OO in Drupal code. But for custom modules, hooks are the most dead-simple way to do it, and I’d probably recommend starting there.

[Farmer-Ed]

@mstenta

But for custom modules, hooks are the most dead-simple way to do it, and I’d probably recommend starting there

Just spent the last few hours trying to figure out event subscribers
Heads a bit melted so maybe I’ll have a look at hooks too.

But on a positive note this part works to send messages to an external server.

	//Creating a httpClient Object.
$client = \Drupal::httpClient();
//Some Data
$json_data = ['log'=>$log_name,'meat_withdrawal'=>$max_meat_withdrawal];
//Sending POST Request with $json_data to example.com
$request = $client->post('http://192.168.1.7:1880/farmos/webhook', [ 'json' => [ $json_data, ]
]);
//Getting Response after JSON Decode.
$response = json_decode($request->getBody());

[Farmer-Ed]

@mstenta

You can do it easily in a simple custom module. Drupal’s hook system makes it super easy!

Really is! Obviously needs a little more to catch exceptions, authentication and filter just the log types that I want. Maybe eventually a UI to configure these options too.

<?php

/**
 * @file
 * Contains farm_events_http.module.
 */
use Drupal\Core\Entity\EntityInterface;
/**
 * Implements hook_ENTITY_TYPE_update().
 */
function farm_events_http_log_update(EntityInterface $entity) {
	
$log = $entity->toArray();
//Create httpClient.
$client = \Drupal::httpClient();
//Sending POST Request with $json_data to external server
$request = $client->post('http://192.168.1.7:1880/farmos/webhook', [ 'json' => $log]);
//Getting Response after JSON Decode.
$response = json_decode($request->getBody());

Really opens options for Node-Red automations.

[Farmer-Ed]

@mstenta

@Farmer-Ed I’d be happy to help you through the process of a contrib module implementation if you’re interested in going down that route! Similar to the process @jgOhYeah and I went through in Is it possible to use custom fields and filtering in asset views and filter by a range of birth dates?

Happy to do that here or in a separate dedicated thread, if that would make more sense (keep this one general)?

I think this thread may be a more appropriate home for that conversation.

@mstenta

First thing I would do is point to some of the other examples of entity event subscribers…

I have been looking at a few already and borrowed @paul121’s farm REI example for my withdrawal module, although I’ve used the hooks method so far in this project.

I’m currently trying to figure out how to add a configuration page and have been looking at form API examples and adding menu items.

[mstenta]

@Farmer-Ed

currently trying to figure out how to add a configuration page and have been looking at form API examples and adding menu items.

Great! If this will be a general notifications settings form, then it probably makes sense to add it as a tab under the /farm/settings path (alongside “Farm Info”, “Modules”, etc).

The Quantity settings might be a good example to work from since it’s simple.

There are a couple of pieces to this that all work together:

1. The QuantitySettingsForm class, which extends from ConfigFormBase: farmOS/QuantitySettingsForm.php at 2.x · farmOS/farmOS · GitHub
2. The quantity.routing.yml file that defines the /farm/settings/quantity path, and points it at the QuantitySettingsForm: farmOS/quantity.routing.yml at 2.x · farmOS/farmOS · GitHub
3. The quantity.links.task.yml file which adds the “Quantity” tab itself (pointed to the route defined in quantity.routing.yml), under the farm_settings.settings_page base route.

The ConfigFormBase class is basically just a slightly more specialized extension of the general FormBase class, specifically for config forms like this. It adds the config.factory service via dependency injection so you can use it in your submit() method to save Drupal config settings.

https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Form!ConfigFormBase.php/class/ConfigFormBase/9.3.x

Here is where the quantity form saves the values: farmOS/QuantitySettingsForm.php at 76a6fdf56326b4412623f106d9500119cdb38b9e · farmOS/farmOS · GitHub

Optionally (but good practice), the module can provide two other YML files for the configuration settings it provides specifically:

1. config/install/quantity.settings.yml sets default values for the configuration settings when the module gets installed: farmOS/quantity.settings.yml at 2.x · farmOS/farmOS · GitHub
2. config/schema/quantity.schema.yml describes what type of data each setting is: farmOS/quantity.schema.yml at 2.x · farmOS/farmOS · GitHub (only look at the first bit in that file quantity.settings, ignore the quantity.type.* section - that’s for something else).

[Farmer-Ed]

@mstenta

Great! If this will be a general notifications settings form, then it probably makes sense to add it as a tab under the /farm/settings path (alongside “Farm Info”, “Modules”, etc).

Yes that would make sense alright.

And thanks again, that one post is probably worth days of Googling for me.

[Farmer-Ed]

I made a start at this.

Still plenty to work out and do but at least now I don’t need to hard code settings anymore. The basic mechanics are working for using ntfy.sh and Node-Red as servers I may add some more in future.

I’ve created a repo here:

GitHub - Farmer-Eds-Shed/Farm-Notifications

[mstenta]

This is great @Farmer-Ed! So cool to see progress!

[Symbioquine]

That looks like a really interesting start!

I wonder if the Node-Red/WebHook/MQTT case might be better described by the term “events” instead of “notifications”. It feels like the “events” case probably belongs in its own module compared with the Ntfy “notifications” case.

Events: Tell some other server that some data has changed so it can run its own code - possibly making farmOS API calls to get more data and/or storing/maintaining its own state somewhere else.

• { "eventType": "entityCreated", "entity": { "type": "log--activity", "id": "be876abf-bbc5-4d10-8822-5be4ce58ce2b", "attributes": { "drupal_internal__id": 1, "drupal_internal__revision_id": 1, "langcode": "en", "revision_created": "2022-05-19T13:41:27+00:00", "revision_log_message": null, "name": "Mow back 40", "timestamp": "2022-05-20T13:41:01+00:00", "status": "pending", "created": "2022-05-19T13:41:27+00:00", "changed": "2022-05-19T13:41:27+00:00", "default_langcode": true, "revision_translation_affected": true, "data": null, "notes": null, "flag": [], "geometry": null, "is_movement": false } } }

• { "eventType": "entityUpdated", "entity": { "type": "log--activity", "id": "be876abf-bbc5-4d10-8822-5be4ce58ce2b", "attributes": { "drupal_internal__id": 1, "drupal_internal__revision_id": 1, "langcode": "en", "revision_created": "2022-05-19T13:41:27+00:00", "revision_log_message": null, "name": "Mow back 40", "timestamp": "2022-05-24T15:00:00+00:00", "status": "pending", "created": "2022-05-19T13:41:27+00:00", "changed": "2022-05-19T17:22:15+00:00", "default_langcode": true, "revision_translation_affected": true, "data": null, "notes": null, "flag": [], "geometry": null, "is_movement": false } } }

Notifications: Tell/remind a user of farmOS about something on the farmOS server. Presumably, farmOS should make the decision about which user to tell, when to tell them, and what to say.

• 2022-05-19T13:41 "Example Farm @Alice - New activity log assigned: May 20th - Mow back 40"
• 2022-05-19T17:22 "Example Farm @Alice - Updated activity log: May 24th - Mow back 40"
• 2022-05-23T15:00 "Example Farm @Alice - Upcoming activity log: May 24th - Mow back 40"
• 2022-05-24T16:00 "Example Farm @Alice - Overdue activity log: May 24th - Mow back 40"

[Farmer-Ed]

This is probably true…
I actually started it off with the module name http_events

[mstenta]

Good feedback @Symbioquine! I would agree that “events” might be better for general event propagation to external APIs (which then handle the “notification” aspect specifically).

This is always one of the challenges with Drupal module development… the balance between general and specific. Sometimes you need multiple modules to achieve a specific goal - building up from general to specific!

Aside from those bigger points, I skimmed the code and have a few specific suggestions, if you are open to them @Farmer-Ed…

First, I notice that it implements hook_log_update() - which is fired when logs are updated. You probably also want to implement hook_log_insert() (fires when logs are created), and hook_log_delete() (fires when logs are deleted).

Minor fixes:

• Change farm notifications settings to farm_notifications_settings_form so that this form can be altered by other modules properly. Farm-Notifications/FarmNotificationsSettingsForm.php at d14c522328eed02462d2fb38011047eea3621878 · Farmer-Eds-Shed/Farm-Notifications · GitHub
• Remove dependency on farm_medical and replace with dependency on log - since this is not specific to Medical logs. Farm-Notifications/farm_notifications.info.yml at d14c522328eed02462d2fb38011047eea3621878 · Farmer-Eds-Shed/Farm-Notifications · GitHub

Nit picks (I don’t think they make a difference functionally - just “best practice” / “coding standards” stuff):

• Rename from Farm-Notifications to farm_notifications for consistency with other modules.
• Rename Config directory to config (this may make a difference on certain platforms that are case-sensitive, I’m not sure)
• Use $entity->label() instead of $entity->name->value Farm-Notifications/farm_notifications.module at d14c522328eed02462d2fb38011047eea3621878 · Farmer-Eds-Shed/Farm-Notifications · GitHub

PHP CodeSniffer is a great way to find and fix coding standards issues! If you are using the dev environment already then it’s pretty easy to use: Coding standards | farmOS

Hope that helps! Keep up the good work @Farmer-Ed !

[Farmer-Ed]

Thanks for taking the time for that detailed critique @mstenta.
I don’t even know where the farm_medical dependency came from, obviously a cut and paste from another project

I will take those recommendations on board.

on the hook_log_update() or hook_log_insert() I was thinking about this and unsure whether or not an update to a log also justified a notification (or event), had contemplated making it an option. Just glad at the moment it fires as expected

[Farmer-Ed]

@mstenta I was looking at the option of plugins that you suggested on the call and love the idea, but suspect it could be beyond my current abilities, have you an example of its implementation? I was thinking of adding additional server types later including Google calendar (although I’ve left that out for now due to the authentication complexity of the API and the fact that I have it working well in Node-Red anyway, but perhaps it may be more worthy of its own module and settings panel in the future).

I want to add a Boolean switch to logs to be able to turn on/off if they should be notifiable events. The switch appears in the logs but its value seems to be always null

/**
 * Implements hook_entity_base_field_info().
 * NOTE: Replace 'mymodule' with the module name.
 */
function farm_notifications_entity_base_field_info(EntityTypeInterface $entity_type) {
  $fields = [];

  // 'log' specifies the entity type to apply to.
  if ($entity_type->id() == 'log') {
    // Options for the new field. See Field options below.
    $options = [
      'type' => 'boolean',
      'label' => t('Notify'),
      'description' => t('Send Notifications for this log.'),
	  'settings' => [
        'format' => 'default',
        'format_custom_false' => '',
        'format_custom_true' => '',
      ],
      'weight' => [
        'form' => 10,
        'view' => 10,
      ],
    ];
    // NOTE: Replace 'myfield' with the internal name of the field.
    $fields['farm_notification_notify'] = \Drupal::service('farm_field.factory')->baseFieldDefinition($options);
  }

  return $fields;
}

I’ve tried various combinations of changing the format options under settings but don’t think that’s the issue. Perhaps an initial default value is needed for booleans?

Edit: setting an initial value does seem to fix it, I’ll do a bit more testing later though.

Another Edit:
Maybe Radio style options would be better to trigger events based on status?

farmfield.factory could do with a little more documentation.

[mstenta]

@Farmer-Ed

I was looking at the option of plugins that you suggested on the call and love the idea, but suspect it could be beyond my current abilities, have you an example of its implementation?

I wouldn’t worry too much about this right now. It can be broken out into plugins (or some other mechanism) pretty easily in the future. I’d probably just open a todo issue as a placeholder to start thinking about it. This submodule of the “Examples for Developers” module might be helpful (but there may be other/simpler ways to accomplish the same thing): modules/plugin_type_example · 3.x · project / examples · GitLab

[mstenta]

farmfield.factory could do with a little more documentation.

Did you see this? Services | farmOS

(Even if you did, it is a very deep-dive - and largely extends from the core Drupal field system, which itself could probably be better documented. IMO we should only document the farmOS-specific bits on top of that, and improve the Drupal documentation upstream as needed.)

[Farmer-Ed]

@mstenta

Did you see this? Services | farmOS

No, No I didn’t

Don’t know how I missed the link to services. Thanks

I did get what I needed directly from farmOS/FarmFieldFactory.php at 2.x · farmOS/farmOS · GitHub

[Farmer-Ed]

@pat

@Farmer-Ed

I have been working on a notifications module too (another work in progresss… maybe some day I’ll get one finished )

That is cool. Task notification is very useful.
Also useful to be able to snooze the notification directly. Dont know if that would be possible but still very useful. Along as marking a task done, right from the notification.

Cool that Node-Red can make use of the notification. That also makes it easier for different users to change the levels to trigger flows in Node-Red.

I wasn’t aware of ntfy. It looked nice. Been looking for something like that to use with Node-Red.
We use Signal. Is it hard work to add support for that? I’ve seen support for it in Node-Red.

@pat hope you don’t mind I’ll drag this conversation over here.

You can enable and disable a notification on a per log basis, don’t have a snooze option as such but those features may be part of some notification servers/apps. Marking a log ‘done’ from the notification is not currently implemented in the module but could be, I have experimented with this using Node-Red as the notification server and it works quite well as it is trivial to send a response back to farmOS through the API.

So far I’ve tested with Node-Red, ntfy and Telegram others can be added easily if they have an API based on http requests. But then again anything which can be accessed via Node-Red is also an option.

Signal as far as I can see would be possible now through Node-Red but not directly from my current module, it has a JS library though, so possibly a future plugin.

I’ve also already experimented with mix and matching the services using one as a Proxy for another.

farmOS → Node-Red
Straight forward, simple to use received Event for any sort of Automation or pass on to another service. Great if you have an always on reliable connection between farmOS and Node-Red.

farmOS → Telegram → Node-Red
This configuration is useful if Node-Red is not always on/available, an encrypted chat can be used to que messages until Node-Red is available. So maybe for Node-Red running on a laptop that is not always on/connected to the internet.

farmOS → ntfy → Node-Red
As above, but no security unless you self host ntfy.

farmOS → Node-Red → Any other service → Node Red → farmOS
Create a 2 way link between farmOS and (almost)any service you like.

Edit: Should probably also note this is a long way from completion, as I plan on redesigning it significantly to fit with the new core-notifications module and make the different communications methods plugins which can work with other modules.

[pat]

@Farmer-Ed

@pat hope you don’t mind I’ll drag this conversation over here.

My thought too. I quickly made it off topic

I’m looking forward to this. It’s very nice to have log notification in my workflow with the pigs among other things.