Authorizing a farm in farmOS aggregator

Hi all! I have two self-hosted test instances of farmOS v3.3 that I’m trying to link to a self-hosted instance of farmOS aggregator (I believe the latest version). I’ve tried authorizing one of the instances both by generating an authorization link and using the “Authorize Now” button. In both cases, it gets as far as directing to the farmOS instance and then stops with a “Page not found” error. See screenshot for the request that fails. Do you have any recommendations for troubleshooting this?

I also tried authorizing via the API. I was able to authenticate my user with the correct scopes to list and delete farms, but I wasn’t able to figure out how to authenticate a farm that way, either. In particular, I was thrown off by the grant_type, code, and state parameters required in the request body for the /api/v1/utils/authorize-farm endpoint. Is there a way to authorize completely through the API? If so, what should be used as values for those parameters?

Thank you for your help!

1 Like

Hi @ktohalloran! In order for an aggregator to connect to a farmOS instance, the farmOS instance needs to have an OAuth2 client configured with appropriate scopes. The Aggregator also needs to be configured to use the same client ID and scopes when it generates the authorization link.

Maybe you are already doing that? Figured that would be the first thing to mention in case you haven’t… (and for others who find this thread in the future).

SurveyStack has their own aggregator, and they provide an OAuth client via their farm_surveystack module here: GitHub - mstenta/farm_surveystack: farmOS SurveyStack.io integration module. - So basically, in order to connect a farmOS instance to the SurveyStack Aggregator, that module needs to be installed on the farmOS instance.

If you look at farm_surveystack.install, when the module is installed it creates the OAuth2 consumer:

  // Create an "SurveyStack.io Aggregator" consumer.
  $consumer = Consumer::create([
    'label' => 'SurveyStack.io Aggregator',
    'client_id' => 'surveystack_aggregator',
    'grant_types' => [
      'authorization_code',
      'refresh_token',
      'password',
    ],
    'scopes' => ['farm_manager'],
    'access_token_expiration' => 3600,
    'secret' => NULL,
    'confidential' => FALSE,
    'third_party' => FALSE,
    'redirect' => 'https://surveystack.farmos.group/authorize-farm',
  ]);
  $consumer->save();

And when it’s uninstalled, it deletes it:

  // Delete the SurveyStack Aggregator consumer.
  $consumers = \Drupal::entityTypeManager()->getStorage('consumer')
    ->loadByProperties(['client_id' => 'surveystack_aggregator']);
  if (!empty($consumers)) {
    $consumer = reset($consumers);
    $consumer->delete();
  }

And then, on the Aggregator side, the backend container’s environment variables are configured to use that client ID:

AGGREGATOR_OAUTH_CLIENT_ID=surveystack_aggregator
AGGREGATOR_OAUTH_CLIENT_SECRET=
AGGREGATOR_OAUTH_SCOPES=[{"name":"farm_manager","label":"farmOS Manager","description":"Allow access to farmOS records."}]
AGGREGATOR_OAUTH_DEFAULT_SCOPES=["farm_manager"]

Note that the SurveyStack Aggregator currently uses the farm_manager scope/role, which grants a high level of access. Ultimately it depends on what your particular application needs, so that’s up to you to determine the right level.

I also tried authorizing via the API.

I offer Aggregator hosting through Farmier and one of the features is the ability to register a new farmOS instance and automatically add it to an aggregator. This all happens via API calls, so it is definitely possible to do everything via the API.

The one catch is that it uses the password grant type for the authorization, which means that you need to know the user’s password ahead of time. The way I do it is I generate a random password, then create a dedicated user for the authorization with that password (using drush), and then I use Python to add and authorize the farmOS instance with the aggregator via that user. Happy to share some of that code if it would be helpful.

Otherwise, you can also manually authorize through the Aggregator UI. Log into the normal frontend (not /docs), select “Manage Farms” on the left, and click the “Add a farm” button. Fill in a name and the URL, check the “Active” box, and click “Save”. It should appear in the “Manage Farms” list, with a red “Unauthorized” button. Click that, scroll to the bottom, and click “Request Authorization”, then click the “Generate Authorization Link” button. Copy the link that is generated into a new tab, and it will take you to the farmOS instance you want to authorize. Make sure you are logged in as the farmOS user you want to authorize as! Once you authorize, you’ll be told that you can close the window, and everything should be wired up.

Also: be sure to configure a cron job on the server that is running the aggregator to refresh the access tokens! Otherwise the authorizations will expire. More details here: farmOS-aggregator/docs/deployment.md at 2.x · farmOS/farmOS-aggregator · GitHub

If you run into issues, let us know! We haven’t had dedicated funding to work on the Aggregator, and there are some known bugs/issues. We have some funding to take a pass through and make some updates soon, so now is a good time to report anything you find. :slight_smile:

Hope that all helps!

Hi Mike, thanks for your response! The info about the consumer and scopes was helpful; I had thought of that but had no idea what settings to use, so it was good to see how SurveyStack is doing it. That being said, even with the consumer and environment variables set to the same as SurveyStack (though I used the UI to create the Consumer, as I’m just trying to get it working at the moment), I’m still seeing the same “Page not found” error in my farmOS instance when I try to authorize. I’m also don’t have a “farm_info” or “farm_metrics” scope in my farmOS instance, which seems to be the default options in the authorization flow even though I’ve replaced them with farm_manager in the .env file, restarted the server, and cleared the instance cache. I tried to add the scopes via the UI, but the permission dropdown threw me–I couldn’t tell which permission to select to represent “all farm info”.

So yeah, having some trouble with this one. Are there any obvious problems here? Do you have a suggestion for how I might add the farm_info and farm_metrics scopes or if I’m missing a module that adds them?

2 Likes

Hmm, I think those two scopes were only available in farmOS v1, so it may just be outdated documentation (and .env.template example). I wonder if your .env file is actually being applied? Can you tell if any of the other configuration is having an effect? I would not expect to see those two other scopes being requested if they aren’t included in .env.

I’m not sure if that would cause a “Page Not Found”… but it might. @paul121 is more familiar with the way all of this works than I am.

For what it’s worth, I run the Aggregator in a Docker container, and set the environment variables via the docker run command. It could be that there’s a bug with other non-Docker methods. You may be one of the first to test this stuff.

I didn’t put this server up so I talked to the engineer who did who said this is running in a docker container in EC2. Based on that, I was able to get my .env to update the env vars in Docker by stopping the containers and running docker-compose up again (though I’ll give docker run a try next time). Now when I start the authorization flow, I see Farm Manager as the only permission available; however, the authorization is still failing with the same “Page Not Found” error :cry:

…I use Python to add and authorize the farmOS instance with the aggregator via that user. Happy to share some of that code if it would be helpful.

I’ve been trying to authorize via the API from Postman–maybe I need to try to use Python. Can you please share that code for reference and I’ll give that a try?

however, the authorization is still failing with the same “Page Not Found” error :cry:

@ktohalloran I took a quick look into this… the authorization endpoint in farmOS should be /oauth/authorize, not /oauth2/authorize. I also searched the farmOS-aggregator codebase, and it uses /oauth/authorize too: farmOS-aggregator/frontend/src/components/FarmAuthorizationForm.vue at 3cef3bcf580a7814b5f7352c7fed647ea4f6f81c · farmOS/farmOS-aggregator · GitHub

If you remove the 2 from the authorization URL, does it work?

I’m wondering where the /oauth2/authorize URL is coming from. :thinking:

Oh! Are you using v1 or v2 of the Aggregator? Make sure you’re using v2!

I’m looking now and realizing that Docker Hub is missing the latest tagged releases of the 2.x branch, but it does have 2.x-dev, which is what I’m using.

This could explain some things… :sweat_smile:

Ah, it looks like we are using the latest code but old Docker images… let me see if I can update the Docker images and I’ll let you know what happens. Thanks for the suggestion!

1 Like

I think that would explain everything you’re experiencing! I’ll open an issue in the aggregator GitHub to update our Docker Hub tagged releases as well. But 2.x-dev should work for now!

Yeah, I’m still having some issues but before adding them here, I’d like to double check the Docker images you’re referring to. I updated our docker-compose.yml file to use the farmos/aggregator:frontend-2.x and farmos/aggregator:backend-2.x images which I saw in Docker Hub. This was a vast improvement, but the authorization is still failing attempting to use a GET request where only POST requests are allowed. But you’re saying to use farmos/aggregator:frontend-2.x-dev and farmos/aggregator:backend-2.x-dev?

You’re using the correct images now - disregard the -dev bit that I suggested. :slight_smile:

FYI: You will need to do docker compose down in order to make sure the new image is used. Simply stopping the containers and restarting them with docker compose up will still use the old ones.

Hmm. What URL is the request being made to?

@ktohalloran I’m also in the farmOS chat on Matrix if it would be easier to debug together on there… https://app.element.io/#/room/#farmOS:matrix.org

Ok, I’ll jump into the chat–that would be easier! Thanks!

1 Like

Finally got it working. To sum up for anyone who ends up experiencing similar problems, the issues were as follows:

  • I needed to add a matching consumer in my famOS instance and update the scopes in the aggregator .env file to match those available in the farm instance so that both sides were using and authorizing the same client with the same scopes
  • Even after that change, the scope I declared in my aggregator’s env file wasn’t being applied, as the Docker image needed to be rebuilt or docker run... command run to apply those changes
  • I was using a very old version of the frontend and backend Docker images. The images that should be running are frontend-2.x and backend-2.x
  • Even after updating .env and the images, I was still seeing old scopes in the aggregator, which I was only able to update by clearing the site data in the browser
  • I needed to update my SSL certificate, as the expired SSL cert on the aggregator was resulting in http POST requests to the farm instance being re-routed as https GET requests, which were then rejected due to the use of an invalid request method
  • The SSL cert renewal process did not go smoothly, as just following the spin up instructions to get a new one resulted in the proxy container failing to start. We ended up just starting over with a new container… so lesson learned: don’t let your cert expire!
2 Likes

That’s great @ktohalloran! Glad you got it working! And thanks for posting all those details. They are sure to help someone in the future. :slight_smile: