Issue adding area from coordinates / kml

Development
1

Hi there!
I tried to create areas copying the coordinate description to another instance of farmos:

POLYGON ((9.7341636746652 53.602429343963, 9.7348547759043 53.602581621115, 9.7349703592438 53.602403251875, 9.7342780039437 53.602247803529))

I press the button “Find using files field” and the area appears correctly on the map

When I press save, I get an error thrown:
Geometry : Specified location data is invalid"

If I get the kmz and try to import it:

<?xml version="1.0" encoding="UTF-8"?> Block 1 9.7341636746652,53.602429343963 9.7348547759043,53.602581621115 9.7349703592438,53.602403251875 9.7342780039437,53.602247803529

I get an HTTP 500 from system ajax with empty ResponseText

The kml works fine in google earth.
Importing the area from a 7.x 1.6 instance into a Farmier one

Might this be a bug?

thanks

1 Like
2

@raleonardo hmmm if you’re simply copying the value from a geometry text area from one site to another you shouldn’t need to click on the “Find using files field” button. If you click save it will use the value that you pasted in. Can you confirm this works?

Ahh actually… I just tried using your geometry and it doesn’t work for me either. I noticed that after clicking “Find using files field”, the map zooms to that geometry just fine. Then if I use the map Edit tool and drag a corner of the polygon, the geometry value is updated just slightly! A 5th set of coordinates is added, equal to the first:

POLYGON((9.734013140792925 53.60242744530984,9.734854775904301 53.60258162111501,9.7349703592438 53.60240325187499,9.7342780039437 53.60224780352897))

Which makes sense… I found this on the wiki page for WKT text (which is the format you see in that geometry text area)

The OGC standard definition requires a polygon to be topologically closed

This might be. It’s certainly a weird behavior that the map can zoom and display the geometry correctly, but upon saving say that it is invalid :thinking:

It would be another bug if farmOS dispalys the WKT Polygon as only having 4 sets of coordinates in that textarea. Is it possible you copied that Polygon string from the KML? Maybe KML doesn’t require the polygon to be “closed” ?

1 Like
3

I copied the polygon string from from the area edit dialog from another instance (self installed on a VM), where I manually created the area beforehand.
There I can use the polygon string to recreate another area, yet not in the Farmier instance.

1 Like
4

Can someone reproduce creating the area in their instance from the polygon description above?

1 Like
5

POLYGON ((9.7341636746652 53.602429343963, 9.7348547759043 53.602581621115, 9.7349703592438 53.602403251875, 9.7342780039437 53.602247803529))

This is not valid WKT. A polygon must have the same point at the beginning and end to close itself.

When I press save, I get an error thrown:
Geometry : Specified location data is invalid"

The validation that happens when clicking “Save” is performed by the Geofield module, which uses GeoPHP to parse the WKT. It shows this message if GeoPHP throws any exceptions during the reading of the WKT.

I press the button “Find using files field” and the area appears correctly on the map

The “Find using files field” button is only for parsing KML files that are uploaded to the Files field. However, it has the secondary effect of re-rendering the map using the geometry in the Data field (which would be populated from the KML file, if there were a KML file. In this case there wasn’t, so it just reads the data that was pasted into it.)

When that happens, OpenLayers is responsible for taking the WKT in the Data field and rendering it on the map. It does not pass through GeoPHP at all.

So it seems that OpenLayers is more permissive of reading polygon WKT. Perhaps they (or one of their dependencies for parsing WKT) have some logic that auto-connects the last point of a polygon if it is missing. That would explain why it then shows up on the map.

Then if I use the map Edit tool and drag a corner of the polygon, the geometry value is updated just slightly! A 5th set of coordinates is added, equal to the first:

When you edit the points using the edit tool in the map, what actually happens is: OpenLayers has its own internal representation of the geometry being edited, and when a change is made to it the farmOS-map library converts it to WKT (using methods provided by OpenLayers) and updates the Data field with the new geometry.

So this indicates that OpenLayers does output correct WKT, even if it allows importing incorrect WKT.

I copied the polygon string from from the area edit dialog from another instance (self installed on a VM), where I manually created the area beforehand.
There I can use the polygon string to recreate another area, yet not in the Farmier instance.

This is interesting… you were able to save the incorrect WKT in your own self-hosted instance of farmOS? It sounds to me like THAT is the bug. You should not have been able to save that WKT. What version of farmOS are you self-hosting?

1 Like
6

And perhaps more importantly: what is the GeoPHP module version? There was an update recently (including in farmOS 7.x-1.6). Also note that farmOS patches GeoPHP to use BCMath, but that’s probably not related to this. But if you made any manual updates to GeoPHP that patch would not be included. Any other details you can provide about your self-hosted instance might help diagnose this!

7

I think it would also matter whether the GEOS integration is enabled for geoPHP…

If so, geoPHP will parse the polygon with GEOS and that would fail because of this code;

if(!LineString::isClosed()) {
    throw util::IllegalArgumentException(
        "Points of LinearRing do not form a closed linestring"
    );
}

Whereas the plain PHP-based geoPHP parsing for WKT polygons doesn’t seem to validate that the linear rings of a polygon are closed.

2 Likes
8

Hi!
I’m running [farmOS 7.x-1.6.] on a debian buster with php7.4

(php-geos) So that’s what I thought too, that this instance did not validate the creation of the polygon properly.
It has all the mentioned php (7.4) dependencies installed from debian packages (including bcmath and php-geos), although I noticed the bed generator is not working (“This area generator tool requires the GEOS libary” warning message). btw this is the only place where this message show up…

I then dig deeper and realized somehow the php-geos debian package only installs the module for php7.3 and not 7.4. :thinking:

What do you mean by patching? Does the application change something in the installed dependencies files? I so it really doesn’t sound like a good thing :confused: . It might work in an air tight container but it’s not decoupled enough from the system packages updates.

2 Likes
9

If so, geoPHP will parse the polygon with GEOS and that would fail because of this code;

Ah ha! Great find @Symbioquine ! That definitely seems like it would explain the difference here.

Thanks for the extra info about your environment, @raleonardo …

although I noticed the bed generator is not working (“This area generator tool requires the GEOS libary” warning message)

Ah yea that’s a good indication that GEOS is not being recognized by the GeoPHP module in farmOS, which would mean it is also not being used in the polygon validation.

I then dig deeper and realized somehow the php-geos debian package only installs the module for php7.3 and not 7.4.

Here is how the farmOS Docker image installs GEOS (built from source), for reference: https://github.com/farmOS/farmOS/blob/f93d4bef58a045d53d1c6583de52954a398d4660/docker/Dockerfile#L26

You could give that a try. Make sure you uninstall the version you installed previously first.

What do you mean by patching? Does the application change something in the installed dependencies files?

We don’t patch GEOS itself (on the server level), but we do patch the Drupal GeoPHP module. This is done by Drush during the drush make process, so as long as you are downloading a packaged release of farmOS, or building it yourself via Drush, then you will have the patch. But if you were to download/update the GeoPHP module alone manually from https://drupal.org/project/geophp, then you would not have the patch. We have a warning against this at the top of the documentation page: Updating farmOS | farmOS - and we disable the ability to automatically update modules through Drupal’s UI (see https://www.drupal.org/node/3136140).

Here is where the patch is included: https://github.com/farmOS/farmOS/blob/f93d4bef58a045d53d1c6583de52954a398d4660/drupal-org.make#L68

We patch a number of other modules as well, including Drupal core itself. All of these patches are included in that file (and Drupal core’s patches are included here: https://github.com/farmOS/farmOS/blob/7.x-1.x/drupal-org-core.make).

Hope that helps and all makes sense! Thanks again for opening this topic and investigating it with us - great to have it all documented in case someone else experiences the same issue.

2 Likes
10

Wow @mstenta thank you for all this information.

I get that all that drupal patching makes sense: being all managed by drush, from my understanding all patches all get into place.

In this case I’m installing directly from the tar.gz and not touching it.

I think this is an issue with the php-geos (debian) package that has a dependency:
php-geos Depends phpapi-20180731, which is provided by php7.3 (buster default).

I’m using php7.4 from sury, but apparently there is no php-geos in sury.
It might work if all is installed with php7.3

I tried to build it from source (as per the docker file), which went fine and the module went into
/usr/lib/php/20190902/geos.so which seems the right place. yet… the ini is nowhere to be found
$ sudo phpenmod geos
WARNING: Module geos ini file doesn’t exist under /etc/php/7.4/mods-available
WARNING: Module geos ini file doesn’t exist under /etc/php/7.4/mods-available
WARNING: Module geos ini file doesn’t exist under /etc/php/7.4/mods-available

I guess should prob use the container directly and that’s that…
Thanks again for all the help and let me know if it’s useful to dig forward

Ah btw, I am still wondering how did I get into the original issue in the first place:

  • why the geometry got generated with a wrong format
  • why I only get a warning about geos misconfiguration in bedgenerator and not in add area section (it doesn’t use it?)

thanks again
cheers

1 Like
11

Glad to help @raleonardo !

I guess should prob use the container directly and that’s that…

Haha that is one of the benefits of Docker containers! We make sure it works in there, and we don’t need to worry about platform specific differences or officially “supporting” them (aka spending time debugging things). We support the Docker image, and the Docker image works the same everywhere (in theory). :slight_smile:

Ah btw, I am still wondering how did I get into the original issue in the first place:

  • why the geometry got generated with a wrong format
  • why I only get a warning about geos misconfiguration in bedgenerator and not in add area section (it doesn’t use it?)

This is because the GeoPHP library does not have a “hard” dependency on GEOS. If GEOS is installed, then it will use it. Otherwise, it provides its own alternative functions and methods.

These alternative functions and methods work for most everything you normally need in farmOS. The one (I think only) exception is the Area Generator. The code that is used to slice a larger polygon into strips (eg: dividing a field up into beds) requires some functions that are only available in GEOS, not in GeoPHP alone. So we explicitly check if GEOS is installed and if it is not then we disable the Area Generator form and display that message.

The reason you were able to save invalid geometry is because the GEOS library is where that polygon validation is taking place, not in GeoPHP. So without GEOS, the validation doesn’t happen, and you are allowed to save an invalid Polygon. So, perhaps a next step here could be to open an issue in the upstream GeoPHP library for adding this validation when GEOS is not present. Feel free to start that ball rolling if you want!

12

thanks for the thorough explanation @mstenta

I think I’ll pass the ball now, have others jugling :stuck_out_tongue: (and serve with docker)

Just a note on the “saving invalid geometry”:
I didnt manipulate the geometry, it got generated (or displayed) into it’s field after creating it through the graphical interface. Then I assume there were 2 issues: it was incorrectly generated and/or saved into the db after not being validated.
That might also being worh looking into by the devs

Alright thanks again,

13

I didnt manipulate the geometry, it got generated (or displayed) into it’s field after creating it through the graphical interface.

That’s a very good point that I overlooked @raleonardo.

Are you able to replicate this behavior? My experience is that drawing a polygon in the farmOS map always produces valid WKT (a closed polygon).

Is it possible that you drew a line instead, and then manually converted it to a polygon (by replacing LINESTRING with POLYGON in the WKT)? That’s the only way I can reproduce it.

14

That makes sense, I don’t think its possible, as I was initially only using the guy to draw polygons, and wanted to copy them from my managed instance to a farmier one.
And I didnt manipulate the geometry WKT…
I tried to reproduce it, but couldnt get an open polygon.

I’d drop i for now. Thank for your help Mike

1 Like
15

Huh! Yea well let us know if it happens again! Very strange! :slight_smile: