Log not Synching to Database

I have been using the Field kit over the past couple week to make field observations. The logs have not synched with main database since Sept 26th. Still on my phone. When I select synch all from the upper right corner of app screen (cloud arrow button), the circle arrow icon starts, runs a few seconds, and the app seems to crash. Restart the app and the logs since Sept 26 are still on the phone. Just updated my Andriod phone to latest system software, no change.

Hi @graffte, thanks for posting.

Do your logs have photos and/or GPS coordinates attached to them? How many are there, roughly?

I’m going to take a closer look at this and the other issue that cropped up:

https://farmos.discourse.group/t/error-the-website-encountered-an-unexpected-error-please-try-again-later

I suspect they’re closely related.

Both, GPS coordinates and photos. I have 11 log entries stuck on my phone. All have photos and GPS.
Tim

Related… I have one log entry I never completed filling out from Oct 9 that appears it did synch. This one did not have photos or a GPS coordinate.
Tim

You’ll also see several on Oct 5th with no info entered. I think I hit the " + " key several times in a row.

However, I think I tried to save logs at least with a couple (including photos) and things appear to have gotten lost on Oct 4th and 5th. I have photos in my phone gallery as I started taking photos with phone camera and then attaching to the Field KIt app given the previous issues with losing some photos.
Tim

Huh, yea, I’m inclined to think this may be due to the GPS coordinates, though I won’t rule out the photos being an issue too. We recently fixed an issue with the way GPS coordinates were formatted, but I think that introduced a bug where older logs that had any GPS data prior to the fix were corrupted into a format that the server would no longer accept.

If you have the time, could you test a few things for me and let me know which of the following procedures work?

  1. Create a new log with just a name and try to sync it.
  2. Create a new log with just a name and a photo, and try to sync it.
  3. Create a new log with just a name and a GPS coordinate, and try to sync it.

Thanks in advance!

Okay ran some tests. All three of the your suggestion synch’d properly. I also tried one with GPS and Photo. It worked too. Here is the difference, today I synch’d from the individual “edit log” screen. When I went back and tried to synch from the “My Logs” screen things crashed again.

Tim

Also… I went back to one of the ones that didn’t previously synch and opened the “Edit Log” screen for it. Then hit the synch button. The old one worked too.

Ah, ok. I suspected new logs might work. So yea, I believe it’s malformed GPS coordinates that the server is rejecting.

I’m not 100% sure that this will work, but it’s possible if you open an old log that’s not working, add another GPS point, then delete that point, it will reprocess that data into the proper format, which may then be able to sync. :crossed_fingers:

The old logs are synching WITHOUT changes. … BUT… I have to select the synch cloud button from the “Edit Log” screen for the individual log entry. NOT the “My Logs” list showing all the unsynched logs.

It may be related to one of the logs “Wet Area NE corner” on Oct 10. All of the other log entries have synch’d properly by doing them one at a time. This one give me an error message “request failed with status code 413”. I tried deleting the GPS coordinate. Still failed to synch.

Hi @graffte - I just saw an error notification on my server that traced back to your farmOS. It looks like you tried to upload a file that exceeds the 50 megabyte limit currently configured on the server. I imagine that’s related to this thread? (Just catching up now.)

Oh actually, to be precise: it was the request BODY that was over 50 MB (!!) - I assume this is because Field Kit base64 encodes files into the JSON body itself and sends it all together, which exceeds the POST size limit.

Oh wow, yea, that might have something to do with it. :upside_down_face:

Maybe. I like to include photos of what I’m looking at out in the field. The one that won’t synch has more photos than the others. Problem is, when you add photos to a log record there doesn’t seem to be an indicator that says you have exceeded the 50MB boundry limit. There also doesn’t appear to be a way to delete a photo from a log entry ??

1 Like

Problem is, when you add photos to a log record there doesn’t seem to be an indicator that says you have exceeded the 50MB boundry limit.

That’s true - and it’s also something Field Kit CAN’T display - because it has no way of knowing what the server is configured to allow. Farmier servers have a default limit of 50 MB. The default PHP config is 2 MB. Anyone setting up their own hosting may set it to something different.

So the “proper” solution would probably be to send the files individually, and confirm that each works on its own. This would require some work to achieve, so we should start thinking ahead for it even if we can’t do it right now (due to time/budget/technology constraints).

There also doesn’t appear to be a way to delete a photo from a log entry ??

Add this to the list as well! :slight_smile:

File handling in general needs to be overhauled in Field Kit. Unfortunately, we most likely won’t be able to make major changes before 2.x.

Certainly you can never know what somebody else might set up. However, seems like anyone tech savvy enough to set up their own server for hosting, should be able to sort out the ramifications of the alternative defaults they might set up. Suggest putting the focus on making sure the combination of Farmier hosting and Field Kit is clean and robust for the non-techies.

While you are adding things to the list… in the Field Kit, the “Notes” field entry box is irritatingly small. You can only see about 3 lines which makes entry and editing on a phone somewhat difficult. Can you triple that? Be nice to be able to read what you’re writing in full.

anyway, I now know I need to keep my photo habit in check and upload the log entries individually. That helps. I can at least avoid the problem.

1 Like

Yes, overhauling image/file storage has been in our issue queue for a while now, as evinced by this issue from 2018:

Happy to say it’s finally on the roadmap, at least, but as a part of the 2.0.0 plan, so probably won’t be available til next spring at the earliest. Incidentally, I’m planning to include it in a stage of development that will be focused on creating a dedicated Observations module in Field Kit. I’d love to solicit your feedback when we get to that stage, since Observation logs seem to be an integral part of your workflow.

Duly noted! I’ll actually be overhauling the “Edit Log” screen starting this week, and I think this should be easy enough to achieve. Here’s a rough prototype of what it should look like, if you’re interested:

No problem. If I can be of help reviewing some stuff let me know.

Tim

1 Like

Field Kit might not know what the limit is ahead of time. However, it seems like the farmOS server should respond with HTTP status 413 which would at least let Field Kit tell the user why the log upload failed after the fact…

1 Like