App to scan paper records into FarmOS (OCR/Computer Vision)

Hi!

I’m a farmer in Alberta Canada, we grow mostly small grain crops under pivot irrigation. I have tried several commercial record keeping apps over the years but have a really hard time getting my guys (my brothers—so I can’t boss them around too much ha) to enter data into an app.

We have a physical notebook in each of our tractors/machines, where we keep pretty detailed records of each operation we perform. Currently I then spend hours inputting that data into FarmAtHand (FarmOS soon?!?) or whatever record keeping software I use.

Here is my idea. How hard would it be to write a piece of software that would look at a scanned page and extract data from it to create entries in FarmOS?

I imagine making custom note-books that would have the same layout on each page (or room for 2 entries per page). These pages could contain target markers to help computer vision software align it. I would then have boxes for each piece of information I want to collect, so the program would know that the information on a ceratain area of the page is meant to go in a certain field in the FarmOS database. Because of this rigid structure, there should’t be much heavy lifting for the software to do other than OCR, and maybe a bit of computer vision to find checkmarks etc.

Here is an example notebook page:

I envision using coil binding or even just normal binders, so I could take pages and scan them into the program at the end of the season or whenever I wanted to update my digital records. I envision saving a copy of the scanned page with each created record so the digitized information could be verified, and to capture any drawings or other stuff written on the forms.

I’m interested to see how feasible some of you think this would be, and approximately how much it would cost to build such a system. We are willing to pay for a solution, just not sure what ballpark it would be in, so it’s hard for me to put a budget estimate on this description.

If anyone thinks this might be interesting to them or have ideas on how to make it even better I’m open to any and all ideas!

Looking forward to learning FarmOS and using it in our operation!

4 Likes

I forgot to mention: currently I have FarmOS hosting thru farmier, but haven’t really dove in and got everything set up for my operation… just poking around at this stage.

1 Like

It’s an interesting idea, but there are plenty of issues to get over.

  • How clear is everyone’s writing?
  • Are you getting these sheets bound into a book to prevent creasing?
  • How clean can the sheets be kept, I know my own note taking on a tractor often has smugs of oil/dirt.
  • Are you hoping to build this yourself?

Maybe you just need a more simplified App? A simple webform with only the fields they need? That can sync with farmOS via the API. That way they don’t need to worry about the complexity of the farm management software but just get the minimum they need for data entry.

This is something I had been working on.

2 Likes

It’s maybe a bit deep in the weeds, but there’s also this related topic where we touch on alternate data formats and input structures: Design Improvements: the Hierarchical Data Model vs the Graph-like UX - #2 by Symbioquine

2 Likes

Yes good points. I plan to have them in a folder or even a spiral bound notebook that I can unbind to scan. Document scanners are pretty amazing these days.

I think I can overcome the bad handwriting risk to since extent by having checkboxes for operations and fields etc so most of the operation details can be determined by seeing if there are any marks on a certain box (check mark, x). Then OCR can attempt to read the text from the other fields, but also append a picture of the relevant box to each record in database, or at least a picture of the whole page, so it can easily be referenced by me when looking through the data later. Computer vision is pretty good these days at recognising writing, if we try to write neatly I’m sure it can at least get the date right, which, combined with the operation and field will get me most of what I need, which is medically the ability to look back and check when certain operations were done in a field.

I intend to hire someone to make this, whether it integrates with farmOS or some other record keeping program or even just a custom made piece of software. Seems like farmOS should be ideal since it’s already made and open source. I need to get my farm set up and try the basic farmOS system more, it seems a bit more detailed than I need, or maybe my brain just hasn’t wrapped itself around how things are structured… assets and movements and stuff seem a bit convoluted maybe? I just need fields, and a journal of operations done on each field…

1 Like

This looks interesting. Definitely a simple web form would work really well, if I can get my guess to use it. Trouble is we kinda all like having paper notes too… Hmm

Oh and no, I don’t intend to make this myself, that’s why it’s in the job board forum, I would love to pay someone else to make it, our at least help me determine how crazy it is :wink:

2 Likes

Opps, I didn’t even notice the category.

I don’t think your idea is too crazy, wouldn’t be my first choice but it would be a boring old world if we all did things the same way. I had a few mad cap ideas at the start myself.

A Telegram chatbot was one of them, even considered making a voice input version.
Telegram_JEOoTZqjJZ.gif

Another farmOS user gave me the idea, they had made an SMS version, but Telegram made it a little more interactive and more responsive.

3 Likes

We are working on a similar solution for Canadian farmers in other provinces. Can we connect? support@planningintelligencegroup.com

4 Likes

Welcome to the forum @erict!

Just to note: farmOS is “modular” which means you can turn off features you don’t need, to simplify things.

Join one of our monthly calls sometime, if you are able! Happy to give an overview to help get you started. :slight_smile:

PS: I shared a link to this on the farmOS microblog… farmOS.org: "#farmOS job opportunity: App to scan paper record…" - Fosstodon

1 Like

Thanks! I definitely should try to join a call and/or find some time to really dive in. I love the concept for sure!

Looks like maybe from the lack of responses from anyone interested in taking money to start on this that there aren’t many developers looking for extra projects hanging around eh?

1 Like

@erict Probably just need more eyes on it. I don’t have any experience with OCR (or time to spare) otherwise I’d be all over it! :laughing:

1 Like

How much is the requirement about having some paper format vs having that paper/handwriting format?

I can imagine strategies where almost all the data - including the date and probably field - could be checkboxes…

e.g. 2024/02/23

image

(pardon my messy mock-up)

1 Like