I took another stab at this post with the idea that the first step is to walk folks through the high level ideas of figuring out what to record and the general thought processes around it.
A later post should probably talk about more specific details of type, naming, frequency of assets/logs/etc.
Hopefully, this strikes an okay balance of being beginner-oriented and not condescending. I know a lot of this is obvious, but I also don’t think it’s written down anywhere related to farmOS.
title: Organizing Farm Data with farmOS
date: 2023-08-03
author: Symbioquine
slug: 2023/organizing-farm-data
Organizing Farm Data with farmOS
Growing things is busy work that requires tracking many small (but critical) details and being responsive to unpredictable factors such as weather, pests, and market forces. A naive outsider may have strong notions about what data is required to keep things running smoothly and how it is recorded/stored. Similarly, farmers will have a pretty wide spread of actual and aspirational record keeping practices.
Here are some common mindsets when it comes to farm record keeping;
- “Obviously farmers should record everything. More data is better. Increasingly fine-grained data will lead to more optimal practices/production.”
- “Record keeping for my farm is easy. I just jot down free-form notes and organize/transcribe them as I go. The unimportant data gets lost naturally along the way.”
- “Record keeping for my farm is near impossible. I can’t predict what data I will need and trying to record or organize it is a waste of already scarce time.”
We can imagine all sorts of challenges and opportunities that might be implied by those mindsets but each of them holds an important lesson too.
Costs and Benefits
It may be obvious, but the first thing to consider is what you are trying to achieve and how is that different from where you are now.
It can be tempting to blindly pursue a comprehensive record system, record ad-hoc data without a plan, or even skip record keeping altogether. Chances are that none of those is a great fit for your farm. Instead, most farms probably need something in the middle of those extremes. You can’t record every minutiae since it would take too much time and have diminishing returns. Completely ad-hoc data entry probably will not serve any particular purpose well, but skipping record keeping entirely would be giving up on a valuable mechanism to keep things running smoothly and improve operations over time.
What to do?
Note: The thought process outlined here is almost certainly not original, but is hopefully presented it in a way that is tailored to be immediately useful to those entering data in farmOS.
Any time spent entering data has a cost, both in dollars - if paying someone - and in opportunity cost since that time could be spent on the primary work of actually taking care of your growing things. That means that we should have a clear understanding of the value proposition for each kind of data. Sometimes the work entering a type of data will be an investment. Other times it will represent regulatory requirement or a hedge against some possible negative outcome. In both cases, try to imagine the expected value for a given type of data as the probability of needing the data multiplied by the value of that data if it turns out to be useful. Those values might be expressed in dollars or time and will necessarily be best guesses, but they can provide a bit of a framework for determining if the opportunity cost for entering the data is worth it.
A simple example would be if having a peice of data up-to-date and in a convenient place will save you 10 minutes per day but takes an hour a week to enter/update, it might only barely be worth it.
More complexly, if having historical seeding rates (perhaps in kg/acre) lets you avoid $100 in wasted seed 50% of years and takes an hour per year to enter and use the data, then the expected value of entering that data is $50/year.
Working Backwards
Now that we have a framework for evaluating the opportunity represented by a type of data, we need to find some to evaluate.
Start by considering your existing operation and where your opportunities are. In other words, what do you want to change that seems like it could be improved with better data/analysis. Perhaps you already have good data for production and profits from a point-of-sale system, but have less visibility into the day-to-day operations that affect those production numbers. Alternatively, you might have great pre-production metrics from another tool or even a spreadsheet, but lack a good way to track day-to-day tasks assigned to workers or the movements/locations of animals/plants.
Whatever those opportunities are, write goals around them like this;
- “I want to improve grazing rotation efficiency by being able to compare historical movements and animal weight gain.”
- “I want to produce more consistent plant starts for sale by having data that will let me optimize the timing and quantity started year over year.”
- “I want to reduce employee time lost looking for tools and materials by keeping track of where they are located.”
Next, take that statement and imagine what data it implies. For instance the grazing rotation example might require knowing the timing of which (and how many) animals were in which pastures along with - at least - starting and ending weights for those animals. The plant starts example might require knowing the dates when new batches of starts were planted along with the quantity and sales records for the resulting starts.
It might also be important to consider the ways in which the data will need to be accessed to achieve the goal. Maybe the data gets used once a month as some sort of report or maybe it is accessed on a day-to-day basis. This can help inform the granularity of the data and the selection of data formats/tools.
Data Structure and Re-usability
This next part is arguably the hardest because it requires researching (and possibly experimenting with) the available tools and data model choices. It also involves weighing the trade-offs between coarser or finer grained data. Here we are discussing use-cases where farmOS is a good fit, but it is important to keep in mind that for other use-cases accounting, GIS, or even spreadsheet/paper tools might be a better fit.
The farmOS data model provides for representing your growing things and related stuff using assets. It also provides logs for recording events. The basic ideas of how to use those primitives (and a few others) are described in the farmOS user guide. Armed with that information, we can start to decide what structure and granularity will help meet your goals.
In the grazing rotation example we need to decide whether to represent the herd/flock/etc as individual animal assets or as a single group asset. Correspondingly the weights might be recorded as an animal count, total weight, and average weight for the whole group or as individual weight logs for each animal. One thing that can help with that decision is determining whether there are other ways to re-use the data. e.g. Would having individual animal pasture locations and weight history be helpful in finding animals or making care decisions day-to-day?
farmOS is very unopinionated about how one uses its data model. This can be helpful but also means we need to exercise some creativity when choosing the best fit for any given use-case. In the employee tool and material finding example above, it might be tempting to track the locations of each tool or material when employees move them around, but it might not be realistic to expect employees to be updating those locations very well in real-time. Instead, you could choose to just model the normal storage location for the tool or material assets that way employees know where to look first and where to put the materials back when they’re done with them. By also referencing the tools and materials on logs for tasks assigned to those employees, other employees could look up who to ask if the tool or material isn’t where it was expected to be.
Developing Conventions
The patterns of what data gets entered and how it is recorded/used are called conventions. It is expected that farmOS will eventually have tools that can work directly with conventions, but for now conventions are a way to capture those patterns and possibly refine them over time. The audience for your convention(s) is yourself and others who will use farmOS with you.
Basically, a convention is just a written description of the purpose of the data being recorded and how to record it. This can be a plain english (or language of your choice) set of instructions for what data to enter in sufficient detail that someone who is generally familiar with farmOS, but not with your use-case can enter or validate the data.
Writing conventions - especially more detailed/formal ones - is optional, but is a good way to capture the work you have done thinking through the what, how, and why of the data you plan to record. It also could be a good way to share your plan with collaborators or employees.
Next Steps
Hopefully this has been a helpful overview of one strategy for choosing what data to record in farmOS. Ideally, it will inspire you to choose something strategic to improve through better record keeping - and start soon.
Remember not to get too wrapped up in making it perfect initially. It can help to set up a rhythm for improvements to your record keeping - possibly yearly/seasonal - so you can feel free to capture data a way that feels good now and improve on it in the next cycle once you know more or have seen a return on your investment in the data already entered.
Also remember that you’re not alone. It is natural to have questions and struggle to fit the messy real-world into the orderly software one. Feel free ask for help and share your victories on the farmOS forum or chat. Chances are, you will inspire others to think differently about their own data or even lead to improvements in farmOS itself.
