This idea has came up a number of times over the years, and again recently with convention work. So I figured I’d start a forum topic to get input from the wider community…
Problem: If I have data that doesn’t have a place in the current farmOS data model, there isn’t a good place to put it while also maintaining structure, visibility, and interoperability.
Example: in a lab_test log, I want to store the “Soil texture” that was included in my lab test results. Sure, I can store % sand, % clay, % silt as three quantity measurements on my log, and use the USDA model (or some other model) to derive the class, but what if I just want to store the name that the lab gave me? Or what if the lab didn’t give me percentages and only gave me the name?
This is one example, but it highlights the fact that we don’t have a good place to put arbitrary, non-quantitative data values.
There are a few options currently:
- I can add a new field to logs via a custom module.
- I can put it in the log
notesfield. - I can put it in the log
datafield.
#1 is very flexible, structured, and visible, but it requires custom module development skills to implement and it is not the farmOS core data model, which makes it more difficult to create interoperable data. It is your custom data model on top of farmOS’s.
#2 is also very flexible and visible, but it is unstructured. Trying to parse data out of an arbitrary text field is a bad idea.
#3 Is flexible and structured, but not visible. The data field is not shown via the UI anywhere, and even if it was, it would probably just show a blob of JSON text (or YML, or XML, or whatever was stored in it).
The idea that has been tossed around to solve this problem is to add a key/value field to assets and logs, which would allow storing arbitrary data as key/value pairs.
There is a Drupal module that provides such a field type: https://www.drupal.org/project/key_value_field
Notably, this module also adds a “Description” alongside each “Key” and “Value” field, which optionally supports human-readable descriptions.
If we added a key/value field to assets and logs (perhaps named extra to indicate that it is for “extra” data that doesn’t fit well anywhere else), it would provide a new place to store arbitrary data that can be semi-structured/interoperable and is visible to the user.
Would this be a useful addition to the farmOS data model?
It should be obvious, but to be very clear: this is NOT going to be the best place to store data in many (maybe even most) cases. It is always better to try to use the existing core data model, or work on extending it with new fields/entity types (either in core or in contrib module).
But is there value in providing an “in-between” place for semi-structured data, that allows experimentation while more “standardized” models are debated/experimented with in the community?
I like the old adage “bad data is better than no data”, and the goal of “always improving data quality over time”. Right now, in some cases, there is a gap in the farmOS data model in that area between “bad data” (eg: stuffing everything in the Notes field) and “good data” (eg: dedicated fields).
The argument against adding a key/value field is that it may encourage making messes. This is true and valid. But is it an opportunity for our existing messes to be a little less messy? If so, then I consider that a step in the right direction.
Very curious to hear all opinions on this!
