Been noodling about how to better manage products explicitly. Talked to Mike, and rolled around some of my own ideas. @paul121 and Mo this is relevant I think to the cases where we want to track inventory. I also know that @Farmer-Ed you’re generating products, curious to know how you manage it also and what you think about this given your experience.
Anywho… here’s where we ended up (currently I’m testing this setup on a test instance, will provide some more feedback as I go) but figured I’d share a summary here for reference –
Problem: The current flexible quantity field method doesn’t distinguish saleable products effectively. This makes generalized reporting difficult, and makes it hard to connect with external systems for product sales (like Open Food Network.)
Proposed Solution: Continue using quantities, but create a product taxonomy that contains all products, and require a reference to the product taxonomy list when creating a quantity. This would be a Product Quantity.
Product Quantity Definition: A quantity referencing a product for sale or internal use. Reporting tools which relate to yield or products will know to look at these types of quantities when performing calculations (efficiency, total yield, etc.)
Create a “Product Type” Taxonomy term
Create a field referencing the “Product Type” (allow for multiple) in the “Plant Types” and “Animal Types” taxonomy terms. This allows a Plant or Animal to have a set of default Products associated with it. This can be used to auto-populate dropdown lists for users selecting a Product Type in the UI.
Create a new “Product Quantity” entity. This either:
Adds a new required field which references the “Product Type” taxonomy. Ideally it filters down the list to those specified in the Crop Type for that crop (or Animal Type for that animal).
Enforces the “label” field to do the same thing as above (this avoids creating a new field) and makes it required. I think (?) this is my preference… but not sure. Then Label is always king and easily searchable…
Now, as a user I can easily product know Product Quantities that
have the flexibility of a quantity (increment, decrement, move to any asset, etc.)
have specificity needed for ease of access (show me all Products I made this week/month/year, show me all production of Corn Stover across my farm, show me all Products from this Animal… etc.). In short - I can now say ‘show me Products’ without knowing the name, and I can say ‘show me product x’ without knowing which assets to look in.
products emerge from a hierarchical, user-managed lists (Product Type taxonomy term) which external systems (like sales platforms) can clearly see and query.
This seems like a great idea and a pretty natural fit for the data model…
I’m curious how it would/should interact with the inventory system though. Specifically, what happens if two separate inventory adjustments (for the same measure and unit) are applied to the same asset with different product types selected.
I agree re. increments / decrements - they pretty naturally should always occur as pairs… but I think the glue programming (JS for SurveyStack, php for FarmOS, etc.) can ensure that works happens correctly for now. But it definitely would quickly be a mess if you screw that up without realizing it
Just expanded the idea a bit, especially to handle better and more natural defaults, put it here:
FYI @gbathree and I were brainstorming some of these ideas over the phone this morning so I sketched up a quick farmOS Product module to play around with. Just a proof of concept, so DON’T USE WITH REAL DATA (yet)…
Basically copies the “material” pattern from farmOS core by providing a Product asset type, quantity type, and Product Types vocabulary.
I currently use quantities to track Material Assets, mostly fodder and bedding, basically use Harvest Logs to increment Hay, Silage or Straw Bales as Material Assets as they are produced, and Activity Logs to decrement as they are consumed. I don’t track saleable products like this at the moment but I’m following this thread with some interest.
I need to read through all this still, but @mstenta just mentioned this to me and I just wanted to make sure to nudge those involved to take a look at the work Data Food Consortium has been doing in this space:
Or if you want to jump straight to the visuals, here’s the product ontology:
100% @jgaehring - I guess my question would be what information needs to come from the farm in order to be useful / productive / helpful up the chain? It’s a question of what a farm generates of that total structure.
Is there someone we could contact to ask that question, who really groks the current product specification that you’ve described?
Thanks @Farmer-Ed , I think Mike described use of a ‘Material Asset’ as a possibility. I think it works, but it feels like kind of an unnecessary intermediary (ie (more complex) a barn asset contains a hay bales asset which references a quantity which is incremented (this requires a new hay bale asset for every barn)… versus (more simple) a barn asset contains a quantity referencing a product_type taxonomy term which is incremented (now, if there are many barns, you don’t create a new product_type taxonomy term for every one).
I could be missing uses / applications / etc. though. And as Mike said, maybe just try things out and see where/how/if things fail in different implementations.
Maybe there is a need for both approaches, depending on the level of granularity required. This is very similar to the reason we have both a “Material quantity” and a “Material asset”, both of which reference a “Material type” taxonomy term. For some people, it’s enough to just say “we added 100 lbs of compost to a field”, without also keeping track of how much compost is leftover (aka managing inventory of a compost or material asset).
The same may be true here. Some people may only need to be able to say “I harvested 100 lbs of corn”, without keeping track inventory of it in a structure (eg: “Grain bin”) or product (eg: “2023 Harvested Corn”) asset. In that scenario, I can understand @gbathree’s desire to have a product quantity type that references a product_type taxonomy term, in the same way that we already have a material quantity type that references a material_type taxonomy term. In both cases, you can also optionally increment/decrement inventory of an asset, but that isn’t required. So it allows for a lot of flexibility.
At first I wasn’t sure about the idea, but I may be coming around to it. I do like the data model consistency of it. I guess my main critique is the product quantity type doesn’t feel very intuitive, but maybe that’s just me. Curious what others think!
Sorry, I’m on a mental tare here, just need to get this all out here in this thread so I can move on with my life.
I also thought about how to manage connections between production yields and product generation - not only to see (in reporting) but to connect the two for understanding productivity and sales.
In addition to what I said before, the below proposal really only adds a few more fields to taxonomy terms (base_units and base_units_per) and establishes a pretty reasonable convention for how to store product_types in the taxonomy list… but the outcome establishes a powerful connection both to reduce effort and producer higher-value estimates and reporting on the sales and production end.
Just want to note another consideration that @paul121 mentioned in chat:
Being able to reference the product type via quantity maybe seems useful if you were using logs for sales receipts? But that wouldn’t really be possible with price quantities eh?
The farm_ledger module provides its own price quantity type for use with Sale/Purchase logs. Open question: if we had both a price and product quantity type, how would they be used to represent the sale of a product?
Ok - partial redo of what I said above… better idea… (hopefully)
Here the focus is reduce complexity by allowing unique package types to be specified and identified (via a is_package flag) within the units taxonomy. Then, the relationship between those package units and base units (like kg, lb, count, etc.) can be established on a product by product basis (which is key!).
In this way, you could now increment/decrement by adding kg’s, boxes, or bags, as long as their relationship was specified in the product taxonomy (requires some php logic, but the info is there). This creates a continuous movement from yield (bulk gallons of milk) to product (cartons) where both an API or UI can immediately provide both values. Example: I could add 25 kgs of carrots to a warehouse in one log, and add 50 bunches (defined in product_type->carrot->package as .5kg each) to the same warehouse, and the result would be 50kgs (25kg + 25kg)! (there’s work here generally to do automatic unit conversions, I’m sure the discussion has happened there also, would love to help that happen too).
This is also important for budgeting purposes also, as total income needs to be estimated quickly against current production / expected production, though they may be in different units that need to be specified on a product by product basis (kg’s peaches → boxes of peaches).
It also significantly simplifies the product taxonomy, keeping the core terms high level (like “peaches”) and enabling multiple packages within it (boxes, bulk, bags).
Finally, these package relationships can be nested for complex products. I could have milk packaged in pints (.125 gal), quarts (2 pints), and gallons (2 quarts), allowing subsequent package units (quarts) to reference previous ones (pints). (I know… this isn’t a great example… you’d probably just reference these all back to gallons but you get the idea).
The Data Food Consortium has a data standard (RDF) and ontological standard (visible and editable in VocBench). These are the standards that many groups, including OFN, will be using to move food product-related data around between services. Basically, if we all agree to this standard, then we all do less API integration work
If farmOS wants to push out product data, it should convert it’s data to this data and ontological standard.
The minimum required information is very minimal - just the product type which is pulled from their product type ontology (so chicken->meat->thighs for example… you can set it to chicken, chicken->meat, or chicken->meat->thighs… they all work!).
There are already some tools for creating data types in pHp using their framework. There may be some work to make a standard drupal module for use by FarmOS and potentially even others.
They also have a very general sense of units - it includes things like kg, lb, gal… but also box, bag, sack… and they don’t convert units (they are just listing them). So that means we can create an internal system for managing package types and units as we want - yay!
How we may implement this
We could make a Serializer to convert from FarmOS’s internal format to DFCs format. This could be minimal, as their required information is minimal. The only added UI element may need to be the user specifying, for each product, the product type from DFC’s ontology list (chicken->meat->thigh for example).
Ensure the API has the required endpoints (get all products, get individual product, etc.) specified in the Getting Started link above.
What to do now?
They aren’t quite ready, but for now we just need to make sure we have product types which are clear and can be provided in the future by the API…
That’s my summary, very helpful… I feel like we’re in good shape to implement something similar to what’s been offered here in this thread and be pretty close to compliant when we’re ready to push to things like OFN.
After talking with mike, I have some updating to the original idea I’ll do in a bit, but hoping to get a conceptual spec proposed again in the next few days.
So I realized before I wasn’t thinking about inventory correctly. After a walk through thanks @mstenta I think I’ve got it together now.
Super short summary
Create a product_type taxonomy. Enable conversion between units (boxes → kg, jugs → gal) specified in that product_type taxonomy, ensure you can add boxes or kg’s, and it converts to the correct base stored unit. Create a new quantity product which references the product_type taxonomy term, and use that to manage inventory (increment / decrement) as per usual. For better traceability, add links to the increment / decrement logs associated with the asset which contains the inventory. Finally, improve the UI so that product-level conversion (boxes → kgs, kgs → boxes) can be visible where users would want / need it.
I know I’m mixing work - this could be pieced out into stages, it’s just all coming out together in my head so I’m putting it together here.
Just wanted to make this point for others (you and I already talked about it @gbathree): unit conversion may not be necessary (yet), at least in the first stage of this…
If a producer is using product inventory units consistently, and they are the same units used in sales, then they may not need to convert between them. There are more advanced use-cases where this might be helpful, but that can probably be a next step.
Some conversions are fundamentally fixed like grams to kilograms, but arguably the majority of unit conversion cases are only valid at a given point in time.
The two obvious cases are currency (changes constantly) and time (changes often enough to get annoying and has a ton of implicit rules) conversions. However, I suspect most other units will change a little from season to season or at least eventually over the life of a farm.
The farm changes how it gets its peaches to market and the partner introduces a new boxing standard which is actually 52kgs
A new hay baler is bought that makes round bales that are 5% denser than the old ones
A state reporting requirement for a slaughtered animal weight changes from the hot carcass weight to the weight after chilling
Obviously, new unit terms could be introduced each time the conversion logic changes, but we don’t have a mechanism for “archiving” units and/or helping make sure the correct units are used. Further, it would also probably be helpful to define what should happen if the conversion logic is changed after products are created or have already moved through the system. Presumably, historical reports should still generate the same data whenever they are re-run.
I’m not sure what my point is here, other than that it is complicated and we should probably move slowly in core…
100% I had less good versions of those thoughts towards the end of my scoping.
I think to your point, the ability to archive a taxonomy term would be really helpful. I would also say we almost (almost) shouldn’t let people modify them after creation, or at least say ‘he if anything is connected to this your past data could get messed up’ kind of message.
I think that’s scopable / implementable. Probably a second phase, but arguably not depending on the failure types and how often they happen.
This is a long complicated string. I have not studied it so this may be off base versus what’s being proposed. Currently, My biggest gap in version 2 is the lack of a product asset. Products are an asset. They have value add and are inventoried. From a farm asset management perspective, Seed becomes Plants (Seed in location/field with activity and materials added), then becomes Product. Grain in my case generated by harvest value added activity. Grain can be further processed (cleaned) to become seed (with new asset name and lot codes). Sort of circular in some circumstances. The farm activities do value added on each asset to transform/create the next. Right now in V2, I need to use a work around asset category to deal with the lack of the Product asset. Not everyone will want to follow this level of asset management rigor. For my organic certification tracking and audits, showing a clear sequence of assets with associated actions and inputs/materials makes the audits cleaner.