CSVs are inherently limited, because they can only represent two dimensions (columns and rows). farmOS represents things in a more complex way, with many-to-one relationships. That said, it is still possible to design CSV importers/exporters that can go back and forth - within those limitations. And in many cases that’s all you need!
For example, a Plant asset in farmOS can have multiple seedings, transplantings, harvests, etc. But perhaps 99% of the time you only need 1 of each. Therefore you can create a CSV that has a column for “seeding date”, “transplanting date”, etc, and that can work just fine. It’s only when you need more complicated relations that CSVs begin to fall short (or need to be represented with multiple CSVs).
So, yes I agree: CSVs are limited in what they can represent. But they can still serve a lot of use-cases quite well.
such simple CSV importers
The goal of this first pass at CSV importers is to match what we have in 1.x, which are pretty simple: each record type has it’s own flat CSV importer.
However, the new system we’re building upon (Drupal’s Migrate API) gives us some new powers to explore and experiment with! So I’m hopeful that, after the first pass, we can also begin to explore more complex CSV templates, which can assist in creating multiple record types at once perhaps.
That said, the APIs will still always provide the most flexibility, because you can work with the raw records in whatever way you need. With the CSV importers, we basically need to write “plugins” for more complex processing of CSV column data. So some things come “for free” and others require development to enable them.
Going back to the Plant+Seeding example above, it’s “easy” to have a Plant CSV importer, which takes a Plant name and Plant type. And it’s “easy” to have a Seeding CSV importer, which takes a seeding date and a Plant asset name/ID. These two are “easy” because they write directly to the record being created, so the Drupal Migrate API gives us that “out of the box”. However, in order to have a single CSV importer for creating Plant assets AND a Seeding log, we need to write a plugin that takes a seeding date (and maybe other columns too, for seeding quantity, location, etc) and uses them to generate a Seeding log, which can then be associated with the Plant asset that is also being created at the same time. This goes above and beyond what Drupal’s Migrate API can do on its own, and requires us to write plugins, which also means we need to define exactly how they should work, which columns they require, etc.
Perhaps we can spend some time on one of the next development calls looking at the underlying considerations to provide a deeper understanding of how these things work behind the scenes. I am excited to explore adding more and more of these features over time! But I expect these enhancements to be a slow trickle of new developments over the coming months and years (unless we get some dedicated funding JUST to focus on CSV importers). That’s why I suggest Python + API calls in the near term to solve your immediate problems.
you can count on me to share whatever i might develop.