Field Kit Status Update

Back in February, I mentioned Field Kit was nearing a 2.0 BETA release. I regret I’ve left many of y’all hanging since then, with little progress and fewer updates. I’d like to make up for that now with some updates on when to finally expect the beta release, what exactly that entails, and what to expect going forward.

Beta Release: Timeline & Features

First and foremost, since I think this is what will most interest folks, here’s the rough timeline for development, starting from the already released alpha.7, or what I called at the time the “developer’s beta”. While I don’t have funding for these, I can’t guarantee anything on this timeline, it’s just what I’m personally hoping to get done when:

Tag Deliverables Est. Release Date
alpha.7 Stable API Mar 24, 2022
alpha.8 Tasks Field Module Jan 2, 2023
alpha.9 Observations Field Module Jan 23, 2023
beta.1 Stable UX & API Docs Feb 2, 2023

These releases will mostly achieve parity with what features were included under the “General” tab of the “Edit Log” screen in 1.0 Field Kit. So logs can be created, given a name and notes, and their status can be toggled between “pending” and “done”. Those features in particular will be part of the Tasks Field Module, focusing on the simple “To-do list” functionality that was originally at the core of Field Kit. Going forward, Field Kit will embrace a more modular architecture, much like farmOS itself, to keep the user interface and experience of each module concise, focused and uncluttered. Meanwhile, viewing log images and camera functionality will be moved to its own, separate Observations Field Module, where such features can be expanded and prioritized within a dedicated context, instead of shoe-horned into the already cramped confines of the Tasks screen. Notably, however, the features formerly under the “Movements” tab on the “Edit Log” screen of 1.0 Field Kit will not be included. Hopefully, a corresponding Movements Field Module (or perhaps more specific modules for pasture rotation or crop transplanting) will be implemented after the beta release, but the integrations with farmOS 1.0 that those features depended upon will need to be drastically rewritten for farmOS 2.0, and without additional funding, I just can’t commit myself to such an undertaking at this time.

Also not included:

  • Field Kit User Guide
  • Language localization & translation
  • Comprehensive Dev Tooling (eg, npx create-field-kit-module)
  • Fix & update Storyook for the component library
  • Resolving sync conflicts

These were of course some tough choices, but the main intent was to reach a level of internal and external stability, in terms of both the Field Module API as well as general user experience, so that others could begin contributing independently (that’s part of the point of the module system) and I wouldn’t have to continue being the bottleneck for the project as a whole. These releases I hope will represent the bare minimum to achieve such stability, even if its at the cost of 100% parity with 1.0 Field Kit. That’s not to say there won’t be more changes to come, but hopefully once we can reach beta, there won’t be huge breaking changes that prohibit users from bringing Field Kit back into their normal workflow.

6 Likes

Field Kit alpha.8 is now up at https://farmos.app. Still not stable, but for those looking for something to play around with.

And the Tasks field module is released at 2.0.0-beta.1:

Up next: the Observations module! :telescope:

4 Likes

Coolness! :

Is that a “stable beta”? and does this mean it will work with a Farmier-hosted instance?

Oooh! double-cool. Same question: will this work, when it lands, with Farmier-hosted instance?

3 Likes

Maybe if we both say “please”? :innocent: :smiley:

I’ve been looking forward to the fieldkit too. Very happy about progress and the info.

3 Likes

Sorry, I should have clarified. I vacillated a bit on how to version the Tasks module, b/c in and of itself, it should be more-or-less complete, and I don’t anticipate any changes will be needed prior to Field Kit itself reaching a stable beta, aside from any minor bugfixes that crop up in the intervening time. However… since Field Kit itself is still unstable, I won’t recommend it for active daily use with critical data. So I guess the answer is no, sorry. :frowning_face: Same for the Observations module when it’s “done”.

And I’ll further clarify my interpretation of “stable” for Field Kit is that it won’t force users to totally wipe their cache and reinstall it whenever there’s a new release. The main reason for that would be if I needed to make any changes to the underlying schemata Field Kit uses for its local (offline) database. I don’t expect any more changes like that before I get Observations done ( :crossed_fingers: ), but the heaviest lift with that module will be working out a new way to cache images (and other files) together their associated data, and until I do that, I don’t want to get bogged down writing database migrations whenever a schema needs to change.

The other way I’m interpreting “stable” here is in what I’ve referred to as “UX Stabilty” in the above chart. That mainly centers on the overall shift in user experience away from the single monolithic app that v1 represented, toward the more modular paradigm in v2, where all significant features are contained in their own Field Modules and Field Kit itself is relegated to mere scaffolding that deals with the nitty gritty problems of keeping things offline, synced and configured correctly with any given farmOS server. And to my mind, I can’t consider that experience “stable” for users while there are fewer than 2 working modules that can come standard with farmOS & Field Kit. I don’t want to “break” users’ workflows or mental models because they just got used to it being a single module, if that makes sense?

The real achievement with Field Kit beta, I hope, is that other developers can make use of that scaffolding and I won’t have to be a blocker to new features and functionality in the future. At least, that’s the dream!

4 Likes

As a naive user for whom most talk of tech under the hood is incomprehensible, i just want to make sure i understand something about data stores & flows between Field Kit and the server with whom she syncs.

Question(s): Is it true to say that server is ALWAYS master of the data? That when FK “phones home,” she can pull any new or updated records from server, and/or write new records or updates to server, but never delete any records from the server? That, if FK were to lose her mind, any or all of her records, the worst she can do is waste some bandwidth by updating herself with another copy of all records, but there’s no way she can mess up the master database?

This has been my assumption-set, and it’s why i did use Field Kit for some time back in the v1.x server days, even though i did have some trouble with loss and/or corruption of data on the FK; rightly or wrongly, i never worried about messing up the mothership’s database -which problem i never had, fortunately.

Now if that is the case w/ these pre-production versions you are releasing these days, @jgaehring , then i’m inclined to want to get FK on my mobile, talking to my (Farmier-hosted) production instance of farmOS ASAP, because this promises to make for some significant improvements in workflows of our field workers, and the sooner we can understand and test the capabilities that FK can bring to our ops, the better. Am i wrong to be thinking this way about it?

4 Likes

This is basically correct. Field Kit won’t be able to do anything irreversible to the server’s database. Especially since farmOS v2 stores all revision data, the worst that could happen is you would need to go in and restore a log or asset to a previous revision.

The greatest risk, however, is that you record a bunch of logs but don’t/can’t sync them, then something breaks and that data never reaches the farmOS server in the first place. Particularly while I’m not enforcing schema changes with db migrations, there could be no way of fixing an issue like that w/o totally wiping the local data before it gets sent, other than manually entering it into farmOS. Now, I think even that possibility is extremely unlikely at this point, but it’s not 0, so I wouldn’t chance it. Of course, if you don’t have data in FK you care if you lose, then I’d say go ahead and experiment. But that’s a determination for users to make, and I just want to be clear what there’s no guarantee that data will be preserved.

Thanks for asking these questions, btw, helps me get this stuff out in the open that I otherwise just take for granted. :wink:

4 Likes

Right, i forgot: given availability of this rollback provision:

Considering this caveat, it’s a risk i’m willing to take w/in scope of the testing i’d like to do. Honestly, @jgaehring : do you really think even your most stable code will ever be able to guarantee a 0% chance of such data loss ever happening? There is no app i have ever had on my phone in which i would place such trust; even core apps like the Contacts database etc., i’ve never trusted completely enough that i don’t have data stored both to phone AND to either SIM card or cloud (or both!).

Seems to me the only way that we’ll ever have a solid Field Kit in production is via a fair number of users out the the field banging on it to root out the bugs that will inevitably appear in some configurations or other, acting out every UseCase you aim to support, along w/ some others that probably never occurred to you -am i right? If so, then let’s have at it! :grin:

2 Likes

Nope! But I guess the other half of the equation is just setting the expectation that there is little to be done if data gets lost in such an instance. I also can’t commit myself to helping anyone recover any lost data, since I’m trying to get this done sooner rather than later, which is the same reason I don’t want to get bogged down writing data migrations in the first place.

Happy to have some guinea pigs, so long as you know the score!

2 Likes

I totally understand and agree with your priority on getting the app done, & damn the torpedos. Again: if there be any users here with the expectation of data migration tools for this sort of mobile app, i have to wonder what gave rise to such expectation, since that is certainly not a norm in the mobile app dev space.

Anyway: this is about farming, after all, not banking or crypto, so… We’re all guinea pigs here, right? :grin:

2 Likes

Hello @jgaehring ,
I tried farmOS Field Kit and login to my farm os 2.0 at https://chienvu.farmos.net/. But still get error CORS. How can i do?

If you have a Farmier account, I believe you still need to enable Field Kit from farmOS first, is that right @mstenta?

2 Likes

That’s correct, but @henry there are no Field Modules available yet, so it would only be useful if you are experimenting with Field Module development yourself - in which case you would be better off setting up both a local farmOS development environment and a local Field Kit development environment.

2 Likes

Thank you @jgaehring @mstenta i will try setup it to local development. by the way let me ask how can i enable Field module in local development mode? is it automatically turned on?

Thank you again!

1 Like

I’m still just hammering out the documentation and tooling for setting up the dev environment, but will make sure to post here when it’s available. Over on in the Matrix room we’re also talking about doing a little walk-through of the process next Thu, Feb 9, either before, during or just after the regular development call.

3 Likes

Within farmOS there is a farm_fieldkit module that adds an OAuth2 consumer for Field Kit to connect to. This can be enabled either via the command line drush en farm_fieldkit or by going to /farm/settings/modules and enabling it via the UI.

2 Likes

@jgaehring @mstenta Thank you so much for your support. I will setup it now.

I’ve been overdue once again for a substantive update, since I’ve fallen behind again on the timeline I posted in my initial update.

Work on alpha.9 and the Observations module is moving forward, it’s just taken far longer than anticipated. That said, I think as of yesterday the most challenging bits are complete, relating to how images and other files are synced between the remote farmOS server and cached in the local Field Kit database. Just now I had the pleasure of closing a couple pesky and truly ancient issues in the queue over on GitHub. As I mentioned in the topic on Fetching images, I’m tempted to merge this into main for further testing, but for now I’ve just started a draft pull request.

Otherwise all that’s left is to update the UI for taking photos and notes on observation logs.

I’ll need to redirect my attention for the next week or so, as some other unrelated work has been postponed for too long as it is, but hope to have alpha.9 and Observations ready another week after that. Here’s my updated timeline:

Tag Deliverables Est. Release Date
alpha.7 Stable API Mar 24, 2022
alpha.8 Tasks Field Module Jan 4, 2023
alpha.9 Observations Field Module Mar 10, 2023
beta.1 Stable UX & API Docs Mar 20, 2023
3 Likes

Thanks for sharing

1 Like

Hi @jgaehring and @walt,

I am happy to test it as well —I have no critical data in FarmOS quite yet and want to be a heavier contributor soon.

I think Field Kit (as well as @Symbioquine 's Assest Link) would be safer in instances where there is a solo user/tester for the instance (no timeline issues / synchro for data as being tested), which is my case for now

Can I help testing it? Thank you

5 Likes