I wanted to share the strategy that I use to host farmOS on my local network (at our “farm”) that makes it internet accessible without requiring a dedicated IP through my ISP.
It’s not farmOS-specific, but farmOS is one application which especially benefits from this strategy since it opens up the possibility of more easily hosting farmOS “on the farm”.
Pros:
You control the data (even more) by keeping the storage hardware physically under your control
farmOS stays available to on-farm users even if the farm loses its internet connection
On-farm users can upload files/photos at local network speeds
It can work well with multiple internet connections. e.g. a cable ISP and a fallback LTE cell modem. Whereas a single static IP through the cable ISP would mean farmOS would stop being accessible from the public Internet if the primary cable ISP goes down.
You’re not beholden to a given ISP to consistently provide a public IP from a block they own
It does require a remote server with a fixed public IP address, but requires minimal configuration on that server and no custom software running there beyond a modern version of the OpenSSH daemon - that makes it trivial to switch remote server providers with almost no migration effort if a faster/cheaper options become available
Cons:
Local power and hardware reliability become limiting factors for the site’s availability
Access through the public Internet will be (maybe significantly) slower than if the site were directly hosted on the remote server
It may not be an option with lower-tier consumer routers that lack certain NAT/Routing configuration options
This has been a dream of mine for some time, @Symbioquine, but -regarding the cons that pertain to the network architecture you describe, whereby data must be served directly from farm, or else not at all -this could be a stopper for those of us with weak network infrastructure to support the kind of uptime & performance we must have.
So i’m wondering: might it be possible to have the farm-based server replicated to a 2nd server in the cloud?
I know this is possible in Zope-world with their ZRS (ZODB Replicated Storage) solution, but the limitation there is that the replicant in cloud serves read-only to clients on the web. Still, could might be a good-enough enhancement to the architecture you describe @Symbioquine, if there’s something like ZRS operative in the Drupal world.
I guess the main question is how you’d want writes (e.g. creating/modifying/deleting assets/logs/etc) to work?
A read-only cloud mirror should be fairly trivial, but it gets a lot trickier if they both need to be read/write capable. Then you would need to consider how writes are handled - especially during network instability.
As things stand, practically all writes are from my farm-based desktop to our Farmier-hosted server, and most data is bulk-uploaded from spreadsheets via API script. Small edits & deletes are sometimes made from other machines, but always (as a rule [1]) on the farm, but there are starting to be calls to read data from off-farm web browsers. Enabling those read-only UseCases is a win worth exploring for us.
A read-only cloud mirror should be fairly trivial, but it gets a lot trickier if they both need to be read/write capable. Then you would need to consider how writes are handled - especially during network instability.
Concurrency control in databases is a Big Hairy Deal, but we can live very well without that for now!
So: i’d be curious to hear more about that fairly trivial solution for a read-only cloud mirror, if you can suggest one, @Symbioquine. With that to supplement the sort of internet functionality that you’ve described, i can see this Local-first solution working pretty well for my farm.
[1] Every rule (even Local-first) has its exception, and the edge case for me requiring full CRUD access via internet is for remote tech support. Not an everyday need, but often enough that it would probably justify the overhead involved in what you’ve described (i took a quick look, but will need to understand better what all that tech means in terms of TCO).
In theory Drupal supports a pluggable file storage system where you could just drop in your own replicated storage (e.g. https://www.drupal.org/project/s3fs), but I suspect that would be an exercise in frustration in terms of tracking down modules that assume files are local. That might be worth exploring some, but for a read-only-remote/local-first strategy I’d also want to explore a simple cron/rsync script - maybe with a hook for queuing up best-effort real-time copies when files change.
Depending on how important it is for the read-only mirror to be up-to-date, an even simpler strategy could involve periodically restoring daily backups.
Perhaps it’s worth noting that the local-first strategy I’ve described could be used in combination with a stale-read-only mirror on the remote machine. i.e. going to myfarm.example.com gives you the local-first version but going to replica.myfarm.example.com gives you the read-only replica when accessed via the Internet. (That would require a bit more NGINX configuration, but should be pretty doable.)
I think the key question is, 'how much would your farm’s “weak network infrastructure” actually be a blocker for getting the level of remote tech support you need?"
In other words, if you had farmOS only running locally and used the strategy I described in the original post to make it accessible via the Internet when your farm’s network is up, how much overlap would there be between the times you need tech support and the times your network is too unstable?
In terms of TCO, it’s probably hard to beat a co-hosted solution like Farmier.
That said, I do believe that the local-first strategy I’ve described can be pretty cost effective - especially where electricity is cheap.
Thanks, @Symbioquine, for the good ideas about read-only replication, and for helping me to think more clearly through the problem -which is not in fact a problem as things stand, but more of an exercise in “scenario planning” (one discipline i never took seriously until this year of extreme drought, war on the continent, spiralling energy prices, etc.)
In fact i’m very happy with Farmier hosting my system, and would never even have ventured to think about local hosting, were it not for the appeal of certain Shiny Things (e.g. Asset Link ) that are not yet ready for co-hosting… And once embarked on that road, then Local-first seems to me the way to go with data management, simply on principle (along with Local Food, Local Money, etc.).
But, for now: i’m just happy to know such a solution exists, and is moreover so nicely documented here -so thanks for that, mate!
Electricity is not going to be cheap in Europe for a while unless you have your own micro Generation. It could well impact on the cost of hosting over the coming few months too though.
Important point there, @Farmer-Ed : though Portugal is more energy-resilient than much of Europe, at current and projected rates of both options, Farmier-hosting is still looking to be cheaper than self-hosting, in terms of TCO -by far!
Love to see the explorations and experimentation here!! That’s always been my hope that we would see a big spectrum of solutions develop.
And my thought is Farmier will always serve as the “simplest” and “cheapest” solution, for folks who don’t need advanced use cases. With the goal of lowering the barrier to just use “default” farmOS as easily as possible.
) that are not yet ready for co-hosting… And once embarked on that road, then Local-first seems to me the way to go with data management, simply on principle (along with Local Food, Local Money, etc.).