I have installed docker version of farmos with sqlite db with following environments
all the files are on my external hard drive, i use 2 computers to use farmos first is windows 10 and other is ubuntu 2204. previously i used to get the synched sqlite db (i was able to see the changes done on windows 10 pc in linux), one day i forgot to docker compose down and shut down the windows pc, since then my changes in windows 10 pc are not seen when i plug that external hard drive to ubuntu pc and run docker compose up,
can someone help me to figure out where is the sqlite db in my windows pc ( the default db in the sites folder is not updated on windows pc, but is updated on ubuntu pc)
my docker compose file reads :
version: ā3ā
services:
www:
# Update this to the latest stable version before deploying.
image: farmos/farmos:2.0.0-beta6
volumes:
- ā./:/opt/drupal/web/sitesā
ports:
- ā80:80ā
restart: always
Canāt say I like the idea of running containers on a removable drive used with 2 different PCās.
I assume you are running the same docker-compose file from the same directory of the removable drive on both machines?
One would think the same SQLite file would work on both, have you physically found 2 versions of it?
Does Data persist for data created on the Win10 machine, if the container is restarted?
Why use it this way on 2 different machines? Itās a Web server, canāt you just use one machine to host?
I think Postgres would be favorable over SQLite and easier to manage in the long run. Migration should be straight forward enough but make sure to take a backup before trying anything.
Canāt say I like the idea of running containers on a removable drive used with 2 different PCās.
Agree with @Farmer-Ed this sounds very risky to me. If they were both Linux or both Windows maybe, but it feels like a good way to find an incompatibility issue and lose your data.
Can you set up one of the machines as an always-on server and connect to it from the other one?
yes, data persisted previously,
the reason for not making one machine server is coz of IT policy of my organization which does not allow for intersubnet access and both machines are on different subnets, besides any one machine cannot be kept ON for 24 hrs as it is shared
i will give a try for data migration, but need support to configure my specific db, any help would be appreciated ( i went thru the stackoverflow post but it overflowed me!)
inorder to take the backup, on windows machine, i copied the whole folder to a different folder and ran the docker compose up, but it was showing readonly (same as linux).
the site from original location in windows machine also is not editable now
can anyone guide me to locate and migrate / backup - restore my self hosted farmos site?
I meant currently, as in is Windows loading the correct SQLite file but its read only?
It sounds like your issue is just permissions.
Have you checked the ownership and permissions of the file/(s)?
using ls -a -l in sites/default/files/ should show the permissions and ownership.
Personally Iād look for another solution as you will likely end up with permissions issues again with different files by using the 2 operating systems.
Why the need for 2 machines? is it one at work and one at home?
Assuming this is your project and not your organizations, here are solutions I think would be better:
A virtual machine image (still some risk on a removable drive but permissions would be simpler)
Hosted on Farmier (both machines would require Internet Access)
Self Host at home on always on low power machine like a Raspberry Pi 4 or NUC (needs internet access at both sites)
Get a laptop and bring with you (doesnāt need to be anything fancy)
Use a Pi 4 or NUC or Nettop but bring it with you and only borrow screen keyboard and mouse at work (maybe use a simple KVM switch).
āportableā means the same Docker image can be run on multiple platformsā¦ not the same Docker container or volumes. Maybe itās possible in theory, but I just want to point out that this is not what is meant by portability, and this is probably not a recommended usage of Docker. So you are taking risks by doing this @perfectinfo - just want to make sure that you are aware and donāt blame us if something goes wrong.
All of the solutions @Farmer-Ed described above would be safer.
Iāve updated the title of this forum post to make it more explicitly about this unusual strategy.
I think Postgres would be favorable over SQLite and easier to manage in the long run. Migration should be straight forward enough but make sure to take a backup before trying anything.
Agreed here as well. SQLite works but the performance is going to be much lower than PostgreSQL.
I have never done a migration from SQLite to PostgreSQL, so I canāt provide any support with this. Itās more of a general database question than a farmOS-specific one, so you could also try other forums.
I will say that I tried to help @Skipper migrate from PostgreSQL to MySQL on farmOS v1 and learned a lot about how difficult it is to go from one SQL database type to another. We ended up giving up. Maybe youāll have better luck.
Confirmed with ls -al? at least one of your OSās seams to disagree if its loading the database as read only.
I have, with your migration tool from SQLite on1.x to PostgresSQL on 2.x, It went without much issue, although admittedly there was not much in the database as Iād only just started using farmOS a short while when youād announced that 2.x was now the preferred installation. It hardly works from 2.x to 2.x?
Notably, this is not really a direct SQLite-to-PostgreSQL migration, but rather a Drupal 7 to Drupal 9 migration. Drupal provides a database abstraction layer, so the farmOS v1->v2 migration uses that to read records into PHP (from whatever db they are in), translate them to v2 records, then save them to the new db (whatever that is).
We donāt have any way to migrate v2 SQLite to v2 PostgreSQL that uses the Drupal database abstraction layer. So it will be a much lower-level process working directly with SQL queries.
I figured that would more or less be the answer, I think pgloader will be the simplest solution.
But at @perfectinfoās own risk and backups before trying anything.
Not much, youāll probably need to seek help in a more SQL orientated forum on migrating from SQLite to Postgres. Maybe open an issue on pgloader github issues, or look at alternative migration tools.
I assume you ran it as a super user su / sudo ?
permission of /tmp does not necessarily mean the permissions of all files and subfolders are the same.
chmod -R 777 /tmp would apply the same permissions to every file and subfolder. use of -R can be useful and dangerous consider what other files and sub folders would be effected
If the drive is flash/solid state Iāve seen disk failures which make files appear read-only.
can someone let me know what am i doing wrong? am i supposed to create a dump of the file sqlite db and then import? ( i am importing the db directly as mentioned in the quote)
i am creating a new pgsql DB and punching all the data from scratch now as suggested by you all (running docker on only one PC, nothing portable)
BUT
am still skeptical for data loss as i am still not aware of the backup and restore process.
any step by step guide for data backup and restore would be appreciated ( at least i can take regular backups and will be sure that i can restore it as and when required !)
Thank you for the reply, but as I am deploying pgsql docker container, need a specific advice in accordance to farmos (the entire site SHOULD look like it was before backup)
