Unraid install, updated, errors and warnings

[Mick]

Transaction isolation level says my data_stream_basic does not have a primary key

Trusted host settings not enabled. I’ve never altered the settings before and don’t know where to start. This is the first time I’m seeing this warning but I might have missed it. I’m still very new to this program and have no experience with programming.

Warning for output buffering not enabled. Same deal, hadn’t noticed this warning before. I was able to download a PHP reading program and looking through my settings.php file but can’t see anywhere I could change this from a false to a true.

I’m also still getting error messages that files and photos cannot be uploaded. I haven’t yet figured out how to get this working.

My installation is “latest” from the Unraid Community Apps. My database is Mariadb latest as well.

Any help is appreciated. Thanks!

[mstenta]

Hi @Mick - welcome to the forum!

I hadn’t heard of “Unraid” until now, but I think this is what you’re referring to?

Community Apps

I also found these related topics on the Unraid forum:

• [Docker Request] FarmOS - Docker Engine - Unraid
• FarmOS docker - Docker Engine - Unraid
• [Support] C4artZ Templates - Docker Containers - Unraid

These are not maintained by the farmOS community officially, but it looks like @ c4ndel4 (who is a farmOS community member here on this forum ) maintains them. Perhaps they can chime in and help to debug these issues. (edit: I misread the username - sorry @c4ndel4! )

I’ll take a first pass at replying to each myself in case that helps…

But first, can you tell us exactly what version of farmOS you are running? Is it the latest (3.0.0)? Or an earlier version (eg: 3.0.0-beta3, 2.2.1, etc)? You can find out at /admin/reports/status

Also: I’m worried that you are not running database updates between version changes if you are using the latest Docker tag and updating automatically. Updates DO require some manual steps, which is why we always recommend pinning the farmOS version and undertaking updates intentionally with BACKUPS EVERY TIME!

See Updating farmOS | farmOS

If this information is not clear to Unraid users it is VERY IMPORTANT to make it clear. I would NOT recommend providing latest images of farmOS for this reason. Happy to discuss more but I’m worried that it could lead to broken systems or data loss, and users will think that it’s a farmOS issue when it is actually incorrect expectations set by the Unraid app maintainer.

Transaction isolation level says my data_stream_basic does not have a primary key

It’s true the data_stream_basic table does not have a primary key (farmOS/modules/core/data_stream/data_stream.install at 3.x · farmOS/farmOS · GitHub). If this is causing an error we should discuss options. The reason it doesn’t have one now is to keep the table minimal. We probably can’t simply create a primary key as a combination of the existing id and timestamp fields, as that would mean it’s not possible to POST two data points with the same timestamp. Maybe that’s reasonable, but at this point there may be existing data. It requires discussion. Curious @paul121’s thoughts…

FWIW I found a similar Drupal core issue here: https://www.drupal.org/project/drupal/issues/3365945

Trusted host settings not enabled.

This is a Drupal-level environment configuration that will be specific to your setup (Unraid), so it isn’t a bug with farmOS. A solution will need to be figured out in the context of Unraid specifically (probably in the Unraid forum).

Here is the official Drupal documentation: https://www.drupal.org/docs/getting-started/installing-drupal/trusted-host-settings

Warning for output buffering not enabled.

I was seeing this error on 3.0.0-beta3, but not on 3.0.0, so I think this may be fixed upstream, but please confirm what version you are running so we can check that.

Here is the relevant Drupal change record: https://www.drupal.org/node/3298550

I’m also still getting error messages that files and photos cannot be uploaded. I haven’t yet figured out how to get this working.

Our official install docs have guidance here: Installing farmOS | farmOS

It requires creating a directory with the right permissions and adding a single line to your setting.php file.

My installation is “latest” from the Unraid Community Apps.

Once gain, I do NOT recommend running the latest farmOS Docker image. Only because it runs the risk of updating automatically and then drush updb / update.php updates are not run… leaving your installation in a partially-updated state which is dangerous.

Hope all that info helps @Mick! I love the idea of systems like Unraid making it easier to run farmOS! But it’s important that updates are run properly. Happy to help figure out a better process if we can…

[mstenta]

Once gain, I do NOT recommend running the latest farmOS Docker image.

This reminded me… @Symbioquine and I had a bit of back and forth on this here: https://www.drupal.org/project/farm/issues/3162807#comment-14011979

Tl;dr: I proposed removing the latest tag entirely, but we came to the conclusion that there are some good reasons to keep it. Maybe we should revisit that discussion in a new thread.

[c4ndel4]

I’m sorry but I’m not the user responsible for that repositories, who has the same 2 initial letters but the rest is different : @C4artZ (and looks like is not a farmOS community member)

I also didn’t even heard about “unraid”, but seems a nice place to distribute broken and malicious software.

And after reading the message I’m not sure that @Mick is an actual human being. Sorry if you actually are.

[mstenta]

Oh whoops! Sorry @c4ndel4 that was my bad. It was very early in the morning here.

(I edited my comment above to remove/amend any references to you.)

[Mick]

Hi again,
Yep human here. I have some metal in my body but not enough to be called android yet.

I don’t have automatic updates on because there are a few other unraid distributed docker apps that have special update requirements. I’m rather new (been messing with self-hosting for only 1 year), so I’m not yet versed on using docker or the nuances of docker inside unraid so please excuse if I do dumb things. Unraid has some great tools for backups and maintaining the whole docker system that allow me to not need to setup every app individually, which is why I prefer to use it. Hopefully once I make it through getting farmos running smoothly I’ll be able to help others using unraid as their OS. Also, installing the farmOS instance on Unraid, other than the small number of issues I’m experiencing, was as simple as clicking install making it easier for me.

To c4ndel4’s point about malicious software, I’m not experienced enough to know any better, but the community doesn’t seem to have a problem and I stick to well known app developers. So basically, I’m just crossing my fingers.

Here are the general system information from the status report:
Drupal 10.2.1
Web server Apache 2.4.57 Debian
PHP 8.2.14 Mem Limit 256M
Database MariaDB 10.11.5
farmOS (farm-3.0.0)

For installing drupal trusted host settings. I used the mysql commands. The database indicated they were successful. I re-ran the Cron run on the status report page and the error was still present.

The file system does appear to be different for unraid.
FarmOS app points to one file location in unraid:
user/appdata/farmos
within /farmos I have
/default
development.services.yml
example.settings.local.php
example.sites.php

within /default I have
/files
default.services.yml
default.settings.php
settings.php

within /files I have
/config
/css
/js
/php
.htaccess (file but I don’t know what this is)

Let me know if I need to go any deeper.

For the settings.php file, I’d need help understanding where to set the filepath for uploading files.

Thanks again in advance.

[c4ndel4]

Sorry @Mick

The only thing I can recommend is to follow the official distribution method for any software, in this case described in here: Installing farmOS | farmOS

If you are not experienced in self hosting apps, the previous rule is even more important. You are not going to get official support and maybe you end with malicious software installed.

If you have problems that you can’t solve, maybe that simple click install is not that simple.

Sorry again

[mstenta]

@Mick

Yep human here. I have some metal in my body but not enough to be called android yet.

Ha! Everyone is welcome here (Androids included) as long as they converse in good faith.

@Mick

I don’t have automatic updates on because there are a few other unraid distributed docker apps that have special update requirements.

That’s good. Is it possible to specify versions in Unraid, instead of using latest? I find that to be a better habit to be in because it forces you to be intentional about updates and follow the steps outlined here: Updating farmOS | farmOS

It’s also good to hear that Unraid provides backup options. The database is the main thing you want to create a snapshot of before updating farmOS to a new version.

Question: When you go to https://[your-farmOS-domain]/admin/reports/status, do you see “Database updates: Up to date”? That will indicate that update.php has been run between updates. You’ll want to make sure that’s always “Up to date” before you use farmOS after an update.

@Mick

For installing drupal trusted host settings. I used the mysql commands.

Hmm can you help me to understand what commands you ran on MySQL?

The trusted host settings are actually just configured in settings.php - they don’t have any relationship to the database, as far as I understand it.

It’s basically a way to protect against a certain type of attack, byt defining what hosts your website is officially available under. You would add something like this to your settings.php file:

$settings['trusted_host_patterns'] = [
  '^www\.example\.com$',
];

(Where www.example.com is the URL of your farmOS instance.)

Worth noting: you can also probably ignore this warning without worrying. I think it is more important for bigger public-facing website installations of Drupal, which farmOS is not.

@Mick

The file system does appear to be different for unraid.
FarmOS app points to one file location in unraid:
user/appdata/farmos
within /farmos I have
/default

OK so it sounds like Unraid is creating a volume to persist data from the /opt/drupal/web/sites directory in the container. That’s good!

Your settings.php file will be in default/settings.php. That’s where things like your database login credentials get stored, as well as the trusted host settings and private filesystem configuration.

@Mick

For the settings.php file, I’d need help understanding where to set the filepath for uploading files.

Try creating a directory called private in your user/appdata/farmos/default directory. This directory will need to have the correct permissions on it, which will depend somewhat on your environment. If you are familiar with running Linux commands I would try this (from a terminal in the user/appdata/farmos/default directory):

chown www-data:www-data private
chmod 0770 private

Then add this line to settings.php (at the bottom is fine):

$settings['file_private_path'] = '/opt/drupal/web/sites/default/private/files';

Finally, clear your caches by going to https://[your-farmos-domain]/admin/config/development/performance in the browser and clicking “Clear all caches”.

If it’s successful, you should see this in https://[your-farmOS-domain]/admin/config/media/file-system under “Private file system path”: /opt/drupal/web/sites/default/private/files and file uploads should work.

Important to note: In your setup there are TWO things you should be including in your backups: your database (farmOS data) and your user/appdata/farmos directory (which includes settings.php and all uploaded files).

[Mick]

@mstenta

That’s good. Is it possible to specify versions in Unraid, instead of using latest?

It’s possible, I haven’t yet figured it out, but when I do I intend to implement it as suggested.

@mstenta

Question: When you go to https://[your-farmOS-domain]/admin/reports/status, do you see “Database updates: Up to date”?

No, however I accidentally wiped my database doing the update method incorrectly, so I will note this down for next time an update is required. Thankfully up to this point there is nothing I can’t replicate. This is effectively a fresh install. I could roll back the database, but there are other apps sharing MariaDB that would lose a week’s worth of data that is far more work than rebuilding my FarmOS instance. I’ll need to also perform a fresh backup as a rule before future farmos updates.

@mstenta

Hmm can you help me to understand what commands you ran on MySQL?

I ran both of these:
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET GLOBAL tx_isolation=‘READ-COMMITTED’;
and got back;
“Query executed OK, 0 rows affected”

@mstenta

The trusted host settings are actually just configured in settings.php - they don’t have any relationship to the database, as far as I understand it.

It’s basically a way to protect against a certain type of attack, byt defining what hosts your website is officially available under. You would add something like this to your settings.php file:

$settings['trusted_host_patterns'] = [
  '^www\.example\.com

(Where www.example.com is the URL of your farmOS instance.),
];

(Where `DISCOURSE_PLACEHOLDER_4` is the URL of your farmOS instance.)

Unraid runs all the docker apps on their own IPs inside of itself. I connect to it by going to my http://local.host:PORT where PORT is the port assigned inside the server to the app. So I would put this at the bottom of the settings.php file?

I’m not currently accessing this remotely but it would be a nice to have in the future. I have other apps that use SSL certificates to port into the server, but within the server all communication is http. And I believe there is little reason to encrypt this communication unless it is required for something to work.

@mstenta

Try creating a directory called private in your user/appdata/farmos/default directory

Done and permissions set to read/write for all.

@mstenta

Then add this line to settings.php (at the bottom is fine):

$settings['file_private_path'] = '/opt/drupal/web/sites/default/private/files';

Finally, clear your caches by going to https://[your-farmos-domain]/admin/config/development/performance in the browser and clicking “Clear all caches”.

This line exists in my settings.php file

$settings[‘file_private_path’] = ‘’;

I removed this line and pasted your code in. This didn’t work. I also tried /sites/default/private/files to mimic what I saw on the file system admin page for this:
Public file system path
sites/default/files
A local file system path where public files will be stored. This directory must exist and be writable by Drupal. This directory must be relative to the Drupal installation directory and be accessible over the web. This must be changed in settings.php

There is also:
Optimized assets file system path
sites/default/files
Private file system path
/sites/default/private/files (which reflects whichever path was in the settings.php file)
Temporary directory
/tmp

I found this line:

$settings[‘file_public_base_url’] = ‘http://downloads.example.com/files’;

and changed it to:
$settings[‘file_public_base_url’] = ‘http://local.host:PORT/files’;

Now files are working but the UI is no longer working.
This is what my interface looks like (uploaded picture)

[Mick]

I restarted with a fresh settings.php file and have done the following:

Added my details to this and uncommented it -
$databases[‘default’][‘default’] = [

• ‘database’ => ‘databasename’,
• ‘username’ => ‘sqlusername’,
• ‘password’ => ‘sqlpassword’,
• ‘host’ => ‘localhost’,
• ‘port’ => ‘3306’,
• ‘driver’ => ‘mysql’,
• ‘prefix’ => ‘’,
• ‘collation’ => ‘utf8mb4_general_ci’,

Added this -
$databases[‘default’][‘default’][‘init_commands’] = [
‘isolation_level’ => ‘SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED’,
];

Uncommented and updated with my file structure -
$settings[‘file_assets_path’] = ‘sites/default/files’;

To fix the UI issue, I turned CSS cache to off.

Files and pictures can now be uploaded. and the only error / warnings I still have are these:

Transaction isolation level
Trusted Host Settings
Output Buffering

I’ll play around with it likely this weekend and report back any successes.

[mstenta]

Great! I’m not familiar with the transaction isolation level error, but I haven’t used MySQL in a while (PostgreSQL is my go-to nowadays).

You can probably just ignore the trusted host settings if you’re only using farmOS locally.

I don’t think you should need to mess with $settings[‘file_assets_path’] or $settings[‘file_public_base_url’] at all… those are for the “public” files (which are where auto-generated CSS/JS files get stored). You only need to add $settings[‘file_private_path’] so that you can upload “private” files (images/files on assets/logs). I bet the problem is permissions on your “public” files directory. It needs to be writable by Apache (www-data) in the same way that the private directory is. You could try running the same chown + chmod commands on your sites/default/files directory, and remove the extra things you did in settings.php. And clear your cache again to make things take affect. Then you should be able to turn JS/CSS aggregation back on. Hopefully that’s it!

Thanks for sharing all this @Mick! It will no doubt help someone else in the future.

[Mick]

Hey, back again.

I setup a new farmos instance using postgresql15 rather than mariadb and out of the gate, there are far less issues. However, I am still getting some issues that might be easy to fix but are currently outside my grasp.

Output buffering. The guides refer to an “ini” file, but I can’t find this file in my installation. There are some references to adding some code to the php file, which I can open and review but there are no commented coding for turning on buffering. If anyone has a script I could copy/paste for testing, it would be greatly appreciated.

Also, my installation webUI keeps flip flopping between the intended UI that is easily navigatable and the picture I posted on Jan10 where it’s just a wall of text. Would this be fixed by the buffering update?

Thank in advance.

Cheers,
Mick

[mstenta]

@Mick

I setup a new farmos instance using postgresql15 rather than mariadb and out of the gate, there are far less issues.

Great to hear @Mick!

@Mick

Output buffering. The guides refer to an “ini” file, but I can’t find this file in my installation.

Yea this has been coming up recently, it’s not your fault, and I think you can safely ignore it for now… we may try to fix it in farmOS Docker images so that the error goes away.

Here are relevant #farmOS chat logs that describe it in more detail and offer a solution if you want:

https://irc.farmos.org/bot/log/farmOS/2024-01-16#T91858

Tl;dr: It’s a recent change to Drupal that adds a warning if output buffering is not enabled, but in the case of farmOS it probably doesn’t hurt anything.

The official PHP Docker image ships with an optional “production” INI configuration file that can be dropped in which includes output buffering. There are various ways to do this in your environment (eg: making a copy of that file outside of your container and using Docker to bind-mount it into the correct place within the container, which is what @shplorf did to fix it (see chat logs).

The way I fix it is by extending the farmOS image with my own Dockerfile that has this line:

RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"

(As described in the official PHP Docker image documentation: Docker)

@Mick

Also, my installation webUI keeps flip flopping between the intended UI that is easily navigatable and the picture I posted on Jan10 where it’s just a wall of text. Would this be fixed by the buffering update?

Hmm, could you share your settings.php file (with the database credentials redacted) as a Gist or Pastebin?

It’s odd that it would be flip flopping back and forth between working and not working.

Generally this issue occurs when the PUBLIC filesystem is not properly configured AND JS/CSS aggregation is turned on. You could try turning off JS/CSS aggregation at /admin/config/development/performance to see if that consistently solves the problem for you.

That’s not a good long term solution, though. It will reduce performance. The way it’s working is: when Drupal builds a page, it figures out all the different JS/CSS files that are needed for that page and loads them. If JS/CSS aggregation is turned on, then it will concatenate all of the files together into a single JS/CSS file and save that in the public files directory (different from the private files directory, which is used for your file uploads). If the public filesystem is misconfigured, or the directories are not writable by Drupal, then you will experience this issue.

Maybe that helps in your debugging… happy to review your settings.php to see if anything jumps out… I recall that you messed with PUBLIC filesystem settings earlier, and I said:

@mstenta

I don’t think you should need to mess with $settings[‘file_assets_path’] or $settings[‘file_public_base_url’] at all… those are for the “public” files (which are where auto-generated CSS/JS files get stored).

Generally speaking, as long as your public files directory has the right permissions you don’t need to do anything else. I’ve never had to add any settings.php configuration for PUBLIC files… only PRIVATE.

[Mick]

Here’s the pastebin link: farmos test unraid - Pastebin.com

In the new instance I did not mess with the assets or public filepaths and went straight to the private file path. That worked for the postgresql database but not the mariadb database for me. This might have something to do with using unraid OS.

[mstenta]

The only unusual thing I see in your settings.php is:

$settings['file_private_path'] = 'sites/default/files/farm';

Two things:

1. This should be an absolute path, eg: /opt/drupal/web/sites/default/files/farm
2. sites/default/files is typically the public filesystem path. Using the farm subdirectory of that for your private files might work, but it might be better to be more explicit by calling it private instead of farm, or by putting it next to the files directory like sites/default/private (that’s what I do).

I’m not sure if this is the cause of your problems, but maybe worth trying just in case. Clear your caches after changing the file_private_path value.

[Mick]

I may be wrong so this is just how I understand it. Unraid uses shares which are mappings that allow the files to be moved around by the OS and put on different disks, backed up on the array, and in the event of a disc failure, rebuilt on the replacement disk. The “sites” is mapped to my farmos_test appdata location. Any call to the filepath will be sent to that location. I did some digging in the files and wasn’t able to find any opt, drupal, web file structure anywhere, so this might be part of the mapping.

After changing to this on the file path, $settings[‘file_private_path’] = ‘sites/default/private’;

I’m now getting this error. Drupal\Core\File\Exception\InvalidStreamWrapperException: in Drupal\Core\File\FileUrlGenerator->doGenerateString() (line 106 of core/lib/Drupal/Core/File/FileUrlGenerator.php).

I did move the files as well out of the old files/farm pathway and then ran the clear cache. Ran the cron report as well and no new errors.

[mstenta]

@Mick That makes sense…

Unraid uses shares which are mappings that allow the files to be moved around by the OS and put on different disks

These are probably “Docker volumes” (Volumes | Docker Docs) which are the standard way to “persist” data outside of a Docker container (which can then be destroyed/rebuilt in a “stateless” way).

In farmOS, the sites directory is where most “state” gets stored (settings.php + uploaded files), so that’s the main volume you need in your Docker container. From what I’ve seen, the Unraid farmOS app is set up correctly to make that directory a volume.

I did some digging in the files and wasn’t able to find any opt, drupal, web file structure anywhere

Yea this is a bit confusing, but basically a Docker container has it’s own entire Linux directory structure inside the container, which is different from any directory structure outside of the container (eg: your laptop/server filesystem). When a “volume” is mounted into a container it’s basically saying “take this local directory from my laptop and map it to this directory inside the container, so that when the container makes any changes to the files inside of it, those changes are happening in my local directory”. In the case of farmOS, the directory inside the container is /opt/drupal/web/sites. So your Unraid setup is mapping your “farmos_test appdata location” to that directory inside the container.

Drupal is running inside the container, and from Drupal’s perspective it doesn’t know it’s in a container!

All Drupal knows is that it’s installed in a Linux filesystem in /opt/drupal (which is in the container).

So… when you are setting the “absolute path” of the private files directory in settings.php, it needs to be the absolute path inside the container, which is /opt/drupal/web/sites/default/private.

@Mick

After changing to this on the file path, $settings[‘file_private_path’] = ‘sites/default/private’;

I’m now getting this error. Drupal\Core\File\Exception\InvalidStreamWrapperException: in Drupal\Core\File\FileUrlGenerator->doGenerateString() (line 106 of core/lib/Drupal/Core/File/FileUrlGenerator.php).

First create a directory called private next to files in your sites/default directory and make sure both of those directories are writable by Apache (www-data). And note: they need to be readable/writable by the www-data user inside the container… which may be different than any www-data user outside the container! Docker containers are tricky magic! But once you understand that they are essentially a self-container Linux box with their own root filesystem and users it start to “click” how awesome they are!

In the end, what you want are the following directories:

• sites/default
• sites/default/files (for “public files” like aggregated CSS/JS)
• sites/default/private (for “private files” like uploaded photos/PDFs)

And you want a line in settings.php to configure the private directory:

$settings['file_private_path'] = '/opt/drupal/web/sites/default/private';

(And clear the cache whenever you change that line!)

That’s how I would recommend setting things up. I use that pattern and it’s always worked well for me. Good luck!

[Mick]

Followed this up with removal of the equipment from the postgresql database. After clearing the cache the error went away and the equipment is gone. Not an ideal solution but ok for testing.

Changed php file to $settings[‘file_private_path’] = ‘/opt/drupal/web/sites/default/private’;

Cleared cache. Made sure all file structures have read/write access.

Went to test uploads and it is now not allowing uploads again. I then changed the “default” folder to allow owner to read/write, where previously, the folder was read only for all permission levels, this also, did not fix the issue.

What might have caused my sites/default/files/farm settings to previously allow uploads where it is now broken?

[mstenta]

Oh I’m sorry I mispoke in my last comment! The settings.php line should be:

$settings['file_private_path'] = '/opt/drupal/web/sites/default/private/files';

Notice the extra files directory at the end. That might be important. Drupal will create that directory itself, if it can, I believe.

Clear your cache after making that change.

As for your directory permissions, can you try making them wide open (0777 / read+write+execute for all users), just to see if that fixes the issue for you? If it does, then we at least know that it’s a permission issue, and we can figure out what the right owner/group/mode needs to be after that.

[mstenta]

@Mick

What might have caused my sites/default/files/farm settings to previously allow uploads where it is now broken?

Just to answer this specifically: I think previously you had your private directory inside your public directory, and your public directory was writable, so therefore your private directory was writable.

And now I think the new directory is not writable (at least that’s the only guess I have that makes sense, hence the suggestion to try 0777 perms in my last comment to confirm that).

It’s fine to have your private directory inside your public directory if you want. I only recommended the separate files + private directories because I’m trying to test a pattern that I know works. But in reality there are a lot of different ways to set this up… just trying to narrow things down in an effort to get to the bottom of this.