farmOs v1 migrate v2 - issue with taxonomy

Hi,
I’m trying to run the migration from FarmOS v1 to v2 and I’m facing this issue when runnning

drush farm_migrate:import

It gives me:

 [notice] Processed 12 items (0 created, 0 updated, 12 failed, 0 ignored) - done with 'farm_migrate_taxonomy_plant_type'

I tried to follow https://farmos.org/hosting/migration/ and https://farmos.org/hosting/install/ very closely.
I think I have not created custom entries in v1 intentionally. However if I check the table I can see some custom ones that must have been created when I crated assets and logs.
Can you help me solve this?

I hope it is ok that I created a separate thread on this forum. If not, I can also post this an and additional comment of existing migration issue posts.

1 Like

Hi @PDorfFarm - sure let’s see if we can figure out what’s wrong…

This command will print out any messages related to the migration that failed:

drush migrate:messages farm_migrate_taxonomy_plant_type

Can you paste the output of that here?

I think I have not created custom entries in v1 intentionally. However if I check the table I can see some custom ones that must have been created when I crated assets and logs.

Can you describe what you’re seeing? Or post screenshots/copy+pastes?

Thank you for taking this up. This is the result

root@d7b4acb2949e:/opt/drupal# drush migrate:messages farm_migrate_taxonomy_plant_type
 -------------- ------------------- ------- ----------------------------------- 
  Source ID(s)   Destination ID(s)   Level   Message                            
 -------------- ------------------- ------- ----------------------------------- 
  11                                 1       [taxonomy_term: 11]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  23                                 1       [taxonomy_term: 23]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  25                                 1       [taxonomy_term: 25]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  31                                 1       [taxonomy_term: 31]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  66                                 1       [taxonomy_term: 66]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  67                                 1       [taxonomy_term: 67]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  69                                 1       [taxonomy_term: 69]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  74                                 1       [taxonomy_term: 74]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  75                                 1       [taxonomy_term: 75]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  77                                 1       [taxonomy_term: 77]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  78                                 1       [taxonomy_term: 78]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             
  82                                 1       [taxonomy_term: 82]:               
                                             langcode.0=The value you selected  
                                             is not a valid choice.             

Seems like it’s language-related…
In my farmos_v1 installation I have installed 2 additional modules (locale and l10_update) for language settings as I mentioned in this post farmOS translation (i18n) - #4 by PDorfFarm

Now I have installed the module farm_l10n in my farmos_v2 installation (it seemed like the most similar module compared to the 2 in farmos_v1). However migration still fails with the same error message.

Update:
After adding German as additional language, I was no able to overcome this error. However now there error:

48/60 [======================>-----]  80% [error]  Migration failed with source plugin exception: Log 187 has both area references and movement area references. See https://github.com/farmOS/farmOS/blob/2.x/docs/hosting/migration.md#movement-logs in /opt/drupal/web/profiles/farm/modules/core/migrate/src/Plugin/migrate/source/d7/FarmLog.php line 138 

investigating further… :-/

Ah that is probably the reason.

I haven’t tested migrations with v1 localization modules enabled since it was not “officially” supported in v1.

I think the best thing to do is attempt the migration without any localization modules installed on EITHER side (v1 or v2). You will need to translate v2 from scratch regardless, the translations do not get migrated. Translations are downloaded from localize.drupal.org.

You will need to disable AND uninstall those modules in your v1 instance (FYI disabling and uninstalling are two separate actions in v1, whereas in v2 there is only “uninstall”). I would recommend taking a backup of your v1 instance before you do so, just to be safe.

Then, try to perform the migration again from scratch. Completely delete your v2 instance and database and start fresh, to be sure that there are no other variables lingering.

Log 187 has both area references and movement area references. See https://github.com/farmOS/farmOS/blob/2.x/docs/hosting/migration.md#movement-logs

This is a separate issue, which is described in detail here: Migrating from farmOS v1 | farmOS

You will need to manually edit that log (187) on your v1 instance and clean it up BEFORE you run the migration. If any of the documentation doesn’t make sense on that particular issue feel free to ask questions here.

Good luck!

1 Like

I had to fix the issues as described in Migrating from farmOS 1.x to 2.x | farmOS .
After that I followed your advice regarding the modules and migration completed successfully! I will check completeness of the migrated data in the following days.

Thank you very much @mstenta ! The speed and the quality of support in this forum is really amazing!

1 Like

That’s great to hear @PDorfFarm!

And regarding v2 translations, you may be interested in: farmOS 2.0 Translation / Localization (l10n) Initiative

1 Like

I just wanted to share with you what I found missing after checking content of my v2 instance and comparing to v1 instance:

  • Log entries of type ‘purchase’, ‘sale’ and ‘soiltest’
  • locations of type ‘plantation’; I think those came from a module called ‘forest’ that I tried out once
2 Likes

@PDorfFarm Ah yea you would need to install additional modules in order to migrate that data.

Both provide migrations for their data into v2.

There is a slight chance that migrating these now (after you’ve already done your other migrations) could lead to asset/log ID conflicts/overwrites. So if you haven’t already started adding/editing records in your v2 I would recommend redoing the migrations from the beginning with those modules installed.

A little more detail: the migration logic tries to maintain asset and log IDs that were used in v1. This means that if you have an asset/log in v2 with the same ID, it will essentially be overwritten by the migration (and maybe only partially, depending on what data is being migrated) which can lead to strange results. This is more of a consideration with assets than logs (although it can affect both, depending on the circumstances), because “Areas” were migrated into “Assets” in v2. This means that every area from v1 gets a new asset ID in v2. This is why the area migration MUST run after all the asset migrations - to ensure that all the old assets have first pick of asset IDs. If you run an asset migration AFTER the area migrations, and an area is granted one of the same IDs that your later asset migration needs, it will get merged into that record.

Logs can be similarly affected, but it’s probably only going to happen if you started adding new logs after the migration, and then tried migrating some old ones again (and the IDs end up conflicting - so lots of things need to line up for this issue to take place).

Hope that all makes sense! If you find yourself in a bind and are worried about overwriting data, let me know and maybe we can think through options to work around it…

1 Like

The other thing to consider with assets: if you didn’t migrate your forest assets during the initial migration, but DID migrate logs that referenced them, then those logs may be orphaned already. So ideally you would re-run the migration from scratch with ALL modules installed that you need.

Thank you for the explanation. In my case it’s only 8 log entries that are affected. I think I will just add them manually - or maybe I do not require them anymore at all.

How would I install these modules? I can’t find neither of them in administration → Extend in my v2 instance.

1 Like

Download them into your web/sites/all/modules directory, then they will appear in “Extend”.

1 Like