Upgrading Drupal Sites

Drupal 8 was a major change to the codebase. Upgrading a site from 7 or older is a major operation because many modules are outdated or have changed to something else. Plus, the theming layer uses twig templates now so that needs an overhaul too. Images, which can be in fields or in wysiwyg content, can be converted to Media, but that requires updating all the places where they're used.

Upgrading sites to Drupal 10 is a common project today. It's best to treat it as a new site build, and importing content into the new structure. This gives you a chance to restructure things, but you can also keep much of the existing config.

Before starting, read up on how the Drupal Migrate API works. This will come in handy when we have to edit the migration config generated by the upgrade. https://www.drupal.org/docs/upgrading-drupal

Environment

To set up the upgrade, you want to have a D10 development site, with a copy of the D7 site that can be accessed by the dev server.

Migration modules

Migrate
Core migration API.

Migrate Drupal
Handles drupal upgrades from d6/7.

Migrate Upgrade
Exports migrations to config.
Difference between Migrate Drupal, Migrate Plus, Migrate Tools, Migrate Upgrade.

Media Migration
Convert files, file entity, old contrib media to media.
Converts embedded wysiwyg images to media.

Migrate Plus
Migrate API extensions.

Migrate Tools
Drush commands.

Upgrading modules

Run Upgrade Status on your current site to see what modules have a D10 upgrade or replacement. https://www.drupal.org/project/upgrade_status

Anything that is changing is likely going to need to be rebuilt. Pay special attention to modules that are incompatible with no replacement available. You will have to find another way to implement the feature if it's still needed.

Install the modules you think you'll need on the D10 site, and uninstall the incompatible modules on the D7 site. This will narrow the scope of the generated migrations on the source, and prepare the destination site with the necessary requirements.

Config and Content tags

Generated migrations are tagged with Content and Configuration. You generally want to run the config updates once to get the fields set up, then make any modifications, then update the content migration to fit the new structure.

drush mim --group=migrate_drupal_7 --tag=Content

Files

The migrate_upgrade module will use the d7_file source plugin to import managed files.

The --legacy-root flag sets the site root for the old site, which can be a path to a copy of the site, or the site url.

Check out Media Migration and Migrate File to Media modules.

If files are missing on the original site, the migration will fail. Use mim --continue-on-failure.

Fields

d7_field and d7_field_instance migrations.

Once these run, you can export your site config and modify from there.

Running them again will throw errors if the fields already exist, and safely ignore these warnings.

The errors to look out for are missing destinations. For example, Corresponding Entity References doesn't have an upgrade path and will throw errors. But you can reinstall the module and configure it on the new site manually.

Here's an example of skipping entities or fields:

  entity_type:
    -
      plugin: get
      source: entity_type
    -
      plugin: static_map
      map:
        file: media
      bypass: true
    -
      plugin: skip_on_value
      method: row
      value:
        - config_pages
        - cer

  skip_fields:
    -
      plugin: concat
      source:
        - entity_type
        - field_name
      delimiter: ':'
    -
      plugin: skip_on_value
      method: row
      value:
        - node:field_photo_credit
        - node:field_timeout

upgrade_d7_view_modes and upgrade_d7_field_formatter_settings also need to be updated with skipped entities and bundles.

Content types

upgrade_d7_node_types

Content

After the fields are mapped and the content migrations updated, you can run them.

Preserve IDs

When migrations are generated, they map existing content IDs to the imported value. This is great because all references can use their current values and any node/ID urls will show the right content.

However, the problem with preserved IDs is that they will collide with any content added in development.

You can remove the nid field from migrations and it will auto increment entity IDs on import. However, all reference fields will need to use migration lookup to get the new values.

Incremental updates

Node complete migrations default to importing the existing node IDs into the new content. This works great if you do a migration after installing, and only create new content after that.

However, on most upgrade projects, the old site is getting content updates while the new one is in development. We need a way to get new and updated content without disrupting any changes made on the new site.

Generate migrations with incremental updates: https://www.drupal.org/node/3105503

The default migration imports node (nid) and revision IDs (vid) as:

source:
  plugin: d7_node_complete
  node_type: blog
process:
  nid:
    -
      plugin: get
      source: tnid
  vid:
    -
      plugin: get
      source: vid

To allow the IDs to update automatically, delete the lines for nid and vid and change the source plugin to d7_node. Now node IDs will be sequential as content is imported. The old IDs are mapped to the new ones in the migration map.

Content that references an old ID will need to use migration_lookup to get the new ID.

#Import existing IDs
field_artwork:
  -
    plugin: get
    source: field_artwork

#Lookup imported ID
field_artist:
  -
    plugin: migration_lookup
    migration: upgrade_d7_node_complete_artist
    source: field_artist/0/target_id

Node reference field

D7 sites used Entity reference and Node reference fields.

Here is the generated migration config.

field_artist:
  -
    plugin: sub_process
    source: field_artist
    process:
      target_id: nid

To convert to incremental updates:

field_artist:
  -
    plugin: sub_process
    source: field_artist
    process:
      nid: nid
  -
    plugin: migration_lookup
    migration: upgrade_d7_node_complete_artist
  -
    plugin: node_complete_node_lookup

Text Formats

If you install the standard drupal profile, you will get some default text formats. These may be different that the origin site, so you will have to decide what you want to keep. If you want to make improvements during the migration, you can set up the text formats you want to have, and map them in the migration.

  body/0/value:
    -
      plugin: get
      source: body/0/value
  body/0/format:
    -
      plugin: static_map
      source: body/0/format
      map:
        filtered_html: full_html
      bypass: true

Use the HTML Tag Usage module to check if your editor config supports the tags in your content.

After generating config and copying the files to your custom module, Media Migration will still generate derivative migrations and insert dependencies for them into your migrations.

I wasn't able to find a good solution for this, other than returning [] on line 205 in FileEntityDeriver.

Tips

• Put a breakpoint on the line shown in the migration error to find out what failed.

Media

Using Media Migration module.

Install with the initial migration modules before running drush migrate:upgrade to generate migration config.

After the config is imported you can remove the config migrations, but the media entity migrations need to remove these lines or else it will try to reference them:

  source_field_name: field_media_image
  source_field_migration_id: 'd7_file_entity_source_field_config:image'
  media_migration_original_id: 'd7_file_entity:image:public'

I changed the Default media type on the old site to import into Document on the new site. In upgrade_d7_file_entity_default_public, change the field names in the processors from field_media_default to field_media_document.

Content model changes

I have fields on a content type that I would like to move into a referenced entity. First I create that entity and set up the fields. Then create a migration based on a copy of the original, and edit it to map fields into the new type.

Steps

1. Set up a local copy of the site to migrate from.
2. Disable incompatible modules and delete any content types, users, anything else you don't want.
   1. Leave modules enabled if they are used for content or config storage. You will have to write a custom migration for them if there is no provided alternative.
3. Install the site with all the modules you need, including migrate upgrade and media migration.
   1. If you miss something and get an error, try installing it and rerunning the migration.
4. Run the migrate drush command with --configure, and export the site config files.
   1. https://www.drupal.org/project/migrate_upgrade
5. If you encounter a field that does not migrate, like old date formats, set up a replacement field and customize the content import (IE upgrade_d7_node_complete_article).
   1. You will need to tell the failed migration to skip that field.
6. The actions migration may warn about some modules like metatags, pathauto, and views bulk operations.