migrating to drupal 8

a brief historydrupal.org/upgrade

update vs. upgradewhat types drupal “renewing” we have?

drupal update

drupal update minor version update

7.24 > 7.25

drupal upgrade

drupal update minor version update

7.24 > 7.25

drupal upgrade

drupal update minor version update

7.24 > 7.25

major version upgrade7.x > 8.x

drupal upgrade

drupal update minor version update

7.24 > 7.25

major version upgrade7.x > 8.x

the migrate module“in service” since 2009

drupal.org/project/migrate

the migrate module

• How it worked?

• Migrations = classes extending Migration.

• Main elements: source, destination, map, mappings, “hooks” (prepareRow, prepare, complete, createStub, etc).

• Each migration has to extend the Migration class or one of its successors.

migrate in D8 core

disclaimer

disclaimer

• the migrate system is under heavy development right now.

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

• not all the current work is pushed to 8.x.

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

• not all the current work is pushed to 8.x.

• The work is in the sandbox athttps://drupal.org/sandbox/chx/2105305

creditsKároly Négyesi (chx)Mike Ryan (mikeryan)

Moshe Weitzman (moshe weitzman)Ben Dougherty (benjy)

drupal 8 migration

note

note

• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to

note

• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to

• The new interaction allows for really nice and powerful migrations but at the same time we are most definitely not reinventing wheel.

structure/modules

Migratecore/modules/migrate/

Migratecore/modules/migrate/

• provides general API for all migrations

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

• provides a plugin manager for manipulation on migration plugins.

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

• provides a plugin manager for manipulation on migration plugins.

• provides the migrate configurable (configuration entity type).

Migrate Drupalcore/modules/migrate_drupal/

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

•Content definitions (vocabularies, node types, etc)

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

•Content definitions (vocabularies, node types, etc)

•Content (noded, terms, users, etc).

understanding migrations

migrations are configurables

small peekinto configurables

what is a configurable?

• “Configurables” are migrations entities.

what is a configurable?

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

\Drupal\Core\Config\ConfigEntityBase

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

\Drupal\Core\Config\ConfigEntityBase

• A configurable is the way Drupal 8 stores the configuration of a specific functionality. E.g. the the definition of a node type is stored in a configuration entity of type “node_type”.

• Configuration entity types are annotated classes, meaning that the object meta information is stored in annotation rather than in info hooks - as it was in Drupal <= 7.

• Imagine configurables as entities storing their data in config YAML files rather than DB.

• The “fields” of a configurable are the public properties exposed by the configurable object.

what is a configurable?

what is a configurable?

how it’s stored?example

migration pluginsparts implemented by specific migrations

source plugins

• plugins returning information and data from the source of migration.

• usually: the list of fields, the source iterator (used retrieve data from source).

• each migration should configure a source.

destination plugins

• are handling data at the destination: import, rollback.

• different plugins for different destination components: entity, config, etc.

• are defined in the base module (migrate) as destination is always drupal 8 but if necessary it can be extended.

• each migration should specify a destination.

id map plugin

• plugins of this type are handling and storing the relation between primary IDs of source and destination.

• without this, rollback and continuous migrations are impossible.

• in 99% of the cases you’ll use the sql id map plugin (Sql) that keeps the map of each migration in a table.

• table name migrate_map_MIGRATION_ID

processors

• plugins that are performing small but very specialized operations against values to be migrated.

• Some simple examples: DefaultValue, Concat, etc.

• The most important interface method: transform().

the anatomy of a migrationmigrating user roles from a dupal 6 site

creating the config fileconfig/migrate.migration.d6_user_role.yml

relative to core/modules/migrate_drupal

config .yml file content

config .yml file content• id: same as the last part of filename (d6_user_role)

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

• process: describe the list of processors to be applied per destination field.

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

• process: describe the list of processors to be applied per destination field.

• destination: destination configuration, usually the destination plugin.

id

id• this is the configurable unique id.

id• this is the configurable unique id.

• it must be exactly as the same as the last part of filename: d6_user_role.

id• this is the configurable unique id.

• it must be exactly as the same as the last part of filename: d6_user_role.

id: d6_user_role

sourceIds

sourceIds• look in D6 schema to find the role primary ID.

sourceIds• look in D6 schema to find the role primary ID.

• lines 107 - 115 of drupal/modules/user/user.install.

sourceIds• look in D6 schema to find the role primary ID.

• lines 107 - 115 of drupal/modules/user/user.install.

$schema['role'] = array( 'description' => 'Stores user roles.',  'fields' => array(   'rid' => array(    'type' => 'serial',    'unsigned' => TRUE,    'not null' => TRUE,    'description' => 'Primary Key: Unique role id.',

sourceIds

sourceIds• use TypedData identifiers for data type.

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

sourceIds:    rid:    type: integer

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

sourceIds:    rid:    type: integer

Note: sourceIds will be removed in the near future and the source plugin will set also the primary id.

source

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

• add the source plugin id in the configuration .yml file.

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

• add the source plugin id in the configuration .yml file.

source:  plugin: drupal6_user_role

process

process• process keys are destination “fields”.

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

• let’s see how it looks! (code).

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

• let’s see how it looks! (code).

process:  id:  label:  weight:  permissions:

destination

destination

• should point to the destination plugin.

destination

• should point to the destination plugin.

• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.

destination

• should point to the destination plugin.

• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.

destination: plugin: entity entity_type: user_role

running a migration

• via drush

• There will be a brief UI implemented in core (to come!)

final notes

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.

• Similarly, Drupal 4.x and Drupal 5.x sites are able to migrate using this same approach.

Questions?Thank you.