Migrating to Drupal 8

97
migrating to drupal 8

Transcript of Migrating to Drupal 8

Page 1: Migrating to Drupal 8

migrating to drupal 8

Page 2: Migrating to Drupal 8

a brief historydrupal.org/upgrade

Page 3: Migrating to Drupal 8

update vs. upgradewhat types drupal “renewing” we have?

Page 4: Migrating to Drupal 8

drupal update

Page 5: Migrating to Drupal 8

drupal update minor version update

7.24 > 7.25

Page 6: Migrating to Drupal 8

drupal upgrade

drupal update minor version update

7.24 > 7.25

Page 7: Migrating to Drupal 8

drupal upgrade

drupal update minor version update

7.24 > 7.25

major version upgrade7.x > 8.x

Page 8: Migrating to Drupal 8

drupal upgrade

drupal update minor version update

7.24 > 7.25

major version upgrade7.x > 8.x

Page 9: Migrating to Drupal 8

the migrate module“in service” since 2009

drupal.org/project/migrate

Page 10: Migrating to Drupal 8

the migrate module

• How it worked?

• Migrations = classes extending Migration.

• Main elements: source, destination, map, mappings, “hooks” (prepareRow, prepare, complete, createStub, etc).

• Each migration has to extend the Migration class or one of its successors.

Page 11: Migrating to Drupal 8
Page 12: Migrating to Drupal 8

migrate in D8 core

Page 13: Migrating to Drupal 8

disclaimer

Page 14: Migrating to Drupal 8

disclaimer

• the migrate system is under heavy development right now.

Page 15: Migrating to Drupal 8

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

Page 16: Migrating to Drupal 8

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

• not all the current work is pushed to 8.x.

Page 17: Migrating to Drupal 8

disclaimer

• the migrate system is under heavy development right now.

• some of the features or APIs may change in the future

• not all the current work is pushed to 8.x.

• The work is in the sandbox athttps://drupal.org/sandbox/chx/2105305

Page 18: Migrating to Drupal 8

creditsKároly Négyesi (chx)Mike Ryan (mikeryan)

Moshe Weitzman (moshe weitzman)Ben Dougherty (benjy)

Page 19: Migrating to Drupal 8

drupal 8 migration

Page 20: Migrating to Drupal 8

note

Page 21: Migrating to Drupal 8

note

• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to

Page 22: Migrating to Drupal 8

note

• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to

• The new interaction allows for really nice and powerful migrations but at the same time we are most definitely not reinventing wheel.

Page 23: Migrating to Drupal 8

structure/modules

Page 24: Migrating to Drupal 8

Migratecore/modules/migrate/

Page 25: Migrating to Drupal 8

Migratecore/modules/migrate/

• provides general API for all migrations

Page 26: Migrating to Drupal 8

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

Page 27: Migrating to Drupal 8

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

• provides a plugin manager for manipulation on migration plugins.

Page 28: Migrating to Drupal 8

Migratecore/modules/migrate/

• provides general API for all migrations

• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).

• provides a plugin manager for manipulation on migration plugins.

• provides the migrate configurable (configuration entity type).

Page 29: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

Page 30: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

Page 31: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

Page 32: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

Page 33: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

Page 34: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

Page 35: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

•Content definitions (vocabularies, node types, etc)

Page 36: Migrating to Drupal 8

Migrate Drupalcore/modules/migrate_drupal/

• the first module using the new Migrate API.

• kind of migrate_d2d successor.

•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.

•Defines migrations for all system components:

•Drupal 6 settings (site name, slogan, roles, etc)

•Content definitions (vocabularies, node types, etc)

•Content (noded, terms, users, etc).

Page 37: Migrating to Drupal 8

understanding migrations

Page 38: Migrating to Drupal 8

migrations are configurables

Page 39: Migrating to Drupal 8

small peekinto configurables

Page 40: Migrating to Drupal 8

what is a configurable?

Page 41: Migrating to Drupal 8

• “Configurables” are migrations entities.

what is a configurable?

Page 42: Migrating to Drupal 8

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

Page 43: Migrating to Drupal 8

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

Page 44: Migrating to Drupal 8

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

Page 45: Migrating to Drupal 8

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

\Drupal\Core\Config\ConfigEntityBase

Page 46: Migrating to Drupal 8

• “Configurables” are migrations entities.

• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.

what is a configurable?

\Drupal\Core\Entity\Entity

\Drupal\Core\Entity\ContentEntityBase

\Drupal\Core\Config\ConfigEntityBase

Page 47: Migrating to Drupal 8

• A configurable is the way Drupal 8 stores the configuration of a specific functionality. E.g. the the definition of a node type is stored in a configuration entity of type “node_type”.

• Configuration entity types are annotated classes, meaning that the object meta information is stored in annotation rather than in info hooks - as it was in Drupal <= 7.

• Imagine configurables as entities storing their data in config YAML files rather than DB.

• The “fields” of a configurable are the public properties exposed by the configurable object.

what is a configurable?

Page 48: Migrating to Drupal 8

what is a configurable?

Page 49: Migrating to Drupal 8

how it’s stored?example

Page 50: Migrating to Drupal 8

migration pluginsparts implemented by specific migrations

Page 51: Migrating to Drupal 8

source plugins

• plugins returning information and data from the source of migration.

• usually: the list of fields, the source iterator (used retrieve data from source).

• each migration should configure a source.

Page 52: Migrating to Drupal 8

destination plugins

• are handling data at the destination: import, rollback.

• different plugins for different destination components: entity, config, etc.

• are defined in the base module (migrate) as destination is always drupal 8 but if necessary it can be extended.

• each migration should specify a destination.

Page 53: Migrating to Drupal 8

id map plugin

• plugins of this type are handling and storing the relation between primary IDs of source and destination.

• without this, rollback and continuous migrations are impossible.

• in 99% of the cases you’ll use the sql id map plugin (Sql) that keeps the map of each migration in a table.

• table name migrate_map_MIGRATION_ID

Page 54: Migrating to Drupal 8

processors

• plugins that are performing small but very specialized operations against values to be migrated.

• Some simple examples: DefaultValue, Concat, etc.

• The most important interface method: transform().

Page 55: Migrating to Drupal 8

the anatomy of a migrationmigrating user roles from a dupal 6 site

Page 56: Migrating to Drupal 8

creating the config fileconfig/migrate.migration.d6_user_role.yml

relative to core/modules/migrate_drupal

Page 57: Migrating to Drupal 8

config .yml file content

Page 58: Migrating to Drupal 8

config .yml file content• id: same as the last part of filename (d6_user_role)

Page 59: Migrating to Drupal 8

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

Page 60: Migrating to Drupal 8

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

Page 61: Migrating to Drupal 8

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

• process: describe the list of processors to be applied per destination field.

Page 62: Migrating to Drupal 8

config .yml file content• id: same as the last part of filename (d6_user_role)

• sourceIds: Source fields, providing a primary ID.

• source: configure the source of data, usually the source plugin to be used

• process: describe the list of processors to be applied per destination field.

• destination: destination configuration, usually the destination plugin.

Page 63: Migrating to Drupal 8

id

Page 64: Migrating to Drupal 8

id• this is the configurable unique id.

Page 65: Migrating to Drupal 8

id• this is the configurable unique id.

• it must be exactly as the same as the last part of filename: d6_user_role.

Page 66: Migrating to Drupal 8

id• this is the configurable unique id.

• it must be exactly as the same as the last part of filename: d6_user_role.

id: d6_user_role

Page 67: Migrating to Drupal 8

sourceIds

Page 68: Migrating to Drupal 8

sourceIds• look in D6 schema to find the role primary ID.

Page 69: Migrating to Drupal 8

sourceIds• look in D6 schema to find the role primary ID.

• lines 107 - 115 of drupal/modules/user/user.install.

Page 70: Migrating to Drupal 8

sourceIds• look in D6 schema to find the role primary ID.

• lines 107 - 115 of drupal/modules/user/user.install.

$schema['role'] = array( 'description' => 'Stores user roles.',  'fields' => array(   'rid' => array(    'type' => 'serial',    'unsigned' => TRUE,    'not null' => TRUE,    'description' => 'Primary Key: Unique role id.',

Page 71: Migrating to Drupal 8

sourceIds

Page 72: Migrating to Drupal 8

sourceIds• use TypedData identifiers for data type.

Page 73: Migrating to Drupal 8

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

Page 74: Migrating to Drupal 8

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

sourceIds:    rid:    type: integer

Page 75: Migrating to Drupal 8

sourceIds• use TypedData identifiers for data type.

• Here are the .yml lines that we need to add.

sourceIds:    rid:    type: integer

Note: sourceIds will be removed in the near future and the source plugin will set also the primary id.

Page 76: Migrating to Drupal 8

source

Page 77: Migrating to Drupal 8

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

Page 78: Migrating to Drupal 8

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

Page 79: Migrating to Drupal 8

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

• add the source plugin id in the configuration .yml file.

Page 80: Migrating to Drupal 8

source

• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.

• let’s see how it should look (code).

• add the source plugin id in the configuration .yml file.

source:  plugin: drupal6_user_role

Page 81: Migrating to Drupal 8

process

Page 82: Migrating to Drupal 8

process• process keys are destination “fields”.

Page 83: Migrating to Drupal 8

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

Page 84: Migrating to Drupal 8

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

Page 85: Migrating to Drupal 8

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

• let’s see how it looks! (code).

Page 86: Migrating to Drupal 8

process• process keys are destination “fields”.

• for configurables: the public properties (except uuid)

• for content: the keys from baseFieldDefinitions

• let’s see how it looks! (code).

process:  id:  label:  weight:  permissions:

Page 87: Migrating to Drupal 8

destination

Page 88: Migrating to Drupal 8

destination

• should point to the destination plugin.

Page 89: Migrating to Drupal 8

destination

• should point to the destination plugin.

• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.

Page 90: Migrating to Drupal 8

destination

• should point to the destination plugin.

• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.

destination: plugin: entity entity_type: user_role

Page 91: Migrating to Drupal 8

running a migration

• via drush

• There will be a brief UI implemented in core (to come!)

Page 92: Migrating to Drupal 8

final notes

Page 93: Migrating to Drupal 8

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

Page 94: Migrating to Drupal 8

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

Page 95: Migrating to Drupal 8

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.

Page 96: Migrating to Drupal 8

final notes• Minor version updates are unchanged. Developers

continue to use hook_update_N() for those.

• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.

• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.

• Similarly, Drupal 4.x and Drupal 5.x sites are able to migrate using this same approach.

Page 97: Migrating to Drupal 8

Questions?Thank you.