Transmogrifier: Migrating to Plone with less pain
33
Transmogrifier Lennart Regebro Plone Conference 2009, Budapest
-
Upload
lennart-regebro -
Category
Technology
-
view
3.578 -
download
1
description
Transmogrifier is a migration framework written by Martijn Pieters, Jarn, that makes migrating to Plone fun again. This talk gives an introduction to Transmogrifier, an explanation of how it works and some hints for usage.
Transcript of Transmogrifier: Migrating to Plone with less pain
Transmogrifier
Lennart Regebro
Plone Conference 2009, Budapest
collective.transmogrifier
Created by Martijn Pieters, Jarn
in 2008
Released inversion 1.0
August 2009
collective.transmogrifier
Import/migration framework
Both from Plone 2 and from other websites or data
Super-simple architecture
You set up a pipeline of sections
Each section send items to the next
section.
Sections do something with the
items (or not)
What are you talking about?
A simple example
1. A section that reads a CSV file with a list of events and creates items based on that data
2. A section that will generate some plone-specific data for each item, like path and type.
3. A section that creates the Plone event4. A section that updates the Archetypes fields5. A section that publishes the event
The CSV file
title,startDate,endDatePlone Conference,2009-10-28 09:00,2009-10-30 18:00Christmas,2009-12-24 00:00,2009-12-26:23:59
Setting it up
1. Define a pipeline in a .cfg file
2. Register it as a name in ZCML
3. Call Transmogrifier from Python
The configuration file[transmogrifier]pipeline = section1 section2
[section1]blueprint = name.of.the.blueprintsize = 5
[section2]blueprint = another.blueprint
ZCML
<configure xmlns:transmogrifier= "http://namespaces.plone.org/transmogrifier"> <transmogrifier:registerConfig name="Zap event import" title="Import of events" description="" configuration="transmogrifier.cfg" />
</configure>
setuphandlers.py
from collective.transmogrifier.transmogrifier import \ Transmogrifier
def setupVarious(context):
if context.readDataFile('zap_various.txt') is None: return
transmogrifier = Transmogrifier(context.getSite()) transmogrifier('Zap event import')
What is a generator
def my_generator(): i = 1 while i < 1000: yield i i = i * 2
for x in my_generator(): print x
Never ending story
def my_generator(): i = 1 while True: yield i i = i * 2
for x in my_generator(): print x
EventSource
class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event
for item in self.previous: yield item
EventSource
class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event
for item in self.previous: yield item
EventSource
class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event
for item in self.previous: yield item
The item
{ 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00',}
EventUpdater
class EventUpdater(object): def __iter__(self): for item in self.previous: id = item['title'].lower().replace(' ', '-') item['_path'] = '/events/'+id item['_type'] = 'Event' item['_transitions'] = ('publish',) yield item
The item
{ '_type': 'Event', '_path': '/events/plone-conference', '_transitions': ('publish,), 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00',}
Registering the section
<utility component=".eventsource.EventSource" name="zap.eventsource" />
<utility component=".eventsource.EventUpdater" name="zap.eventupdater" />
The configuration file[transmogrifier]pipeline = eventsource eventupdater
[eventsource]blueprint = zap.eventsourcecsvfile = /path/to/events.csv
[eventupdater]blueprint = zap.eventupdater
The configuration file cont.
[constructor]blueprint = collective.transmogrifier.sections.constructor
[schemaupdater]blueprint = plone.app.transmogrifier.atschemaupdater
[workflow]blueprint = plone.app.transmogrifier.workflowupdater
The configuration file cont.
[transmogrifier]pipeline = eventsource eventupdater constructor schemaupdater workflow
SectionsCreate items
Modify items
Drop items
Split the pipeline in two
Construct content
Modify content
Migrating from another CMS
Similar to the simple examplebut using SQL instead of CSV
Migration from Plone 2
1. Write an export script2. Write an import section
Migrate from Plone 2
importsourceconstructor
criterionadderschemaupdaterbrowserdefault
workflowsavepoint
Migrating from HTML
1. Get HTML from web
2. Extract fields like title, publish date, etc.
3. Make all urls either relative or into UID's
From small to huge
Largest migration:28 sections
527 lines of configuration16 custom sections
Caveat Emptor
Don't migrate from HTML on disk
Caveat Emptor
Newly created objects are not indexed.
Caveat Emptor
Debugging is backwards
