Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform”...

7
Summary • Service Catalogue VM Update • Taverna “Platform” hackathon • 1-day SCAPE “Platform” workshop in Berlin • Taverna -> MapReduce thoughts

Transcript of Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform”...

Page 1: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Summary

• Service Catalogue VM Update• Taverna “Platform” hackathon• 1-day SCAPE “Platform” workshop in Berlin• Taverna -> MapReduce thoughts

Page 2: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Service Catalogue VM - Update

• AIT Vienna decides not to host it as Paris partner has the resources to do it and is responsible for hosting the Platform– Paris will not have the necessary hardware till September– Possibly longer than that to actually start hosting the service

• Currently negotiating with the Uni’s IT Services to host it here until Paris takes over – IT Services want to be paid for this (possibly)– Might not be done before September anyway

• Other possibility is to host it in our office– Only expose it as a test server to a selected group of users– We do not have man-power for maintenance– Nor is it our job

Page 3: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Taverna “Platform” hackathon

• Mid August – mid September• Finish off SCUFL2 and the new Taverna

Platform• We need to get a “sign off” from all other

duties during this time

Page 4: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

SCAPE “Platform” workshop in Berlin

• Get to know the cloud work done by the Berlin partner• 1 day workshop• Sometime between 4th and 14th October• David will probably go

Page 5: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Taverna -> MapReduce thoughts

• Started a wiki page: – http://tinyurl.com/42sw4qp

• Various issues with converting Taverna workflows to MapReduce:– Different data and iterations models

Page 6: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Taverna -> MapReduce Issue Summary

• MapReduce is designed to take a single input, Taverna processors can have multiple– How do we pass the input ports map to a

MapReduce function?• MapReduce is designed to return a single

output, Taverna processors can have multiple– How do we save the outputs – in a directory

structure in HDFS?

Page 7: Summary Service Catalogue VM Update Taverna “Platform” hackathon 1-day SCAPE “Platform” workshop in Berlin Taverna -> MapReduce thoughts.

Taverna -> MapReduce Issue Summary

• Passing data down the workflow arcs:– To benefit from MapReduce parallelization and local execution

we need to let MapReduce handle data/data references– Taverna processors understand T2References:

• Create a wrapper + special Reference Service to convert between the two

• Save intermediate results into HDFS and pass them around the workflow

• Replace Taverna’s implicit iteration with MapReduce’s parallelization framework– Determine which processor input to parallelize over– Can have several processors in a workflow to parallelize over