1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

29
Interoperability A quick chat, a few war stories...

description

Succeed Interoperability Workshop. 2nd October, 2014. The Hague (The Netherlands).

Transcript of 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Page 1: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Interoperability

A quick chat, a few war stories...

Page 2: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

The OPF teamCommunity Manager● Events (face-to-face/virtual)● Training (staff development)● Comms (web/email/social)

Executive Director● Membership (engagement/value)● Open preservation advocacy● Operational management

Technical Lead● Infrastructure (host/test)● Software stewardship (roadmap/maturity/packaging)● Data corpora

Page 3: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

About OPF

Trying to help institutions create robust digital preservation workflows by:

● sharing knowledge and best practise;

● improving the quality of digital preservation software.

Page 4: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Introduction

Here to set the scene by:

● Defining interoperability

● Providing a personal perspective

● Covering a few engineering fundamentals

● Looking for an easier way

Page 5: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Interoperability

“is the ability of making systems and

organisations work together (inter-operate).”

straight from Wikipedia

Page 6: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Origins?

Coined in IT and system services to allow for

information exchange.

Page 7: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

But it’s not just an IT thing

A broader definition also considers:

● social issues● political & organisational factors

that “impact system to system performance”

Page 8: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Another “definition”

“Task of building coherent services for users

when the individual components are technically

difficult and managed by different

organisations.”

Wikipedia again

Page 9: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Who’s in the interoperability game?

● Information professionals● Software engineers● Staff who work on distributed projects● People dealing with complexity

Page 10: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Sound like anyone we know?

So those of us lucky enough to work in IT for Information professionals on collaborative projects it’s welcome to interoperability X 3…..

Page 11: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

It’s an older problem than IT

=?

Page 12: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Old and international...

Page 13: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

But IT does it better...

The UK NHS National Program for IT

● biggest civilian IT project of its kind● “mashups” for health records ● in theory 2003 - 2007 costing £2.4 billion● in reality 2003 - 2011 costing £13 billion● incidentally it didn’t work…….

Page 14: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

But it’s possible in real life

Page 15: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Think of a system that is:

● decentralised

● fault tolerant

● built on simple building blocks

And it’s possible in IT…..

Page 16: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

What would that look like?

Page 17: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Things to think about

● simplicity

● standards

● clarity

● test early

● test often

Page 18: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Small is beautiful

● small parts can build beautiful robust wholes

● it’s one way of tackling complexity

● BUT the parts need testable, tested and reliable

Page 19: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Standards are your friends

It’s always worth checking if there’s a standard

available and in use but:

● there’s often more than one

● they’re not always easy to understand

● they’re hostage to commercial interests

Page 20: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Be clear in your intentions

Again simplicity / small size helps here

● Document clearly and publicly● Keep it up to date● If you’re not willing to explain how it works….

Page 21: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Test early and often

And test the parts together rather than as well as separately

Technologies to automate testing:

● virtualisation● online public continuous integration

Page 22: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Interoperability is the future

The Internet of Things is gathering pace

Information exchange happening between more and more devices, more regularly.

REST, XML, Json

Page 23: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

But it’s NEVER finished

Two aspects of interoperability:

● Syntactically interoperable● Semantically interoperable

That’s to say speaking the same language is one thing, understanding what the other is saying is another…..

Page 24: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Why data is a special problem

In the cultural heritage sector

● shared data == shared problems● legal issues (IPR) means we don’t share as we

should

Page 25: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

Moving the mountain

Moving the large volume takes way too long….

● If I want to move > 5 TB for processing● THEN move the results back…..

Page 26: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

A better way…..

It’s now MUCH easier to move the application to the data:

● It’s more secure● virtual machines are approx. 300 MB● Docker - application containers

Page 27: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

So the oldest interoperability story?

● It might not be the oldest but it’s close,● about old school integration issues.● Something of a cautionary tale,● it didn’t end well…..

Page 28: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

The Tower of Babel

Page 29: 1. 'Interoperability. A quick chat, a few war stories'. Carl Wilson, Open Planets Foundation.

My Details

● Carl Wilson ● Technical Lead● Open Planets Foundation● Email : [email protected]● Skype : carl.f.wilson● GitHub : carlwilson● Twitter : @openplanets● Google+ : [email protected]