David De Roure

End-to-End Semantics Sensors

Semitones Science

Social Machines

•  This  workshop  series  has  successfully  demonstrated  that  Seman7c  Web  technologies  can  be  used  with  sensor  networks  

•  It  hasn’t  necessarily  demonstrated  that  they  are  the  technology  of  choice  

•  What  can  we  learn  from  the  applica7on  of  Seman7c  Web  elsewhere?  

Mo7va7on  

http://musicnet.mspace.fm/blog/music-linked-data-workshop/

A digital lab book replacement that

chemists were able to use, and liked. 1 1 2 2 1 3 1 4

Sample of 4-flourinatedbiphenyl

Add CoolReflux

Butanone Sample ofK2CO3Powder

Weigh

grammes0.9031

Measure

40 ml

Add

Weigh

2.0719 g

text

3 5

Add

g

Sample ofBr11OCB

2 6

Reflux

2 7

Cool

Water

Measure

30 ml

9

Liquid-liquid

extraction

DCM

Measure

3 of 40 ml

10

Dry

MgSO4

11

Filter(Buchner)

12RemoveSolvent

by RotaryEvaporation

13

Fuse

Silica

14Column

Chromatography

Ether/PetrolRatio

Butanone dried via silica column andmeasured into 100ml RB flask.

Used 1ml extra solvent to wash outcontainer.

Started reflux at 13.30. (Had tochange heater stirrer) Only reflux

for 45min, next step 14:15.

Inorganics dissolve 2layers. Added brine

~20ml.

Organics are yellowsolution

Washed MgSO4 withDCM ~ 50ml

Measure

excess

Observation Types

weight - grammes

measure - ml, drops

annotate - text

temperature - K, °C

Key

Process

Input

Literal

Observation

Add CoolRefluxAddAdd Reflux Cool Dry Filter Remove

Solventby Rotary

Evaporation

Fuse ColumnChromatography

Dissolve 4-flourinatedbiphenyl inbutanone

Add K2CO3powder

Heat at refluxfor 1.5 hours

Cool and addBr11OCB

Heat atreflux untilcompletion

Cool and addwater (30ml)

Combine organics,dry over MgSO4 &filter

Removesolvent invacuo

Liquid-liquid

extraction

Extract withDCM(3x40ml)

Fuse compound to silica &column in ether/petrol

4 8

Add

Add

text

Annotate

Annotate

text

Weigh

Annotate

g

Annotate Annotate

text text

Future Questions

Whether to have many subclasses of processes or fewer with annotations

How to depict destructive processes

How to depict taking lots of samples

What is the observation/process boundary? e.g. MRI scan

1.5918

Combechem

30 January 2004gvh, hrm, gms

Ingredient List

Fluorinated biphenyl 0.9 gBr11OCB 1.59 gPotassium Carbonate 2.07 gButanone 40 ml

image

To

Do

Lis

tP

lan

Pro

ce

ss

Re

co

rd

Jeremy  Frey  

•  Annota7on  should  occur  within  the  produc7on  process  

•  Integra7ng  knowledge  of  the  produc7on  workflow  

•  Managing  and  exposing  this  metadata  using  modern  seman7c  web  and  linked  data  technology  

•  Empowering  human  producers  and  consumers  

Content Navigation throughout the Content Life-Cycle

semanticmedia.org.uk

Some  Social  Machines  

Science

Sensor Networks

Music

Social

1

2 3

More people

Mor

e m

achi

nes

A  Big  Picture  

Big Data Big Compute Conventional Computation

The Future! Social Networking

e-infrastructure

online R&D

The  Fourth  Quadrant    

Science

Sensor Networks

Music

Social

1

2 3

http://www.slideshare.net/moustaki/linked-data-on-the-bbc-2638734

“Twelve months ago, there were three of us in the new Olympic Data Team: … Today, we are a team of 20, we have built five applications, provide 174 endpoints, manage 50 message queues and support ten separate BBC Olympic products - from the sport website to the Interactive Video Player.”

http://www.bbc.co.uk/blogs/bbcinternet/

INT. VERSE VERSE VERSE VERSEBRIDGEBRIDGE OUT.

ê

Ichiro  Fujinaga  

Digital  Music  Collec7ons  

Student-­‐sourced  ground  truth  

Community  SoLware  

Linked  Data  Repositories  

Supercomputer  

23,000 hours of recorded music

Music Information Retrieval Community

Structural Analysis of Large Amounts of Music Information

class structure

Ontology models properties from musicological domain •  Independent of Music Information Retrieval research and

signal processing foundations •  Maintains an accurate and complete description of

relationships that link them

Segment  Ontology  

Kevin  Page  and  Ben  Fields  

Digital  Music  Collec7ons  

ground  truth  

Evalua7on  Infrastructure  (sociotechnical)  

Exper7se  Exper7se  Exper7se  Exper7se  

Community  SoLware  Community  SoLware  Community  SoLware  

Digital  Music  Collec7ons  Digital  Music  Collec7ons  Digital  Music  Collec7ons  

ground  truth  ground  truth  

Evalua7on  Infrastructure  (sociotechnical)  

Results  Results  Results  Results  

Evalua7ons  Evalua7ons  Evalua7ons  

papers  papers  papers  Papers  

story arcs and timey-wimey stuff

Mike Jewell Faith Lawrence

•  Industry  adop7on  (fits  with  prac7ce  and  solves  a  problem?)  

•  Significant  benefit  of  automa7on  within  the  enterprise  and  for  public  use  

•  R&D  community  has  created  socio-­‐technical  infrastructure  

•  Pushing  towards  capture  at  source  

Music  

Science

Sensor Networks

Music

Social

1

2 3

...the imminent flood of scientific data expected from the next generation of experiments, simulations, sensors and satellites

Source: CERN, CERN-EX-0712023, http://cdsweb.cern.ch/record/1203203

data  

method    

Research  Objects  

Computa7onal  Research  Objects  

Co-­‐Evolu7on  of  Research  Objects  

Workflows  Packs   O

AI  ORE

 

W3C  PRO

V  

Notifications and automatic re-runs

Machines are users too

Autonomic Curation

Self-repair

New research?

•  Systema7c  processing  of  (big)  data  •  Par7cular  aWen7on  to  trust,  provenance,  reproducibility  

•  Assistance  versus  automa7on  –  Taylorisa7on,  provisional  outcomes  –  Trust  through  authority  or  the  crowd?  

•  Scholarly  communica7on  supported  by  Seman7c  Web  “social  objects”  –  e.g.  Models,  mashups,  narra7ves,  …  

Science  

Science

Sensor Networks

Music

Social

1

2 3

Real life is and must be full of all kinds of social constraint – the very processes from which society arises. Computers can help if we use them to create abstract social machines on the Web: processes in which the people do the creative work and the machine does the administration… The stage is set for an evolutionary growth of new social engines. Berners-Lee, Weaving the Web, 1999

The  Order  of  Social  Machines  

Physical  World  (people  and  devices)  

Building  a  Social  Machine  

Design and Composition

Participation and Data supply

Model of social interaction

Virtual World (Network of social interactions) Dave  Robertson  

http://www.bodleian.ox.ac.uk/bodley/library/special/projects/whats-the-score/research-context

•  Sociotechnical  systems  perspec7ve  

– Fundamental  to  sensor  networks?  

– Closing  the  loop  needs  applica7ons  and  users  

•  Ci7zen  sensing  as  social  machine  

•  Developing  theory  and  prac7ce  – Design  and  construc7on  of  social  machines  

Social  

Science

Sensor Networks

Music

Social

1

2 3

•  ICT-­‐enabled  manufacturing  (e.g.  a  phone,  car  parts)  

•  Sensors  throughout  produc7on  process  •  Monitoring  en7re  product  lifecycle  –  life  of  objects  (internet  of  things)  

•  Collec7ng  data  from  discourse  in  collabora7ve  engineering  process  

•  Informa7on  privacy  

•  Pursuing  “Crowd  and  cloud”  philosophy  •  Big  Data:  seek  to  detect  “signatures”  in  the  data  in  order  to  op7mise  the  manufacturing  process  

Case  Study  1  

•  Studio  as  sensor  network  •  End  to  end  seman7cs  from  instrument  (sensor)  to  consumer  (almost  to  the  brain…)  

•  Reproducible,  repurposeable,  reuseable,  remixable  music  

•  Business  models  and  DRM  •  Social  Objects  =  Score,  MP3  file  Research  Object  =  mix  

•  Interoperability  standards  (analogue  good  too!)  •  Dimension  of  performance,  crea7vity  •  Already  using  Seman7c  Web!  

Case  Study  2    

The brain opera technology: New instruments and gestural sensors for musical interaction and performance. Journal of New Music Research, 28(2), 130–149.

1.  Do  we  have  examples  of  end-­‐to-­‐end  seman7cs  in  SSN?  2.  How  do  we  share  the  methods  for  processing  sensor  

data?  (soLware,  workflows,  automa7on,  …)  –  Provenance,  trust,  reproducibility?  (Linked  Science,  Provenance)  

3.  Are  we  considering  SSN  as  sociotechnical  systems?  –  What  are  the  social  objects  in  SSN?  (data?)  –  Social  life  of  objects  (bikes)  –  Sensors  as  Ci7zens!  S3N  =  Seman7c  Social  Sensor  Nets?  (Ruben)  –  What  are  the  social  machines  in  SSN?  

4.  What  can  we  do  in  concert  with  science  and  music?  –  The  studio  as  sensor  network?  Crea7vity?  (ISWC2013  jammers)  

5.  Can  our  community  build  a  sociotechnical  infrastructure  to  advance  the  work?  

–  More  than  (one)  ontology  as  social  object?  (SSN  ontology)  

Provoca7ons  

david.deroure@oerc.ox.ac.uk  www.oerc.ox.ac.uk/people/dder  

www.scilogs.com/eresearch  

@dder    

Slide  credits:  ,  Jeremy  Frey,  Chris7ne  Borgman,  Faith  Lawrence  &  Mike  Jewell,  Ichiro  Fujinaga,  Stephen  Downie,  Kevin  Page,  Ben  Fields,  Carole  Goble,  Dave  Robertson      www.myexperiment.org/packs/347      

•  Seman7c  Media  hWp://seman7cmedia.org.uk/    

•  Music  Informa7on  Retrieval  Evalua7on  eXchange  (MIREX)  hWp://www.music-­‐ir.org/mirex/    

•  Workflow  Forever  project  (Wf4Ever)  hWp://www.wf4ever-­‐project.org/    

•  Future  of  Research  Communica7on  (FORCE11)  hWp://force11.org/    

•  Theory  and  Prac7ce  of  Social  Machines  (SOCIAM)  hWp://sociam.org/    

•  W3C  SSN  Community  Group  hWp://www.w3.org/community/ssn-­‐cg/    

Links  

•  D.  De  Roure,  C.  Goble  and  R.  Stevens.  The  Design  and  Realisa7on  of  the  myExperiment  Virtual  Research  Environment  for  Social  Sharing  of  Workflows  Future  Genera/on  Computer  Systems  25,  pp.  561-­‐567.    

•  S.  Bechhofer,  I.  Buchan,  D  De  Roure  et  al.  Why  linked  data  is  not  enough  for  scien7sts,  Future  Genera/on  Computer  Systems  

•  D.  De  Roure,  David  and  C.  Goble,  Anchors  in  ShiHing  Sand:  the  Primacy  of  Method  in  the  Web  of  Data.  WebSci10,  April  26-­‐27th,  2010,  Raleigh,  NC,  US.  

•  D.  De  Roure,  S.  Bechhofer,  C.  Goble  and  D.  Newman,  Scien7fic  Social  Objects,  1st  Interna/onal  Workshop  on  Social  Object  Networks  (SocialObjects  2011).  

•  D.  De  Roure,  K.  Belhajjame,  P.  Missier,  P.  et  al  Towards  the  preserva7on  of  scien7fic  workflows.  8th  Interna/onal  Conference  on  Preserva/on  of  Digital  Objects  (iPRES  2011).  

•  Carole  A.  Goble,  David  De  Roure  and  Sean  Bechhofer  Accelera7ng  scien7sts’  knowledge  turns.  Will  be  available  at  www.springerlink.com  

•  Khalid  Belhajjame,  Oscar  Corcho,  Daniel  Garijo  et  al  Workflow-­‐Centric  Research  Objects:  First  Class  Ci7zens  in  Scholarly  Discourse,  SePublica2012  at  ESWC2012,  Greece,    May  2012  

•  Kevin  R.  Page,  Ben  Fields,  David  De  Roure  et  al  Reuse,  Remix,  Repeat:  The  Workflows  of  MIR,  13th  Interna7onal  Society  for  Music  Informa7on  Retrieval  Conference  (ISMIR  2012)  Porto,  Portugal,  October  8th-­‐12th,  2012  

•  Jun  Zhao,  Jose  Manuel  Gomez-­‐Perezy,  Khalid  Belhajjame  et  al,  Why  Workflows  Break  -­‐  Understanding  and  Comba7ng  Decay  in  Taverna  Workflows,  eScience  2012,  Chicago,  October  2012