End User Mashup End User Mashup Programming EnvironmentsProgramming Environments

Oleg BeletskiOleg BeletskiHUT, Telecommunications Software HUT, Telecommunications Software

and Multimedia Laboratoryand Multimedia Laboratory28.03.2008 28.03.2008

ContentContent

IntroductionIntroduction

Approach – mashup methods and ways to Approach – mashup methods and ways to comparecompare

Case study – overview of mashup Case study – overview of mashup environmentsenvironments

Results + DiscussionsResults + Discussions

ConclusionsConclusions

IntroductionIntroduction

Mashup – web application that combines data Mashup – web application that combines data from online sources of data and perform new from online sources of data and perform new tailored tasktailored task

Mashup creationMashup creation– Hard way – programmer skills requiredHard way – programmer skills required– Easy way – use end user programming tools and Easy way – use end user programming tools and

visual environmentsvisual environments

ApproachApproach

Find environments for a comparisonFind environments for a comparison

Define criteria for comparisonDefine criteria for comparison

Analyze each siteAnalyze each site

Sum-up and analyze results (drop if Sum-up and analyze results (drop if environment does not fit well to our scope)environment does not fit well to our scope)

Make conclusionsMake conclusions

Approach – criteria for comparisonApproach – criteria for comparison

LifecycleLifecycle

Richness of functionalityRichness of functionality

UsabilityUsability

Target type of usersTarget type of users

Publishing formatsPublishing formats

TechnologiesTechnologies

Data sourcesData sources

Case study – list of environmentsCase study – list of environments

Yahoo! PipesYahoo! Pipes

Microsoft’s PopflyMicrosoft’s Popfly

MarmiteMarmite

DapperDapper

Google mashup editorGoogle mashup editor

MashmakerMashmaker

QEDWikiQEDWiki

Case study – Yahoo! PipesCase study – Yahoo! Pipes

Pipes is a powerful composition tool to aggregate, manipulate, Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web. and mashup content from around the web. Visual drag and drop environmentVisual drag and drop environmentDoes not require knowledge of programming languagesDoes not require knowledge of programming languagesRequires good understanding of a data formats. Requires good understanding of a data formats. Sample application:Sample application:– Combine many feeds into one, then sort, filter and translate it. Combine many feeds into one, then sort, filter and translate it. – Geo code your favorite feeds and browse the items on an interactive Geo code your favorite feeds and browse the items on an interactive

map (use location extractor)map (use location extractor)– Power widgets/badges on your web site. Power widgets/badges on your web site. – Grab the output of any Pipes as RSS, JSON, KML, and other formats. Grab the output of any Pipes as RSS, JSON, KML, and other formats.

Case study – Yahoo! PipesCase study – Yahoo! Pipes

Case study – Microsoft PopflyCase study – Microsoft Popfly

Microsoft® Popfly™ is a web site and tool to help people create and share Microsoft® Popfly™ is a web site and tool to help people create and share web sites, mashups, and other kinds of experiences. Popfly is the fun and web sites, mashups, and other kinds of experiences. Popfly is the fun and easy way to build and share mashups, gadgets, and Web pages. easy way to build and share mashups, gadgets, and Web pages. It’s made up of online visual tools for building Web pages and mashups It’s made up of online visual tools for building Web pages and mashups and a social network where you can host, share, rate, comment and even and a social network where you can host, share, rate, comment and even remix creations from other Popfly users.remix creations from other Popfly users.

It is fun, it is simpleIt is fun, it is simpleUses Silverlight runtimeUses Silverlight runtimeHas a lot of predefined modules for existing services and social networkHas a lot of predefined modules for existing services and social networkCool visualizationsCool visualizations

Case study – Microsoft PopflyCase study – Microsoft Popfly

Case study – Microsoft PopflyCase study – Microsoft Popfly

Case study – MarmiteCase study – MarmiteMarmite, a tool that will let everyday end-users create mashups by making it easy to extract Marmite, a tool that will let everyday end-users create mashups by making it easy to extract content from web pages, process it in a data-flow manner, integrate it with other data content from web pages, process it in a data-flow manner, integrate it with other data sources, and direct it to a variety of useful sinks, such as databases, map services, and sources, and direct it to a variety of useful sinks, such as databases, map services, and compilable source code that can be further customized. compilable source code that can be further customized. This proposal focuses on three high-risk issues: This proposal focuses on three high-risk issues:

– making it easy to select what content to crawlmaking it easy to select what content to crawl– developing a hybrid dataflow / spreadsheet UI that shows what content has been extracted and how developing a hybrid dataflow / spreadsheet UI that shows what content has been extracted and how

that content is transformedthat content is transformed– developing techniques for handling exceptions in the dataflow. developing techniques for handling exceptions in the dataflow.

Structure comprises of sources, processors, sinksStructure comprises of sources, processors, sinksSources: add data into Marmite by querying databases, extracting information from web Sources: add data into Marmite by querying databases, extracting information from web pages, and so on. pages, and so on. Processors: modify, combine, or delete existing rows/columns (geocoding, filtering). Processors: modify, combine, or delete existing rows/columns (geocoding, filtering). Sinks: redirect the flow the data out of Marmite. (showing data on a map, saving it to a Sinks: redirect the flow the data out of Marmite. (showing data on a map, saving it to a file/web page) file/web page) Ownership: Jason I. HongOwnership: Jason I. HongHuman Computer Interaction InstituteHuman Computer Interaction InstituteSchool of Computer ScienceSchool of Computer ScienceCarnegie Mellon UniversityCarnegie Mellon University

Case study – MarmiteCase study – Marmite

Case study – DapperCase study – Dapper

Dapper stands for data mapper. The main purpose of the service is to Dapper stands for data mapper. The main purpose of the service is to convert any type of content into standard form that can be reused (RSS, convert any type of content into standard form that can be reused (RSS, XML). It also has the set of publishing features that turn that content into XML). It also has the set of publishing features that turn that content into Google Gadget, Netvibes Module, iCalendar, Flash widgets and so on.Google Gadget, Netvibes Module, iCalendar, Flash widgets and so on.Visually it run is a wizard mode Visually it run is a wizard mode Results is to create a dapp (data imported). Results is to create a dapp (data imported). Dapps can be shared and there are many of then for popular servicesDapps can be shared and there are many of then for popular servicesDapp often is sufficient and can be tuned in map mashup or image loopDapp often is sufficient and can be tuned in map mashup or image loopOutput or visualization is defined by userOutput or visualization is defined by userNext level of development is to combine those dapps into aggregator Next level of development is to combine those dapps into aggregator service. The typical example is to combine search result from several service. The typical example is to combine search result from several search engines or video clips from alternative video services similar to search engines or video clips from alternative video services similar to magg movie aggregator (http://www.dapper.net/dapplications/Magg/). magg movie aggregator (http://www.dapper.net/dapplications/Magg/).

Case study – DapperCase study – Dapper

Case study – GMECase study – GME

Google Mashup Editor is an AJAX development framework Google Mashup Editor is an AJAX development framework and a set of tools that enable developers to quickly and easily and a set of tools that enable developers to quickly and easily create simple web applications and mashups with Google create simple web applications and mashups with Google services like Google Maps and Google Base. Google Mashup services like Google Maps and Google Base. Google Mashup Editor is a great tool for grabbing information from feeds and Editor is a great tool for grabbing information from feeds and letting users see and manipulate it. letting users see and manipulate it. Creating applications with Google Mashup Editor is “simple” Creating applications with Google Mashup Editor is “simple” for developer for developer This is a software developer tools and not an end user This is a software developer tools and not an end user mashup environment.mashup environment.Google mashup editor will be excluded from the section Google mashup editor will be excluded from the section where we compare different mashup environment because it where we compare different mashup environment because it did not quite fit to out purpose.did not quite fit to out purpose.

Case study – GMECase study – GME

Case study – MashmakerCase study – MashmakerIntel® Mash Maker is an extension to your existing web browser that allows you to Intel® Mash Maker is an extension to your existing web browser that allows you to easily augment the page that you are currently browsing with information from easily augment the page that you are currently browsing with information from other websites. As you browse the web, the Mash Maker toolbar suggests other websites. As you browse the web, the Mash Maker toolbar suggests Mashups that it can apply to the current page in order to make it more useful for Mashups that it can apply to the current page in order to make it more useful for you. For example: plot all items on a map, or display the leg room for all flights.you. For example: plot all items on a map, or display the leg room for all flights.Community can contribute to mashup DB. Any user can teach Mash Maker new Community can contribute to mashup DB. Any user can teach Mash Maker new mashups, using a simple copy and paste interface, mashups, using a simple copy and paste interface, New user created mashups are suggested to other users. New user created mashups are suggested to other users. Relies on the community to teach it about the structure and semantics of web Relies on the community to teach it about the structure and semantics of web pages, using a built in structure editor.pages, using a built in structure editor.There is no dedicated page on the web where to have to go and construct the There is no dedicated page on the web where to have to go and construct the mashup application. All what user has to do is to install the toolbar and start mashup application. All what user has to do is to install the toolbar and start browsing pages. So the paradigm could be called “annotate and mix while browsing pages. So the paradigm could be called “annotate and mix while browsing”.browsing”.Users themselves tell to Mashmaker servers about the page structures that they Users themselves tell to Mashmaker servers about the page structures that they visit and purpose content from other sites to mix with current page.visit and purpose content from other sites to mix with current page.The service is in beta and not available to wide public at the moment.The service is in beta and not available to wide public at the moment.

Case study – MashmakerCase study – Mashmaker

Case study – QEDWikiCase study – QEDWiki

QEDWiki is a browser-based assembly canvas used to create simple QEDWiki is a browser-based assembly canvas used to create simple mashups. mashups. The same steps: Assemble, Wire, ShareThe same steps: Assemble, Wire, ShareA mashup maker is an assembly environment in which the creator of a A mashup maker is an assembly environment in which the creator of a mashup uses software components (or services) made available by mashup uses software components (or services) made available by content providers. content providers. Wiki framework in that it provides both Web users and developers with a Wiki framework in that it provides both Web users and developers with a single Web application framework for hosting and developing a broad single Web application framework for hosting and developing a broad range of Web 2.0range of Web 2.0Sample applicationsSample applications– Web content management for a typical collection of Wiki pagesWeb content management for a typical collection of Wiki pages– traditional form processing for database-oriented CRUD traditional form processing for database-oriented CRUD

(Create/Read/Update/Delete) applications(Create/Read/Update/Delete) applications– document-based collaborationdocument-based collaboration– rich interactive applications that bind together disparate servicesrich interactive applications that bind together disparate services– situational applications (or mashups).situational applications (or mashups).

Case study – QEDWikiCase study – QEDWiki

Results + DiscussionsResults + Discussions

Programming paradigmProgramming paradigm

Richness of functionalityRichness of functionality

TechnologyTechnology

Ease of useEase of use

Openness for integrationOpenness for integration

Results table 1/2Results table 1/2Pipes PopFly Marmite Dapper QEDWiki Mashmaker

Paradigm Data flow Data flow Flow + spreadsheet Flow + wizard Application on canvas

Browsing enrichment

Maturity Mature Mature(beta) Research project Mature Mature Close, beta

Functionality Rich but techy

Rich, integrated with services

Poor Limited Rich N/A

Easy of use For coders, not for an end users

Simple and friendly, makes a suggestion about linking parts together

Cryptic Easy, but still not 100% end user tool

Easy, lazy developer tool

Toolbar is simple to use if site structure know

Technology Standard web, YUI

Silverlight, proprietary Browser plug-in, runs in own windows, allows point at the parts of web pages to scrap

Standard and/or popular WEB technologies

Standard In browser mashing up, needs a toolbar, enabled by community maintained DB of sites structure

Results table 2/2Results table 2/2Openness for

integrationHtml code to embed in

blog, RSS, JSON

HTML code to embed

n/a New API creator. Export to Google gadget, RSS, XML, Flash Widget

Can embed public WEB services and widgets

N/A

Extensibility Yes Yes, user can contribute

n/a Dapps can be reused by other dapps

New sites and widgets can be added

Advanced features Extension by JavaScript

?

Other Claim to have unique web page scrapping approach

Results – programming paradigmResults – programming paradigm

Data flow with various visual codingData flow with various visual coding– Wizard basedWizard based

DapperDapper

– Visual connected functionalVisual connected functionalPipesPipesPopflyPopfly

– SpreadsheetSpreadsheetMarmiteMarmite

Visual widget assemblyVisual widget assemblyMashmakerMashmakerQEDWikiQEDWiki

Results – richness of functionalityResults – richness of functionality

Data sourcesData sources– RSS, ATOM feeds (low level that is supported by all tools)RSS, ATOM feeds (low level that is supported by all tools)– Page scrapping (Most tools support, except Mashmaker & QEDWiki)Page scrapping (Most tools support, except Mashmaker & QEDWiki)– Service wrappers (Pipes, POPfly)Service wrappers (Pipes, POPfly)

Data processing optionsData processing options– Feed mergingFeed merging– Location extractionLocation extraction– Numeric operationNumeric operation– String manipulationString manipulation

Visualization of results and output formatsVisualization of results and output formats– Multiple visualization alternatives (POPFly only)Multiple visualization alternatives (POPFly only)– Visual appeal Visual appeal – Export to external services via RSS/ATOM feeds, XML, JSONExport to external services via RSS/ATOM feeds, XML, JSON– Blog integrationBlog integration

Results - technologyResults - technology

AJAX – standard web technologies, runs in AJAX – standard web technologies, runs in any browser (Pipes)any browser (Pipes)

Silverlight – proprietary but highly visual Silverlight – proprietary but highly visual technology from Microsoft (Popfly)technology from Microsoft (Popfly)

Browser add-on, dependent on browser Browser add-on, dependent on browser (Mashmaker, Marmite)(Mashmaker, Marmite)

Results – ease of useResults – ease of use

User rolesUser roles– Mashup authorMashup author– Mashup userMashup user– Environment developerEnvironment developer

Mashup author is the user of tools we analyzeMashup author is the user of tools we analyze– Average user that uses provided blocksAverage user that uses provided blocks– Advanced user that adds modules and new services to the Advanced user that adds modules and new services to the

environmentenvironment

Goal – make system as simple as possible for use Goal – make system as simple as possible for use by average userby average user

Results – openness for integrationResults – openness for integration

Can results of the mashup be reused?Can results of the mashup be reused?

Popfly – supports only html code for adding to Popfly – supports only html code for adding to external blogsexternal blogs

Pipes are more open -RSS, KML, JSONPipes are more open -RSS, KML, JSON

Dapper is designed for integration - XML, Dapper is designed for integration - XML, RSS, Google Gadget, Flash, Google Map, RSS, Google Gadget, Flash, Google Map, Image Loop, iCalendarImage Loop, iCalendar

Mashmaker and QEDWiki is a bit out of scopeMashmaker and QEDWiki is a bit out of scope

ConclusionsConclusions

Mashup environments target different user groups – Mashup environments target different user groups – programmers, technology enthusiasts, non technical programmers, technology enthusiasts, non technical usersusersBest visual tools for non technical users hide data Best visual tools for non technical users hide data exchange formats and provide blocks for every exchange formats and provide blocks for every service that user is likely to integrateservice that user is likely to integrateAdvanced functionality will keep environments on the Advanced functionality will keep environments on the float by letting users provide support for new float by letting users provide support for new servicesservicesStandard technologies are preferredStandard technologies are preferredNew direction – enhancing browsing experienceNew direction – enhancing browsing experience