WYSIWYG Development of Data Driven Web Applications

Fan Yang, Nitin Gupta, Chavdar BotevCornell University, Ithaca, NY

Elizabeth F Churchill, George Levchenko, Jayavel Shanmugasundaram

Yahoo! Research, Santa Clara, CA

Abstract

• Social Networking sites and Web portals have opened up their APIs

• External developers can design their own tools and share it with others

• Targeted towards developers with programming expertise and database knowledge

• Solution– AppForge , a WYSIWYG application development platform

AppForge

• WYSIWYG application development platform• Users graphically specify the components of web

pages• The database schema and application logic are

automatically generated• Provides instantaneous feedback• Not yet open to the public

Solution Space

Stateless

Data Complexity of Applications

Simple State Relationships

SophisticatedDevelopers

Developers

Powerusers

J2EE.NET

NingRuby-on-RailsWebMLHilda

Yahoo! PipesMicrosoft PopFly

JotSpotOracle Forms

App2YouCogHeadZohoCreatorDabbleDBWyaWorks

…

Users

Ta

rge

t Au

die

nce

AppForge

WYSIWYG developmentGeneral relationships

Trends

• Web pages are turning into computing platforms

• So websites have opened up their API’s for users

• But beyond the reach of majority of users

• Current solutions:

– Yahoo! Pipes, Microsoft Popfly, App2You, CogHead

– Zoho Creator, Ning, Dabble DB, WyaWorks, JotSpot,

SaleForce

Drawbacks

• Non-WYSIWYG development environment

– Two modes

• Development mode

• Execution mode

– Developer has to visualize while in development mode

– E.g. LaTex and Microsoft Word

Drawbacks (contd..)

• Limited support for creating stateful applications

with complex structures

– Support stateless web applications

– Predefined structure

– Support sophisticated stateful applications

• But require developer background

Drawbacks (contd..)

• Limited support for publishing views over multiple

related entities

– Current systems don’t support views

– Require database knowledge

AppForge

• WYSIWYG environment– Integrates design, deployment and testing– Instantaneous feedback– Generates database schema and application logic– WYSIWYG to target the general user

AppForge (contd..)

• Users can create sophisticated stateful applications– Developers focus on display items– Entities and relationships are inferred– Algorithm translates developer actions into database

schema

AppForge (contd..)

• Users create complex views over multiple entities– Schema Navigation Menu– Hierarchical graph– Algorithm to convert E-R graph to Schema Navigation

Menu– Algorithm to translate developers actions on Menu into

views

System Architecture

System Components

• Application Creation System – Creates and updates the application model– Page View Creation module

• Interface for creating and updating web pages

– Automatic Schema Generation module• Generates the relational database schema from page views

– Application model stored in the file system– Application state is stored in a relational database system

System Components (contd..)

• Application Runtime System– Loads the application model– Interprets the model– Issues SQL queries over the database

AppForge GUI

Abstractions• Application

– Create and manage multiple applications

• Role– Users are divided into multiple roles

• Page – Each page can contain one or more Forms and Views

• Form– Used to enter new data

• View– View and update the application state

• Container– Entities in an application

Application Model

• Extension of E-R model

Application Model (contd..)

• Nested Relational Algebra– Extends Relational model– Relaxes first normal form– Model Hierarchical Data– Two operators

• Nest (v)• Unnest( µ )

Nested Relational Algebra

Application Model

• Database Model• Specifies the application state

• Schema• Constraints

• Page Model• Specifies the content, structure and presentation of web

pages• Content and Structure• Presentation

Constructing Views

• Page View Creation Module• Challenges

– Show database schema without exposing the complexity of an E-R graph

• Solution: Schema Navigation Menu

– Create views without exposing the details of NRA operators such as join and nest.

• Solution: Three graphical primitives

Schema Navigation Menu

• Tree structured menu • Root is an entity in the E-R graph• Algorithm applied to each level of tree• Output is list of checkable options

Algorithm

AttrForNextStep link, currentStepItems = currentStep.attributesItems += link.attributesforeach relationship r that currentStep is involved in doif r is 2-way relationship then

nextStep = r.otherSide(currentStep)if nextStep is not an aggregation then

Items += nextStepelse

Items += all entities in the nextStep aggregationelse if navigationPath of currentStep contains allentities participating in r then

Items += r.attributes

Algorithm (contd..)

if link forms an aggregation Agg thenforeach relationship r that Agg is involved in do

if r is 2-way relationship thennextStep = r.otherSide(Agg)

if nextStep is not an aggregation thenItems += nextStep

elseItems += nextStep.allEntities

else if navigationPath of Agg contains all entitiesparticipating in r thenItems += r.attributes

Schema Navigation Menu

cS

cSattributes

cSdirectrelnships/entities

cSindirectrelnships/entities

Algorithm Output

• Entity Attributes– Attributes of currentStep

• Relationship Attributes– Attributes of link with currentStep

• Navigational Link– Entities involved in relationships

Graphical Primitives

• Select Menu Item• Move up/down columns• Create filter

View Generation

( ) ) )

Speaker

Presentation

Attend (Users)useridEvent

(eventid

eventid, topic, date, {username}

( name, {event}

Select Menu item

• Entity Attributes and Relationship Attributes– Only select attributes to show– Corresponds to the projection operator in NRA

• Navigational link– Joins the new entity through the navigation relationship

Graphical Primitives (contd.)

• Move up/down columns– Developers can change the view structure

• If column is moved down, they will be asked about the nested column it should be put in.

• Create Filter– Specify a filter predicate– Can be any comparison operator

Automatic Schema Generation

• Generates complex schemas based on two actions– Creating forms/views– Adding columns to forms/views

• Algorithm– Input when adding new forms/views to a page

• Name: Name of the form/view• Attrs: Columns in the new form/view

– Input when adding columns to views• Target: The position in the view where developer clicks• newAttrName: The name of the column to be added• Type: The type of the column to be added

Add new forms/view

/* Triggered when developers add new forms/views to a page. */

Input : name : Name of the new form/view attrs : Columns in the new form/view

onNewFormViewEvent name, attrsAddEntity (name, attrs)

Algorithm to edit view

Input : target : The position in the view where the developer clicks

newAttrName : The name of the column to be added

type : The type of the column to be added

onAddAttributeEvent target, newAttrName, typeif target is a non-nested column of the view or beside theview then

targetEntity = root entity of the viewElse

targetEntity = the entity that the target column belongs to

if NOT isEntity(type) thenif targetEntity is root entity then

AddAttribute (targetEntity, newAttrName, type)else

navigationPath = getNavigationPath(targetEntity)if navigationPath contains two entities then

r = the relationship that connects the two entities in navigationPath

else if exists relationship r that connects all entities in the navigationPath AND exists a constraints that r depends on all 2-way relationships in the navigationPath then

r = getTheRelationship(navigationPath)else

r = createRelationship(navigationPath)create a constraint that r depends on all 2-way

relationships in navigationPath.AddAttribute (r, newAttrName, type)

elseif targetEntity is root entity then

createRelationship (targetEntity, getEntity(type),newAttrName)else

navigationPath = getNavigationPath(target)if exists an aggregation over the navigationPath) then

aggregation = getAggregation(navigationPath)else

aggregation = createAggregation(navigationPath)createRelationship(aggregation, getEntity(type),newAttrName)

Creating/Editing Entities

Creating Relationships

Creating Aggregations

Preliminary Study

• The user study consisted of three groups of two people, pairs, who were given three tasks to complete.– Group 1: Our first pair were two researchers who have advanced

degrees in computer science. Both are actively involved in designing programming and using databases.

– Group 2: Our second pair were both researchers trained with advanced degrees in computer science, but neither is a database expert.

– Group 3: Our third pair were both experienced computer users.

Test Application

• Members of a Yahoo! Group would like to give away unwanted stuff for free. Please create an application that provides the following functionality to members– Post items that they want to give way. Each item includes a name, a

description and the original owner (who posted the item).– List all the items posted by everyone up to now. Each listing should

include the name, description and the owner of the item, and the list of members who have placed a request for the item. The current member can add herself to the requesters list.

– List the items given away by the current member. Each listing should show the name and description of an item, and the persons requesting the item.

Results

• People who had extensive database experience found mapping and logic very easy

• Others did not find the visual presentation quite so intuitive• Group 3 were the most challenged.

Feedback from Tests

• Redesigned the interface • Personalization pop-window• Exposed the notion of containers• Improved database terminology

Related work

• WebML• Visual Query Builder• DreamWeaver• Zoho Creator• Ning• SAP Visual Composer