Programming the Interactive Web

Programming theInteractive Web

Shriram KrishnamurthiBrown University

The Interactive Web

• The Web is increasingly “dark matter”

• Numerous Web APIs:– The Common Gateway Interface (CGI)– Java Servlets– Active Server Pages, Java Server Pages– Scripting languages (Perl, PHP, etc)– Microsoft’s Web Services

Where You See This

• URLs become simple:https://onepass.continental.com/asp/statement.asp

• URLs become complex:http://maps.yahoo.com/py/ddResults.py?Pyt=Tmap&tarname=&tardesc=&newname=&newdesc=&newHash=&newTHash=&newSts=&newTSts=&tlt=&tln=&slt=&sln=&newFL=Use+Address+Below&newaddr=3007+Santa+Monica+Boulevard&newcsz=santa+monica,+ca&newcountry=us&newTFL=Use+Address+Below&newtaddr=2815+Santa+Monica+Boulevard&newtcsz=Santa+Monica,+CA+904042409&newtcountry=us&Submit=Get+Directions

Why Dynamic Content?

• Server maintains large database

• Continuous upgrades (software and data)

• Platform independence for clients

• Sometimes, just a Web interface to an existing program (eg, airline reservations)

Red Herring Says

“No software? No problem. You should be moving all your business processes onto the Web anyway.” (The Angler, Anthony B. Perkins, October 2002)

Discusses successful online subscription-based service:“No CD to install, no maintenance, no backup, and no need to upgrade!”

The Orbitz Problem

Not limited to punk script monkeys!

Also found on Web sites of• Microsoft• Apple• the National Science Foundation• …

Programming InteractiveWeb Scripts

Printing a Message(Console)

print

“Hello, World\n”

exit

Printing a Message(Web)

print <html> <head><title>Test</title> </head> <body> <p>Hello, World!</p> </body> </html>exit

Printing Uptime(Console)

print

“Uptime: %s\n”

system (“uptime”)

exit

Printing Uptime(Web)

print <html> <head><title>Uptime</title> </head> <body> <p>system (“uptime”)</p> </body> </html>exit

Area of Circle(Console)

r = read “Enter radius: ”

print

“area is %d\n”

(3.14*r*r)

exit

Area of Circle(Web)

Enter radius:

r = get_binding “radius” bindings<p>area is (3.14*r*r)</p>

Adding Two Numbers(Console)

n1 = read “Enter first: ”

n2 = read “Enter second: ”

print

“sum: %d\n”

(n1 + n2)

exit

Two User Interfaces

Enter first:

Enter second:

Enter first:

Enter second:

Interacting with Web Scripts

Adding Two Numbers(Web)

Enter first:

n1 = get_binding “n1” bindings<form>…</form>

A Central Problem

• Web scripts write a page, then terminate

• When the user replies, another script reads the form’s bindings and performs the next step


Enter first:


Enter second:

n2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>


Enter first:


Enter second:


“free variable”

In Practice

• System signals an error– The user doesn’t get a useful answer – The user may not understand the error– User expended a lot of effort and time

• Program captures variable by accident (i.e., it implements dynamic scope!), or

• “internal server error”


Enter first:


Enter second:


Enter second:

n1::


Enter first:


Enter second:

n1 = get_binding “n1” bindingsn2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>

The Actual Form

<html><head><title>The Addition Page</title><body><p>Enter the second number:</p><form method="get" action="http://www. .../cgi-second.ss"><input type="hidden" name=“n1" value=“1729"><input type="text" name=“n2" value="0"></form></html>

Problems

• Generating forms is a pain

• Programmer must manually track these hidden fields

• Mistakes can have painful consequences(Worst, silently induce dynamic scope)

Bad News

That’s the easy part!

What’s in a URL?

Let’s go back to this URL:http://maps.yahoo.com/py/ddResults.py?Pyt=Tmap&tarname=&tardesc=&newname=&newdesc=&newHash=&newTHash=&newSts=&newTSts=&tlt=&tln=&slt=&sln=&newFL=Use+Address+Below&newaddr=3007+Santa+Monica+Boulevard&newcsz=santa+monica,+ca&newcountry=us&newTFL=Use+Address+Below&newtaddr=2815+Santa+Monica+Boulevard&newtcsz=Santa+Monica,+CA+904042409&newtcountry=us&Submit=Get+Directions

Breaking it Down

Write it differently:http://maps.yahoo.com/py/ddResults.py? newaddr=3007+Santa+Monica+Boulevard&

newcsz=santa+monica,+ca&

newcountry=us&

newtaddr=2815+Santa+Monica+Boulevard&

newtcsz=Santa+Monica,+CA+904042409&

newtcountry=us&

Submit=Get+Directions

Breaking it Down

Or:http://maps.yahoo.com/py/ddResults.py?

newaddr 3007+Santa+Monica+Boulevard

newcsz santa+monica,+ca

newcountry us

newtaddr 2815+Santa+Monica+Boulevard

newtcsz Santa+Monica,+CA+904042409

newtcountry us

Submit Get+Directions

It looks an awful lot like a function call!

The Real Picture

The scriptand the user

arecoroutines!

script user

Event lines

Control Flow: Back Button

script user script user

A silent action!

Control Flow: Cloning


Control Flow: Bookmarks


What Programmers Need

“Multiply-resumable and restartable coroutines”

• No language has exactly this – the new control operator for the Web

• How do we implement it?

How to Reengineer Programs

for the Web

What we Want to Write

n1 = read

“Enter first: ”

n2 = read

“Enter second: ”

print

“sum: %d\n”

(n1 + n2)

exit

What we are Forced to Write:1 of 3

Main () = print

<form action=“f1”> Enter first:

<input name=“n1”>

</form>


f1 (form) = print <form action=“f2”>

<input hidden name=“n1”

value=form.n1>

Enter second:


</form>


f2 (form) = print The sum is

form.n1 + form.n2

Sensitive to Interaction

Why Does this Work?This form has both• a reference to the next program (f2)• the value of the previous input (1729)

Program Structure Destroyed

n1 = read

“Enter first: ”

n2 = read


print

“sum: %d\n”

(n1 + n2)

exit

Main () = print <form action=“f1”> Enter first: <input name=“n1”> </form>

f1 (form) = print <form action=“f2”> <input hidden name=“n1” value=form.n1> Enter second: <input name=“n2”> </form>

f2 (form) = print The sum is form.n1 + form.n2

The Reengineering Challenge

• Web interfaces have grown up:from “scripts” to “programs” (or

“services”)• Need debugging, maintenance, evolution, …

• We would like a “Web compiler” that– Automatically splits programs by form– Automatically propagates fields– Preserves behavior in the face of bizarre control

flow

The Key Insight

The manual conversionsimply implements the

continuation-passing style transformation!

Change I/O to HTML

n1 = read

“Enter first: ”

n2 = read


print

“sum: %d\n”

(n1 + n2)

exit

n1 = read <form>Enter first: </form>

n2 = read <form>Enter second: </form>

print <p>sum: (n1 + n2)</p>exit

Change the Input Operator

n1 = read <form>Enter first: </form>

n2 = read <form>Enter second: </form>


n1 = read/web <form>Enter first: </form>

n2 = read/web <form>Enter second: </form>


CPS: Create Function for the

“Rest of the Computation”n1 = read/web

<form>Enter first: </form>

n2 = …

read/web/k

<form>Enter first: </form>

function (n1)

n2 = …

The Result

read/web/k <form>Enter first: </form>

function (n1)

read/web/k <form>Enter second: </form>

function (n2) print <p>sum: (n1 + n2)</p>

Lift Functions

Main () = read/web/k <form>Enter first: </form> f1f1 (n1) = read/web/k <form>Enter second: </form> f2f2 (n2) = print <p>sum: (n1 + n2)</p>

Propagate Free Variables

Main () = read/web/k <form>Enter first: </form> f1f1 (n1) = read/web/k/args n1 <form>Enter second: </form> f2 n1f2 (n1, n2) = print <p>sum: (n1 + n2)</p>

Convert to Web API

f1 (n1) =

read/web/k/args n1

<form>Enter second: </form> f2 n1

f1 (form) = print

<form action=“f2”>


value=form.n1>

Enter second:


</form>

Resulting Web Application

Main () = print


Enter first:


</form>

f1 (form) = print



value=form.n1>

Enter second:


</form>

f2 (form) = print

<p>sum:

form.n1 + form.n2</p>

Summary

Three transformations:• Make all value receivers explicit

functions• Make all functions top-level, replace

free variables with explicit parameters

• Replace first-class functions with first-order representations

A Remarkable Coincidence

These are known as:• Conversion to continuation-passing

style• Lambda lifting • Closure conversion

All are standard compiler transformations

The Payoff

• Implementations available for many languages

• Not too hard to implement• Correctness preservation can be

put on a formal foundation• Enables next level of theorems

(robustness in the presence of unreported events)

Moral

Thinking about Web programming as a programming languages question helps

• better explain the nature of the Web• understand common problems• identify and implement solutions• point to shortcomings in language

semantics also!

Broader Payoff

The same results apply to:• GUIs (especially “wizards”)• network i/o (“upcalls”)• blocking to non-blocking i/o• event-driven server construction and a host of other interactive

programs!

Why Can’t Languages do Better?

Program Structure Reinstatement

n1 = read

“Enter first: ”

n2 = read


print

“sum: %d\n”

(n1 + n2)

exit

n1 = read/web

<form>Enter first:

</form>

n2 = read/web

<form>Enter second:

</form>

print

<p>sum:

(n1 + n2)</p>

exit

API Implication

Build a richer API!

• Standard APIs offer get_binding, maybe also cookie store and access

• Demand read/web as a primitive

Programmers: Stand up for your rights –make server implementors work harder!

The PLT Scheme Web Server

The Real Primitive

read/web lies slightly:n1 = read/web

<form>Enter first:</form>

We provide send/suspend:n1 = send/suspend

function (k)

<form action=k>Enter first:</form>

send/suspend generates theURL that resumes computation

Addition Servlet

n1 = send/suspend function (k) <form action=k>Enter first:</form>n2 = send/suspend function (k) <form action=k>Enter second:</form>print <p>sum: (n1 + n2)</p>exit

In Scheme

Parenthetical syntax aside, this is a functioning PLT Scheme servlet

But we do more than just add numbers …

Scenario

GDB Servlet

driver_loop (prev_cmd) =

let next_cmd =

send/suspend (i/o_page prev_cmd)

driver_loop (next_cmd)

i/o_page (cmd) =

<p><strong>cmd</strong><br>

read( send (cmd, gdb_process))</p>

function (k)

<form action=k>Next cmd:</form>

Broader Applicability

• Can generalize over other shell-like programs (just provide a parser)

• Can allow tree-shaped exploration with about ten more lines of code (try doing this in GDB!)

• Retrofitting Web interfaces:– “easy” for databases– tricky for already interactive

programs

The CONTINUE Server

• Conference paper manager• Name inspired by START server (UMd)• Handles submission and review

phases• Used by several conferences: PADL

2002, PADL 2003, FDPE 2003, Scheme 2003, …

• Two interesting scenarios

Scenario

“register my email”

visits URL

“visit this URL!”

“okay, registered!”

Email Confirmation Servlet

addr = send/suspend

function (k)

<form action=k>Email:</form>

send/suspend /* ignore response */

function (k)

send_mail (addr, k)

<p>We sent mail to addr</p>

add_to_database (addr)

Scenario

One-Shot Guest URLs

guest_review =

send/finish

function (k)

<form action=k>

Overall Rating: <input …>

Expertise: <input …>

</form>

Embedded Server

Embedding the server in the DrScheme programming environment enables

• sophisticated servlet debugging• implement all documentation

features through “the Web”:– indexing– searching– user-sensitive customization

Safety Errors

Stepping Through Code

Other Research Issues

• Type systems for forms (general problem for scripting applications)

• Soundness/safety lifted to level of Web interactions

• Formal verification (theorem proving, model checking) – “impossible” at present

Summary

Interactive Web Programs

• Web interactions are more complex than we might suppose

• Programming language semantics offers analysis and solutions – and theorems for formal software engineering!

• A custom server with a better API enables more natural and more complex programs

Future Work

• Some languages inhibit the transformations

• send/suspend and send/finish only scratch the surface

• Lots of work to be done on type systems• Many other kinds of interaction may be

amenable to a similar analysis• Formally verify the server and CONTINUE• Keep eating our own dogfood!

Collaborators

• Matthias Felleisen (Northeastern)

• Robert Bruce Findler (Chicago)

• Matthew Flatt (Utah)

• Paul T. Graunke (Northeastern)

Potential Problem

• Problem: Transmitted Web pages contain the values of free variables, maybe including the current account balance.

• Question: so what?

• Solution: Encrypt data when sending, decrypt when it returns.

Potential Problem

• Problem: Where does the state of mutable variables and mutable records go?

• Solution 1: Use store-passing transformation to make store explicit. Pass it along to the client (as hidden field in Web form).

Potential Problem

• Problem: Users can clone browser windows, copy pages, … and with it the store.

• Solution 2: Register cells (store) and retain on the server or move over as a cookie.

Programming the Interactive Web

Documents

Transcript of Programming the Interactive Web