Post on 04-Jan-2016
description
Programming theInteractive Web
Shriram KrishnamurthiBrown University
The Interactive Web
• The Web is increasingly “dark matter”
• Numerous Web APIs:– The Common Gateway Interface (CGI)– Java Servlets– Active Server Pages, Java Server Pages– Scripting languages (Perl, PHP, etc)– Microsoft’s Web Services
Where You See This
• URLs become simple:https://onepass.continental.com/asp/statement.asp
• URLs become complex:http://maps.yahoo.com/py/ddResults.py?Pyt=Tmap&tarname=&tardesc=&newname=&newdesc=&newHash=&newTHash=&newSts=&newTSts=&tlt=&tln=&slt=&sln=&newFL=Use+Address+Below&newaddr=3007+Santa+Monica+Boulevard&newcsz=santa+monica,+ca&newcountry=us&newTFL=Use+Address+Below&newtaddr=2815+Santa+Monica+Boulevard&newtcsz=Santa+Monica,+CA+904042409&newtcountry=us&Submit=Get+Directions
Why Dynamic Content?
• Server maintains large database
• Continuous upgrades (software and data)
• Platform independence for clients
• Sometimes, just a Web interface to an existing program (eg, airline reservations)
Red Herring Says
“No software? No problem. You should be moving all your business processes onto the Web anyway.” (The Angler, Anthony B. Perkins, October 2002)
Discusses successful online subscription-based service:“No CD to install, no maintenance, no backup, and no need to upgrade!”
The Orbitz Problem
Not limited to punk script monkeys!
Also found on Web sites of• Microsoft• Apple• the National Science Foundation• …
Programming InteractiveWeb Scripts
Printing a Message(Console)
“Hello, World\n”
exit
Printing a Message(Web)
print <html> <head><title>Test</title> </head> <body> <p>Hello, World!</p> </body> </html>exit
Printing Uptime(Console)
“Uptime: %s\n”
system (“uptime”)
exit
Printing Uptime(Web)
print <html> <head><title>Uptime</title> </head> <body> <p>system (“uptime”)</p> </body> </html>exit
Area of Circle(Console)
r = read “Enter radius: ”
“area is %d\n”
(3.14*r*r)
exit
Area of Circle(Web)
Enter radius:
r = get_binding “radius” bindings<p>area is (3.14*r*r)</p>
Adding Two Numbers(Console)
n1 = read “Enter first: ”
n2 = read “Enter second: ”
“sum: %d\n”
(n1 + n2)
exit
Two User Interfaces
Enter first:
Enter second:
Enter first:
Enter second:
Interacting with Web Scripts
Interacting with Web Scripts
Interacting with Web Scripts
Interacting with Web Scripts
Interacting with Web Scripts
Interacting with Web Scripts
Adding Two Numbers(Web)
Enter first:
n1 = get_binding “n1” bindings<form>…</form>
A Central Problem
• Web scripts write a page, then terminate
• When the user replies, another script reads the form’s bindings and performs the next step
Adding Two Numbers(Web)
Enter first:
n1 = get_binding “n1” bindings<form>…</form>
Enter second:
n2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>
Adding Two Numbers(Web)
Enter first:
n1 = get_binding “n1” bindings<form>…</form>
Enter second:
n2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>
“free variable”
In Practice
• System signals an error– The user doesn’t get a useful answer – The user may not understand the error– User expended a lot of effort and time
• Program captures variable by accident (i.e., it implements dynamic scope!), or
• “internal server error”
Adding Two Numbers(Web)
Enter first:
n1 = get_binding “n1” bindings<form>…</form>
Enter second:
n2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>
Enter second:
n1::
Adding Two Numbers(Web)
Enter first:
n1 = get_binding “n1” bindings<form>…</form>
Enter second:
n1 = get_binding “n1” bindingsn2 = get_binding “n2” bindings<p>sum: (n1 + n2)</p>
The Actual Form
<html><head><title>The Addition Page</title><body><p>Enter the second number:</p><form method="get" action="http://www. .../cgi-second.ss"><input type="hidden" name=“n1" value=“1729"><input type="text" name=“n2" value="0"></form></html>
Problems
• Generating forms is a pain
• Programmer must manually track these hidden fields
• Mistakes can have painful consequences(Worst, silently induce dynamic scope)
Bad News
That’s the easy part!
What’s in a URL?
Let’s go back to this URL:http://maps.yahoo.com/py/ddResults.py?Pyt=Tmap&tarname=&tardesc=&newname=&newdesc=&newHash=&newTHash=&newSts=&newTSts=&tlt=&tln=&slt=&sln=&newFL=Use+Address+Below&newaddr=3007+Santa+Monica+Boulevard&newcsz=santa+monica,+ca&newcountry=us&newTFL=Use+Address+Below&newtaddr=2815+Santa+Monica+Boulevard&newtcsz=Santa+Monica,+CA+904042409&newtcountry=us&Submit=Get+Directions
What’s in a URL?
Let’s go back to this URL:http://maps.yahoo.com/py/ddResults.py?Pyt=Tmap&tarname=&tardesc=&newname=&newdesc=&newHash=&newTHash=&newSts=&newTSts=&tlt=&tln=&slt=&sln=&newFL=Use+Address+Below&newaddr=3007+Santa+Monica+Boulevard&newcsz=santa+monica,+ca&newcountry=us&newTFL=Use+Address+Below&newtaddr=2815+Santa+Monica+Boulevard&newtcsz=Santa+Monica,+CA+904042409&newtcountry=us&Submit=Get+Directions
Breaking it Down
Write it differently:http://maps.yahoo.com/py/ddResults.py? newaddr=3007+Santa+Monica+Boulevard&
newcsz=santa+monica,+ca&
newcountry=us&
newtaddr=2815+Santa+Monica+Boulevard&
newtcsz=Santa+Monica,+CA+904042409&
newtcountry=us&
Submit=Get+Directions
Breaking it Down
Or:http://maps.yahoo.com/py/ddResults.py?
newaddr 3007+Santa+Monica+Boulevard
newcsz santa+monica,+ca
newcountry us
newtaddr 2815+Santa+Monica+Boulevard
newtcsz Santa+Monica,+CA+904042409
newtcountry us
Submit Get+Directions
It looks an awful lot like a function call!
The Real Picture
The scriptand the user
arecoroutines!
script user
Event lines
Control Flow: Back Button
script user script user
A silent action!
Control Flow: Cloning
script user script user
Control Flow: Bookmarks
script user script user
What Programmers Need
“Multiply-resumable and restartable coroutines”
• No language has exactly this – the new control operator for the Web
• How do we implement it?
How to Reengineer Programs
for the Web
What we Want to Write
n1 = read
“Enter first: ”
n2 = read
“Enter second: ”
“sum: %d\n”
(n1 + n2)
exit
What we are Forced to Write:1 of 3
Main () = print
<form action=“f1”> Enter first:
<input name=“n1”>
</form>
What we are Forced to Write:2 of 3
f1 (form) = print <form action=“f2”>
<input hidden name=“n1”
value=form.n1>
Enter second:
<input name=“n2”>
</form>
What we are Forced to Write:3 of 3
f2 (form) = print The sum is
form.n1 + form.n2
Sensitive to Interaction
Why Does this Work?This form has both• a reference to the next program (f2)• the value of the previous input (1729)
Program Structure Destroyed
n1 = read
“Enter first: ”
n2 = read
“Enter second: ”
“sum: %d\n”
(n1 + n2)
exit
Main () = print <form action=“f1”> Enter first: <input name=“n1”> </form>
f1 (form) = print <form action=“f2”> <input hidden name=“n1” value=form.n1> Enter second: <input name=“n2”> </form>
f2 (form) = print The sum is form.n1 + form.n2
The Reengineering Challenge
• Web interfaces have grown up:from “scripts” to “programs” (or
“services”)• Need debugging, maintenance, evolution, …
• We would like a “Web compiler” that– Automatically splits programs by form– Automatically propagates fields– Preserves behavior in the face of bizarre control
flow
The Key Insight
The manual conversionsimply implements the
continuation-passing style transformation!
Change I/O to HTML
n1 = read
“Enter first: ”
n2 = read
“Enter second: ”
“sum: %d\n”
(n1 + n2)
exit
n1 = read <form>Enter first: </form>
n2 = read <form>Enter second: </form>
print <p>sum: (n1 + n2)</p>exit
Change the Input Operator
n1 = read <form>Enter first: </form>
n2 = read <form>Enter second: </form>
print <p>sum: (n1 + n2)</p>exit
n1 = read/web <form>Enter first: </form>
n2 = read/web <form>Enter second: </form>
print <p>sum: (n1 + n2)</p>exit
CPS: Create Function for the
“Rest of the Computation”n1 = read/web
<form>Enter first: </form>
n2 = …
read/web/k
<form>Enter first: </form>
function (n1)
n2 = …
The Result
read/web/k <form>Enter first: </form>
function (n1)
read/web/k <form>Enter second: </form>
function (n2) print <p>sum: (n1 + n2)</p>
Lift Functions
Main () = read/web/k <form>Enter first: </form> f1f1 (n1) = read/web/k <form>Enter second: </form> f2f2 (n2) = print <p>sum: (n1 + n2)</p>
Propagate Free Variables
Main () = read/web/k <form>Enter first: </form> f1f1 (n1) = read/web/k/args n1 <form>Enter second: </form> f2 n1f2 (n1, n2) = print <p>sum: (n1 + n2)</p>
Convert to Web API
f1 (n1) =
read/web/k/args n1
<form>Enter second: </form> f2 n1
f1 (form) = print
<form action=“f2”>
<input hidden name=“n1”
value=form.n1>
Enter second:
<input name=“n2”>
</form>
Resulting Web Application
Main () = print
<form action=“f1”>
Enter first:
<input name=“n1”>
</form>
f1 (form) = print
<form action=“f2”>
<input hidden name=“n1”
value=form.n1>
Enter second:
<input name=“n2”>
</form>
f2 (form) = print
<p>sum:
form.n1 + form.n2</p>
Summary
Three transformations:• Make all value receivers explicit
functions• Make all functions top-level, replace
free variables with explicit parameters
• Replace first-class functions with first-order representations
A Remarkable Coincidence
These are known as:• Conversion to continuation-passing
style• Lambda lifting • Closure conversion
All are standard compiler transformations
The Payoff
• Implementations available for many languages
• Not too hard to implement• Correctness preservation can be
put on a formal foundation• Enables next level of theorems
(robustness in the presence of unreported events)
Moral
Thinking about Web programming as a programming languages question helps
• better explain the nature of the Web• understand common problems• identify and implement solutions• point to shortcomings in language
semantics also!
Broader Payoff
The same results apply to:• GUIs (especially “wizards”)• network i/o (“upcalls”)• blocking to non-blocking i/o• event-driven server construction and a host of other interactive
programs!
Why Can’t Languages do Better?
Program Structure Reinstatement
n1 = read
“Enter first: ”
n2 = read
“Enter second: ”
“sum: %d\n”
(n1 + n2)
exit
n1 = read/web
<form>Enter first:
</form>
n2 = read/web
<form>Enter second:
</form>
<p>sum:
(n1 + n2)</p>
exit
API Implication
Build a richer API!
• Standard APIs offer get_binding, maybe also cookie store and access
• Demand read/web as a primitive
Programmers: Stand up for your rights –make server implementors work harder!
The PLT Scheme Web Server
The Real Primitive
read/web lies slightly:n1 = read/web
<form>Enter first:</form>
We provide send/suspend:n1 = send/suspend
function (k)
<form action=k>Enter first:</form>
send/suspend generates theURL that resumes computation
Addition Servlet
n1 = send/suspend function (k) <form action=k>Enter first:</form>n2 = send/suspend function (k) <form action=k>Enter second:</form>print <p>sum: (n1 + n2)</p>exit
In Scheme
Parenthetical syntax aside, this is a functioning PLT Scheme servlet
But we do more than just add numbers …
Scenario
GDB Servlet
driver_loop (prev_cmd) =
let next_cmd =
send/suspend (i/o_page prev_cmd)
driver_loop (next_cmd)
i/o_page (cmd) =
<p><strong>cmd</strong><br>
read( send (cmd, gdb_process))</p>
function (k)
<form action=k>Next cmd:</form>
Broader Applicability
• Can generalize over other shell-like programs (just provide a parser)
• Can allow tree-shaped exploration with about ten more lines of code (try doing this in GDB!)
• Retrofitting Web interfaces:– “easy” for databases– tricky for already interactive
programs
The CONTINUE Server
• Conference paper manager• Name inspired by START server (UMd)• Handles submission and review
phases• Used by several conferences: PADL
2002, PADL 2003, FDPE 2003, Scheme 2003, …
• Two interesting scenarios
Scenario
“register my email”
visits URL
“visit this URL!”
“okay, registered!”
Email Confirmation Servlet
addr = send/suspend
function (k)
<form action=k>Email:</form>
send/suspend /* ignore response */
function (k)
send_mail (addr, k)
<p>We sent mail to addr</p>
add_to_database (addr)
Scenario
Scenario
One-Shot Guest URLs
guest_review =
send/finish
function (k)
<form action=k>
Overall Rating: <input …>
Expertise: <input …>
</form>
Embedded Server
Embedding the server in the DrScheme programming environment enables
• sophisticated servlet debugging• implement all documentation
features through “the Web”:– indexing– searching– user-sensitive customization
Safety Errors
Safety Errors
Stepping Through Code
Stepping Through Code
Other Research Issues
• Type systems for forms (general problem for scripting applications)
• Soundness/safety lifted to level of Web interactions
• Formal verification (theorem proving, model checking) – “impossible” at present
Summary
Interactive Web Programs
• Web interactions are more complex than we might suppose
• Programming language semantics offers analysis and solutions – and theorems for formal software engineering!
• A custom server with a better API enables more natural and more complex programs
Future Work
• Some languages inhibit the transformations
• send/suspend and send/finish only scratch the surface
• Lots of work to be done on type systems• Many other kinds of interaction may be
amenable to a similar analysis• Formally verify the server and CONTINUE• Keep eating our own dogfood!
Collaborators
• Matthias Felleisen (Northeastern)
• Robert Bruce Findler (Chicago)
• Matthew Flatt (Utah)
• Paul T. Graunke (Northeastern)
Potential Problem
• Problem: Transmitted Web pages contain the values of free variables, maybe including the current account balance.
• Question: so what?
• Solution: Encrypt data when sending, decrypt when it returns.
Potential Problem
• Problem: Where does the state of mutable variables and mutable records go?
• Solution 1: Use store-passing transformation to make store explicit. Pass it along to the client (as hidden field in Web form).
Potential Problem
• Problem: Users can clone browser windows, copy pages, … and with it the store.
• Solution 2: Register cells (store) and retain on the server or move over as a cookie.