Web Interfaces and Data Web Interfaces and Data PortalsPortals

John Porter

Department of Environmental Sciences

University of Virginia

Roadmap

Goals for web data access The web interface "tool box" Putting the pieces together Content management systems Exercises

Goals for Ecological Databases

Easy to use – no specialized training required

Able to locate/discover data Able to produce data in desired forms

» Data files» Graphs and Maps» Specialized formats

Users can understand and use data

Circa 1994- The World Wide Web

Availability nearly universal » access rapidly increasing

Easy to use – point and click Difficult to update source

documents» Need to edit static text documents

Circa 1994 - Databases

Difficult and specialized interfaces Powerful capabilities

» Query» Update» Output

Suitable for “about 10%” of ecological data management problems (Gorentz, pers. comm.)

Now!

Web+

DatabaseNew Power!

ubiquitous easy to use easy to query and

update

Examples

An early use, in the Internet Movie Database (http://www.imdb.com) links movies with actors and vis versa» Search for a movie» Get a list of actors» Click on actor’s name to get list of movies

they were in

Retrieve customized data via a web form linked to a database

The Web Portal Toolkit

We have all seen (and used) web pages that were linked to databases

But, HOW IS IT DONE?» Actually there are MANY ways to link

databases and web pages using a wide variety of tools

Piece #1 Web Forms<H1>A Simple Form</H1>

<FORM ACTION=“/cgi-bin/test.cgi" METHOD=get >

<H2>Simple input of a string</H2>

<P>Input a sentence:

<INPUT NAME=inputstring SIZE=60> </P>

<INPUT TYPE=SUBMIT>

</FORM>

Tools: Stand-alone Computer Languages

Visual Basic » PC’s only

C & C++ » powerful (but with power comes responsibility)

Pattern Extraction and Regular expression Language – PERL» Strong search and parsing capabilities

JAVA – object oriented Many more (Python, etc.)

Tools: Computer Languages you can Insert in a Web Page

Active Server Pages – ASP» Works primarily with Microsoft products» Insert Microsoft Visual Basic into web

pages PHP

» open source» Works on a wide number of platforms

Specialized languages associated with specific products (e.g., LITE)

Tools: Languages that Run on your Browser

Active X» Microsoft Internet Explorer only

JAVAscript» Lightweight programs

JAVA» Full-featured object-oriented programs

Tools: Relational Databases

Commercial Databases» Oracle» Microsoft SQL Server

Open Source Databases» MySQL» SQLite (lightweight, limited functions)

All provide:» Structured Query Language (SQL)» Application Programming Interfaces (APIs) to

interact with computer languages

Putting the Pieces Together

Web forms Interface» Web authoring tools like DREAMWEAVER

can automate linking web forms to databases

Computer Programs» Stored & run on Web Server» Stored in web page & run on Web Server» Stored on Web Server & run on Server» Stored in web page & run on Client

Database

Client-Server Architecture

WWW Browser

WWW ServerWWW Server

JAVA

Internet

HTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML file

WWW Server Software

CGI-BINScripts

WWW ClientWWW Client

CGI or SCRIPTS

Apache and others» If you see /cgi-bin/ in a web address, it is running a

program stored in the cgi-bin directory of the server

» CGI stands for “Common Gateway Interface” » it defines the ways to communicate information

between the server and the client Microsoft IIS

» Uses the “scripts” directory on the server to store programs

Client-Server Architecture

WWW Browser

WWW ServerWWW Server

Internet

HTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML file

WWW Server Software

WWW ClientWWW Client

Basic configuration for serving static documents

Adding a Database

WWW Browser

WWW ServerWWW Server

Internet

HTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML file

WWW Server Software

CGI-BINScripts

WWW ClientWWW Client

Database

Database Programs in HTML Documents

Usually webmasters do NOT allow users to add programs to the cgi-bin directory» Dangerous – programs run on the server so only

trusted programs should be in cgi-bin

So, instead a general CGI program (such as PHP) is used to read specific database programs that are integrated into a web page

Adding a Database

WWW Browser

WWW ServerWWW Server

Internet

HTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML fileHTML file

WWW Server Software

CGI-BINScripts

WWW ClientWWW Client

Database

Example Program

This example program uses PHP to talk to a MYSQL relational database

The details of each step will differ between databases and languages, but will share many similarities

Here we insert information from a web form into a database and retrieve an observation number for later use.

/* Connecting, selecting database *//* Connecting, selecting database */ $link = mysql_connect("data.vcrlter.virginia.edu", “myID")$link = mysql_connect("data.vcrlter.virginia.edu", “myID") or die("Could not connect");or die("Could not connect"); mysql_select_db("www") or die("Could not select mysql_select_db("www") or die("Could not select database");database");

/* Performing SQL query *//* Performing SQL query */$query = "insert into waiver $query = "insert into waiver (date_req,station,name,isVert,healthtype) (date_req,station,name,isVert,healthtype) values('" . date("Y-m-d") . ','$_REQUEST[station]', values('" . date("Y-m-d") . ','$_REQUEST[station]', ‘$_REQUEST[fullname]','Y', $_REQUEST[health])"; ‘$_REQUEST[fullname]','Y', $_REQUEST[health])";

$result = mysql_query($query) or die("Unable to log waiver $result = mysql_query($query) or die("Unable to log waiver creation, Query failed");creation, Query failed");

$query = "select max(waiver_num) as waiver_num from $query = "select max(waiver_num) as waiver_num from waiver";waiver";$result = mysql_query($query) or die("Unable to get waiver $result = mysql_query($query) or die("Unable to get waiver number, Query failed");number, Query failed");extract(mysql_fetch_assoc($result));extract(mysql_fetch_assoc($result)); mysql_close($link);mysql_close($link);

Open Open connectionconnection

Set up Set up insertinsert

Store result Store result as PHP as PHP variablevariable

SelectSelect databasedatabase

Run Insert Run Insert QueryQuery

Set up querySet up query

Run QueryRun Query

DON’T PANIC!

By now, you may be swamped with new information, so keep in mind:» There are good tutorials or books to walk

you through the intricacies of SQL, PHP, etc.

» Example programs can frequently be adapted for your use

» Some tools for creating database-enabled web pages can automate many of the basic tasks (e.g., Dreamweaver has basic web form and database interfaces)

Web Portals Made Easy –Content Management

Systems

Content Management Systems (CMS)» Store ALL the web content in a database (not just

data and metadata)» Provide pre-written “modules” containing

– Web form-based interfaces– Associated database programs

» These modules are designed to be easy to use by even novice webmasters to provide specific functions on a web page

Examples are PostNuke, SlashDot, OpenCMS, WIKI

A sample web site developed using POSTNUKE

Administration Page for PostNuke Web Site

Content Management Systems

Advantages» Easy to configure and use» Allow input from users (subject to login)» Many pre-existing modules

Disadvantages» Existing modules may not meet your needs» May be difficult to extract content or

transition to other systems

System Requirements for CMS

To use a CMS, you will need» A Relational Database (e.g., Oracle,

MySQL)» A computer language (JAVA, PHP etc.)» A web server

For example, PostNuke, by default, requires PHP and a MySQL database

Web Services

For some applications your “user” is not a person, it’s a program on another computer» E.g., Provide data to run a model or

graphical package, without human intervention

Web pages work fine for people, but are hard for programs to interpret. For this reason Web Services have developed

Features of Web Services

A Web Service provides the following functions: It “advertises” its methods (it tells you what it

can do for you) Once you select a method to invoke, the Web

Service does the required processing (often accessing a database)

It returns the result as a structured eXtensible Markup Language (XML) document, suitable for use with many types of software

Web Services

Web services tools and techniques are still undergoing rapid development and evolution. Here are some tools and standards related to that development:

SOAP – Simple Object Access Protocol» A “envelope” for packaging requests and receiving

results WSDL – Web Services Description Language

» A language for describing web services and their methods

Useful References & Tutorials

PHP – http://www.php.net MySQL – http://mysql.com HTML Forms, Web Services (SOAP,

WSDL), SQL, XML - http://www.w3schools.com/html/

SQL – http://sqlcourse.com Dreamweaver –

http://www.macromedia.com

Exercises

Learn Structured Query Language at: http://www.sqlcourse.com

Learn to Administer a real PostNuke web site at:

http://mareo.org/~jhp7e/ESA04