Mastering Cgi

MASTERING CGI

The cgiemail home page

Welcome to the cgiemail home page. For detailed information, consult one of the following pages:

The cgiemail webmaster guide introduces the software and gives details about downloading and installing it.

The cgiemail ISP page lists web-hosting services that already have cgiemail.

The cgiemail user guide gives complete instructions for using the software.

The cgiemail example page will aid you in the design of your own forms.

Information that pertains both to webmasters and to users follows.

Where do I turn for help?

Why are you giving this software away?

This is great! How can I thank you?

Status of cgiemail, December 2002

The original author of cgiemail, Bruce Lewis, is no longer working for MIT. Although cgiemail does have an official maintainer on the MIT I/S staff, there are no firm plans for future development, and support is limited/slow.

For this reason, we recommend that you ask cgiemail questions on the newsgroup noted below before writing to the cgiemail address. We do not personally get to this newsgroup nearly as often as we'd like, but there are generally other readers there who can provide help.

This website will continue to be available, and is updated occasionally.

Updated status of cgiemail, September 2010

Cgiemail is not actively receiving attention. These pages are being left up because they have useful information, but be aware that the usenet newgroup mentioned in "Where do I turn for help?" is now defunct.

1. Where do I turn for help?

If you have trouble using cgiemail, please ask for help on the newsgroup comp.infosystems.www.authoring.cgi, and include the following information:

Be sure to put "cgiemail" in the subject line.

Be sure the "Distribution" is set to "world". Too narrow a distribution may prevent people from seeing your message.

Include the URL of the form you're having trouble with. If you don't want to post this URL, please try to create another form which exhibits the same problem and post its URL instead. It is much easier for readers to try the form live than to save it locally and modify the ACTION to their own servers. Emailing the HTML isn't good enough.

Include the exact error message you're getting - cut and paste if possible.

Name the WWW server you're using if you know it (e.g. CERN 3.0).

If you don't have access to Usenet, try Deja News. As a last resort, send e-mail to cgiemail at mit.edu.

2. Why are you giving this software away?

The purpose of this software is to function efficiently and reliably for MIT. It is advantageous to give it away, because people outside MIT are often the best testers of the software. Bugs can be found and fixed before they affect MIT. Bug fixes and improvements have been supplied by non-MIT people, enhancing the software's reliability and functionality. For more details, see Free Software: Solving the Buy/Build Dilemma.

Another good reason for giving software away is that it provides an example that encourages other people to freely share their own software, some of which may be useful to MIT. This spirit of openness and cooperation will make the Internet a better place for everyone.

3. This is great! How can I thank you?

Help other users who ask cgiemail questions in the comp.infosystems.www.authoring.cgi newsgroup. To facilitate this, we have a FAQ list with answers you can cut and paste. (If you see something you think should be on the FAQ, let us know.)

cgiemail

Last modified: Fri Dec 20 12:11:34 EST 2002

The cgiemail webmaster guide

This guide answers the following questions:

What is cgiemail and why would I want it?

If I don't want cgiemail, what else is there?

What security issues are there?

Can I just download a binary?

Where can I download cgiemail source code?

How do I build and install it?

How do I tell if the installation worked?

How do I find out about new releases?

What's new in release 1.6?






What is cgiemail and why would I want it?

The purpose of cgiemail, a CGI program written in C for Unix, is to take the input of WWW forms and convert it to an e-mail format defined by the author of the WWW form, so that the resulting e-mail messages are not difficult to sort through. Using it requires slightly more work than forms-to-email programs with less flexibility in e-mail format. Nevertheless, users with no programming experience can follow the simple step-by-step instructions in the cgiemail user guide.

It's easy to get cgiemail up and running. Ten minutes is the median time reported. Webmasters who are familiar with Unix frequently report getting cgiemail up in two minutes.

You do not want cgiemail if either of these is true:

Controlling the layout of e-mail messages is unimportant to your information providers.

Your WWW server is not running a variant of Unix. Sorry, there is no version for Windows or MacOS here. See the next section for other possibilities.

You do want cgiemail if any of these is true:

Controlling the layout of e-mail messages is important to your information providers.

You have a number of information providers putting forms your web site and you don't want to install a perl script for every form.

You want to facilitate the creation of forms accessible to users with disabilities.

If I don't want cgiemail, what else is there?

Others have developed programs with the same functionality:

Gypsy Mail is a free cgiemail clone written in Python.

MailPost for Windows 32 bit webservers (primarily O'Reilly's Website)

If cgiemail isn't what you're looking for, check the following sites for programs that might fit your needs more precisely.

At Yahoo, see ``Internet:World Wide Web:Gateways:Email Gateways''.

The CGI Resource Index lists programs and scripts sorted by language, e.g. C and C++.

ScriptSearch catalogs scripts by language and purpose, e.g. C/C++ Communication Mail utilities.

In the WWW FAQ, see ``How can users send me comments and/or email?''.

What security issues are there?

Here are some notes about cgiemail in relation to the three types of security listed in The WWW Security FAQ (January 1998 version). I hope you find these notes helpful, but you should refer to the file mit-copyright.h re. MIT's non-warranty of this software.

There are basically three overlapping types of risk, listed here with the extent to which cgiemail has each risk.

Bugs or misconfiguration problems in the Web server that allow unauthorized remote users to:

Steal confidential documents not intended for their eyes.

Small risk: A private or confidential document stored in the Web site's document tree (not elsewhere in the filesystem) can fall into the hands of unauthorized individuals if all of the following conditions are met: The web server must allow PATH_TRANSLATED to point to said document even if direct access to the document would be denied.

The document must look like a template file to cgiemail, i.e. contain text enclosed in square brackets.

The text in at least one set of square brackets must be known to the attacker.

The URL of the private document must be known to the attacker.

Execute commands on the server host machine, allowing them to modify the system.

Non-risk: Since cgiemail does not pass any form data to the shell, it is not vulnerable to ``special character'' holes. The code has been checked for buffer overflows. I believe there are none, but I must mention that MIT owns the software and makes no guarantees. Refer to the file mit-copyright.h in the distribution.

Gain information about the Web server's host machine that will allow them to break into the system.

Non-risk: cgiemail does not give users such information.

Launch denial-of-service attacks, rendering the machine temporarily unusable.

Non-risk: cgiemail does not increase vulnerability to denial-of-service attacks.

Browser-side risks, including:

Active content that crashes the browser, damages the user's system, breaches the user's privacy, or merely creates an annoyance.

Non-risk: does not apply to cgiemail

The misuse of personal information knowingly or unkowingly provided by the end-user.

Small risk: form creators can trace the origin of mail sent with cgiemail to about the same extent regular e-mail can be traced. They can also choose to see what browser software the end-user uses, since most browser software identifies itself to the web server. Most modern mail software also identifies itself in each mail message (the X-Mailer header), so this is not considered a breach of privacy.

Interception of network data sent from browser to server or vice versa via network eavesdropping. Eavesdroppers can operate from any point on the pathway between browser and server.

Risk: With cgiemail as with any form-to-mail program, eavesdroppers can also operate on any point on the pathway between the web server and the end reader of the mail. Since there is no encryption built into cgiemail, it is not recommended for confidential information such as credit card numbers.

Can I just download a binary?

If your Unix platform can run binaries from one of the following, and sendmail is in the same place as listed below, then you can download the cgiemail and cgiecho binaries, place them in your cgi-bin directory and "chmod 755 cgiemail cgiecho" to make sure they're executable.Platform sendmail location Release

HP-UX A.09.01 /usr/lib/sendmail 1.5

IRIX 5.3 /usr/lib/sendmail 1.5

Linux 2.0.32 /usr/sbin/sendmail 1.5

NetBSD 1.2 /usr/sbin/sendmail 1.5

SunOS 5.5.1 /usr/lib/sendmail 1.5

Where can I download cgiemail source?

You can download it right here, and unpack it with the following commands at the Unix shell prompt in any directory where you have write permission. Please note the time you download it. Later you will be asked for the number of minutes from the time you download it to the time when you get it up, running and tested.

cgiemail-1.6.tar.gz (28k compressed with the GNU software utility gzip)

gzip -dc cgiemail-1.6.tar.gz | tar xvf -

cgiemail-1.6.tar (120k uncompressed, in case you don't have gzip)

tar xvf cgiemail-1.6.tar

To pick up the latest beta version and try it out:

cgiemail-beta.tar.gz (38k compressed with the GNU software utility gzip)

gzip -dc cgiemail-beta.tar.gz | tar xvf -

How do I build and install it?

The simplest way to build it is by typing the following commands at the Unix shell prompt.

cd cgiemail-1.6

./configure

make

Before you do the configure step, you may want to type ./configure --help and decide if you want to enable/disable any optional features.

If cgiemail fails to compile with a message like header file 'unistd.h' not found, then your C compiler is not installed correctly. (It could also mean your version of Unix hasn't been upgraded since 1990, but probably not.) Consult whoever gave or sold you the C compiler.

Once you have built cgiemail, simply copy the cgiemail and cgiecho binaries into your cgi-bin directory. If you are a non-webmaster installing cgiemail on a server that allows user CGI programs, you will usually need to add the extension .cgi to cgiemail and cgiecho. Contact your webmaster for details.

Note: An additional program, cgicso, is built. You can delete this program and ignore any warnings that occur while it compiles. It is only useful at MIT, and is included only to avoid having separate source trees for internal and external use.

How do I tell if the installation worked?

Try opening the URL for where you've installed cgiemail, e.g.

http://www.myname.com/cgi-bin/cgiemail/

You should be taken to a Using cgiemail page.

If you get an error that doesn't say cgiemail at the bottom, you either didn't configure your WWW server to run CGI programs, or you put cgiemail in the wrong directory. Consult your server documentation.

If you do get the right error, install the files testce.html and testce.txt in your top-level htdocs directory. (If you install them elsewhere, be sure to set the correct ACTION in testce.html.) This test form gives you the opportunity to send feedback, e.g. how many minutes it took from the time you downloaded cgiemail to the time it's up and running. (If it isn't up and running, obviously the mail won't be sent.) You can also choose not to send the feedback except to your own address for testing purposes.

If you get no errors, feel free to remove testce.html and testce.txt. You can also throw away the cgiemail-1.6 directory and its contents. Go ahead and notify your users that cgiemail is up and running. If, on the other hand you get errors when you try the test form, follow the appropriate link below:

503 cannot write temporary file

500 sendmail exit with error message (cgiemail 1.2 and later)

500 Exit status of sendmail (cgiemail 1.1)

How do I find out about new releases?

It is in your best interest to find out when new releases come out. They may have features you have been waiting for, and might have fixes for a bug you haven't encountered yet but would encounter later. If you run a web-hosting service, you get better prominence on the ISP page if you keep up-to-date. The mailing list has only announcements. You will only get a message every few months, so there is no reason not to subscribe.

Send a message to [email protected] with the following in the body (not the subject line) of your message:

SUB CGIEMAIL John Doe

That's if your name is John Doe; otherwise substitute your own name. You'll get a confirmation message. Ignore the instructions on how ``to send a message to all the people currently subscribed''; the list is for announcements only.

A reference.com archive is available.


This release on 1998-04-14 fixes these bugs:

Two code errors prevented compilation on certain platforms.

For cgifile, the incoming.txt file is not allowed to be a link.


This release on 1998-01-26 makes success/failure templates work consistently and be more supportable.

A bug is fixed that often prevented success/failure templates from working. Thanks to Ed Orcutt of eosys for tracking it down.

Failures in a success template are reported correctly.

Any failure in a failure template falls back on the default failure page.

Error information is better divided between cgierrmsg and cgierrinfo.

Although it still needs to be installed by hand, cgifile is compiled by default.


This release on 1997-10-27 substantially increases the functionality of the cgiemail package.

A new program, cgifile, supports appending incoming information to an incoming.txt file.

Support is added for success/failure templates incorporating form input values.

Support is added for %U and %H formatting (URL and HTML encoding)

Support is added for special variables: cgidate, cgirelease, cgierrmsg, cgierrinfo.

Required fields are enforced entirely through the template file. This fixes a bug in which cgiemail would reject a form with multiple same-name required inputs if only one of them was left blank.

Leading whitespace in the mail template is removed that otherwise would have confused sendmail.

Macintosh users are no longer required to upload template files as text. End-of-line markers are automatically converted to Unix format at runtime.

A configuration option (turned off by default) allows bounced mail to go to the owner of the template file rather than the web server's mailbox.

The required prefix no longer requires a hyphen (better interaction with JavaScript).

If a pipe to sendmail cannot be opened, the error message includes the command that failed.


This release on 1997-05-29 includes the following changes:

Feature: A test form and template are included to facilitate feedback and simplify testing.

Feature: CGI environment variables are enabled unless explicitly disabled at configure time.

Feature: A mail header, X-Mailer, gives cgiemail release number, the URL of the HTML form, and the form ACTION.

Bugfix: A workaround is included for a bug in NetBSD /bin/sh that prevented configure from finding the path for sendmail.

Bugfix: Messages from sendmail are ignored if its exit status is available and indicates success.


This release on 1997-02-20 eases debugging of various problems with sendmail.

A new hidden variable, cgiemail-mailopt, is introduced. Its only option so far is to allow synchronous mail delivery, to ease debugging when no mail gets through and the user doesn't have access to the bounce message.

NetBSD users should not encounter any more problems with the configure script, which was generated with autoconf 2.10.

If the configure script can't find the path to sendmail, it defaults to sendmail rather than /bin/false, which gives some chance of success, and makes the error message more understandable if sendmail can't be found.

Because of difficulty on various OS/httpd combinations in getting a meaningful return value from pclose(), sendmail is assumed to succeed if and only if it produces no output. Any output is treated as an error and displayed to the user. No more groveling through server error logs!

The mkcmtform script and manpage are no longer included. They were never useful outside MIT and only confused people.

If the ACTION is not set correctly, a page is displayed pointing the user to the user guide.


This release on 1996-07-25 is primarily bugfixes. The most visible change is that the release number appears at the bottom of the default success/error pages, to facilitate debugging problems at sites running unknown versions. The release name has been changed to a more conventional format than that used in mit-dcns-cgi950822 and prior.

Other changes are:

fix a bug that prevented the required- prefix from working for checkboxes

prevent cgiemail from spinning if the client goes away

automatically determine where the sendmail executable lives

fix the signal mask in order to get the exit status of sendmail

send CGI headers as proper HTTP headers (ending in CRLF)

cgiemail <cgiemail at mit.edu>

Last modified: Fri Dec 20 12:28:48 EST 2002

Validate me

The cgiemail user guide

This guide will help you write a WWW form that sends an e-mail message to you. The following steps are required:

Create an e-mail template.

Put a link to the template on your page.

Decide if a mailto: link will do.

Create the HTML form.

Create more advanced HTML forms.

Make sure the ACTION is correct.

Try out your form with cgiecho.

Go live with cgiemail.

Debug if you don't get mail

The following steps are optional.

Add text to the success page.

Use an alternate success page.

Make some inputs required.

Specify formatting for some inputs.

Use CGI environment variables.

1. Create an e-mail template.

Before you start receiving e-mail messages through the web, you should decide what these messages should look like. Create an ASCII file, called an e-mail template, that looks something like this:

To: [email protected] HEADER LINES

Subject: questions three

blank line

What is your name? [yourname]

What is your quest? [quest] BODY

What is your favourite colour? [colour]

In one sense, this template is free-form. People who want to send you e-mail can download this template, fill it out, and mail it to you. However, the template will also be used by the cgiemail program, so before you upload the file to your WWW server, be careful to follow these guidelines:

Wherever you want the user of your form to supply information, use a single word inside square brackets with no spaces, e.g. Your name: [yourname]. Not [Put your name here].

Make sure the address in the To: field is correct.

If there are blank lines among the header lines, remove them.

If there are blank lines before the header lines, remove them.

Make sure all your header lines are valid. The first character on the line must be a letter. Most information should go in the message body; don't make up your own headers.

Make sure there is a blank line between the header lines and the body.

Make sure you save it as ASCII text. For example, if you are using Microsoft Word, use "Save As" and choose "Text Only with Line Breaks."

If you created the file on a Mac, be sure to upload it as text, i.e. CR's translated. (Unix computers have different codes denoting the end of a line than Mac's do, so your file might look like one long line to the Unix computer.)

Within these guidelines there is a lot of flexibility. You can put Bcc:, X-Face:, or any other header in the headers. You can put things like Cc: [yourname] in the headers. Be creative. Just don't put anything in there you wouldn't want your webmaster to see, because that's where bounced messages go.

Now go ahead and upload your e-mail template to the WWW server and look at it with your WWW browser.

2. Put a link to the template on your page.

Here's an example:

Would you like to cross the bridge? Download my "questions three" form and send it to <[email protected]>.

Even after you create your WWW form, you will want to leave this link in to increase accessibility to users with disabilities.

3. Decide if a mailto: link will do.

Already, without any complicated HTML, you have a way for people on the WWW to send you the information you want. Before you go to the effort of making an HTML

form, decide if it's really worth it. Forms on the WWW have two particular disadvantages:

You will get a lot of frivolous e-mail from people who are merely ``surfing the web.''

The user's e-mail address is typed manually, and is often mistyped, so that you have no way to reply. This is less of a problem with mailto: links.

4. Create the HTML form.

If you've decided to create an HTML form, you need to give people a way to supply an e-mail address. With the mailto: link, their mailer would supply the From: address for them. But now you need to add a line to the top of your e-mail template like this:

From: [email]

Here is an example HTML form.

Your e-mail address:

Your name:

Your quest:

Your favourite colour:

(This example doesn't actually send e-mail.)

This is the HTML source:

<FORM METHOD="POST"

ACTION="http://web.mit.edu/bin/cgiecho/wwwdev/cgiemail/questions3.txt">

Your e-mail address: <INPUT NAME="email"><p>

Your name: <INPUT NAME="yourname"><p>

Your quest: <INPUT NAME="quest"><p>

Your favourite colour: <INPUT NAME="colour"><p>

<INPUT TYPE="submit" value="Send e-mail">

</FORM>

This is a very simple example. Note that the NAME of each input corresponds to what you previously put in the e-mail template. In this example they are email, yourname, quest, and colour. This is the key concept in using cgiemail. Be careful to make them exactly the same; if you put NAME="colour" in your HTML form and [color] (note the spelling difference) in your e-mail template, the input will not show up in the e-mail.

5. Create more advanced HTML forms.

To learn to create more complicated forms, read NCSA's guide and/or an HTML book. All of their example forms can be converted to cgiemail forms merely by changing the ACTION. Unlike other forms-to-email programs, you are not required to use hidden inputs with special names.

All types of inputs (radio buttons, etc.) work the same way. Each input needs a NAME, and that name must appear within square brackets in your e-mail template. It's that simple. To get more ideas, see the cgiemail example page.

6. Make sure the ACTION is correct.

The trickiest part of the HTML form is getting the ACTION set correctly. Start with the URL of your e-mail template, then split it into two parts, e.g.

http://web.mit.edu/wwwdev/cgiemail/questions3.txt

\ /\ /

`--- Part 1 ---' ` -------- Part 2 ----------'

First type the URL of your e-mail template into a web browser and make sure it's correct. Then put the script name in the middle. Usually this is ``/cgi-bin/cgiecho'', but it depends on how your server is configured. On web.mit.edu it happens to be ``/bin/cgiecho'', thus my ACTION looks like this:

http://web.mit.edu/bin/cgiecho/wwwdev/cgiemail/questions3.txt

\ /\ /\ /

`--- Part 1 ---' script name ` -------- Part 2 ----------'

For simplicity, you may leave out part 1, but you must include it if you want to test your form as a local file. If you don't know what that means, just feel free to omit part 1.

7. Try out your form with cgiecho.

Pop your form into your favorite WWW browser, fill in the inputs, and submit it. You should see what the processed form looks like. If instead you see an error with a number near 500, your ACTION is probably set wrong. Go back to the previous step.

If some of your inputs don't seem to be showing up in the processed form, make sure that the inputs have the exact same names in the HTML form as in the ASCII template. E.g. NAME="yourname" in the HTML form and [yourname] in the e-mail template.

8. Go live with cgiemail.

Now change cgiecho to cgiemail in the ACTION of your HTML form. Try it out. You should receive an e-mail message with the processed form. If you get a success page but don't receive mail, there is some problem with your template file. Go back and make sure you correctly followed the guidelines in step 1.

If it works, congratulations!

9. Debug if you don't get mail

Normally, mail gets sent asynchronously, meaning it goes into a queue to be sent at at a convenient time. Asynchronous mail is sent more efficiently and reliably, but has the disadvantage that problems can only be reported by mailing an error message back to the sender. To the mail system, it appears that the sender of the mail is the web server, so the error message won't get to you.

If you are getting a success message but aren't getting mail, you can temporarily use synchronous mail delivery by creating a hidden input named cgiemail-mailopt and giving it a value containing "sync", e.g.

<INPUT TYPE="hidden" NAME="cgiemail-mailopt" VALUE="sync">

Be sure to remove this variable when you are done debugging, because it slows things down for the end user and possibly for the mail system.

Note: For release 1.1 and prior, this won't work. Ask your webmaster to install a newer release. Some mailers have a nonstandard extension that sends bounces to an address in an Errors-To: header, so you might try using that header in your template if you're stuck with an old version of cgiemail. However, some errors make this header line unreadable, so there's no way to make absolutely sure the bounce will go to you.

Optional: Add text to the success page.

When mail is sent, a page titled ``Success'' appears with the text of the e-mail message. You may use a hidden variable called ``addendum'' to add your own text. Here is a simple example:

<INPUT TYPE="hidden" NAME="addendum" VALUE="Thank you!">

If you are willing to assume that readers of your form are using recent browser software like Lynx 2.6 or Netscape 3.0, then you may put HTML markup into this variable using the appropriate character entities. For example, if you wanted to add

Thank you!

then the HTML markup would be

<em>Thank you!</em>

meaning you would need the following in your form:

<INPUT TYPE="hidden" NAME="addendum"

VALUE="<em>Thank you!</em>">

Note that besides being difficult to write, this feature won't work for people using older browser software.

Optional: Use an alternate success page.

If you don't like the default page that comes up when email is successfully sent, you can specify an alternate URL using a hidden variable called ``success'' in your HTML form, e.g.

<INPUT TYPE="hidden" NAME="success" VALUE="http://web.mit.edu/">

Note: Start your URL with / or with http://. Otherwise cgiemail will direct your browser to a second invocation of cgiemail, resulting in the error No variable substitutions.

As of release 1.3, there is no way to make this alternate success page contain information the user submitted in the form. This feature is likely to be added in a future release.

Optional: Make some inputs required.

If you would like to automatically reject forms with certain inputs left blank, add the prefix ``required-'' to the name of the input in both your HTML form and your e-mail template. Here is an example:

In the HTML form:

Your name: <INPUT NAME="required-yourname">

In the e-mail template

Your name: [required-yourname]

Note: With release 1.4, the prefix cgiemail uses to recognize a required field is ``required'' without the hyphen. You should still add a hyphen for readability, but you have the option of using an underscore (better for interaction with JavaScript) or nothing at all.

Optional: Specify formatting for some inputs.

If, in your e-mail template, the text inside square brackets begins with %, cgiemail will use the printf() function in C on the field name after the comma. If you're not familiar with this function, look in a book on C. If you are familiar with it, please note these two differences:

The first character in the format string must be %.

Characters like \n and \t must be literal. If you want a newline, you have to put a newline just before the comma, even though this looks strange. For example, if Godzilla's Pizza wanted toppings listed one per line, they would put the following in their e-mail template:

[%s

,topping]

Optional: Use CGI environment variables.

This feature may or may not work, depending on whether or not your webmaster enabled it when configuring cgiemail.

In addition to form inputs, your e-mail template can include CGI environment variables simply by preceding the variable's name with a dollar sign. For example,

[$HTTP_USER_AGENT]

will put the name of the user's browser and/or gateway in your e-mail message. In order to be respectful of privacy, your HTML form should warn users about any information about them that will be included in the e-mail, e.g. HTTP_USER_AGENT, REMOTE_ADDR.

cgiemail

Last modified: Tue Oct 31 09:24:21 EST 2000

Mastering Cgi

Documents

Transcript of Mastering Cgi