Five Minutes to a Python CGI

Python is a freely available, high-level, interpreted language developed by Guido van Rossum. It combines a clear syntax with powerful but optional object-oriented semantics. Python has a lot of the same strengths as other script languages used for Web programming: good text processing tools; dictionaries (hash tables) and other versatile types; and a broad range of modules (libraries) relating to Web programming.

Why Python?

Compared to Perl, most people find Python code easier to read and maintain. Compared to VBScript or ColdFusion, Python packs more powerful basic constructs. Compared to PHP, TCL, or REXX (or C for that matter), it's a lot easier to make modular and object-oriented code in Python. Compared to JSP, Python is concise, dynamic, and loosely typed—in short, a lot quicker to develop. Compared to Bash...well.

Okay, you figured me out. I am a Pythonista, a convert to all things Pythonic. I have had the opportunity to do a bit of programming in a lot of languages, and have found Python my favorite among them for most purposes. Of course, there are many more languages I've never yet managed "Hello World!" in, so who knows. But let me avoid proselytizing, and pass on a few hints for getting started with CGI programming in Python. Before I start, let me mention that CGI has sometimes gotten a bad reputation. This reputation is mostly ill-deserved.

To be fair, plain CGI certainly has some overhead to it (in the need to "fork" processes mostly), but you can't beat old-fashion CGI for rapid development and server portability. If speed turns into a real issue, a number of solutions are available to speed things up: Python/ASP, fastcgi, mod_python, JPython servlets, Medusa, and Zope. Or, you can write your own solution by using the module CGIHTTPServer. See the sidebar for links to resources.

Using the CGI Module

Python's cgi module—in the standard distribution—is usually the best place to start writing CGI programs in Python. The main use of the cgi module is to extract the values passed to a CGI program from an HTML form. Most typically, one interacts with CGI applications by means of an HTML form: A site visitor fills out values in the form, then the CGI is called upon to perform its action according to the script you've set up with individual specifications.

You may include many input fields within an HTML form, and the fields can be a number of different types (text, checkboxes, picklists, and radio buttons). Chuck Musciano wrote a nice series of articles for Web Review explaining all the form elements. Here's the first piece.

Your Python script should begin with import cgi to make sorting out its calling form easy. One thing this module does is hide any details of the difference between GET and POST methods from the CGI script. By the time the call is made, this is not a detail the CGI creator needs to worry about. The main thing the CGI module does is to treat all the fields in the calling HTML form in a dictionary-like fashion, defining each item such as a "username," then explaining what the script will do with it in different scenarios. What you get is not quite a Python dictionary, but it is close enough to be easy to work with. I play around with it in the following sample:

Example of working with Python [cgi] module

importcgiform = cgi.FieldStorage()   
# FieldStorage object to
# hold the form data
# check whether a field called "username" was used...
# it might be used multiple times (so sep w/ commas)
if form.has_key('username'):username = form["username"]usernames = ""
if type(username) is type([]):
# Multiple username fields specified
for item in username:
if usernames:# Next item -- insert comma
usernames = usernames + "," + item.valueelse:
# First item -- don't insert commausernames = item.value
else:# Single username field specified
usernames = username.value
# just for the fun of it let's create an HTML list
# of all the fields on the calling form
field_list = '<ul>\n'
for field in form.keys():field_list = field_list + '<li>%s</li>\n' % field field_list = field_list +
'</ul>\n'

We'll have to do something more to present a useful page to the user, but we've made a good start by working with the submitting form.

Getting the Output Right

After parsing the query form that called your Python CGI, the next thing you need to do is send something back to the client browser. Judging from questions on the comp.lang.python newsgroup, the most common mistake made by beginners is forgetting to include a blank line between the HTTP header(s) and the HTML document, or forgetting the header altogether. Be sure to put something like the following in your Python CGI:

Writing HTTP header in Python

print 'Content-type: text/html\n\n'

Of course, if you want to send back something other than an HTML page, the header should indicate that. But be sure to have a header in any case. For example, a dynamically generated image using Python's PIL module, for example, might start with:

Writing HTTP header in Python

print 'Content-type: image/jpeg\n\n'

Once the header is there—and it might include other header lines, such as one to set cookies—we need to compose an HTML page. It's perfectly acceptable to use a bunch of print statements in a row to output the whole page, like:

Step-by-step HTML creation in Python

print '<html><head>'
print '<title>My Page</title>'
print '</head><body>'
print '<h1>Powers of two</h1>\n<ol>'
for n in range(1,11):
print '<li>'+str(2**n)+'</li>'
print '</ol></body></html>'

A technique that is often more readable and easier to work with is to use Python's 'sprintf()' style string formatting on a page template of the whole HTML page—usually as the last thing in the script, after the variables have been computed. You can do this with tuples and use Python's nifty triple quoting for multiple line expressions:

Formatting sprintf()-style in Python

print """<html><head>
<title>%s</title>
</head><body>
<h1>Famous irrational numbers</h1>
<dl><dt>Pi</dt>
<dd>%2.3f</dd>
<dt>Square-root of 2</dt>
<dd>%2.3f</dd></dl>
</body></html>""" % ("Another Page", 3.1415, 1.4142)

Python has an even better trick up its sleeve, however. In addition to using positional '%' expressions in a string, you can use named expressions that are pulled from a dictionary:

Dictionary sprintf()-style in Python

mydict = {"title":"Formatted from Dict",
"pi": 3.1415, "e": 2.7182,
"sqrt3": 1.73205, "sqrt2": 1.4142}
template = """<html><head>
<title>%(title)s</title>
</head><body>
<h1>Famous irrational numbers</h1>
<dl><dt>Pi</dt>
<dd>%(pi)2.3f</dd>
<dt>Square-root of 2</dt>
<dd>%(sqrt2)2.3f</dd></dl>
</body></html>"""
print template % mydict

Tricks for Debugging

As easy as Python makes writing a CGI script, there's always the possibility some mistakes will creep into the code. Fortunately, it's not hard to design a Python CGI program to catch a helpful traceback. Depending on what your needs are, you might either want to log errors to server storage, or display them in the client browser.

The simplest case is coaxing a CGI to display errors in the client browser if displaying the desired page fails. The first thing to know for this is that Python errors and tracebacks are sent to STDERR, while Web servers normally pick up the output of STDOUT. It might seem like we have a problem, until we notice that redefining STDERR is simple in Python. Here's what a script might look like:

Debugging CGI script in Python

import sys
sys.stderr = sys.stdout

def  main():
import cgi
# ...do the actual work of the CGI...
# perhaps ending with:
print template % script_dictionary

print "Content-type: text/html\n\n"
main()

This approach is not bad for quick debugging. Unfortunately, the traceback (if one occurs) gets displayed as HTML, which means that you will need to view source in a browser to see the original linebreaks in the traceback. With a few more lines, we can add a little extra sophistication.

Debugging/logging CGI script in Python

import sys, traceback
print "Content-type: text/html\n\n"
try:# use explicit exception handling
import my_cgi  # main CGI functionality in 'my_cgi.py'
 my_cgi.main()
except:
import time
errtime = '--- '+ time.ctime(time.time()) +' ---\n'
errlog = open('cgi_errlog', 'a')
errlog.write(errtime)
traceback.print_exc(None, errlog)
print "<html><head><title>CGI Error Encountered!</title></head>"
print "<body><p>Sorry, a problem was encountered running MyCGI</p>"
print "<p>Please check the error log on the server for details</p>"
print "</body></html>"

The above approach is a generic wrapper for any real CGI functionality we might write, you'll of course customize it for your needs. Just import a different CGI module as needed; and maybe make the error messages more detailed or friendlier.

Wrapping Up

Our five minutes should help get you started on Python CGI. Play with what we have gone over, put up a few CGIs on a Web site you have access to, and try to get a feel for what is going on. Make some mistakes even, there is no better way to learn.

After you are comfortable with the basics, I am sure you'll want to move on to some of the really fancy things you can do with Python and CGIs. Maybe you'll want to generate information from a SQL database—be sure to check out the database modules available at locations listed in the sidebar. Maybe you'll want to dynamically generate images, so be sure to check out PIL. Whatever direction you want to go, there is plenty of room to grow, and the learning curve is an easy one to master.

David dabbles in a lot of things. You can find out copious biographical details by rooting around at http://gnosis.cx/publish/.