Simon is a freelance programmer and author, whose titles include Beginning Perl (Wrox Press, 2000) and Extending and Embedding Perl (Manning Publications, 2002). He's the creator of over 30 CPAN modules and a former Parrot pumpking. Simon can be reached at simon@ simon-cozens.org.
We've all done it. We need to whip up a web application based on a database or some other structured data source; we drag out our favorite templating library, our favorite database abstraction layer, and write a bunch of code to glue the two together, maybe using something like CGI::Application to handle some of the front-end work. We extend the application in particular ways as the specifications change, but at least we have something working.
Then, the next time we need to create a database-backed web application, we drag out the same tools and do it again. And again. Wouldn't it be nice if there were a decent, extensible web application framework like Java's Struts? Well, there are one or two: OpenInteract and OpenFrame, to start with. But in this article, and its follow-up next month, I'd like to talk about a new one: Maypole.
Maypole started when I couldn't find any tools to "magically" put a web front end on a database without the need for lots of code or customization. Since then, it's become generalized to support extensions in all directions: adding functionality to the application itself using different templating modules and database abstractions. However, it's retained its focus of allowing you to get as much done as possible without writing very much code.
To introduce Maypole and some other friends, we'll look at an application I described a few months ago—the house-sharing database. I mentioned then that I planned to extend the application to track what food was in the cupboards. We're going to do that now, and our eventual application will also collect a database of recipes and have the system suggest the best recipe to use up the food before it goes off.
Tracking Provisions
We'll start, as usual, by developing our database schema, beginning with the recipes. The recipes will come in as XML, as we'll see in a few moments, so the table looks like this:
CREATE TABLE recipe (
id int not null auto_increment primary key,
name varchar(255),
xml text
);
Now, each recipe will have multiple ingredients, so we need a many-to-many table linking ingredients to recipes:
CREATE TABLE ingredient (
id int not null auto_increment primary key,
recipe int,
food int,
quantity varchar(255)
);
Then, we're going to need a table for the names of food. One of the things you notice about recipes is that they're not always consistent about their use of the names of ingredients. "Lamb mince" and "minced lamb" are the same thing, so we'll use a column in the table to identify the "canonical" name of a food to help us when we're importing recipes. We'll call this the normalized name, and so our schema looks like this:
CREATE TABLE food (
id int not null auto_increment primary key,
name varchar(255),
normalized varchar(255)
);
The recipes have categories as well as ingredients, so here's another many-to-many linking table:
CREATE TABLE category (
id int not null auto_increment primary key,
name varchar(255)
);
CREATE TABLE categorization (
id int not null auto_increment primary key,
recipe int,
category int
);
And finally, there's a table for the food in the cupboards. I called my database "larder," so "larder.contents" is a good name for this table:
CREATE TABLE contents (
id int not null auto_increment primary key,
food int,
quantity varchar(255),
use_by date
);
Now I could use Class::DBI to set up the database, create classes for each of the tables, load them up with the column names, declare the relationships between them, and start writing my templating controller code. Or I could just say (see Listing 1):
package Larder;
use base 'Apache::MVC';
Larder->setup("dbd:mysql:larder");
Apache::MVC is a subclass of Maypole that is specifically suited to creating Apache mod_perl handlers, and it takes care of pretty much everything.
We still need to declare our relationships between the tables, but we'll do this in several parts: First, we'll get a system going where we can add and edit the contents of our larder, then we'll look at displaying recipes as well; finally, in next month's article, we'll look at how to link the recipes with their ingredients and also the ingredients with what's in the larder.
To relate the food types in the contents table to the food table, we use a has_a relationship:
Larder::Contents->has_a( food => "Larder::Food" );
Deploying the Web Site
Believe it or not, we're almost done with setting up a simple interface to the database. All we need to do is tell the system where it's going to live, so that it can construct URLs to itself, and we're done:
package Larder;
use base 'Apache::MVC';
Larder->setup("dbi:mysql:larder");
Larder->config->{uri_base} = "http://localhost/larder/";
Larder::Contents->has_a( food => "Larder::Food" );
1;
Well, almost done. That's as much Perl as we need to write for the moment. The rest are HTML templates but, thankfully, Maypole includes a useful bunch of those, too. We copy the factory templates directory inside our web root:
% cp -r templates/factory /var/www/larder/
We probably want a CSS file in there also; the sample one provided with Maypole works well. And then set up the Apache configuration to use the Larder Perl module:
<Location /larder>
SetHandler perl-script
PerlHandler Larder
</Location>
That's all.
Restart Apache, go to the site, and you should be able to view, search, and browse the database. This works equally well for any database, and it's a great way to look around an established database that has a lot of relationships defined. But, of course, our larder database doesn't have any data in it at the moment.
Editing and Untainting
To allow us to add and edit records, we need to tell Maypole more about the kinds of data we're expecting. To do this, Maypole makes use of Class::DBI::Untaint, an application of CGI::Untaint.
CGI::Untaint is a mechanism for testing to make sure that incoming form data conforms to various properties. For instance, given a CGI::Untaint object that encapsulates some POST parameters, we can extract an integer like so:
my $h = CGI::Untaint->new(CGI->Vars); $h->extract(-as_integer => "food");
This checks that the food parameter is an integer and returns it if it is; if not, $h->error will be set to an appropriate error message. Other tests by which you can extract your data are as_hex and as_printable, which tests for a valid hex number and an ordinary printable string, respectively; there are other handlers available on CPAN, and you can make your own, as documented in CGI::Untaint.
To tell the Maypole handler what handler to use for each of your columns, you need to use the untaint_columns methods in the classes representing your tables. We have a class representing food in the cupboard, Larder::Contents, which has a string column quantity and a date column use_by, so we declare these:
Larder::Contents->untaint_columns(
printable => [ "quantity" ],
date => [ "use_by" ]
);
This will allow us to edit these columns, but there's one column we've forgotten—the food column will have to contain an integer that refers to a primary key in the food table. Even though this will be displayed as a drop-down list, we need to ensure that the number passed in to the edit process is an integer:
Larder::Contents->untaint_columns(
printable => [ "quantity" ],
date => [ "use_by" ],
integer => [ "food" ]
);
Now we can add and edit rows in the contents table.
Tomatoes and Triggers
The food table itself is slightly different. We have three columns, id, name, and normalized. The only one we need to edit is name, as the others should be hidden. Maypole automatically hides the id column, but we can specify the columns to be displayed by overriding the display_columns method in the Larder::Food class:
package Larder::Food;
sub display_columns { "name" }
We also need to make sure that the normalized column gets set every time there's an update to name. We'll use a very simple canonicalization subroutine for demonstration purposes, but a proper foodstuff canonicalization routine needs to know about specialist cooking terms; for example, that chopped tomatoes and diced tomatoes are both tomatoes. Our routine is a common one for doing fuzzy searches: It ditches vowels and punctuation characters, and compresses whitespace and repeated letters.
sub normalize {
my $word = lc shift;
$word =~ s/[aeiou]//g;
$word =~ s/[^\s\w]//g;
$word =~ tr/a-z /a-z /s;
return $word;
}
To arrange this subroutine to set the value of normalized, we use a feature of Class::DBI called "triggers." These are subroutine references fired off by particular events; the Class::Trigger class helps you to define these. For instance, every time a record is created: Class::DBI-based classes will call their after_create trigger. We can use this trigger to make sure that the normalized field is set to the right thing:
Larder::Food->add_trigger( after_create => sub {
my $self = shift;
$self->normalized(normalize($self->name));
} );
Similarly, when we update the name from the web interface, we want to ensure that the new name is correctly normalized, if the normalized version is now different:
Larder::Food->add_trigger( after_update => sub {
my $self = shift;
my $old = $self->normalized;
my $new = normalize($self->name);
if ($old ne $new) {
$self->normalized($new);
}
} );
Finally, we set our untaint_columns so we can update the name of a foodstuff:
Larder::Food->untaint_columns( printable => [ "name" ] );
and now we have two tables we can display, edit, and add to. With a few bits of data in it, the site looks like Figure 1.
RecipeML
The next stage of this is to get the recipes into the system. The state of machine-readable recipes is somewhat lamentable at the moment, but I did find a large archive of recipes in RecipeML, an application of XML. A RecipeML recipe looks like Example 1.
We want to extract the categories and the ingredients, then dump the data into the database. First, we fire up our old friend XML::Simple to parse the XML into a Perl data structure:
use XML::Simple;
use File::Slurp;
for my $recipe (<xml/*>) {
my $xml = read_file($recipe);
my $structure = XMLin($xml)->{recipe};
}
XML::Simple has some quirky features, and one of these is that if there's an <ingredients> tag with several <ing> tags inside it, it will present these as an array, as one might expect; unfortunately, if there's only one ingredient, it presents it as a hash. Naturally, this causes problems when we come to dereference it. To get around this, we force XML::Simple to present everything as arrays, regardless of the number of subelements. This has the unfortunate side effect of making the rest of the code ugly, but at least it's consistently ugly:
my $structure = XMLin($xml, ForceArray => 1)->{recipe}->[0];
my $name = $structure->{head}->[0]->{title}->[0];
my @ingredients = @{$structure->{ingredients}[0]{ing}};
my @cats = @{$structure->{head}[0]{categories}[0]{cat}};
We have all the data we need.
For the time being, we'll only worry about the name and the XML; the ingredients and categories require slightly trickier many-to-many relationships, so we'll deal with those in next month's column.
To create the database rows, we will load up the same Larder module that we've been using as an Apache handler, since this does the hard work of setting up the Class::DBI classes for us.
use Larder;
use XML::Simple;
use File::Slurp;
for my $recipe (<xml/*>) {
my $xml = read_file($recipe);
my $structure = XMLin($xml, ForceArray => 1)->{recipe}->[0];
my $name = $structure->{head}->[0]->{title}->[0];
my @ingredients = @{$structure->{ingredients}[0]{ing}};
my @cats = @{$structure->{head}[0]{categories}[0]{cat}};
Larder::Recipe->find_or_create({
name => $name,
xml => $xml
});
}
Now we have a load of recipes in our system and we can list them by visiting the URL /larder/recipe/list. However, we have one problem. Since Maypole tries to display all columns in a table by default, it shows the recipe XML alongside the name. We don't want it to display the XML by default, and we need to work out a way to turn that XML into HTML.
The first stage in this is to use the display_columns method as before to have it only display the name. We also need to put the recipe back into the list of allowable tables:
package Larder::Recipe;
sub display_columns { "name" }
Our final version of Recipe.pm will be shown next month. The next stage, actually displaying the XML as HTML, requires a bit of trickery.
Cooking the Source
We only want the recipe to be displayed in full when we go to the view page for an individual recipe; we can do this by writing our own custom view template. Maypole searches for templates in three directories to allow flexibility in overloading the default templates. When you go to a URL such as /recipe/view/10, it looks for the view template in the recipe directory; if there isn't one specific to the recipe table, it looks in the custom directory to allow you to specify site-wide defaults; finally, it checks the factory directory, which contains the generic templates that come with Maypole.
So we want to write our own custom template specifically for recipes, which should live in recipe/view. However, we still need to get the RecipeML into HTML somehow. What we'll use to do that is called XSL.
XSL is a language, itself expressed in terms of XML, for turning one XML document into another. For instance, here's the fragment of an XSL stylesheet for transforming a recipe:
<xsl:template match="recipe">
<html>
<head><title><xsl:value-of select="head/title"/></title>
</head>
<body>
<xsl:apply-templates />
</body>
</html>
</xsl:template>
Most of the tags here are supposed to be output verbatim, apart from the tags with the namespace xsl:, which are magic and refer to the transformation process itself. In these cases, the first line says, "if you see a recipe tag, spit out this chunk of XML." The third line outputs ordinary HTML head and title tags, then looks for the first title element inside a head tag in the source recipe XML, and outputs the value inside those tags. The apply-templates in the middle means "keep working through the source document and apply any other fragments for tags that you find."
To apply the XSL stylesheet to a recipe, we need to use an XSL processor. One of the best in the business (or at least, in the Perl business) comes from the GNOME project and is wrapped in the XML::LibXSLT module. We can take an XML::LibXML parser and parse a document:
use XML::LibXML;
my $parser = XML::LibXML->new();
my $source = $parser->parse_file("recipe.xml");
And then parse a stylesheet because that's just plain XML, as well:
my $xsl = $parser->parse_file("recipe.xsl");
And now we use XML::LibXSLT to turn that into a stylesheet object:
use XML::LibXSLT; my $stylesheet_parser = XML::LibXSLT->new(); my $stylesheet = $stylesheet_parser->parse_stylesheet($xsl);
Now that stylesheet object can be used to transform the original recipe:
my $results = $stylesheet->transform($source); print $stylesheet->output_string($results);
This should output some nice HTML for our recipe. Now we need to know how to get that nice HTML into the context of our web application.
Simmer and Serve
Since we'll be processing our Maypole templates using the Template Toolkit, the most natural way to do this is as a Template plugin. We're going to write our own plugin using the XML::LibXSLT module to transform the RecipeXML into HTML. We'll start by envisaging the syntax we want out of it, which will look something like this:
[% USE transform = XSLT("recipe.xsl") %]
[% recipe.xml | $transform %]
We can inherit from the basic filter class Template::Plugin::Filter and plan to override the two methods init and filter with methods that set up our XSLT parser and apply the stylesheet to the filtered text, respectively.
The init method needs to load the stylesheet, and may as well parse it and turn it into an XML::LibXSLT::Stylesheet object there and then. First, we get the name of the stylesheet we want to use; this will be provided in the _ARGS slot of the filter object:
sub init {
my $self = shift;
my $file = $self->{ _ARGS }->[0]
or return $self->error('No filename specified!');
Next, we load up the parsers and try parsing the stylesheet's XML:
$self->{ parser } = XML::LibXML->new();
$self->{ XSLT } = XML::LibXSLT->new();
my $xml;
eval {
$xml = $self->{ parser }->parse_file($file);
};
return $self->error("Stylesheet parsing error: $@") if $@;
return $self->error("Stylesheet parsing failed") unless $xml;
If that works, then we can try feeding the stylesheet to XML::LibXSLT:
eval {
$self->{ stylesheet } =
$self->{ XSLT }->parse_stylesheet( $xml );
};
return $self->error("Stylesheet not valid XSL: $@") if $@;
return $self->error("Stylesheet parsing failed")
unless $self->{stylesheet};
return $self;
This handles what happens when the USE call is made; the filter method is called when the returned object is used as a filter. This gets handed some text and needs to parse it:
my ($self, $text) = @_;
my $xml;
eval { $xml = $self->{ parser }->parse_string($text); };
return $self->error("XML parsing error: $@") if $@;
return $self->error("XML parsing failed") unless $xml;
And then it needs to apply the stylesheet to it:
return $self->{ stylesheet }->output_string(
$self->{ stylesheet }->transform( $xml )
);
That's essentially the core of the Template::Plugin::XSLT module, which I wrote precisely in order to display these recipes. Now we can write our recipe/view template, based on the generic one in factory/view:
[% INCLUDE header %]
<h2> [% recipe.title %] </h2>
[% INCLUDE navbar;
USE transform = XSLT("recipe.xsl");
recipe.xml | $transform %]
We can now view a recipe using an XSL template. Figure 2 is one I prepared earlier.
Serving Suggestion
We've looked at a wide variety of things in this article: using XSLT to transform XML into HTML; writing Template Toolkit filter plugins; using Class::DBI::FromCGI to restrict the possible input for form fields; and, of course, using Maypole, a new web application framework.
While we've not gone into much depth about how Maypole does its stuff—we'll look at that next month—I hope I've given you the flavor (ho, ho) of how easy it is to construct web applications in Maypole. We've put together an interface to a larder inventory system, together with an XSL-based recipe display in around 40 lines of Perl code.
Next month, we'll look at linking recipes with their ingredients and searching for optimal recipes to use up fading food! Until then, happy cooking!
TPJ
package Larder;
use strict;
use base 'Apache::MVC';
Larder->setup("dbi:mysql:larder");
Larder->config->{display_tables} = [qw[food contents recipe]];
Larder->config->{uri_base} = "http://localhost/larder/";
Larder::Contents->has_a( food => "Larder::Food" );
Larder::Contents->untaint_columns(
printable => [ "quantity" ],
date => [ "use_by" ],
integer => [ "food" ]
);
package Larder::Food;
sub display_columns { "name" }
Larder::Food->add_trigger( after_update => sub {
my $self = shift;
my $old = $self->normalized;
my $new = normalize($self->name);
if ($old ne $new) {
$self->normalized($new);
}
} );
Larder::Food->add_trigger( after_create => sub {
my $self = shift;
$self->normalized(normalize($self->name));
} );
Larder::Food->untaint_columns( printable => [ "name" ] );
sub normalize {
my $word = lc shift;
$word =~ s/[aeiou]//g;
$word =~ s/[^\s\w]//g;
$word =~ tr/a-z /a-z /s;
return $word;
}
package Larder::Recipe;
sub display_columns { "name" }
1;
Back to article
