Creating Perl Application Distributions
The Perl Journal March 2003
By brian d foy
brian is the founder of the first Perl Users Group, NY.pm, and Perl Mongers, the Perl advocacy organization. He has been teaching Perl through Stonehenge Consulting for the past five years, and can be contacted at [email protected].
Most Perl users have run a Makefile.PL file when installing a module, and module authors know how to use one to make their module distributions, but most people do not know that they can use the same mechanism to distribute their scripts. The Makefile.PL file can help configure, test, and install scripts as well as modules.
The Makefile.PL file is just a Perl script that invokes the ExtUtils::MakeMaker::WriteMakefile function, which creates a Makefile that actually does all of the work (the make utility comes with most UNIX platforms and is available for Windows and other platforms). The typical installation sequence consists of four commands:
perl Makefile.PL make make test make install
The first line, perl Makefile.PL, creates the Makefile, which controls all of the action. The make line runs the Makefile with the default target ("all"), which copies the right files to the right places—usually a subdirectory of "blib" (build library) to prepare for installation. The make test line is optional, and runs either a test.pl file or the t/*.t files, which test the script. Other people have written plenty about testing; I'll focus on the other parts in this article. The make install command takes the files in blib and puts them in the right place based on various configuration directives and defaults.
Usually, the end user needs to put the script in a particular place and set the script file permissions—the Makefile can do this automatically. To illustrate this, I start with a fictitious Perl script called "buster."
I create a directory for this script and the files that will go with it, and create a simple Makefile.PL:
prompt$ ls Makefile.PL buster
To create the Makefile, Makefile.PL loads ExtUtils::MakeMaker and calls the WriteMakefile() function, which takes a list of key-value pairs that describes how it should create the Makefile. Each key is explained in the ExtUtils::MakeMaker documentation, which I have printed and keep near my desk.
use ExtUtils::MakeMaker;
WriteMakefile(
'NAME' => 'buster',
'VERSION' => '0.10',
'EXE_FILES' => [ 'buster' ],
'PREREQ_PM' => {},
'INSTALLSCRIPT' => "$ENV{HOME}/bin",
);
The NAME key gives a name to the distribution and does not have to be the name of the script. I'll show why the NAME key is important later. I set the version of the distribution with the VERSION key (modules typically get their distribution from a module file with VERSION_FROM). The EXE_FILES key has an anonymous array of file names as its value. When I run make, the Makefile will move those files into the blib/script directory. The PREREQ_PM key has an anonymous hash as its value and lists all of the modules on which the script depends. Installers like CPAN.pm can automatically fetch and install these modules. The INSTALLSCRIPT key names the directory in which to install the script. I hard code a value for INSTALLSCRIPT so the script will show up in my personal bin directory.
Once I have my Makefile.PL, I can go through the steps I listed earlier. The script ends up in my personal bin directory (/Users/brian/bin/), and the default file permissions are 0555, meaning the owner, group, and everyone has read and executed permissions. The script is in the right place and ready to run, and the end user did not have to learn about copying files or setting permissions; see Example 1.
If I want to install buster in a different location, I have to tell Makefile.PL where to install it. I specify an alternate value for INSTALLSCRIPT on the command line when I run Makefile.PL. The value on the command line overrides the one in WriteMakefile().
$ perl Makefile.PL INSTALLSCRIPT=/usr/local/bin
The Makefile gives me many other benefits. I use it to automatically create a distribution file that I can give to other people so they can install my script easily. Before I create the distribution, I need to create the list of files that should go into the distribution. The make manifest command creates a file named MANIFEST and adds files from the current directory to it. It has a default list of files it ignores (like Makefile), so it only puts the relevant files in MANIFEST.
prompt[3036]$ make manifest /usr/bin/perl "-MExtUtils::Manifest=mkmanifest" -e mkmanifest Added to MANIFEST: MANIFEST Added to MANIFEST: Makefile.PL Added to MANIFEST: buster
I run the make dist command to create the distribution. It looks at the values I specified in the NAME and VERSION keys when I ran WriteMakefile and puts them together with a hyphen (-), then creates a temporary directory with that name. It looks in MANIFEST and copies the listed files into the new directory, tars and gzips the directory, and finally removes the temporary directory.
prompt$ make dist rm -rf buster-0.10 /usr/bin/perl "-MExtUtils::Manifest=manicopy,maniread" -e "manicopy(maniread(),'buster-0.10', 'best');" mkdir buster-0.10 tar cvf buster-0.10.tar buster-0.10 buster-0.10 buster-0.10/buster buster-0.10/Makefile.PL buster-0.10/MANIFEST rm -rf buster-0.10 gzip —best buster-0.10.tar
If I want to use ZIP instead of tar and Gnu zip, I run make zipdist:
prompt$ make zipdist rm -rf buster-0.10 /usr/bin/perl "-MExtUtils::Manifest=manicopy,maniread" -e "manicopy(maniread(),'buster-0.10', 'best');" mkdir buster-0.10 zip -r buster-0.10.zip buster-0.10 adding: buster-0.10/ (stored 0%) adding: buster-0.10/buster (stored 0%) adding: buster-0.10/Makefile.PL (deflated 25%) adding: buster-0.10/MANIFEST (deflated 2%) rm -rf buster-0.10
I am not satisfied with that, though. I want to create some tests for my script so I can see the damage that I do when I work on it. The Makefile gives me the testing framework for free through Test::Harness. So far Test::Harness has simply told me that there are no tests to run.
I create a directory named "t". Test::Harness will look in this directory for files matching the pattern "*.t" and run those as test files. In this example, I make a simple test to ensure that the script compiles. I call this test "compile.t". In the test script, I load the Test::More module, which insulates me from most of the testing details. I tell Test::More that I have one test to run. In the next line, I call perl -c in backticks so Perl runs itself with the -c switch on the buster script in the blib/script directory (which is what make did). The backticks return the output, which I store in $output. The Test::More module provides a like() function that compares the first argument to the regular expression in the second argument. If the regex matches, the test passes; if not, it fails. The third argument to like() is the name I choose for the test.
use Test::More tests => 1; my $output = 'perl -c blib/script/buster 2>&1'; like( $output, qr/syntax OK$/, 'script compiles' );
The make test output looks much different now; see Example 2.
To see the test fail, which is always a good idea so I know it will catch failures, I introduce a small syntax error into buster. Once I make the change, the buster script is different than the one in the blib/script. The Makefile catches this and copies the updated version into blib/script and then runs the tests. The Test::More module shows me the output that it actually got, and what it expected ("syntax OK"); see Listing 1.
So now my distribution has its first test file, and I need to add it to MANIFEST. I use make manifest again, which recognizes the new files and adds them to MANIFEST. Sometimes extra files get into MANIFEST, especially if I have been doing other things in that directory (in which case I edit them out of MANIFEST.)
prompt$ make manifest
/usr/bin/perl "-MExtUtils::Manifest=mkmanifest" \
-e mkmanifest
Added to MANIFEST: t/compile.t
After testing my script, I want to create a new distribution, but I first use the make disttest command, which creates a distribution, then unwraps it in its own directory and runs make test on it. This tests the distribution to make sure all of the tests pass when it only has the files from the distribution (those in MANIFEST), rather than all of the files I have in my working directory. This test catches omissions from MANIFEST.
three_brian[3064]$ make disttest rm -rf buster-0.10 /usr/bin/perl \ "-MExtUtils::Manifest=manicopy,maniread" \ -e "manicopy(maniread(),'buster-0.10', 'best');" mkdir buster-0.10 mkdir buster-0.10/t cd buster-0.10 && /usr/bin/perl Makefile.PL Checking if your kit is complete... Looks good Writing Makefile for buster cd buster-0.10 && make LIB="" LIBPERL_A="libperl.a" LINKTYPE="dynamic" PREFIX="/usr/local" OPTI MIZE="" PASTHRU_DEFINE="" PASTHRU_INC="" cp buster blib/script/buster /usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/script/buster cd buster-0.10 && make test LIB="" LIBPERL_A="libperl.a" LINKTYPE="dynamic" PRE FIX="/usr/local" OPTIMIZE="" PASTHRU_DEFINE="" PASTHRU_INC="" PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Com mand::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/compile....ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.16 cusr + 0.04 csys = 0.20 CPU)
The ExtUtils::Makemaker framework provides much more functionality than I have covered here—these are just the basics to get you started. My buster script now has a full-fledged, full-featured distribution framework. I can create distributions, test them, and easily distribute them. Other users can easily install them because the distribution uses the familiar Perl installation sequence.
TPJ
Listing 1
prompt$ make test
cp buster blib/script/buster
/usr/bin/perl "-MExtUtils::MY" -e "MY->fixin(shift)" blib/script/buster
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/compile....NOK 1# Failed test (t/compile.t at line 5)
# 'String found where operator expected at blib/script/buster line 6, near # "yprint "Hello World!\n""
# (Do you need to predeclare yprint?)
# syntax error at blib/script/buster line 6, near "yprint "Hello World!\n""
# blib/script/buster had compilation errors.
# '
# doesn't match '(?-xism:syntax OK$)'
# Looks like you failed 1 tests of 1.
t/compile....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
Failed 1/1 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------
t/compile.t 1 256 1 1 100.00% 1
Failed 1/1 test scripts, 0.00% okay. 1/1 subtests failed, 0.00% okay.
make: *** [test_dynamic] Error 2
