Free as in Music

Randal is a coauthor of Programming Perl, Learning Perl, Learning Perl for Win32 Systems, and Effective Perl Programming, as well as a founding board member of the Perl Mongers (perl.org). Randal can be reached at [email protected].

I've heard a lot of noise being generated lately about the Record Industry Association of America (better known as the RIAA) cracking down on Internet music-file sharing. They claim that such activity takes the profits away from their artists and the record labels who produce those artists and distribute the music. Critics suggest that most of the money never gets to the artists anyway, and that the artist would often do better by simply publishing directly on the Internet.

One of the articles I read on the subject suggested that we ignore RIAA-member labels and artists completely, and listen only to songs freely available on the Internet, putting our ears where our mouth is, so to speak. But what songs, and where? And how do I keep from downloading so much junk, and find the gems within?

Enter the collaborative filtering model and the power of the masses. In that same article, I found a pointer to iRate, an "internet radio" that learns my preferences, feeding me more and more of what I like. iRate works by downloading a few MP3s to my disk, playing them using a simple embedded player, and then letting me vote on the songs. The songs are selected from publicly available free song sites like IUMA (http://iuma.com/) and Artist Direct (http://artistdirect.com/), so I can feel safe and legal in downloading and listening to them. The player then uploads my votes, and using a collaborative filtering system, sends me more of what I like. As I vote on more songs, the selections more accurately reflect my taste and provide a variety that far exceeds any commercial station I've frequented.

Try it yourself: Visit http://irate.sourceforge.net/ and start your personalized Internet radio station today! I've found that setting it on "continuous download" lets me grab as many songs as I can while I'm attached to a fast Internet connection, and then rate songs even when I'm offline later. Be sure to rate quite a few songs before letting it get ahead of you, though, or you'll find the mix to be somewhat unpleasant.

So where does Perl fit in to this? Well, after downloading and playing a few dozen songs using the built-in miniplayer, I longed for the flexibility of my Mac's iTunes player to play the songs, and the ability to listen to these new free songs on my iPod when I was away from my computer.

But I didn't just want to take the entire download directory out of iRate and import it to iTunes because I had already given the lowest vote to some of those songs ("This sux") and didn't want to waste my disk space on them, and I wanted to preserve my vote rating on the rest of the songs. Luckily, I noticed that the iRate player maintains an XML file, and after a bit of examination, I saw that it contained everything I needed to know to move the files into iTunes, including filename and rating, organized something like this:

<Track
artist="MOTION PICTURE"
url="http://www.insound.com/media/motion_picture_a_paper_gift.mp3"
file="/Users/merlyn/irate/download/motion_picture_a_paper_gift.mp3"
rating="5.0"
last="10/9/03 5:11 PM"
played="10"
title="A Paper Gift"/>

From this record, I could simply tell iTunes to add the songfile and give it a particular rating, using Mac::Glue and a little help from Chris Nandor to work out the messiness. And parsing the XML was easy using libxml2 through the XML::LibXML interface. The result is found in Listing 1.

Lines 1 through 3 begin nearly every program I write, turning on warnings, enabling the usual compiler and runtime restrictions for large programs, and disabling the output buffering.

Lines 5 and 6 pull in the two modules found in CPAN. Mac::Glue is fun to install because it makes my laptop go through a lot of steps as the AppleScript interface gets exercised. Unfortunately, XML::LibXML is a bit finicky, but seemed to work fine with the fink-installed libxml2 on my machine.

Line 9 creates a Mac::Glue handle for iTunes. Method calls against this object will send messages to iTunes, and the responses get mapped back into values and objects. We'll be using this handle to add the songs as we find them.

Lines 11 and 12 change to the "irate" subdirectory below my home directory. Using an empty chdir, I first end up in my home directory without having to explictly look up the home. The next chdir is then relative to the home directory.

Line 14 creates an XML parser using the XML::LibXML library. Using the default settings for how things get parsed, I then create the document object in line 15, parsing the "trackdatabase.xml" file, which contains many elements similar to the one presented earlier.

From watching the XML file, I was able to determine that a track goes through a few different stages. First, when a song gets suggested by the central server, the song record is added with no rating or local filename attributes: just a remote URL. Next, when a song has been successfully downloaded, the file attribute is updated to reflect the location on local disk. Finally, when I've listened to the song (or enough of the song to rate it) and provide a rating, the rating attribute also gets added. In the current release, the rating seemed to be always one of "0.0," "2.0," "5.0," "7.0" or "10.0." Additionally, if a song couldn't be downloaded, it would have a rating attribute (of 0.0), but no file attribute or a file attribute of an empty string.

All of the songs I want move into iTunes are therefore songs that have a rating and a nonempty filename. I can select those using an XPath expression, as shown in line 17. The resulting list consists of all the nodes that have been rated and are ready to move. Each node ends up one-by-one in $track.

The next step is to pick out the file to see if we've already moved this one. Line 18 looks for the value of the @file attribute of the given node, using an XPath-ish way of describing the attribute. A DOM-ish way of getting at the same value might have looked like:

my $file = $track->getAttribute("file");

You have the option of using whatever you feel is clearer, which is one of the nice things about XML::LibXML.

If this string names a nonempty file, then we have a candidate for adding to iTunes; this is tested in line 19. Why would the file be empty? Well, I discovered that if I simply remove the music file once I've moved it, iRate gets mad and redownloads the file to replace it. (If only my real CD collection worked that way...) But if I replace the music file with an empty file, iRate "plays" the file very quickly, and moves on to the next one, causing only a slight pause in the scan for new music. (Perhaps iRate's behavior will change eventually. That'd be nice.)

Line 20 pulls up the iRate rating for this track. Lines 22 to 27 map this rating into the "star" rating for iTunes. A 5-star song in iTunes is an iTunes rating of 100, while no stars (or unrated) is a 0, and everything else is evenly mapped in between. I decided to map the four nonsucking levels of iRate to iTunes levels of five stars through two stars.

If the file has an iRate rating of 0.0, I don't even want to waste the disk space, so line 29 checks this value, and does nothing to the song unless it has a nonzero rating. Otherwise, it's time to move the file, which we ask iTunes to do in line 31. If iTunes isn't running, this starts it up and sends it an AppleScript to add the given file to its library. (To make this work, I have my preference set to always copy played music files into the iTunes folder.) In the iTunes status window, I see "Copying..." with my music file.

The returned object of $s represents a song in the library. In line 32, we further ask iTunes to set the rating for that song to our new value. This preserves the iRate rating all the way through to the iTunes rating.

Finally, lines 38 and 39 null out the file by opening it for writing and closing it. We're left with an empty file.

Running this program, I get a series of lines like:

adding with 60 for /Users/merlyn/irate/download/31_Capricorns_-_She_Says.mp3
adding with 80 for /Users/merlyn/irate/download/Welcome To Tuesday-None-Love 		Crime.mp3
adding with 100 for /Users/merlyn/irate/download/E.C._Powers_-_Baby_I_Do.mp3
adding with 40 for /Users/merlyn/irate/download/Danny Baker Band-Mama%27s 		Cookin%27-Heaven In Your Eyes.mp3
tossing /Users/merlyn/irate/download/brusta-Fresh Interpretation-Fresh 	Interpretation.mp3

And whenever it says "adding...", I see iTunes copying the file into the archive. Because these files are now empty, the test in line 19 now skips them, so it's safe to keep rerunning this program as often or as infrequently as I want. The only place I've found a problem is when iRate is playing the file that I'm also moving into iTunes, causing iRate to get a bit upset. The workaround is to have iRate be closed, or to be playing an "unrated" song.

Another problem I noticed is that iTunes grabs the internal MP3 tags from the file, falling back to the filename for artist and title, but I still sometimes get songs called "track 01" in my playlist. Although I have the artist and title information from iRate, I'm not (yet) using it. Maybe in a future edition of this program I'll add those as well. I'd also like to put the URL into a comment for later reference, and maybe even put all such added music into a separate playlist. But this is good enough for now.

So, the next time someone asks "Where is all this free music I keep hearing about on the Internet?", you can now point at your legitimate personally tailored collection and enjoy!

TPJ

Listing 1

=1=     #!/usr/bin/perl -w
=2=     use strict;
=3=     $|++;
=4=     
=5=     use Mac::Glue;
=6=     use XML::LibXML;
=7=     
=8=     
=9=     my $itunehandle = Mac::Glue->new("iTunes");
=10=    
=11=    chdir or die $!;
=12=    chdir "irate" or die $!;
=13=    
=14=    my $x = XML::LibXML->new or die;
=15=    my $d = $x->parse_file("trackdatabase.xml") or die;
=16=    
=17=    for my $track ($d->findnodes(q{//Track[@rating and @file != ""]})) {
=18=      my $file = $track->findvalue('@file');
=19=      next unless -s $file;
=20=      my $rating = $track->findvalue('@rating');
=21=    
=22=      my $irating =
=23=        ($rating >= 10) ? 100 :
=24=          ($rating >= 7) ? 80 :
=25=            ($rating >= 5) ? 60 :
=26=              ($rating >= 2) ? 40 :
=27=                0;
=28=    
=29=      if ($irating) {
=30=        print "adding with $irating for $file\n";
=31=        my $s = $itunehandle->add($file);
=32=        $s->prop("rating")->set(to => $irating);
=33=      } else {
=34=        print "tossing $file\n";
=35=      }
=36=    
=37=      ## next part is to fool it from downloading again
=38=      open F, ">$file" or warn "Cannot create $file: $!";
=39=      close F;
=40=    };