Welcome to this issue of Dr. Dobb's Data Compression Newsletter! This newsletter keeps you up to date with news items, product information, and the occasional editorial aside about the world of Data Compression. Your input is always welcome in the form of comments, submissions, or original material.
PNG Format: Going, Going, Gone!
June 15, 2003
The folks at CNET are prognosticating about the possible demise of the PNG graphics format. Admittedly, PNG has never been a huge success, but it's there, and it has decent support in browsers and graphics programs. So why would it suddenly go belly-up on us?
By CNET's line of thinking, the expiration of the key LZW patents held by Unisys will suddenly make GIF files a lot less of a liability than they are today. It's well known that if you make a product that supports the GIF file format, you are going to get a letter from Unisys insisting upon a little render-unto-Caeser action. And they have the lawyers to make it happen.
Given the legal situation, there was actually a monetary incentive to switch to PNG. But once those nasty letters from Unisys are gone for good, will anybody see any reason to continue using PNG?
Well, they might want to consider that it's head and shoulders above GIF in the technical arena. GIF files are limited to a crummy 256 color palette, they're stuck with old-school LZW compression, they don't have a true alpha channel - they just lose to PNG all over the map. (We won't talk about the dirty little secret known to PNG advocates: no support for animation.)
Do any of these factors matter more than that patent issue? We might know better in a year or so. But my guess is that PNG will continue struggling with its single-digit brain-share, much as it has today. Too bad, it deserved better.
CNET: Bell tolling for PNG graphics format?
ZipDrop
June 15, 2003
This Mac OS X Droplet accepts files or folders, then uses the command line Zip interface to create a corresponding zip file. In other words, it's the world's simplest zip program. And completely free, of course.
Unzip? Well, I suppose that's another Droplet.
The author of this program, Markus Weimer, appears to be a native speaker of German, but he was nice enough to document this program in both English and German language versions. Thanks, Markus! (The program itself is language-neutral, apparently having no UI whatsoever.)
Real Player on Palm Tungsten
June 14, 2003
If you've forked over enough dough to purchase one of those nice Palm Tungsten PDAs, you certainly are going to want to be able to use it as a portable MP3 player, right?
If you are, that's no problem, because RealNetworks is now shipping a version of the RealOne Mobile Player which is designed to work properly with this and other Palm PDAs. And it appears to be free, so you can download it and start playing right away.
If you go to the RealNetworks Web site to download it, not only do you get a free player, you can also download some free tunes, including works by Moe, Johnny Marr, the Andrew Speight Quartet, and Orbiter.
Yahoo! News: RealNetworks Announces the RealOne Player For the Palm Tungsten C
RealOne Mobile Player for Palm
Leadtools ePrint 3.0 Ships
June 14, 2003
So, you are working on an image in your favorite program, but you need to save it as TIFF file, and that format isn't supported. What do you do?
If you're like me, you start digging around through all your file conversion programs, hoping to find just the right program, you know, the one that knows how to covert format A to to format B. Sometimes you find that you might even need multiple hops to make the trip.
Well, forget about all that hassle. Lead Technologies has a program called ePrint that looks to your system like a print driver, but actually diverts your output to an image file that will be created in one of dozens of formats. So if your program knows how to send data the printer, it automatically picks up the ability to create files in all sorts of different formats, including JPEG2K, TIFF, PNG, and more.
Even if you aren't worried about compressed file formats, you have to appreciate the fact that this program lets you create PDF files. That alone is worth the price of admission!
And yes, you can get a trial version of ePrint before you fork over the full price for the real thing. (The full price is a modest $99 direct from the publisher.)
GIF Animation for Beginners
June 13, 2003
Animated GIFs are of course the backbone of the Web, so it's kind of surprising that Win32 developers don't have a slam-dunk way to display these bits of eye-candy in their MFC apps. Sorry, but it's true; Microsoft is happy to display animated GIFs in their Web browser, or a Web Browser control, but it's not such a simple matter to do so in a normal app.
Fortunately, an Iranian programmer named A. Riazi was nice enough to publish an article on The Code Project with some source code that does the job just fine. It looks pretty good, but it's a new posting, so it doesn't have many miles under it yet. Try it out and see what you think. And let's see who can find out what the "A" stands for in the name "A. Riazi."
The Code Project: Gif Animation Control
MPEG-4 Audio Royalty Streaming Begins
June 13, 2003
Really, when we talk about streaming media, we can't lose sight of the fact that the most important stream out there is the royalty stream. Complicated standards such as MPEG-2 and MPEG-4 are covered by dozens of patents, and trying to figure out who to pay when you use the technology is potentially a nightmare.
Fortunately, we have the ever-faithful invisible hand of the free enterprise to thank for its help in this problem. It turns out that most of these complicated Intellectual Property payment problems are handled in the same way: a Licensing Authority.
The job of the licensing authority (LA) is quite simple. They get all the patent holders together, and each agrees to a specific cut of the income for the technology. The LA then sells one-stop license shopping to all concerned, collects the royalties, and issues checks to the patent-holders. This has worked to the benefit of all concerned for MPEG-2, the technology used in DVD players.
Via Licensing Corporation is lucky enough to be the licensing authority for MPEG-4 audio, and they have announced that they now have a green light to start collecting royalties from those who would like to use MPEG-4 audio in their applications. And it's pretty cool, you don't have to spend a lot of time negotiating with some oily salesman. No, you can go right to the Web site and learn that including an AAC decoder in your app is going to cost you $2/copy shipped. Not too shabby!
Yahoo! News: MPEG-4 Audio Licensing Begins
Via Licensing
PNGWriter
June 13, 2003
I can't imagine a better description of this product than is found on the first line of their Web page: "PNGwriter is a C++ class for creating PNG images." And yes, it's a free library, and it's portable as all get-out, working on Win32, Linus, Mac OS X, and my favorite O/S, "Other."
Basically, this library exists for people who want to quickly create graphical images in their programs. As an example, imagine you want to print plots of the data in your latest scientific experiment. PNGWriter is there to do the job with a minimum of fuss.
From the FAQ, PNGWriter enables you do the following with an image:
- Plot
- Get the color of a pixel
- Open an already-existing PNG file.
- Plot in the HSV color space.
- Get the color of a pixel in the HSV color space.
- Plot filled or hollow lines, circles and rectangles.
- Adjust the image's gamma.
- Choose the level of compression.
- Change the file name.
- Get the height, width and bit depth of an image.
MP3FileInfo
June 12, 2003
This nice tutorial on The CodeProject Web site includes a class that lets you access the ID3 tag info in your MP3 files from C++ projects running under Win32. This is of course a very useful thing to do, and it's complicated enough that you don't want to write the code from scratch.
The only negative thing I have to say about this article is that it is pretty terse. There is a bare minimum of discussion on how to use the class the author provides, and not much to read about how the ID3 tags are actually stored, and nothing to speak of on how the author's class does its work.
But perhaps there is a ton of documentation in the source code that will guide you through the process of reading ID3 tags, making you an expert in a matter of hours. One can always hope.
The Code Project: MP3FileInfo - Extract Header and ID3 Tags of a MP3 File
Pinnacle Edition 5 Launches
June 12, 2003
Pinnacle Edition is a DVD authoring system that competes with the likes of Adobe Premier and Apple Final Cut Pro. The folks at Pinnacle Systems would like you to know that they have just released version 5 of this product.
What's new in Version 5, you ask? Well, it's kind of hard for me to tell you, exactly. Pinnacle says the new release has features that help "streamline the editing workflow" and "help content producers create compelling professional-grade video." But the various Web pages and press releases are a litte short on details. Read them yourself and see if you don't agree.
ZDNet: Pinnacle updates DVD software
Pinnacle Edition product page
Pinnacle Press Release
Pinnacle Press Release
Compreso
June 12, 2003
A fellow named Rolando Herrero has developed a Win32 daemon that implements compressed sockets. It sits as a layer between applications and the kernel driver, compressing and decompressing packets like mad.
It isn't entirely clear how the compression is being done, Roland says LZ78 in one place and LZ77 in others, but I suspect LZ77 is what he's using. It is followed up with a Huffman coder for maximum efficiency.
This is lovely freeware, and Herrero is currently shipping version 2.105. He lists it as running under Windows NT, 2K, XP, and 2003, so I guess the 9X train is out of luck. But you knew that anyway.
Apple Releases QuickTime 6.3
June 11, 2003
It's easy to be lulled into thinking that the media player market is a two-horse race between RealNetworks and Microsoft. Easy, that is, until you realize that the there are a lot of Mac users out there who are quite happily using QuickTime as their player for choice. And more than a few Windows users are in the same boat.
Apple has just released version 6.3 of QuickTime, which seems to be mostly notable for supporting media streaming designed for the 3G phone system, with the media known as 3GPP. Apple has a nice Web page describing the sort of content you can push out to the new generation of wireless phones.
As a PC user, support of 3GPP might not mean all that much. But you should still pick up a copy of QuickTime 6.3, if only for minor enhancements and bug fixes. It's a pretty nice player.
Apple's QuickTime product page
Yahoo! News: QuickTime 6.3 adds 3GPP, improves iApp support
FAAC Ships New Version
June 11, 2003
FAAC is a freeware audio codec supporting AAC coding. The goal of the project is to support the MPEG-4 LOW and MAIN profiles. This seems like a pretty good thing, and apparently its twin, FAAD, an AAC decoder, is being used in a few products. It's also compatible with WinAmp.
Given that Apple has legitimized the AAC format as a good thing, it's probably time we all familiarized ourselves with this format. I vow to start real soon now.
Home of AudioCoding.com, which makes FAAC
Speex 1.0.1 Ships
June 11, 2003
Speaking of audio codecs, looks like another one just released a new version. Speex 1.0.1 shipped at the start of the month.
The Speex project aims to build a patent-free, Open Source voice codec. Unlike other codecs like MP3 and Ogg Vorbis, Speex is designed to compress voice at low bitrates in the 8-32 kbps/channel range. Possible applications include VoIP, internet audio streaming, archiving of speech data (e.g. voice mail), and audio books. In some sense, it is meant to be complementary to the Ogg Vorbis codec.
AOL 9.0 Beta Tests Media Player
June 10, 2003
Your friends at AOL appear to be testing a beta of the next major upgrade of their client software, which will presumably be labeled as version 9.0. If we stay on the current trends, the online division of Time-Warner ought to soon be producing roughly three or four CDs of this release for every man, woman, and child in this hemisphere.
So why bother with a new release? Well, part of the reason is to offer support for AOL's new proprietary streaming technology, which the content company will now be offering alongside media players from Microsoft and RealNetworks.
It looks as though this new media player, code-named Lllama, is coming from the Nullsoft division, producers of the ever-popular WinAmp MP3 player. The folks at NullSoft have created their own streaming audio and video formats, called NSA and NSV. AOL is going to stream data in those formats, but won't force other publishers on AOL to use the format.
What's it all mean? The view from this vantage point says that based on recent reports, AOL has no idea what their strategy should be regarding streaming media. So rather than trying to refine it to a sensible position, they are instead going to try to keep juggling as many balls as possible, and will keep on hoping for a divine flash of revelation.
CNET:
Latest AOL 9.0 beta released
Internet.com: AOL 9.0 Represents Streaming Sea Change
FlyZip XR
June 10, 2003
Do you really need to zip up files that you're storing on your new 100G drive you got with your latest desktop system? Probably not - you've got storage space to spare.
But things are a bit different on that Palm PDA. It has a very limited amount of storage space, maybe 32 MB. Download a couple of games, store some data in a local database, and poof! it's all filled up.
The folks at Elegant Algorithms want to take care of that with their new product called FlyZip XR 2.5. This utility, customized for Palm OS, is advertised as giving world-class compression on the handheld platform. It can compress both applications and data, up to 1 MByte/second on the latest Palm devices.
Had my wife been nice enough to buy me a new Palm Tungsten, I could check out the public beta version of this product and tell you how well it performs. Alas, all I see is a stack of ties. So you tell me, okay?
PowerArchiver Gets A Plug
June 10, 2003
Fred Langa was at one time the editor of Windows Magazine, which probably represents the second marquee job on his resume. The first? Editor-in-chief of the renowned BYTE magazine (now reincarnated as BYTE.com).
Enough of this trip down memory lane. The important thing now is that Fred keeps himself busy freelancing and publishing his little newsletter. The June 2 issue of his newsletter featured a plug for both Power Archiver 8.1 and FreeZip.
Good to see a couple of nice products get touted by a newsletter with a big readership. Checking the Power Archiver site shows that after its last release in February, 2003, Power Archiver has concentrated on pumping out localized versions in German, Italian, Finnish, Polish, and Czech.
I was particularly happy to see the reference to FreeZip, because somehow I did not have a link to this product in my database. Freezip, by Dariusz Stanislawek, is a completely free Win32 application that does the usual job of zipping and unzipping files, and includes full integration with Windows Explorer. You go, Dariusz.
Article from the LangaList of June 2, 2003
FreeZip home page
SheerVideo Pro 0.9.9
June 9, 2003
Proprietary video codecs. Why would anybody create one? Don't we have perfectly good standards for video, like MPEG-4? Maybe Bill Gates can get away with releasing proprietary codecs for Windows Media, but how can a little no-name company really think they can play ball with the big boys?
Well, companies like BitJazz seem to think they can do just that! BitJazz has a video codec called SheerVideo that works with QuickTime, and they claim that they provide lossless compression, which is just what you need for video production and postproduction, medical apps, and scientific research.
The 0.9.9 release of SheerVideo Pro is now available for evaluation from the BitJazz Web site. Give it a try and send me an email. Of course, you had better be using a Mac - the Windows and Linux versions are promised for sometime in the future, but not today.
JZLib 1.0.2 Ships
June 9, 2003
JZlib is a nice project - an attempt to port the entirety of zlib to pure Java. You may properly note that Java already has Zip archive support, so certainly this has already been done, right?
Not exactly. Java does ship with a copy of zlib in native form, and that zlib code is there to enable Java programs to read and write Zip and Jar files. But it is most definitely C code, not Java, and the library is not directly available to Java programmers.
With JZLib, programmers can use any function in zlib, and can step right into the code to see what's happening. The 1.0 release of the library shipped back in February of this year, and version 1.0.2 just went out the door at the start of this month.
It appears that since the 1.0 version shipped there have been just a couple of bug fixes and added support for a ZInputStream and ZOutputStream class. Worth a look.
Data Compression Hits the Big Time
June 9, 2003
Okay, sometimes when you look at your interest in data compression you feel as though you're paddling your canoe around a real industry backwater. Nothing much going on back here.
Well, forget about that, you are now front page news. You have been legitimized! Check out the USA Today crossword puzzle for June 2, 2003, and browse over to 18 across. What's that clue? Open a compressed file.
SPOILER WARNING!
Yes, the answer is UNZIP, but you knew that, right?
USA Today: June 2, 2003 Crossword Puzzle
Archos 300 Series Leaves Columnists Drooling
June 8, 2003
Archos has been a minor player in the portable media player market for a few years now. They've enjoyed a bit of popularity for their hard-disk based music player, but they are still relegated to also-ran status.
I don't know if their new AV300 series of media players is going to change that, but I think it is going to turn a few heads. It has definitely got my attention.
This handheld player comes with either 20 or 40 GBytes of storage, which means you can store hundreds of thousands of digital photos, hours of MPEG-4 video, or thousands of tracks of MP3 or WMA format audio. The unit features a lovely 3.8" color LCD and a very attractive package.
There are a host of standard features and extras that really make this device stand out. It has the ability to record to MP3 directly from analog inputs. You can add a DVR package to record video directly from your TV or video camera. A special remote control has an FM radio built in, which can of course record your music to MP3 format. Plus, the FM recorder has Tivo-like capabilities, which means you can record an entire song even though you pushed the REC button long after it started.
There's a lot more to like about this unit, and I certainly would like to tell you more. But that would require a review unit, wouldn't it Archos? I can always hope!
Archos AV300 Product Page
SFGate.com: Digital video goes portable with Archos gadget
The Register: Archos ships 'cinema-to-go' mobile video player
MP3 Appliances For Dummies
June 7, 2003
PCMag.com writer Lance Ulanoff decided that perhaps he might like to take a stab at trying one of these networked MP3 players that are starting to bubble up into the Matrix. He picked the cd3o C300 Network MP3 player, gave it a spin, and shared the results with those of us who follow his exploits on the PC Magazine Web page.
Good news for all, Lance found it pretty easy to install, pretty easy to use, and now claims to be a believer in networked media players. His list of recommendations at the end of the article are pretty much a call to arms for the MP3 generation.
Only a couple of sour notes in the whole article. First, Lance does fess up to the fact the he'll be returning his c300 eval unit to the manufacturer, so he's not a total believer in this technology. Not total in the sense that he's willing to pay for it.
The second sour note is a bit of bad advice from Lance. He recommends that you start collecting MP3 files from reputable sources, which is certainly good advice. But he includes as an example of a good source the Apple iTunes Music Store, which is definitely not good advice. Any tunes you buy from Apple are not going to play on any networked audio player that I know of! This may change in the future, but there's certainly no guarantee of it at this time.
Lance also recommends buying individual tracks from Listen.com, PressPlay, and others, and again, I have to say this is not good advice. Listen.com, for example, does sell you individual tracks for the sweet price of only 79 cents. But those tracks are burned directly to an audio CD, so you never actually have an MP3 file in hand that you can use. Anything you buy will have to be re-ripped from CD. For that trouble, you're better off buying the retail CD and getting full quality.
Lance's heart is certainly in the right place, but much like the New York Times, it appears that PCMAG.com could use a decent fact checker. Love ya, Lance!
PCMag.com: MP3 Goes Old School
Jpegpixi
June 7, 2003
You just took the greatest snapshot in the world, perhaps of George Bush giving Jacques Chirac noogies at the G-8 conference. The only problem is, your camera has a single bad pixel, and you need to correct the resulting image.
You might think your only choice in this case would be to load the photo into your favorite image editor, manually correct the bad pixel, then save it. This corrects the problem, but is a big hassle, and it subjects the picture to a decompress/recompress cycle, giving you a bit of a hit on quality.
Fear no more! We've got a Linux application called Jpegpixi that is designed for just this situation. It will fix that bad pixel, automatically interpolating from its nearby neighbors, and update the image without a decompress/recompress cycle!
Best of all, jpegpixi runs from the command line. So if you have 200 pictures from this same camera, with a little script you can run through them all in no time flat while cooking your scrambled eggs for breakfast. Good deal.
DRM: Don't Bother
June 6, 2003
Readers of this space have been tracking a lot of stories on Digital Rights Management lately. Much of the fuss is a result of Apple's successful rollout of iTunes, a pay-per-download music service that depends on DRM to ensure users don't share their purchases with the entire world.
The big players in the distribution of content have to be believers in DRM - because there is currently no Plan B that allows them to make lots of money without it. At least in the minds of the RIAA and the major record labels, distribution without rights management of some kind is instant Napster, and that makes them very depressed.
Turns out then that a study by market research firm In-Stat/MDR is going to be causing heartburn all up and down Laurel Canyon. Their study indicates that consumer acceptance of DRM-based schemes is so bad as to make it untenable.
The study anticipates major announcements of pay-per-view and pay-per-listen services later this year, and expects consumers to react in a lackluster manner. The article says:
"Unfortunately, most of these services will be virtually ignored by consumers who are used to accessing and downloading their digital content for free," Mike Paxton, a Senior Analyst with In-Stat/MDR, said in a statement.Somebody needs to tell Steve Jobs. After all, he's probably figuring that the $2M bucks he raised in the first couple of weeks of iTunes meant he was on to something big. Too bad he didn't read this report first!
Interntnews.com: DRM Makes Minimal Impact, Study Says
On2 Technologies Dreams Big
June 6, 2003
I don't know whether it's hubris or just wishful thinking. On2 Technologies is the developer of a proprietary video compression scheme known as VP6, which was covered in May in this space. As part of the rollout of this technology, On2 suggested to the DVD forum that it might not be a bad idea to use VP6 as the format for High Definition DVD compression.
It's interesting to note that On2 has also submitted their earlier standards, VP5 and VP4, to the same body. No word on what sort of feedback they received for these submissions, but give the blokes credit for not giving up.
The article says that On2 is attempting to create a royalty structure that the DVD forum can live with, but no word on specific numbers. Personally, I have to imagine they can afford to go pretty low. A revenue stream of $1 per HD-enabled DVD player would probably work out to a pretty tidy sum after a few years of distribution.
Of course, this is the same forum that Microsoft has been wooing with their Windows Media 9 video codec. Nobody has given Microsoft much chance of success in their venture, not because their codec is inferior, but rather for political reasons. Perhaps On2 is thinking that they can slide under the radar on those nobody-likes-Mr.-Big issues?
Yahoo! News: On2 Technologies Submits VP6 to DVD Forum For Use in High-Def DVD
DivXWMAConverter
June 6, 2003
Let's say you're a happy Mac user, and your Windows buddy just sent you a hilarious DivX movie you've been dying to see. You fire up your media player, then find out that your inconsiderate pal encoded his audio using Windows Media Audio! No codec available for Mac OS X, so you either have to pretend you're back in the silent film era or watch it on your spare Windows PC. Either way is kind of yucky.
This must have happened to Patrice Bensoussan one time too many, and he decided to take action. Enter DivXWMAConverter, a free program for the Mac that will strip out that nasty WMA audio and replace it with something that QuickTime might be happy with.
Thanks, Patrice, for your noble efforts.
Fickle AOL To Switch to WM9?
June 5, 2003
It was just a couple of days ago that I was reporting on AOL's switch from Real Networks to Dolby AAC for their streaming radio service. Now it turns out that the media giant has another ace up its sleeve.
It seems that part of the big settlement between Microsoft and AOL that you've been reading about includes granting AOL the right to use Windows Media 9 for content delivery.
Apparently the deal doesn't require AOL to start using the proprietary format, but the author of the ZDNet article seems to think we are going to start seeing AOL using this technology quite soon.
Not surprisingly, RealNetworks didn't think the agreement means much of a change at AOL. Of course, what else would they say?
ZDNet: Microsoft wins in digital media
WinRAR 3.20 Ships
June 5, 2003
WinRAR is probably the most popular archiver that is designed around a non-Zip format. Why? If I were to guess, I'd say first because of its excellent compression performance, and second because of a decent user interface.
You can count on RAR format archives to be considerably smaller than ZIP archives in almost every case. Although this is not a free archiver, the price is just $29, which is not too shabby.
WinRAR 3.20 shipped sometime in May, with a nice set of improvements, which includes enhancements in timestamps in RAR archives, the ability to rename files inside RAR archives, and a raft of other features. Definitely worth a look.
Hifn Introduces New Compression Processor
June 5, 2003
Even if you are a bit of a compression enthusiast, you may not have ever heard of Hifn, which makes compression hardware.
The Hifm compression engines I'm familiar with seem to be targeted towards tape drives and other backup systems that support the LZS compression algorithm.
Hifn has just released a new chip that implements this algorithm, and is able to compress or decompress data at a rate of 160 Mbytes/s. I think the company is hoping that with this kind of speed they can get design wins outside their normal territory - perhaps in devices such as SCSI disk controllers, network interfaces, and other mass storage devices.
Nice idea, but I have to be a bit skeptical. The press release I'm working from says that this chip is going to sell for $39 in 10K quantities. At that price, it isn't likely that you will see it showing up in mass storage devices selling for less than premium prices - perhaps anything under $1K. So for right now, I think this is still a specialty technology.
Yahoo! News: Hifn Introduces the 9620 High Performance LZS Compression Processor
Hifn home page
AOL Says AAC Is the Real Deal
June 4, 2003
RealNetworks took a kidney punch last week, as AOL finalized their plans to drop their use of Real's audio format in favor of Dolby AAC. The folks at News.com offer up some interesting speculation that the source of this rift is AOL's attempt to develop their own streaming server technology, known as Ultravox. Since Real aims to derive real revenue from selling server software, this could be a source of friction.
So the future at AOL appears to be music downloaded over Radio@AOL from UltraVox servers in AAC format. And yes, if you are an AOL listener I expect you are going to be needing a new client.
Things are looking pretty good for the AAC streaming format. Apple has given it a nice boost as the codec of choice for their iTunes format, it has been endorsed as the audio channel for MPEG-4, and now AOL has bellied up to the bar. Who wins in this exchange? Looks to me that the royalty money will be going to Dolby Labs, AT&T, Sony, and the beloved Fraunhofer Institute. Small fees, but a penny here, a penny there, and next thing you know you're talking about real money.
CNET.COM: AOL drops RealNetworks for Dolby
PATRICIA trie implementation
June 4, 2003
This CodeProject article describes the development of a PATRICIA trie in the .NET framework. The actual code is written in C#, but naturally, it can be used with any of the .NET languages.
Tries are data structures that turn up pretty frequently in the world of compression, and the venerable PATRICIA trie has been keeping it real for something like thirty years. It's still a keeper, and you might find uses for it when developing PPM algorithms, or perhaps even for your string searching code in an LZSS program.
Of course, it's an open question as to whether you'd want to do this in the .NET framework. Just like with Java, there are always going to be those of us who sniff "Seems a bit slow." Dirty little secret. But, just like with Java, I also hear the familiar refrains that the 90% of the optimized code is "just as fast as C++." In any case, I guess that's not a compression issue anyway.
The Code Project: PATRICIA trie implementation
UFUP
June 4, 2003
Open Source sometimes reminds me of Mother Nature and the Theory of Evolution. Much of the process seems to consist of putting together random packages, throwing them against the wall, and seeing what sticks. After all, how else can you explain Michael Swaine's Cousin Corbett?
Indeed, how else could you explain UFUP? This is a Win32 project to decompress popular UNIX archive formats, including bzip2, gzip, and tar. Now, there's nothing wrong with wanting to do that, but we do have to consider that there are probably at least a hundred Windows archivers that already do just that - with a pretty good representation of free software in the mix.
So what is UFUP going to bring to the party to avoid being a complete wallflower? To tell you the truth, I have no idea, but the initial 0.11 Alpha release notes boast "New installer. - Nicer looking icon." So that's something.
IMTC Fighting Multimedia Patents
June 3, 2003
We all know about the kind of submarine patents that make life difficult for users of technology such as GIF files. But lately we've seen the same kind of patent system abuse leading to judgments against the users of standardized multimedia technology. For example, in April I reported that Sony had lost a $25M judgment to Forgent, a company that did absolutely nothing to further the JPEG standard. All Forgent did to advance the state of the art was to purchase the rights to an obscure patent that was determined by a jury to cover part of the technology.
Most of us just sit around whining about stuff like this, but the International Multimedia Telecommunications Consortium (IMTC) is doing a lot more. They've chosen to put together a comprehensive historical archive of prior art covering multimedia standards.
According to the article on Yahoo! News, the IMTC has managed to put together good documentation on all the technology covering various standards such as H.261 and H.320, which in turn covers much of MPEG-2 and MPEG-4.
An archive such as this is no guarantee of protection against submarine patents, but it does at least raise the bar a bit. Any company that needs to defend themselves against this kind of legal challenge at least knows there is a good source of ammunition available for their defense.
And we deem that a good thing.
Yahoo! News: IMTC Fights Patent Chaos in Multimedia Standardization
Parsing MPEG Streams For Fun and Profit
June 3, 2003
If you're like me, and you're developing a tool that spits out MPEG streams, your testing tools consist of trying to load the stream in a popular media player and hoping it runs. The problem is that when this doesn't work, you have absolutely no clue what went wrong. Is there a better way to test your output?
Turns out there is. A set of tools collectively packaged under the name VK Tools is designed specifically for just this type of use. VKTools is a set of programs that are used to analyze MPEG streams, helping you check out the compliance of your work. There are three different tools that help you poke around in MPEG streams, see their contents, and dump their headers.
If you want to use VK Tools, you had better be running Linux.
Neuros to Support Ogg Vorbis
June 3, 2003
If only I had the readership of a John Dvorak or Dan Gillmor! If I were a widely-read industry pundit I could expect to get a flood of review hardware in the mail on a daily basis. Instead, I see something like the Neuros MP3 Digital Audio Computer and all I get to do is drool.
This new hard-drive based MP3 player is really quite the business, equipped with features even Steve Jobs hasn't thought of yet. Beaming music to your FM radio is pretty nifty, but I really like the HiSi feature, which identifies songs you've recorded from the radio via random 30-second clips uploaded from your PC.
Anyway, there are plenty of portable audio players out there, and since I don't get freebies, there isn't really any reason to cover them. But the Neuros breaks through by virtue of the fact that it supports the ultra-cool, open source Ogg Vorbis audio format.
Although it's not easy to get hard data on this, the Vorbis audio codec wrapped in the Ogg package reportedly performs much better
than standard MP3, and perhaps is in the loftier realms of AAC and Windows Media Audio. Regardless, we have an obligation under the terms of political correctness to give it major props, and as a result that means an obligatory nod to the folks at Neuros.
Buy one, check it out, and tell me what you think. Even better, buy one, send it to me, and I'll tell you what I think!
MP3 Newswire: Neuros Review
Neuros Web site
Apple Fighting Piracy
June 2, 2003
Apple quick-like pumped out a point-point release of iTunes that they hope stymies some streaming activity that was making them not so happy.
As was reported in this space a few days back, there are various pieces of software out there that allow copying of files from iTunes servers, which is of limited utility, since the encrypted files are not playable on unauthorized machines. Of course, music that the user has ripped by hand from a CD has no such protection, which means their iTunes machine is suddenly a Napster-like server!
The other piece of news about iTunes that stuck in the craw of the RIAA and other industry types was the possiblity that users could stream their tunes across the Internet. Again, it doesn't seem possible to locate any specific instance where this capability led to piracy. After all, you can only stream to three authorized clients, and as far as I know there have been no instances where anything other than this has happened.
Nonetheless, ponytails were quivering in the boardrooms of the major music labels, which seems to have resulted in iTunes 4.0.1, which disables sharing across the net. It may also have tightened up on the security holes which enabled file copying.
What does this all add up to? Not much. About the only people affected will be those who wanted to stream tunes at home from their server at work, or vice-versa, surely a small (but no doubt unhappy) minority.
Nonetheless, the move seems to have sorely irked Industry Gadfly Dan Gillmor, who sees it as nothing more than an direct attack on basic human rights. (Don't forget Dan, this type of sharing is in fact a violation of copyright law, like it or not.)
Personally, I think it's pretty remarkable that Apple has been able to roll out this service in a big way with no major security problems whatsoever. Imagine what a fiasco we would have on our hands if the DRM in AAC proved easily broken? Every one of those two million songs already shipped would be instant Kazaa-fodder, and the industry would have iTunes shut down in no time flat. And don't think there aren't people out there trying to break the service.
No, I'd say the security aspects have to be declared to be a smashing success. Apple has shown that we now have an infrastructure that can distribute signed music files in a way that makes both consumers and record labels happy. Nobody has done that before, and I sure hope the whole house of cards manages to stay upright. If it were to topple I don't know if we'd ever see another attempt to retail tunes online in a big way.
PCWorld.com: Apple Takes Aim at Music Piracy
Dan Gillmor: Some Upgrade
Yes, KDC Files Can Be Converted
June 2, 2003
If there is anything that will get a developer riled up, it is an undocumented file format. Everyone from the most ardent Free Software Richard Stallman clone all the way to Microsoft's biggest proponent of Free Enterprise surely has nothing but scorn for anyone who thinks they need to protect the structure of their data files.
Anyone who doesn't feel this way just isn't a real programmer. There, I said it.
The heroes in this philosophy I seem to be over-selling are those who reverse-engineer the closed formats. And if I'm not mistaken, the folks who developed kdc2tiff are among that crowd. Kodak cameras apparently produce files in some sort of native Kodak format. (Perhaps because there just aren't any adequate formats out there?) Folks who want to work with these files are stuck using Kodak software for conversion purposes.
To the rescue: kdc2tiff. This Linux app takes care of converting KDC files to formats that the rest of the world understands. And that is undoubtedly a good thing.
The bad news is that kdc2tiff has not scaled every obstacle that the bad citizens at Kodak have placed in their way. It turns out that DC40/50 KDC files are using a proprietary compression format called RADC that has yet to be cracked. Sigh.
CHM2PDF Pilot
June 2, 2003
Speaking of proprietary formats, here's a program that converts Microsoft's proprietary CHM help files into Adobe's proprietary PDF format!
While I really like using the p-word three times in one sentence, it isn't quite as ominous as one might think. Altough both of these file formats are owned lock, stock, and barrel by corporate titans, I do believe that they are fully documented. I further believe that there is a fair amount of free software that will let you work with these files, at least to some extent.
In any case, this nice piece of shareware acknowledges that someday you just might have a need to convert a Microsoft CHM-format help file to Adobe's PDF format. (If this sounds to you like a solution in search of a problem, you're not alone.) Since it's shareware you can give it a spin for free and see if it does the job for you.
Benchmarking Compressors
June 1, 2003
So you've got a new compressor that you're working on, and you're wondering how to see where you stack up in the world of compression.
A chap who posts under the nom de net of Sim posed just this question to comp.compression in late May and got a few decent answers, which are found in the link below.
Although I was a little late posting my two cents, I did get in a plug for the Benchmarks page on DataCompression.info, which has links to a lot of great file corpi, as well as some good benchmarking sites. Shameless self-promotion? Yes.
comp.compression post asking about benchmarks
DataCompression.info Benchmark page
DivX Subtitles with GTKsubtitler
June 1, 2003
The DivX codec naturally supports embedded subtitles, and this program lets you manipulate them. The Web site says this includes moving, merging, and converting to ISO-88591-1 and -2.
This gets me wondering - if the program converts the text to these ISO character sets, what is it converting from? It can't be plain ASCII, because in that case there's no conversion to be done - ASCII is a subset of these two character sets. Are the subtitles in DivX stored as Unicode? If you know, please drop me a line!
Netpbm 10.15
June 1, 2003
Out of respect for netpbm, I feel obligated to report on each and every update. Version 10.15 shipped at the end of April, and 10.11.9 of the stable source release shipped at the end of May.
As always with netpbm, the roster of changes and bug fixes are pretty idiosyncratic - if you don't know the package already the fixes aren't going to make too much sense. But trust me on this point: if it involves a graphics file, and it can be done from the command line, netpbm probably does it.
Nice Try, But...
May 31, 2003
USENET newsgroups are notoriously tough on the naive, foolish, or inexperienced folks who are foolish enough to jump in headfirst with a Grand New Idea. And comp.compression, the major newsgroup covering data compression, is certainly no exception to this rule.
A fellow going by the the name TekCat (bad sign already) posted a question to the newsgroup asking if anyone had ever given thought to the notion of using mathematical formulae to churn out sequences of data matching up with that which is to be compressed.
Has anyone thought of using mathematical formulas that can potentially represent the data set (lets say a sequence of bytes) as a number, and using these formulas (like exponents, factorials, and such) to break the big number into smaller number or set of numbers ( they would have less number of bits than the source number).Of course, this is the essence of the idea I've termed Magic Function Theory. It's a seductive notion, but even if you don't take the trouble to rigorously disprove its utility, a few days of experimentation is usually enough to show you the error of your ways.
But who needs that when we have the Internet to disprove our hopeful ideas? And of course, that's what the regulars on comp.compression attempt to do for our brave poster. And I have to really give credit, the responses to this post were pretty civil, and seemed to be doing their best to explain things in a helpful fashion.
It's worth a read through the thread, which seems to have petered out after 20 or so comments. Eventually it goes to the only place where this can really end, which is by falling back on Kolmogorov complexity.
Specter Raises Specter
May 31, 2003
I reported a few days back on the fact that WinZip had taken the bold step of releasing a beta version of their product that is incompatible with the gold standard, PKZip 6.0. Looks like columnist Lincoln Specter from PCWorld found this a bit alarming as well, and posted an article pointing out some of the excitement we might have ahead of us.
Spector took the time to contact company representatives, read through release documents, and so on. This is great, but in the end we reach the same conclusion: the lid is off the can and the worms are crawling out.
Yahoo!: Updated WinZip Alters Zip Format
Maaate: The Australian audio analysis toolkit
May 31, 2003
Maaate consists of a set of libraries that let you analyze the audio streams encoded in MPEG files. Besides simply parsing the audio out of MPEG files, it also does some sort of energy detection, silence analysis and more. But reading between the lines I think this library's strength is supposed to be a nice architecture that lets you easily add the modules you need for your work.
If that's not enough to get you a little fired up, how about the fact that Maaate works hand-in-hand with Bewdy, a visualization tool and GUI? And if you're wondering about the choice of names for these two products, yes, I'm afraid you're right: they come from the Land of Oz. Enough said.
DivXNetworks Lining Up Chip Partners
May 30, 2003
I remember when the DivX;-) codec was a friendly little open source implementation of the MPEG-4 compressor in a format compatible with Windows Media Player. Those were innocent times indeed, but today we are in a radically different world.
Now the DivX codec seems to be in control of a company called DivXNetworks, and they are very serious about seeing their product designed into software and hardware products wherever possible. There is already movement afoot to get the DivX codec into DVD players, where it could provide High Definition movies on standard DVDs.
To facilitate this kind of work, DivXNetworks is working with ALi Corporation, described as a leader in several IC market segments, including DVD players and PC peripherals. If ALi can come up with chipsets supporting the DivX format, presto, you have a whole generation of DVD players that have built-in support for HD movies.
For what it's worth, this article claims DVD quality at 7-10 times better compression than MPEG-2, which is what we see right now on DVDs from Blockbuster.
Yahoo! Finance: DivXNetworks Partners With Leading IC Supplier ALi Corporation
FreeImage 2.6.1 Ships
May 30, 2003
As far as I'm concerned, we can never have enough free libraries for reading, writing, and manipulating compressed graphical images. FreeImage is yet another product that fits the bill, and it appears to be fairly mature and nicely feature-laden.
FreeImage is yet another free library for image reading, writing, and manipulation. FreeImage supports a long list of compressed formats, including JPEG, PNG, TIFF, and more. Claims to support multiple platforms, but it does apear that most of the experience with this project is on Win32 systems.
The 2.6.0 and 2.6.1 releases offer the usual bug fixes, some minor optimizations, and two new conversions: FreeImage_Threshold and FreeImage_Dither. If you really want to know what those do I'm afraid you're going to have to dive into the docs.
Animal - AN IMAging Library
May 30, 2003
I'm guessing that back in 2001, a bunch of programmers went out one night and saw Rob Schneider's comedy classic The Animal, and naturally thought, we should really do something to honor this film.
Whether that's how it happened or not, we now have an alpha release of Animal - AN IMAging Library, that says it supports over 80 compressed formats. That's certainly a nice number, and we're always happy to see that kind of figure in a free library.
It's a bit early on to pass judgement on this library, but why not give it a shot? If you're developing on its target platform, Win32, maybe you could use a nice free library. If so, download it, use it, write a review, mail it to me, and I'll post it here. I might also see if Columbia TriStar films will send us some promotional materials left over from the movie. Or not, it's undoubtedly a dumb idea. But I can always dream.
DirectX 9.0a SDK Runtime Ships
May 29, 2003
DirectX is Microsoft's technology for high-performance rendering, be it graphics, video, or audio. You can use the DirectX SDK to cozy right up to the hardware on a Win32 machine, which means it is indispensible for game developers.
It's also pretty important if you are developing applications that render or encode media. If you are of a bent to do that, you usually need to be sure that you're working with the latest and greatest release of the SDK, which has now moved up to 9.0a.
This version of the SDK from Microsoft contains both the debug and release libraries. The debug libraries have additional error checking and logging capabilities that the release libraries don't. From personal experience, I can tell you that it's nice to have the debug libraries let you know when you've done something monumentally stupid. It happens.
Nothing too exciting in the move from 9.0 to 9.0a. Looks like the usual mix of minor performance enhancements and bug fixes. But in this bug-rich area of development, you need to have the latest stuff on your machine at all times, so download it now and save yourself from trouble down the road.
Microosft's DirectX 9.0a download page
MPEG-4 Entering DVD World via Stealth
May 29, 2003
Ever heard of Sigma Designs? The proverbial man on the street will undoubtedly have to answer "no" to this question, but apparently there is a good chance that the DVD player you bought last year for Christmas contains a Sigma decoder chip.
In an article on Yahoo! News, Sigma lets us know that their EM8500 decoder chips are being used by eleven different DVD manufacturers, which gives them the ability to play MPEG-4 and DivX data streams.
Interesting story, but there is one hitch. Check out the list of manufacturers that Sigma lists as users of this chipset:
- Cosmic Digital
- Jamo
- Kiss Technology
- Link Concept
- Lite-On IT
- Momitsu
- Mustek
- Neuston
- Shenzhen Skywood
- V Inc
- Xoro
Yahoo! News: Sigma's DVD Decoder Chips Proliferate into a Widening Range of Advanced DVD Players
CleanZipAndGo
May 29, 2003
>From one of my favorite sites, The CodeProject, comes this handy utility called CleanZipAndGo. This is a somewhat unique utility that will only be useful to those of you working on Visual Studio programming projects. Running it erases all your temporary project files, then zips up all the leftovers so you can backup your work.
If you're not a programmer, you'll not find this to be very useful. If you do work in Visual Studio, you might want to think about the fact that backing up your work from time to time is a good idea. Here's an easy way to do it.
The Code Project: CleanZipAndGo
Microsoft Feeling iTunes-envy
May 28, 2003
You're Bill Gates. You wake up in the morning, and yes, you are still the world's richest man. But it doesn't matter - all day long you hear nothing but people raving about how great Apple's iTunes music business is. You hear about how Steve Jobs is the greatest marketing genius the world has ever seen. You get an angry tension headache and it just won't go away. Something must be done or the world's richest head will explode.
Solution? One-up Mr. Media-friendly boutique computer-maker at his own game. At least that appears to be the plan. Microsoft has pretty good Digital Rights Management in its Windows Media Format, which lets services such as PressPlay control your ability to play downloaded music for a fee. But Microsoft comes up short in the portable player arena, which prevents PressPlay from offering true device portability for their customers.
It turns out that WM9 does have DRM that works with a few popular portable media players, but the DRM is effectively only useful for determining if you are a purchaser of a given track. What's missing is the DRM capability that lets PressPlay indicate that you are a subscriber to their service, but might not be tomorrow. This could be accomplished through some sort of timeout capability, and Microsoft is ready to add that to their WMA format.
So today I can pay $10/month to PressPlay, and download thousands and thousands of WMA music files to my PC. They're mine to play, as long as I keep paying my monthly fee, but the second I stop, the files are so much random garbage. By this time next year, I ought to be able to do the same thing with WMA files on my Rio or Zen portable player - keeping all those files playable as long as I dock with my PC once in a while to provide my bona fides.
And once I can do this, Bill's headache will slowly disappear, leaving him with a happy, contented smile. That is, until the next time Steve shows up on the cover of Newsweek with yet another marketing triumph.
CNET: Microsoft prepares reply to iTunes
Slashdot has at it
Amazon Cozies Up To Windows Media Player
May 28, 2003
Microsoft and Amazon have issued a press release telling the whole world that they've found true happiness together. For years Amazon has been letting customers listen to 30 second samples of songs for sale on their Web site. But we poor folks had to wait for long seconds for the entire sample to download before they heard anything.
We've all heard that kids these days were raised on Sesame Street and MTV and no longer have any patience for anything but instant gratification. Amazon must have bought into this theory, because they have chosen to rework their server infrastructure to offer instant-on music samples with Windows Media 9.
One interesting note in all this is a single line in the press release: "Loudeye Corp. was instrumental in deploying this technology..." I've never heard of Loudeye before, but apparently they have enough swing with Bill Gates to muscle their way into this press release. A peek at their Web site turns up this interesting press release, indicating that somebody at Loudeye was industrious enough to rip 30-second samples of 3.3 million songs from several hundred thousand CDs. Impressive, but I think my son's college pals do that every weekend.
ZipGenius 5.0.1
May 28, 2003
Product names are important, so why not be a little self-congratulatory when you are thinking up a moniker for your next little app?
Whoever named ZipGenius was apparently in that frame of mind. But all cynicism aside, this is a nice little product, and it is completely free. No ad-ware, no nag-ware, no shareware, this is the real thing.
ZipGenius supports over 20 different compressed file formats, has built-in email, FTP, and backup functions. It ships in at least three different languages, and even has a slide-show feature available as a plug-in. Let me know what you think of this free product from Italy.
PC Magazine on Archivers
May 27, 2003
PC Magazine had their giant review of utility programs in their most recent issue. Naturally, compression utilities get the better part of a page. The folks in the testing labs took a look at 9 popular archivers, putting them through their paces, then awarded the gold medal.
I'll spoil the results for you right now by letting you know that PKZip 6.0 was the editor's choice. The editors liked the user interface, were happy with its speed, and appreciated the support for the Bzip2 format, offering superior compression.
Editorial space in magazines is tough to get, so we don't get detailed reviews on all the products, but at least you get some of the highlights. Worth a look.
PCMag: Compression Utilities Comparison
iRad DICOM Viewer
May 27, 2003
iRad is an open source project written to view DICOM files on your Mac systems running OS X. For those of you not in the know, DICOM is a set of standards for storing and accessing medical imagery. It is written to help with the unique requirements that the medical industry has for imaging.
It appears that at this time iRad only supports Uncompressed and JPEG format files in both color and gray scale. Version 1.4.0 is available in beta format, I suggest you give it a whirl.
iRad: DICOM Viewer for Mac OS X
OpenIPMP Tackles MPEG-4 DRM
May 27, 2003
The folks at OpenIPMP are busy trying to create open standards for Digital Rights Management integrated with MPEG streams. It appears that MPEG-4 is where the work is being done right now. Read the "Background" page for a pretty good synopsis of what they are doing and where they are going.
DivX Pro Adds Data Rate Allocator
May 26, 2003
The folks who sell the DivX professional codec have souped it up a bit in the latest release with a feature called EKG. EKG allows the person doing the production work on the DivX stream to reallocate data to specific scenes within a stream.
Seems like this ought to be a useful feature, but I'm not sure I would be willing to describe it as revolutionary. If I understand the feature correctly, it sounds as though it simply allows me to modify the quality dial in the middle of a stream, instead of just setting it up once at the beginning and leaving it there for the duration.
My first reaction to the feature is something like: You mean they didn't have this all along? My apologies to the programmer who slaved away to get this into the 5.0.4 release and now feels unappreciated. I've been there too.
Yahoo!: New DivX Pro Video Codec Includes Data Rate Allocator for Enhanced Encoding Quality
Plugin Advice From Redmond
May 26, 2003
Windows Media Player 9 has a completely new plugin architecture, with the folks at Microsoft attempting to open the thing up for maximum monkey business. Everybody and his brother has a media player that has user interface plugins and a skinning interface, but Microsoft tops that with DSP plugins and rendering plugins, which actually manipulate and render the digital media streams the player is using.
I don't know about you, but I find that just because Microsoft offers an interface like this doesn't actually mean it's usable. Designing a friendly API for third-party developers is an art, and Microsoft has had plenty of misses to go along with the occasional hit.
This new plugin architecture for the media player could be a winner or the old gray mare, but at least it has some online documentation. If you are planning on developing in C++, and have the endurance to read through some lengthy text, perhaps you can find yourself the proud author of a clever new accessory to Microsoft's media player. A free subscription to the newsletter for the first person who creates a workable Max Headroom AV filter.
MSDN: Making Windows Media Player Plug-ins Work Together
Gr8 Player
May 26, 2003
If somebody is willing to put in the work to create a free media player, who am I to complain about the name? Yes, calling your product Gr8 Player is a bit self-aggrandizing, but I guess I can always ask for my money back if don't like it, right?
This player runs on Win32 (although you'd have a tough time figuring that out by reading the Web page) and supports quite a few of your better known media formats, including MP3, WMA, DivX, and more. Looks like it even has some minimal slideshow capabilities.
Political correctness demands that we give props to this kind of effort, and I will certainly follow form, without investing the time and energy needed for a test drive. I leave that to you, dear reader!
PC Mag on Enhanced Dialup
May 25, 2003
You've seen a number of stories in this space over the last couple of months on high-speed dialup schemes. Typically, for a few extra bucks per month, an ISP like EarthLink or NetZero gives you some client-side software that works in conjunction with server-side secret sauce to speed up our otherwise pitifully slow dialup connection.
How do they do this? There are a lot of different techniques at work here, but data compression definitely plays a big part in the mix. For example, graphic images and JPEG files on the Web sites you are viewing might be compressed down to shadows of their former selves. And of course, HTML sites that were being delivered in clear text will be compressed before being downloaded.
So the big question is whether these compression techniques, along with server-side caching, DNS tricks, etc. add up to an improved experience. The folks at PC Magazine turned their formidable testing talents to the problem, and if you read the article you can see whether it's worth the extra bucks.
PCMag.com: High-Speed Dial-Up: Does It Work?
Windows Media Audio 9 Professional Codec Features
May 25, 2003
When Microsoft rolled out Windows Media 9, they included a batch of nifty new codecs. Included in this was a new codec designed for professional quality audio work, which includes a nice set of features that ought to appeal to content producers.
This article on Microsoft's Web site goes into many of the interesting features included in this codec, such as fold-down from multiple channels to stereo, high resolution multichannel content, and more.
This article doesn't hold back on the technical content, so if you're a music producer, you might want to find a buddy who can read C code to help you over the rough spots. Nonetheless, it might be worth it. As I've said before, Microsoft intends for Windows Media 9 to be their ticket to total world domination.
Microsoft page: Windows Media Audio 9 Professional Codec Features
Why Use HTTP Compression?
May 25, 2003
If you read the PC Magazine article linked above, you'll see that there may in fact be some gains to be had from this compression stuff. This Web site discusses some straight-up approaches to doing this on your Web site. It includes a couple of useful links.
Web article discussing HTTP compression
MPEG-4 Optimization With the Xtensa Processor Core
May 24, 2003
This article is basically a pitch from Tensilica for their Xtensa Processor Core. Keeping in mind that it is basically a paid advertisement, you can still glean a good bit of useful knowledge from the article. In particular, the author does a good job of pointing out just where you are going to be burning up CPU cycles when encoding or decoding MPEG-4 streams.
This article uses what's known as the MoMuSys reference implementation of an MPEG-4 decoder as a testbed. MoMuSys is short for the "Mobile Multimedia System" implementation of MPEG-4, which apparently includes some reference code.
I'm not exactly sure where this reference code can be found, although I found a fair amount of information about MoMuSys systems here. In what may be a classic case of a bad link, the article attempts to point you to the OpenSource Initiative (OSI) and instead links to Ontario Swine Improvement, Inc., whose mission statement reads: To develop and deliver programs, products and expertise to achieve excellence in the global pork industry. My only disappointment at the OSI site was the absence of a nice page of product photos.
WinZip 9.0 beta
May 24, 2003
Market segment leader WinZip has announced a public beta of version 9.0 of their popular utility. There are numerous small enhancements for improved usability under Windows, but the most important features to be added are:
- Support for 256-bit AES encryption. This is a proprietary encryption format that is only supported by WinZip. Archives created using this encryption can not be used with other Zip archivers. WinZip clearly hopes this will not be the case forever, as they have published a complete spec for their format.
- 64-bit Zip files. No more 4GB size limits for files included in the archive, or for the archive itself. This is an enhancement blessed by PKWare with their 6.0 release.
- Improved compression with the deflate64 algorithm blessed by PKWare with their 6.0 release.
WinZip's beta page
Internetnews.com
article on the beta release
Robert Jessop Keeps Fractal Compression Alive
May 24, 2003
Robert Jessop's final year project at Southampton University involved some research on fractal compression and the creation of a new variation on the basic algorithms. Robert reports that he had some success, and has detailed the results on this page. The page includes his full report, Win32 source code, a Windows executable, and an additional report on Lossy Image Compression.
Fractal image compression was a very hot topic in the 90s, but lost momentum when the JPEG standard was released. With the release of the JPEG 2000 standard, I've been wondering if it was time to put a final nail in the coffin of Fractal image compression. Robert's paper makes me wonder if perhaps Fractal compression plus secret post-processor X might revive this fading glory.
Robert's page at Southampton University
Roxio Buys PressPlay
May 23, 2003
PressPlay, the online music service owned by Sony and the Universal Music Group, appears to have been purchased by Roxio, a company whose previous expertise in digital media seems to be limited to CD-burning software.
There is some strange connection here with Napster, as Roxio purchased the name and technology of the disgraced file-sharing company, while dodging the potentially multi-billion dollar liabilities.
Apparently the idea here is that Roxio is going to combine the potent power of the Napster brand with the legitimate downloading features of PressPlay, in hopes of creating a media superpower. PressPlay uses Microsoft's DRM to protect downloaded songs, which means you can have local copies of as much music as you like on your PC. As long as your PressPlay subscription is up to date, the songs play just fine. Stop making your minimal monthly payment and the songs are suddenly nothing more than random bits on your PC.
Full disclosure: I'm a happy subscribe to the Rhapsody music service from Listen.com, a direct competitor to PressPlay.
Wired
article: Roxio Buys Pressplay, Napster Lives
What the Slashdot crowd thinks of the deal
Ogg an RFC
May 23, 2003
For those of you who haven't been paying attention, Ogg is a wrapper format for media streams. Typically we see Ogg in conjunction with Vorbis, the Open Source audio codec. Both Ogg and Vorbis are considered very cool.
Ogg seems to have made it to the big time, having now been enshrined as RFC 3533. Of course, this doesn't mean that we should now be expecting a flood of support for Ogg streams in handheld audio players or Win32 media clients. No, Ogg is going to remain mired in the backwaters of audio streaming for the forseeable future. It would be nice to be able to predict its sudden emergence based on technical superiority and moral virtuousity, but from experience we know that things just don't work that way.
RFC 3533: The
Ogg Encapsulation Format Version 0
The Slashdot crowd mulls it over
libmpeg3
May 23, 2003
MPEG streams were designed for maximum compression of streaming video data, not for easy editing. In fact, if you analyze the structure of an MPEG stream, you might think that the format was built with the express goal of thwarting editing! An MPEG file could start with a single I-frame and never transmit a complete frame again, meaning any editing software would effectively have to read the entire stream into memory before doing any meaningful work on it.
You'd have to be a fool or a masochist to want to work with this format, right? Well, maybe so, but perhaps you'd just have to be one of the contributors to libmpeg3, because they've created a library that that performs basic editing functions in this not-so-friendly format. This library is the magic behind the advanced compositing and editing system known as Cinelerra
Microsoft Portrait 2.1 Ships
May 22, 2003
Microsoft Portrait is kind of a neat project. It's a prototype videophone out of Microsoft Research, which must be a fun place to work. I don't think Microsoft sells or supports Portrait, they simply toss it out to the world on their Web site and let you have at it.
There are two versions, one for a standard Win32 platform, and another for the PocketPC platform. In both cases it is designed to work with both slow and fast network connections. I believe that with a slow connection you get a single still picture of the remote end on your screen, and with a high speed connection you get some approximation of real video.
With release 2.1 they've added a video codec API, which is of course of great interest to DataCompression.info readers. There are four DirectShow filters included that supply the various video connections.
Microsoft Portration project page
OpenQuickTime
May 22, 2003
Wouldn't it be nice to have a fully functional QuickTime client that worked under Linux? I believe the closest thing to this that you can get right now is the QuickTime 4 Linux project, which appears to be useful only for uncompressed video files. This limitation is in place because the project is aimed at people doing production work on Linux.
So how about the Great Unwashed? Aren't they entitled to watch movies on their Linux boxes? I think so, and so do the folks at the OpenQuickTime project. They've started an effort to create a nice, friendly client that will play all the movies you can watch on your Mac or Windows machine.
The project has released a client with a 0.5 designation, which I suppose means its beta code, but they say that it already works pretty well. Looks worth a try to me.
ReaCompressor 1.5 Ships
May 22, 2003
ReaCompressor is a commercial image compression tool that is designed to assist you in preparing images for the Web, emailing, etc. It lets you choose from various formats: JPEG, PNG, GIF, and TIFF, and shows you the results of the compression on screen.
I'd love to tell you more about the great things in ReaCompressor 1.65, but apparently the folks at ReaSoft are still a bit unclear on one of the most essential things needed when sellling updates of a product: release notes. Yes, if you go to the ReaSoft Web page and try to figure out what's new in this release, you are going to be stymied.
Maybe that's not so bad, as ReaSoft has a lifetime free upgrade policy. So you don't have to make a big decision as to whether to cough up the bucks for a new release.
Still. It's Marketing 101, guys. You ship a new product, you tell people what's new. If you look at what the successful companies do when they ship a new product, you'll find that new features represent 90 percent of their talking points. Think about it, it makes sense!
ReaCompressor page on the ReaSoft Web site
Zip Support in the .Net Framework
May 21, 2003
Here's a nice article in MSDN magazine that describes the author's quest to work with Zip files in his .NET programs. He finds out quickly enough that the .NET framework itself doesn't have inherent support for the Zip format, but does find a few commercial libraries that do the job.
After a little poking around, it turns out that support for Zip files shows up in the J# libraries, which makes sense, since J# basically implements JDK 1.1.4. (Which is a version of Java that goes back so far into the stone age that Microsoft should be beet-red from the shame of their retarded stance.) Naturally, JDK 1.1.4 has full support for the Zip format in the java.util.zip packages.
The rest of the article is basically a rundown of how to use the J# classes in the .NET framework. One nice thing this article points out is how easy it is to use libraries written for one language (Java) with any .NET language, including C#. This is one of the few places where the .NET framework enjoys a very nice advantage over archrival Java.
Is iTunes Piracy for Real?
May 21, 2003
You've seen a lot of verbage in this space about the DRM associated with Apple's iTunes music service. In a nutshell, Apple has set things up so that you can only play your downloaded tunes on the machines that you register them for, with a limit of three. You can burn CDs that will play anywhere. Not too many calls for Steve Jobs's head on a pike, so my guess is that the system is playing out fairly well.
Or is it? Yahoo! News ran an article just a few days ago titled "Song Swappers Taking Bite of Apple's Service." The article implies that file swappers have found a way to bootleg Apple's AAC files from legitimate purchasers of these tunes.
Unfortunately, the article is short on details on exactly how this is happening. It is true that anyone can copy files that have been purchased from iTunes, but they can't be played on non-registered computers, so this is really a non-problem. The article did reference someone named Rob Lockstone as a Web site owner who had shut down his Web site after discovering an application that would allow users to download his files.
I checked Rob's Web site for information on this and became even more confused. Yes, he did shut his Web site down, but he clearly states that file-copying programs do not help pirate AAC files. After copying to another system they will not be playable.
Near as I can tell, the story may be that others have found a way to stream music to unauthorized systems by reverse-engineering the DAAP protocol. If so, this is certainly a small security breach, but it is essentially the equivalent of sharing copies of burned CDs. It still doesn't let you use pristine AAC files.
Update: Perhaps one of the programs known to cause the trouble is iLeech. This piece of free software does indeed trick an iTunes server into letting you copy the file instead of streaming its contents. But again, this doesn't help you play the AAC files on a different system - the DRM in place prevents
it as designed.
Corona Image Library Ships 1.0.1 Update
May 21, 2003
Corona is a nice open source image library for working with a couple of different compressed image formats. It can read and write PNG and TGA files, and can read JPEG, PCX, BMP, and GIF files. The Corona team released their 1.0.0 release in January, and have now followed it up with a 1.0.1 release.
The 1.0.1 release includes the following changes:
- added IsDirect(), IsPalettized(), and GetPaletteSize() functions
- added FlipImage() function
- made FlipImage() operate in-place on the image object
- added writing support to the MemoryFile implementation
- made OpenFile (local filesystem access) a public function
- added support for saving TGA images
On2 Releases VP6
May 20, 2003
The day of the proprietary video codec is apparently not over yet. Microsoft is of course doing pretty well with its Windows Media 9 codec, and now the folks at On2 Technologies have released their VP6 codec.
So why would you use VP6 instead of something off-the-shelf, like MPEG4 or Windows Media? Well, On2 has a pretty good list of reasons, ranging from computational complexity, lower royalty payments, optimization for computer applications, and more.
Regardless of the quality of this codec, is it destined to be nothing more than a little fish in a big pond? As is so often the case, you might be able to figure out where this is going by reading the in-depth
analysis at Slashdot.org.
On2 VP6 page
Internetnews.com
article
Next Generation DVD Format
May 20, 2003
In theory the news feed on this Web site is dedicated to compression issues. If you are a regular reader, you know that based on what's happening in the world, the feed tends to be a bit skewed towards digital media. Sometimes an article of interest manages to slip in with no compression hook at all, just a lot of almost-compression type buzz.
This piece is a good example. A post to the ever-popular Gizmodo Web site raises the flag on the current haggling over the next-generation HD-DVD format. If you have a nice HD-capable TV, you would no doubt love to have beautiful High Definition DVDs to play on your box, but the current combination of MPEG-2 and 7 GB DVDs make it impossible.
So something has to change, and there are lot of different options: blue lasers, H.264, Windows Media 9, etc. This article on Gizmodo will point you towards a few places where these issues are being discussed.
Coursey on DRM
May 20, 2003
As long as we're printing stories only marginally related to compression, here's a piece of ZDNet pundit David Coursey on Digital Rights Management. Coursey puckers up a bit to Steve Jobs by presenting Apples iTunes DRM as a contractual miracle, with absolutely no downside for consumers. He even goes so far as to imply that Jobs really got the major labels to make huge concessions by agreeing to the Apple system.
Sorry, David, but let's not forget one important thing. Apple makes you pay a price that is on par with what you pay for a full-quality CD. In exchange you get seriously degraded music that you can lose forever with nothing more than a hard drive crash. Is that really a great deal? And let's not forget that there are quite a few music services out there that are already letting you purchase songs and burn them to CD, so this is not exactly a breakthrough. Not to mention that an outfit like PressPlay lets you download all the music you want to your PC for a flat monthly fee.
Well, anyway, enough of my grousing. You read the article and see what you think.
TiVo for Radio
May 19, 2003
It's not like TiVo has taken the world by storm. Sure, it has a fanatic base, but that base is not particularly large. (Although I have to admit the Nelson family is a major consumer of hard-drive video storage.) Nonetheless, the TiVo buzz is strong enough to have apparently spilled over into a slightly different market: radio.
Turns out that Wired Magazine managed to turn up not just one, but multiple companies that are creating MP3 recorders that are going to let you capture your favorite radio program.
I'm not so sure this is going to go anywhere. There are two big problems here. First, the vast majority of radio listeners tune into to a station, not a program. They expect to hear a certain mix of tunes, but there's not much difference between the mix at 1:00 PM and the mix at 3:00 PM.
The second problem is with commercial-zapping. It's easy to zap commercials on TiVo, because you can see exactly when the commercial ends. You aren't going to be able to do this with recorded audio, which I have to believe degrades the attraction quite a bit.
But see what Wired thinks about it, and maybe you'll decide I'm out of touch.
A Few Comments on the T2 WM9 Release
May 19, 2003
Earlier this month I told you about the fact that Terminator 2 was going to ship with a High Definition copy of the movie in Windows Media 9 format. I still don't think I was being overly paranoid when I characterized this as part of Microsoft's attempt to take over the world. That's clearly their aim, and they are entitled to give it a shot.
The long running Good Morning Silicon Valley column from the San Jose Mercury News ran an item about this promotion, and several readers apparently wrote in with various gripes. They're found in the last paragraph of the page linked to below.
SiliconValley.com article - page down to graph headed with T2
ImageMagick 5.5.7 Ships
May 19, 2003
ImageMagick 5.5.6 shipped just a a couple of months ago, but this free library is so big and packed full of features that I feel obligated to post a notice for every upgrade, no matter how minor.
Not that this is a minor upgrade. It's just kind of hard to tell. The release notes detail a huge number of changes, but whether any of them are significant or not really depends on whether or not you are using that particular feature.
In any case, it's a great library, and whether you need this release or not, next time you need to handle a bunch of different compressed graphics formats, download ImageMagick and give it a spin.
A Win for RealNetworks
May 18, 2003
Regular visitors to this space know that RealNetworks has been downplaying their server business as of late, in favor of new deals providing content. An announcement from a company named Speedera shows that Real is not completely dead when it comes to providing media server software.
Speedera has customers that provide streaming media to their listeners. Examples include National Public Radio and the National Hockey League. (NPR does a great job of providing their content online in realtime, and of making archived shows available, and all for free!)
In any case, Speedera finds that Real's Helix servers provide some economic benefits when compared to the competition, and they seem to be particularly happy with the fact that the Helix software runs on eleven different operating systems. Now who do we know that might have trouble in that area?
Nvideo-tech MPEG decoder SDK
May 18, 2003
Nvideo-tech is an organization that I know nothing about, but they have released something called MpegDecSdk 1.0.0. This decoder can be used to build a simple MPEG player, and there is a nice article about just that on the CodeProject Web site.
nvideo-tech home page
CodeProject
article on nvideo-tech's sdk
dcm4che - DICOM in Java
May 18, 2003
Medical imaging apps have different requirements then your average desktop application. Standards such as DICOM have addressed this by defining the formats used by various applications in the industry. Odds are good that your average piece of imaging gear used in a hospital setting creates images that conform to this standard.
The dcm4che library is a set of Java code that helps programmers address some of the things covered by DICOM. It appears to this unpracticed eye that much of this relates to the extraction and storage of data in various types of repositories, but surely there's some image compression in here somewhere!
Could Apple Digest TiVo?
May 17, 2003
Just yesterday we were reporting on TiVo's new marketing strategy for their basic technology. Today, along comes a suggestion in Business 2.0 that perhaps TiVo is so undervalued that they are ready for a takeover. And who better to take over the consumer product manufacturer than Apple?
The theory here is that 1) Apple is still sitting on a ton of cash and 2) Apple's iTunes debut shows that they're incredibly adept at dealing with content providers.
Good theories, both, but really, does Apple want to become a content company? First we had the rumors about the company going into the music business by purchasing Universal Music Group, then they started selling music downloads at the retail level, and now Business 2.0 has got them selling TiVo units at Best Buy. The company could end up spending all its time and energy working on new releases of Top 40 tunes and forget about the evolution of OS/X!
Personally I think it's a crazy idea. TiVo's stock is depressed for a good reason, and just because it gives the company a pitiful market cap doesn't make it a good buy. Steve Jobs needs to keep working on developing insanely great computer systems and software, and not get sidetracked on low-margin consumer goods. That's a no-win situation for a company like Apple.
CodeProject Article: Writing Images in PNG Format
May 17, 2003
This CodeProject article provides a pretty brief tutorial on how to write images in PNG format. The .NET framework is chosen to do the job, which I think means this code can probably be easily ported to C#, C++, VB, and various script languages.
Reverse-Engineering iTunes
May 16, 2003
Apple's iTunes 4.0 player is able to play music across a local network or the Internet, using a proprietary protocol. This protocol not only tells machines how to stream audio content, it provides access to playlists and track information as well.
The folks at the Digital Audio Access Protocol project are determined to reverse engineer that protocol so they can use the iTunes protocol on other operating systems or platforms. Sounds like a great idea!
What Chris Boot has released so far is a C++ library that is a work in progress. He doesn't have a full-fledged client yet, but that appears to be in the works.
The best news of all on this project is that Apple has told Chris they intend to document the protocol, meaning an end to reverse-engineering and more time to spend on the important part of the project, which is writing portable code. Good Apple.
Note too that the DAAP library is released under the commercial-friendly LGPL, so if you have a notion to write a commercial iTunes client, you are welcome to include the DAAP library without infecting your proprietary code. (No opinion implied or expressed as to whether this is a good thing or a bad thing.)
Digital Audio Access Protocol project page
