Database Performance
Maintaining satisfactory database performance has been a challenge for Datasystem administrators, especially during missions when students log on to see the pictures they have requested. As EarthKAM has grown more popular, and its software more stable, the missions have grown both in the number of participating schools and the number of photos taken. For example, the first few missions averaged 500 images each. But on the fifth mission, the project recorded over 2,700 images in that single Space Shuttle flight.
Mission 5 averaged 200,000 requests per day to the Web site. The system was bogged down under the load. Because of the launch and construction of ISS, Mission 5 was the EarthKAM's last one aboard the space shuttle. Moving the EarthKAM camera to ISS in 2001 meant reconfiguring the software and testing the new environment with several scaled back missions. Dropping back into pilot mode meant that fewer schools could benefit from the fewer pictures taken, which temporarily took the pressure off the Datasystem. However, Lopes and Andres anticipate that in the long run, the move to ISS will greatly increase demands on the system.
The increased demand will stem from the fact that the camera on ISS is available continuously, not just for a few days during space shuttle missions. Although there are no plans to have EarthKAM operate continuously, the level of activity will still increase significantly. "Our goal is to have the camera functional once a month for a period of a few days, and to have thousands of middle schools all over the country and the world control the camera," explains Dr. Karen Flammer, the UCSD researcher who helped Dr. Ride create the EarthKAM program. Flammer now oversees the project. Once-a-month missions would represent a tenfold increase in site traffic and image retrieval. More frequent flights will also result in a much larger archive of images, eventually consisting of hundreds of thousands of photos.
In addition to serving as a resource for participating middle schools, the Datasystem archive is open to schools that don't take pictures, and in fact to anyone in the world. Hits from non-participants will no doubt increase as the system becomes better known. That's just fine by EarthKAM organizers. "We want to tell the world: Look at this incredible library of information and images we have, for students everywhere—not just middle school students," says Flammer.
Given these ambitions, it became clear during Mission 5 that something drastic had to be done about server and database performance. The team took steps in this direction early this year. First, in conjunction with a move from SDSC to the UCSD Science and Engineering Research Facility, team members put both the Datasystem's Web front end and the database back end on the same Sun Ultra 60, thus reducing access times. They also changed the file access method from a network file system (NFS) to a host-attached RAID system. Naturally, moving from networked storage to local storage led to substantial performance improvements.
The original Web front end had been designed to generate pages dynamically based on user input to HTML forms. To decrease the server load, the team instead created a large number of static pages that offered the types of information users most frequently requested.
The dynamic system, based on Perl scripts, already had some performance measures in place, like a three hour cache of each dynamically created page. That made it possible for teachers to "preview" pages that they knew their students would be requesting, to speed the site up. There were few if any cache misses as long as students requested the pages within three hours. After that time, however, performance would drop as the system had to access the database and dynamically generate new pages. With the new static system, caching is limited by available disk space, not by time. And even if there is a cache miss, it doesn't require database access or dynamic page generation—the page is pulled from the static repository.
Students may still access the Datasystem when images are initially downloaded from the space station, before static pages have been created. Once the static pages are there, they don't need database access anymore. Although precise numbers aren't available, the new system has significantly improved performance. The trade-off is increased disk space requirements and less flexibility in the possible kinds of searches.
Future Technologies
Currently, the most human-intensive task associated with the EarthKAM Datasystem is determining the exact center latitude and longitude for each image. This helps target the precise location to which each image corresponds—a task performed by UCSD students.
Latitude and longitude can be determined approximately, based on the time at which the image was captured. However, that time is known only to the nearest second, due to the delay introduced by software on the ThinkPad. Because the characteristic speed of low-Earth orbit is 8 km/second, one second can mean a difference of eight kilometers in the image's positioning. Unfortunately, an error of 8 km would greatly reduce the accuracy of map matching, a feature on the Datasystem Web site.
Map matching is the overlaying of ISS EarthKAM images on jet navigation charts, digital elevation models, and satellite maps. This lets students more easily identify known geographical features in the photos like rivers, lakes, and mountains. Because the maps are coregistered, they can be used to turn the EarthKAM images into VRML models. These models let users interact with the images in their browsers. With the help of a VRML plugin, like the Cosmo player, students can manipulate their viewpoint of an image in its surrounding area.
To aid in the map matching, the group at UCSD may deploy automatic pattern-recognition software. Although the science of pattern recognition is fairly advanced, it's only beginning to be applied to satellite imagery. Dr. Flammer predicts that we're years away from the day when satellite images will be automatically and routinely matched with maps.
As for short term projects, the UCSD group plans to upgrade the camera onboard ISS to a Kodak DCS 760. The camera will use a 400Mbps FireWire interface to transfer files more quickly than the current SCSI interface. However, the notebook on ISS doesn't support FireWire yet, and UCSD is still developing its Flight Software package to control the DCS 760.
With these upgrades, the ISS EarthKAM program will be able to provide students with an even more efficient method of selecting, retrieving, and researching Earth images. The project's parameters and audience have provided creators and maintainers of the Datasystem with a fun and interesting technical challenge. Best of all, they know that their work on the system is having a direct, positive effect by motivating young students to learn about the universe—all with the aid of some awe-inspiring images of the Earth.
Michael is a freelance writer based in East Sound, Washington. Write him at [email protected] or visit his Web site at www.hurwicz.com.
