|
A technique for finding security vulnerabilities
February 09, 2007
URL:http://drdobbs.com/security/fuzzing/197004850
Gadi Evron is a Security Evangelist for Beyond Security.
Hackers make finding security holes look easy. A new product comes out, and not a week goes by before hackers expose embarrassing, sometimes trivial, security weaknesses in the product. Sometimes it's as if hackers can find those vulnerabilities "at will." In one case security researcher H.D. Moore announced a project he named "a month of browser bugs" where every day during July a new security-related bug was exposed in one of the leading browsers from Microsoft, Apple, and Opera.
How can a single researcher, working in his spare time, find what countless professional software testing people, working full time for the largest software companies in the world, can't find? The secret lies within a methodology called "black box testing," a term that is known in technical circles as "fuzzing".
Fuzzing is a concept that, until recently, has mostly been used on the wrong side of the fence. Fuzzing is a testing technique that automates the search for security vulnerabilities in software without having access to the source code of the application. The lack of source code and other design information is why this testing method is called "black box" testing. It's like looking at a black, opaque box and trying to find holes in it, without having access to the blueprint or design documents.
Most security holes that are published are a result of using home-grown and target-specific fuzzers. These tools have been responsible for discovering many vulnerabilities in the past in products ranging from ZIP archiving software to web servers.
The interesting thing about these discoveries, is the fact that they were made by individuals that were not affiliated with the software vendor in any way, and thus had no special access to the product's source code. In most cases, the discoveries were made when the product was already in the market, resulting in large costs to the software vendors in both bad publicity and the costs of developing and distributing patches to the problem.
For years, development and testing teams watched in envy as hackers and security researchers used fuzzers to perform effective black box testing to find tricky bugs that slipped through formal product testing.
Today, fuzzing has matured and is being offered to software companies as an off-the-shelf tool they can add to their arsenal, aiming to eliminate tricky security holes such as buffer overflows and other such security vulnerabilities in products. The goal is to find and mitigate problems before products are released.
During software testing, a fuzzer allows a qualified tester to act much like a hacker. The fuzzer sends a large amount of requests to the program, trying various forms of malformed input iterations, each iteration a distinct testing scenario attempting to trigger a bug in the program; for instance, to bypass memory restrictions or change the program's execution flow. The number of tested scenarios is huge. In typical QA testing we look for a certain amount of likely usage scenarios. Here we are testing a huge amount of scenarios unlikely to appear in regular usage, but scenarios that might be used as part of an attack.
While fuzzing does not require prior knowledge of the software being attacked, how it generates the attacks is critical. For many years, the only form of fuzzing was to send random data to the software. Today, advanced fuzzers have full knowledge of the protocol they are attacking. They simulate valid protocol sessions while stretching the term "valid" to mean legal but very unlikely scenarios. Fuzzing is similar to landing on the dark side of the moon -- we are trying to see what happens when we do something no one has tried before.
The fuzzer's attack on the application depends on what network protocols are being used. As an example, to attack a mail server product, a fuzzer would use knowledge of SMTP (Simple Mail Transfer Protocol) to send valid, however strange, requests to the mail server.
In theory, a fuzzer's attack could go on forever, attempting virtually unlimited scenarios on every possible combination. Because this is not practical for testing, most advanced fuzzers attempt millions or billions of attack combinations, but use special algorithms to try only those combinations that are likely to trigger a vulnerability. Further, they can start with the most likely attack candidates and proceed to the scenarios less likely to expose a security hole. These "less likely" combinations are not to be ignored; a vulnerability may be hiding some place nobody has thought of, and this is what a malicious hacker would use to break in.
Fuzzers can be used in different phases of the development process, but mainly they apply to:
Fuzzing technology provides the necessary assurance that security vulnerabilities that would be found after a product release will be found while it is still in development. This is important both from the costs of potential technical support (to handle recall/patching) to the costs incurred by losing credibility with customers or from receiving a reputation for bad security practices.
Now that fuzzing has reached a stage of maturity and is being used as a reliable tool for organizations to fight against 0-day vulnerabilities (unpublished vulnerabilities used by hackers to attack while evading detection by signature-based defense tools), it is also likely that hackers will utilize this technology to attack products more than ever before. The difference is that now the tools that hackers use are available to developers to integrate into testing suites that are implemented by organizations and applied to their testing methodology.
Although fuzzers are exhaustive in their testing, and are likely to find many security vulnerabilities, they can only prove the existence of bugs -- not the absence. A fuzzer can not guarantee that every vulnerability was discovered and fixed.
In addition, fuzzers find bugs in programs with a certain likelihood of false positives. The fact that a certain request has triggered an internal program error, raised an exception, or caused the application to hang or crash, does not necessarily mean that the problem is a real security vulnerability. Test cases found by automated tools should be investigated by a developer that knows exactly how serious a threat it really is.
Despite those shortcomings, black box testing remains a popular testing mechanism and has demonstrated a successful record of finding security holes. In fact, this is the "secret sauce" used for many years by both hackers and security researchers to uncover many of the thousands of security vulnerabilities published in recent years.
On the practical side, while black box testing can stand on its own merits, it can also complement other testing techniques such as source code auditing of different forms. It is a good idea to develop a testing methodology that includes black box testing, in addition to the rest of your current QA testing process.
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.