Report: Improving Software Tools that Improve Software

Vendors compared their program analysis techniques for the benefit of the entire market.

Published: Monday, July 20, 2009 - 11:33

(NIST: Gaithersburg, MD) -- If the shiny new software on your computer or mobile phone runs without crashing, you may have another computer program to thank—a static analyzer. Static analyzers try to find weaknesses in other programs that could be triggered accidentally or exploited by hackers. A new report by the National Institute of Standards and Technology (NIST) documents the Static Analysis Tool Exposition (SATE), an exercise by NIST and static analyzer vendors to improve the performance of these tools.

The report is the culmination of a lengthy effort to host and then digest the results of SATE, begun in February 2008 to help toolmakers assess their products’ ability to find security defects in other software. Eight tool developers, along with a ninth team of professional reviewers, participated in SATE, which provided a noncompetitive environment for the vendors to compare their program analysis techniques for the benefit of the entire group.

Software assurance tools may be obscure outside the world of professional software development, however, their importance has increased as programs grow longer, more sophisticated, and increasingly are required to interact with other programs over computer networks. The number and subtlety of attacks from hackers has also increased. Because it is impossible to anticipate every combination of inputs a given piece of software might receive, static analyzers attempt to use mathematical and logical tools to rigorously predict the behavior of the program and examine it for weaknesses based on its code or set of instructions.

NIST software assurance expert, Vadim Okun, says SATE was a long-overdue idea. “Most modern software is too lengthy and complex to analyze by hand,” explains Okun. “Additionally, programs that would have been considered secure ten years ago may now be vulnerable to hackers. We’re trying to focus on identifying what in a program’s code might be exploitable.”

The participating vendors brought a range of tools that possessed different features and analyzed programs written in two different languages. According to Okun, the depth of the field made SATE as much a learning experience for the NIST team as it was for the participants.

“We intend to hold more expositions in the future and will use this experience to help shape their focus,” Okun says.

According to the organizers and several participants, a good deal of research remains to be done. The effort was not only highly demanding, but it also showed some goals may be out of reach. While users want static analyzers to find all the problems in a piece of software, but also raise no false alarms, “That’s not achievable,” Okun says. “We want to show people that this isn’t a trivial process, but the tools are improving and it makes good sense to use them.”

The SATE report is available online at http://samate.nist.gov/docs/NIST_Special_Publication_500-279.pdf.

About The Author

NIST

Founded in 1901, the National Institute of Standards and Technology (NIST) is a nonregulatory federal agency within the U.S. Department of Commerce. Headquartered in Gaithersburg, Maryland, NIST’s mission is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.