Automating Cyber Defenses: A Round Table with Bishop, Cheswick, Porras and Jackson

by rthieme on March 15, 2002

March 2002

Roundtable

Automating Cyber Defenses

Four of infosec’s cutting-edge thinkers examine how intelligent systems might be made to analyze, understand and automatically respond to attacks.

MATT BISHOP (bishop@cs.ucdavis.edu ) is an associate professor in the Department of Computer Science at the University of California, Davis. He serves on the executive committee of the National Colloquium for Information System Security Education and on the editorial board of the Journal of Computer Security.

WILLIAM R. CHESWICK (ches@lumeta.com ) is chief scientist and cofounder of Lumeta Corp., a spin-off of Lucent/Bell Labs that offers intranet topological and perimeter verification services. He’s the coauthor of Firewalls and Internet Security: Repelling the Wily Hacker (Addison-Wesley, 1994).

GARY M. JACKSON (gjackson@air.org ) is director of the Center for the Advancement of Intelligent Systems at the American Institutes for Research in Washington, D.C., and CEO and president of PsynapseTechnologies LLC.

PHILLIP PORRAS (porras@sdl.sri.com ) is program director at the System Design Laboratory of SRI International in Menlo Park, Calif. He is the principal investigator on the EMERALD (Event Monitoring Enabling Responses to Anomalous Live Disturbances) Project, researching and developing systems and components for anomaly and misuse
detection in computer systems and networks.

Moderator RICHARD THIEME (rthieme@thiemeworks.com ) is a contributing editor for Information Security. He writes, speaks and consults on the human dimensions of technology and the workplace.

INFORMATION SECURITY MAGAZINE (ISM): The purpose of this roundtable is to discuss new technologies and methodologies for automated cyberattack prediction and response, as well as the shift in compsec thinking from a reactive/defensive framework to a predictive/proactive one. Let’s begin with the general ideological landscape. Where did we come from? How are we changing?

BISHOP: The state of the art in automated response–in terms of predicting new attacks–is abysmal. If we can describe attacks, we can detect them when they occur or after they occur. But identifying new attacks is very difficult to do in any kind of automated fashion. In some sense, that’s the ideal for a good intrusion detection or intrusion response system. Once we know an attack is under way, assuming we can detect it in real time, various kinds of responses can take place. But the nature of the response should vary according to the specific policy of the site.

It’s not clear to me that people understand the tight binding between policy and response. People often assume a specific security policy when they describe a response and aren’t explicit about the particular policy. As a result, their actions may be appropriate in one circumstance, but not in another.

ISM: They are not contextual.

BISHOP: Correct. They’re not related to the specific context of a policy. In the research community, that’s understood. However, the impression I get from many vendors and training programs is that this isn’t understood or openly discussed. How does policy affect response? How does response affect the continuing functioning of the organization trying to respond?

As for automatic attack prediction, one problem is trying to predict how you’re going to be attacked. Without knowledge of the attackers, all you can do is analyze your system and try to find weak points. There are methodologies for doing that, but it boils down to having a good knowledge of the system, and there are no guaranteed automated ways to do this. It requires human intelligence to try certain things. There are a lot of automated tools that will help, but you need humans in the loop.

PORRAS: Until 1996, I tended to see intrusion detection as a self-contained system. People were trying to build full systems that would do detection, fire primitive scripts to respond to attacks, provide visualization, maybe do data management, all within a single component. From ’96 to ’99, the community shifted toward decomposing the intrusion detection process into alert collection, sensing and all the post-analysis capabilities that would follow sensing, separating them logically and developing standards for interoperating between them.

In the area of intrusion report correlation, I’m seeing momentum in groups emphasizing advanced visualization, trying to get more insight into what sensors are telling them. The more we can make sensors commodities, the more we can embed them into applications, operating systems and network infrastructure, where they’ll generate alerts. Then we can provide better automated processing of those systems. We believe that’s better than developing a single approach to detecting attacks through monitoring network traffic, for example.

There are people developing methodologies for automating the process of managing alerts. When you get into the automated world of processing alerts, we now deploy sensors that generate thousands or tens of thousands of alerts on a daily or weekly basis. There’s no expectation that we’ll develop automated tools that will fire thousands of responses that will make thousands of changes to your system on a daily basis.

JACKSON: My background is different. I’m a Ph.D. psychologist who crossed over into artificial intelligence design. I’ve developed applications for 15 years, primarily in government. The Center for the Advancement of Intelligent Systems is funded by the U.S. government to look at predictive methods that combine behavioral science and computer science.

We’re developing technology to address these issues. If we’re to become proactive in the area of intrusion detection, we need to unite behavioral sciences and computer science. Systems don’t break into systems; people do, so we want to look at what people do.

Our approach, however, is a radical departure from the kind of hacker profiling done in the past. We’re not studying hackers per se, but rather high-level hacking expertise. We want to model not what hackers have done in the past, but hacker intent. We’ve looked at proactive methods and have had some success in the area of asymmetric warfare. We’re applying those methods to intrusion detection, using insights about prediction from weather prediction, physics and other sciences. If we’re going to determine what someone might do next, we have to be able to predict intent, and all we have to look at is the current activity of that person once he’s in the network.

Within the government, I worked at developing predictive methods. When we applied these techniques to computer intrusion, we discovered some principles. One is that we seem to be able to assess the intent of someone entering a system and can deal with it accordingly. If someone appears to be exhibiting harmful intent, we want to track or block that person, depending on the analysis. So, we work at the atomic level of activity and convert that into what that means from a behavioral standpoint. The assessment system operates in real time and makes assessments that are very close to human assessment. We’re currently throwing attacks at the system to determine true-false positives and true-false negatives.

ISM: In [the book] Mind Over Machine, Stuart and Hubert Dreyfus illuminated why expert systems could only go so far in emulating human expertise. Human experts often go far beyond what rule-based systems can determine. Are you saying that you’re close to that level of intuitive functioning, and if so, what enabled you to reach that level?

JACKSON: We have hacker-knowledgeable people on our team. We’ve discovered that when we run certain activities against the system, and then run the same activities against people with hacker expertise and ask them to characterize the activity, the results in both cases–although blind and independent–are extremely close.

“The more we can make sensors commodities, the more we can embed them into applications and network infrastructure, where they’ll generate alerts.”
-PHILLIP PORRAS, SRI International

ISM: Ches, what do you bring to this conversation?

CHESWICK: I bring 10 years of skepticism. I started out running Bell Labs’ firewall around 1988. Around 1990, I read Clifford Stoll’s Cuckoo’s Egg. Stoll learned a lot just by watching, and I decided to do the same. I spent several years watching people bang up against the Bell Labs firewall until I got bored. It was like counting bugs on a windshield. It became clear that the way you do intrusion detection is to record everything, throw away everything you understand and look at the rest. I’ve seen dozens of papers over the past decade that have tried to collect this data and do it automatically. They all use terms like “pattern matching,” “AI” and “neural networks,” and they can do it to some extent, but they’re all characterized by high false positives, which means that as a practical guy, I don’t want to run this system.

Our questions were, “Who was attacking the outside?” and “Who was attacking the inside?” It was assumed that the intranet was a safer place than the Internet. After we threw away everything we didn’t know about, we were usually left with a long list of administrative errors. Of course, there was evil stuff, too, but most was administrative. Some we fixed, and some things weren’t worth it. It wasn’t real time and didn’t shut down the machine if an attack was taking place. My approach–which I still think is the best approach, although it’s hard to do in a world of Microsoft software–is to try to anticipate the kinds of software errors that we know are the source of many intrusion vulnerabilities. Then design systems that are robust, highly resistant and unlikely to be hacked that way. And then use the IDS supplementally to check our assertions.

Bruce Schneier talks about comparing computers to safes. Safes are rated for a certain amount of time. The whole point is to not let the bad guys have too long a period of time to do their work, so you need alarms and IDSes to make sure the cops arrive within 60 minutes, or the safe is broken.

I don’t think we’re getting much closer to automating the hard part, which is the human decision piece, recognizing the intent of what’s going on. Phil’s EMERALD Project attempted this, and Gary says he has made progress, but there are big jobs yet to be done.

“People buy the ‘hottest’ IDS tool that will be very good about telling them about DoS in the network, but is useless detecting problems inside the host.”
-MATT BISHOP, University of California, Davis

JACKSON: I agree.

CHESWICK: The intent of the attackers is an interesting problem if you’re looking at a lot of attacks. If you’re in a secure area–say you’re in the firewall around the payroll section, where you have the crown jewels–a single packet may be sufficient to shut down the machine and call the cops. It depends on where you put these things. The Honeynet Project suggests putting honeypots in places where people will be attracted to them. I have been advocating honeypots for 10 years. They are sometimes the last level that alerts people that they’ve been “had.” It’s another layer, and we want a lot of layers of defense.

The question of intent is interesting. The point about people breaking into machines rather than machines breaking into machines isn’t entirely true. Look at Code Red, for example. Sure, there was a human at the end of it, but it was essentially an automated network battlebot that wandered in and out of networks all over the world. There’s a human who comes behind, and maybe there you can follow the intent. Last year, it was a new thing for compromised machines to persistently announce their presence to addresses around the world. You didn’t have to go hunting for weak computers, because they would send packets to you.

JACKSON: Let me qualify my statement. People build scripts, and, of course, scripts can break in, but this isn’t autonomous, spontaneous behavior. The behavior within a script is geared toward a specific purpose, and as a result there are specific kinds of signatures that do occur.

If we can observe activities and are accurate enough at identifying them as meaning something from a behavioral standpoint in terms of intent, then combinations of them might mean something, too. One of our concerns isn’t so much the methodology as the speed: How much can you see at one time, especially on an active network? Our initial results are good.

ISM: Is the whole human-machine context a moving target, then? This reminds me of conversations about ‘Net war focused on the intent of the actor as the only way to distinguish a hostile intruder from someone who belongs. We do better protecting against attacks that have happened in the past, but isn’t the terroristic intent to be innovative and design novel attacks?

CHESWICK: I’m not sure that we’ve seen a really new attack in years–or at least one that we were not expecting. Most attacks relate to underlying vulnerabilities or programming problems we’ve known about for decades.

BISHOP: If anything, Bill, you’re too conservative. The problems we’re seeing now are the same problems we saw in the ’80s and ’70s and, according to my colleagues, the ’60s, just in different arenas.

PORRAS: I agree with that wholeheartedly. In the space of doing sensing, the observation space isn’t really that large. We’re seeing the same sorts of symptoms produced by the hosts being attacked or the same sorts of activities coming across hosts that are generating attacks. We’re having some success in being able to detect variations and even new attacks that are effectively generating the same sorts of symptoms. For example, in our own environments, we’ve been able to detect Code Red and the variations of Code Red that came afterward without having to write a whole new series of heuristics to detect them, not because we’ve been able to detect exactly what systems it’s going after, but rather the symptom of attacks. There’s a lot of similarity in the kinds of damage these automated attacks try to do, and we’re trying to leverage as much as we can from our observations of symptoms.

BISHOP: I agree fully that it’s necessary to look at the people launching the attacks and study them to predict where attacks will come from. I teach computer security at the graduate and undergraduate levels, and one year my texts consisted of Sun Tzu’s The Art of War, Machiavelli’s The Prince and Saul Alinsky’s Rules for Radicals. On the surface, they have nothing to do with computer security….

“If the future of intrusion detection is to improve, it must become more interdisciplinary and include the human side of hacking.”
-GARY M. JACKSON, Center for the Advancement of Intelligent Systems

ISM: In fact, they have everything to do with computer security.

BISHOP: Yes. Like it or not, my experience is that humans haven’t changed how they interact with one another. Sun Tzu’s work is a blueprint for how to attack systems. To defend them properly, you have to understand the attacks and the attackers.

ISM: You all agree that the computer-human interface is critical. But how much research is done in the context of how people actually behave as opposed to how they ought to behave? How much weight is given to the training of the “human firewall?”

JACKSON: We take off where many intrusion detection systems stop, at the point of detection. We look at the whole human network in terms of possibilities and probabilities and ask, what are the possibilities after detection has occurred? We take it as a given that good programmers and good science can build good detectors, so detection isn’t the issue for us so much as what you do once you’ve detected an intrusion. We’re trying to proceed from the detection of multiple activities at different times. It’s how you assess and deal with it after the fact that determines, we think, the amount of risk.

It’s amazing what you can determine about a person from a limited amount of information. That’s the whole basis for psychological assessment. We look at the human framework post-detection and ask what it means when we see this kind of activity from the point of view of our expert hackers. We wanted to depart from studying ankle-biters for obvious reasons. We want to think about the best–or worst–possible ways to generate harm and model that, so if we see some indication that that might be occurring, we can sound an alert. The big question is, are we achieving our goal of increasing true positives and true negatives and decreasing false positives and false negatives?

PORRAS: The work I’ve seen in the general area of discovering attacker intent and profiling what the adversary is after is manifest in ongoing research in, for example correlated attack modeling. Groups pursue red team tree building and try to recognize typical activities that might characterize amateurs as opposed to nation states trying to compromise systems. Sensors will generate, for example, alerts that might indicate recon, say, followed by an aggressive attempt to take remote authority of a network. You may not get all that information from sensors, but people are trying to develop models for how to bring together alerts produced by today’s sensors and develop complex attack modeling languages.

“The Internet will continue to get more and more dangerous. We need to build systems and programs that aren’t obviously broken.”
-WILLIAM R. CHESWICK, Lumeta Corp.

ISM: A cyberattack might be one aspect of a larger “swarming” attack. Is work being done to correlate network intrusions with symptoms of an attack, say, on the power grid?

PORRAS: That kind of research area is in proposal format right now. I’ve seen it at the local domain or, at best, the computer enterprise level. The idea of developing regional correlation systems or systems that can compare activities at the Bank of America with activities at Charles Schwab and give some kind of guesstimate as to what’s happening in the financial community, or ask if the chip industry is under attack because Motorola, Intel and Texas Instruments are passing alerts to an escrow service doing regional or financial sector correlation–that’s a very hot proposal topic right now. But we have to walk before we can run.

BISHOP: My experience with companies trying to do correlation across organizational boundaries is that competition is the problem. Texas Instruments and Intel, for example, are competitors, and they are going to be very careful about what they share. Sometimes the information they consider sensitive is exactly what you need to do the correlation. I’ve not yet heard of anyone affirming the level of sharing needed for this to work.

CHESWICK: The kind of inference correlation we’re discussing is something at which the human brain excels. Automating the process is a high target. It’s very hard to match the capacity of the human brain.

A week after Sept. 11, a number of us were contacted by someone from the National Security Council who wanted to know how to answer the question, “Is the Internet under attack right now?” With our Internet mapping and a number of other tests, we came up with a process that would give an idea if that was true. One answer might be, “20 percent of the usual routers aren’t reachable right now.” It’s not clear how you take that kind of information and correlate it with information like, “a dozen oil refineries blew up in Chicago at the same time.” It takes a human to do that sort of thing.

JACKSON: I disagree. I worked specifically in the area of human decision making versus automated systems. If the problem is very constrained and we know the parameters, it’s difficult to beat an automated assessment system.

The problem isn’t the human brain so much as the information provided to us.

Information in a complex situation, like asymmetric warfare, is a serious problem. There are many sensors, but it takes tremendous cooperation among all of those sensors coming to a central point to reach an accurate decision. It’s extremely difficult. Second, there is very little ground to compare new information with, so we can make comparisons in terms of class recognition issues. My experience with some of the systems we’ve constructed is that if it’s very constrained, and you need speed and accuracy, the human tends to fall off at that point.

ISM: The complexity of these issues indicates how hard it might be for some in the private sector to get a handle on what’s doable. How can someone without your expertise make a decision about what’s possible and what their business needs are?

CHESWICK: They have to (a) ask their expert and (b) choose lots of solutions. You don’t want a monoculture in your computer systems or IDS or anything else. You want to have belts and suspenders on these things in case something does go wrong.

BISHOP: People who try to purchase IDSes don’t ask themselves, “What is the ‘must-not’ activity that I must get out of bed for at four in the morning?” People tend toask, “Who is the market leader in intrusion detection software?” Then they buy that and say, “Now that I’ve installed that network-based IDS, we’ll no longer have someone illegally transition to root. If they do, my network-based IDS tool will tell me about it.”

Those who develop and market IDSes often don’t explain the “sweet spot” of what IDS software is capable of doing. Most IDSes can do pretty well in recognizing privilege violations, privilege subversions–things that happen inside the host and inside the process. But people often buy the hottest, most recently reviewed IDS tool that will be very good about telling them about port scans or DoS in the network, but are virtually useless at detecting problems happening inside the host. They need to ask, “What must I know and what’s the ideal software to tell me?”

ISM: Where should they get that advice? Where can they find informed, impartial answers to those questions?

BISHOP: Vendors will tell them that they detect everything. I really don’t know. It’s difficult. I’d like to see operating systems come with their own misuse detectors so that the OS can tell you when someone is corrupting that system or doing something they shouldn’t be doing. I want to see applications become smarter at this, but I don’t expect that to happen.

ISM: I get notices of computer security workshops, but I don’t see workshops teaching people to ask the questions you say are most important.

BISHOP: It’s a failure on the part of the people developing this technology to express what the technology is good at and when you should use a different component. People want to sell you an entire solution. The motivations of commercial vendors differ completely from the motivations of security researchers.

ISM: So managers making these decisions know that if they buy the leading product, they will not be fired. They aren’t rewarded for thinking the right way or making the right choices for the right reasons.

JACKSON: One problem is that we’ve been solution-oriented instead of integration-oriented. There are many people out there who are confused, and their job seems to be to buy something that lets them check the box that shows they have done their job by deploying an IDS.

BISHOP: All of this ties into knowing your policy and knowing how your vendor’s solution fits in with your policy. People need to make that connection.

ISM: So for now, the human in the loop is critical, and managed security services are the human form of automating the response we need. Where should we go in the future?

JACKSON: If the future of intrusion detection is to improve, it must become more interdisciplinary and include the human side of hacking. The computer science is there, but the field is lacking in terms of bringing in the knowledge and expertise of the behavioral sciences so we can get better anticipation of responses of those intending to do harm.

CHESWICK: The Internet will continue to get more and more dangerous. We need more tools in the area of software design. We need to build systems and programs that aren’t obviously broken. Unless we have those tools, a lot of this is bailing a ship with a hole in the bottom.

PORRAS: Security is ultimately a human endeavor. Technology can help, but you’re dealing with people. We need to improve technologies, but we cannot neglect the human element.

BISHOP: When all is said and done, a skilled administrator is still the best intrusion detector.

March 2002 Table of Contents

{ 0 comments… add one now }

Leave a Comment

Previous post:

Next post: