[ISN] Recent Gartner Report on IDS/IPS

From: InfoSec News (isnat_private)
Date: Mon Jun 16 2003 - 02:13:01 PDT

  • Next message: InfoSec News: "[ISN] Hacker tips CERT's hand on Linux/PDF flaw"

    Forwarded from: Gary Golomb <gee_twoat_private>
    To: focus-idsat_private, isnat_private
    
    Ok, this is going to be long. Also, this email is being written
    entirely on my own impetus and **definitely does not** reflect the
    views of my employer. (In fact, I'll be surprised if I make it through
    this one without any bruises.)
    
    Gartner, Inc. has recently released a document authored by Richard
    Stiennon entitled, "Intrusion Detection Is Dead - Long Live Intrusion
    Prevention." (So I'm guessing we don't need to cover what that
    document is about.) Gartner is self-described as, "For 20 years,
    Gartner's Research & Advisory services have been recognized as the
    definitive source for objective technology thought leadership." Ok,
    fair enough. I'm a fair person and everyone makes mistakes.
    
    Unfortunately, this is not Gartner's first mistake along these lines.
    Here's a quote from paper now a year and a half old (also from
    Gartner):
    
    "Intrusion Prevention Will Replace Intrusion Detection. Enterprises
    should delay new large investments in intrusion detection systems --
    which have failed to provide additional security -- until intrusion
    prevention systems emerge that provide a stronger defense against
    'cyberattacks.'"
    
    No, this is not the first time Gartner has displayed such a grotesque
    misunderstanding behind detecting and defending against *real*
    threats, but this is definitely the most horrible.
    
    So, for all those who take statements like the above seriously, let's
    define WHY people use Intrusion Detection technologies in the first
    place.
    
    Intrusion Detections systems are used for one reason. It’s your last
    chance to be notified about a potential break-in; a virtual safety
    net. Once an organization has invested massive amounts of time, money,
    and resources into setting up "PROTECTIVE" technologies such as (but
    not limited to) firewalls, encryption, authentication, proxies,
    gateways, PKI, VPN, access control, virus detection/removal, etc...
    The IDS serves the single purpose of sitting back and watching over
    everything to see if people are still getting though. And here's a
    curveball for you: After all the protective technologies just
    described, attackers (both automatic like worms/viruses and live
    people) were/are STILL getting through! Whether it's because of
    vulnerabilities in network designs, application vulnerabilities, or
    unknowingly misconfigured devices, they do get through. And this is
    why IDS's were invented...
    
    The main difference between an IDS and other security devices is the
    fact that it's out-of-band, or passive in nature. It passively watches
    all traffic looking for SIGNS of attacks, compromise, or other misuse.
    The key benefit to being out-of-band is that you have the ability to
    flag traffic that looks even the slightest bit "suspicious." If you
    have an IDS that is telling you that too much is "suspicious," then
    tune it! What's suspicious in one environment might not be in another.
    Vendors try to compensate as best as possible, but only YOU know YOUR
    environment the best! Once it is flagged, it is usually logged and
    followed up by automated processing, or people-based responses.
    
    So, now that we're on relatively the same page when it comes to ID,
    let's look at Gartner's reasons for stating that we don't need this
    technology anymore.
    
    --- 
    Statement #1 
    "Contrary to the philosophy that it is impossible to protect a network
    from all of the attacks leveled against it..."
    ---
    
    Ok, this one is more comical than anything else. It's the first
    sentence in the document. By starting off by telling us that it *IS*
    indeed possible to protect a network from ALL attacks leveled against
    it, I had to chuckle. It also set the stage for the rest of the
    document.
    
    --- 
    Statement #2 
    "The 'demilitarized zone' (DMZ) architecture has been punctured by
    many exceptions to security policies. It poses a threat to
    mission-critical services."
    ---
    
    Since DMZ's [apparently] pose a threat to critical services, Richard
    proposes (what he dubs as) a new nomenclature and architecture for
    replacing the DMZ. The new name is: The Transition Zone. (TTZ?) The
    way TTZ works is by taking your public resources (like a firewall,
    mail serer, or whatnot) and placing it on a network that is logically
    between the Internet and your internal network. This middle ground is
    separated from the Internet via a firewall or gateway that allows
    limited access to the public resources. There is a second firewall
    that separates the TTZ from the internal network which I presume is
    more restrictive.
    
    Interestingly enough, that's what the rest of the world calls a "DMZ."
    I saw no difference between the proposed TTZ and how most
    organizations that I have seen implement their DMZs.
    
    --- 
    Statement #3 
    Regarding another problem with hosts in the DMZ: "Because of the
    constant exposure of these assets to the outside world, they must be
    protected by a greater investment in security devices, rather than
    treated as untrusted, even sacrificial hosts."
    ---
    
    I just called a couple Fortune 50 and some smaller customers of ours
    to ask if their assets in their DMZs are sacrificial hosts. They said
    no.
    
    ---
    Statement #4
    "By 2005, 90 percent of Global 200 gateway firewalls will do 100
    percent deep packet inspection, enabling them to block application
    attacks."
    ---
    
    Now this statement is onto something! We'll get back
    to this in just a minute.
    
    ---
    Statement #5
    "IDSs were proposed as the suspenders in the 'belt-and-suspenders'
    approach to perimeter defense."
    ---
    
    In this short sentence there are two significant errors.
     
    One- 
    IDS is NOT the "belt-and-suspenders" to perimeter defense, although
    the mental image is quite entertaining. Quite the contrast, they are
    the "checks-and-balances" to defense technologies. They do NOT
    "support" protection, they "detect" when detection mechanisms are
    failing. They also help to create audit data useful for the final part
    of the security cycle - "reaction."
    
    Two - 
    IDSs are not designed just for the perimeter. Many organizations place
    them throughout the network, at server farms, groups with large IP
    caches or other data, and even in partner locations. I made a comment
    at the top of this which stated, "If you have an IDS that is telling
    you that too much is 'suspicious,' then tune it!" This is the reason
    why. Not only is each environment different, but different traffic is
    seen in different locations within the same environments. We do our
    best to compensate, but only YOU know YOUR environment best.
    
    ---
    Statement #6
    "State awareness will enable network agents to scale to the
    multigigabit speeds needed."
    ---
    
    This statement shows and obvious and gross misunderstanding of
    implementation and design issues in IDS development. A robust
    state-tracking implementation can take just as much overhead as a good
    pattern matching, protocol decoding, or anomaly detection
    implementation. Look at firewalls for proof of this.
    
    IDS vendors need to find a balance in implementing these
    methodologies, without crushing sensor performance. No single solution
    (such as state tracking) is good enough to be used as a single
    detection methodology, or to state it enable multigigabit speeds.
    
    ---
    Statement #7
    IPS needs to do this: "It requires efficient detection of malicious
    attacks. Well-designed network agents should use a combination of
    signature, protocol anomaly detection and traffic analysis to minimize
    false positives. State awareness will enable network agents to scale
    to the multigigabit speeds needed. They should be in line to allow
    them to drop sessions."
    ---
    
    This statement appears to show that now even Gartner has succumbed to
    marketing hype. You would think they would have based a paper like
    this on analysis of some new vulnerabilities, or trending exploit
    development over time, or looking deep into the geopolitical
    developments and sociological impacts on attackers and hacking, right?
    Based on the above "design requirements," it sounds as if this
    document was written to be a marketing glossy for an IPS vendor.
    
    Marketing is my favorite topic!
    
    Here is a statement taken from a leading IDS/IPS vendor's website. Not
    a kind-of leading vendor, VERY leading. ;)
    
    "...provides broad-based detection, prevention and response for
    attacks and misuse that originate from across a network. ...using a
    combination of sophisticated protocol analysis and pattern matching to
    interpret network activity it detects known attacks, previously
    unknown attacks, and is immune to tools that attempt to evade pure
    pattern matching systems."
    
    So we'd expect this true statement, right? Here are the results of a
    test performed several months ago. (Maybe 60 to 9 months ago now, and
    I'm sure the IDS has been corrected by now.) First, the two most
    critical vulnerabilities where picked from a 4-6 month time period.
    This was done to cover the past two years. Two were picked so one
    Windows and one Unix vulnerability could be tested for each 4-6 month
    time period.
    
    Then, exploits were harvested for those vulns. Only exploits from the
    most public websites like www.packetstormsecurity.com and
    www.securityfocus.com were taken. The final exploit chosen was the one
    in each category that was the most easy to use and most destructively
    robust. This ensured we were testing the exploits that the kids were
    most likely to be using.
    
    The IDS was also fully configured and updated. The point was *not* to
    evade it or make them look silly, but to see HOW it viewed certain
    events compared to other IDSs. There was a problem though. The
    following exploits were missed COMPLETELY:
    
    Moderators: I am sending this from a different account
    than what I'm subscribed under. If this is a problem,
    I will make other accommodations. 
    
     - IIS 5.0 .asp overflow wrapped inside of chunked-encoding exploit
    with port binding shellcode
    http://packetstormsecurity.org/0206-exploits/DDK-IIS.c
    
    For any "Protocol Decoding IDS" this attack should have triggered all
    kinds of HTTP alarms, which it did not.
    
     - UPnP remote shell-binding exploit 
    http://packetstormsecurity.org/0112-exploits/XPloit.c This exploit was
    chosen for two reasons. One is that it affects all unpatched Windows
    ME and Windows XP systems. The second is that it uses shellcode which
    is also used in many other windows-based exploits so it should be
    easily identifiable.
    
     - FrontPage 2000 Server Extensions .asp source disclosure 
    vulnerability
    http://packetstormsecurity.org/0008-exploits/srcgrab.pl.txt
    
    This was chosen mainly because of it's prevalence in security
    scanners. This is a vulnerability that many scanners check for since
    there is a wealth of information in many .asp scripts.
    
     - Apache Chunked Encoding Vulnerability
    http://packetstormsecurity.org/worms/apache-worm.c 
    Not only is Apache the most widely deployed web server on planet
    Earth, but this vulnerability was the basis for MANY different
    exploits and Internet worms.
    
     - Compromise: Command prompt and shell on high port This test was
    done because realistically you cannot expect an IDS to detect EVERY
    attack out there. However, you should expect the IDS to detect the
    most basic and generic signs of a successful compromise.
    
    Now if you have missed the significance of these test results, let me
    paste the statement that same vendor made about their technology on a
    main webpage again:
    
    "...provides broad-based detection, prevention and response for
    attacks and misuse that originate from across a network. ...using a
    combination of sophisticated protocol analysis and pattern matching to
    interpret network activity it detects known attacks, previously
    unknown attacks, and is immune to tools that attempt to evade pure
    pattern matching systems."
    
    All of those attacks were 3-24 months old at the time.
    All of those listed above were missed entirely - not even an unrelated
    false positive was triggered from the attacks. Many others not listed
    here were only detected as something else, and not the actual attack.
    
    We'll elaborate a little more on the subject of IDS vs. IPS in a
    moment, but I just wanted to make a note about vendors who claim to
    have silver-bullet solutions.
    
    Also, if IPS was the end-all, why do you think that every
    market-leading IDS vendor hasn't adopted it yet?
    
    
    ---
    Summary page of document:
    Is a diagram showing something like a firewall that
    can do application content inspection and filtering.
    ---
    
    Now we have two points pending from above that tie into the summary of
    this slide. One is the pros/cons of IDS. Ie: those things that would
    cause a company of Gartner's stature to release a paper with the title
    of this one.
    
    ALL IDS methodologies have to deal with false positives. It's the way
    the technology works. If you have a device that is going to tell you
    about any potentially suspicious activity, that is exactly what it is
    going to do. Just because communications might be suspicious, does not
    mean it's going to always be an attack, but at least you have
    something there to inform you about it when everything else on the
    network fails to protect you.  And the IDSes that are better with
    handling false positives are WORSE when it comes to the category of
    producing false negatives. If you are going to tune down on the amount
    of things you consider "suspicious" then of course you increase the
    chance of tuning down real attacks also. I could show you a
    one-for-one relationship why. Now which prospect is scarier?
    
    The whole point of an IDS is taking advantage of the luxury passive
    analysis affords you. You can be highly sensitive to anything that
    looks slightly suspicious. You can spread your analysis over a time
    period spanning several fractions of a second, several packets, or
    even several months. You have the ability to be highly sensitive to as
    much (or as little) as you need to be - to find and detect violations
    of policy compromise.
    
    Now don't get me wrong, IPS is an awesome technology. Bill Boyle from
    Intruvert put it the most elegantly on the focus-ids list when there
    was a thread of people (including myself) bashing IPS. He said
    something to the effect of, "[paraphrasing] We're not claiming to stop
    everything, but if we can stop a lot of attacks, then why wouldn't
    you?" (Sorry about the previous thread Bill!)
    
    This was the best point I've ever heard made about nIPS, but does it
    *replace* nIDS as Gartner has stated? Absolutely not! That idea is
    about as ridiculous as stating a DMZ is going to be more secure if you
    change the name of it.
    
    There's another point to made along this vein... How is an IPS going
    to block attacks that aren't attacks? I mean, totally valid traffic
    that is only dangerous because of a policy misconfiguration? If you
    refer to a poll done by zone-h (arguably the most active defacement
    mirror on the net), most defacements (if you can rewrite data on a
    server, I'd count that as a hack) are accomplished because of
    misconfigurations. Think about that. That's somewhere around 300
    attacks a day (average) that are reported to zone-h because of
    misconfigurations. How many more do you think happen *every day* that
    aren't reported?
    
    In summary of point one:
     -Good security design follows the Protection ->
    Detection -> Reaction paradigm. 
    
    IP = Protection
    ID = Detection
    IP not= Reaction. 
    
    Point two:
    Richard made it himself which is why I can't believe he went on with
    the paper. "... gateway firewalls will do 100 percent deep packet
    inspection, enabling them to block application attacks."
    
    An IPS, being in-line, does not have the indulgence of being able to
    be highly sensitive to everything an IDS can. Since it is making the
    decision to pass or not pass traffic, it has no room for misjudgment.
    As such, that places a severe limitations on its ability to find
    things off-line analysis offers. In addition, analysis is limited to
    what can be accomplished in fractions of a second. There is no
    opportunity for *real* analysis and correlation.
    
    To make an IDS into an IPS border-lines a silly idea, but to go so far
    as to say that IPS will replace IDS entirely is absolute ignorance.
    And we haven't addressed the issues of politics, availability,
    management, etc...
     
    An IPS is not an extension of an IDS, it's an extension of a firewall.
    And, that does NOT mean a firewall with an IDS on/next to it. The
    discussion of making a firewall an IPS is kind of an entertaining one.
    Most people think they have firewalls all figured out until you start
    heading down the path of all the problems they have. It's funny how
    the solution to all of a firewall's problems seems to mirror most
    people's conception of what an IPS is...
    
    A paper on that topic might be a good read. Greg Shipley recently
    brought the point up in a Network Computing column
    (http://www.nwc.com/1411/1411colshipley.html), and I'd love to see a
    technical analysis of the points he and others have raised. If Gartner
    decides to take on the task of writing this, I hope it's done in a
    more responsible manner than this was.
    
    That is what upsets me the most about incidents like this. Because of
    the long history Gartner has with industry reporting, their documents
    carry a lot of weight for many organizations. Although, this recent
    track record of negligence is disturbing to say the least.
    
    -gary
    
    
    Gary Golomb
    Senior Research Engineer
    Dragon Intrusion Detection Group
    Enterasys Networks
    
    
    
    -
    ISN is currently hosted by Attrition.org
    
    To unsubscribe email majordomoat_private with 'unsubscribe isn'
    in the BODY of the mail.
    



    This archive was generated by hypermail 2b30 : Mon Jun 16 2003 - 04:18:59 PDT