Ok, I'm going to say this. I agree that the the finger service should not be run if at all possible but I at the time was under the impression from what Solar Designer had written that the fix mentioned in the advisory was indeed weak but could be used as a preventive measure for those who could not prevent the use of finger due to either impracticality or politics in the workplace. These were of course all very general comments. The excerpt you have given us is both informative and explains alot of things to me that I did not understand about the attack the first time around. It also shows me the potential this attack has. I have been able to take out large parts of my own network going by the original security advisory. This included the NIS master host which happened to be the DNS host and not just the attacked machine. You are implying that you were able to take out a large number of machines with this attack and I'm starting to understand why. I hope this clears things up a little bit. ------------------------------------------------------------------------------- Raymond R Medeiros II email: medeirosat_private Junior Systems Administrator www: http://www.eng.usf.edu/~medeiros Engineering Computing University of South Florida On Mon, 13 Jul 1998, Michael H. Warfield wrote: > Hello, > > Raymond Medeiros enscribed thusly: > > I would have to only completely agree with you. This fix which was > > contained in the ISS security announcement was indeed very weak. My > > suggestion was to at the very least deny access to finger requests from > > the outside. This attack really isn't that bad however I have been able > > Huh? Did you read the entire advisory or just Solar Designer's > excerpt. The above recommendations were all in the original advisory. I > agree with Solar Designer's opinion that the shell script idea was rather > weak, but it was the last of several suggestions listed. > > Here are the two relavent paragraphs taken fron the original > advisory: > > ] Disable finger service on any systems connected to any NIS based > ] network. Disable access to internal finger services at all perimeter > ] defenses (firewalls, gateways, filtering routers, etc.). If the finger > ] service is required for some specific purpose, limit it to the minimum > ] number of restricted hosts or to hosts which are NOT participating > ] in NIS. > > The above suggestions were the ones recommended first. > > ] For those who wish to continue to run the finger service on their > ] systems, there are other possible actions that could be taken, like using > ] a public domain fingerd that doesn't do ambiguous lookups. The finger > ] service can also be protected by even something as simple as wrapping > ] finger to not do ambiguous lookups like as follows: > > The questioned script then followed, last. > > More on the recommendations below... > > > to take out a machine on my own subnet using a simple perl script. In > > reality it doesn't appear to be more of a threat than a ping flood. I > > have also looked into using it as part of the beginning to a spoofing > > attack (under controlled conditions of course) and it has no apparent > > value. Never the less it should be brought to everyones attention as it > > is such a simple implementation and just one more reason to be suspicious > > of the use of yp. > > Sigh... I was afraid of this... > > Keep trying! > > My contacts at Sun were concerned that I was revealing too much > information in this advisory (the complete text of which is attached below). > My concern was that I was revealing too little and would fail to convey > the magnitude of the problem or run into people who would be unable to > reproduce the problem and discount it. That appears to be the case here > so I will provide a few more hints and guidance to help reproduce this. > (Gee I feel like that faint voice in the game "Adventure" that asks you > if you would like a hint. :-) ) > > Just as a side note... This was not the result of an academic > exercise. This was the result of the analysis of several independent > instances of serious network disruption on large, independent, networks. > The aftermath left numerous systems disabled and hundreds of finger > processes in evidence on those systems which were left viable enough > to respond to interactive input. > > Obviously, I can not go into any more detail about the networks > originally affected by this problem. > > To reproduce this problem, you are going to have to do some tricky > juggling. > > 1) Make sure you have a thousand or more account names in your > NIS password map (this is very IMPORTANT) Obviously this affects only > large networks in practice. You need to add that many fake names if you > are attempting to reproduce this in a test environment on a small network. > > 2) Start a few process, for each address to attack, to attack each > of a large number of hosts. > > - In each process - > > 3) Open up a finger connection and feed down the request. > > 4) If the request returns data within a certain timeout, sleep > for a period of time before launching the next request. > > 5) If the request does not return data within the timeout, close > the connection and start another connection immediately (this is the first > indication that you are suceeding in congesting the network). > > 6) Cycle through this process, steps 3-5, for the specified number > of names in your attack list or for the attack period of time. > > Now... This is something like "tickling the dragon's tail", an > old, very dangerous, no longer practiced, experiment in nuclear physics > where one would determine the critical mass of a radioactive element by > piling it up and shaping the pile until it went critical (several scientists > died doing this). Everything behaves resonably quiet until you hit that > right combination and then it goes through the roof. In what I refer > to as "tuning for maximum smoke", you play with the number of processes > attacking each host, the number of hosts, the timeout before abandoning > the connection, and (important) the sleep time between successful requests. > When you find that right combination, the results are pretty impressive. > It's also likely to piss off anyone you failed to warn when starting a run. > It may even piss off the ones you warned. I heard a few "WHAT DID YOU DO?" > and "YOU DIDN'T TELL US IT WOULD DO THIS!" even from people I warned. > After a few runs it became standard for people to stop working for a while > ("ducking behind barracades" was the term used more than once) when I > announced that I was testing an nis-nuke run. > > The reason I am not providing these numbers is that it's not trivial > to determine them if you do not have a large enough network to test on. I > would prefer not to release that much information at this time (I suspect > I would be scraping my buddies at Sun off of the ceiling if I did :-) ). > That is also one reason why I am not releasing the exploit script... Sorry... > > A typical test run would take less than one minute against 10 > attacked hosts. The network wide disruptions would last anywhere from a > half an hour to over an hour and some systems had not recovered for > over two hours. Several systems, AIX and IRIX boxes in particular, > never seemed to recover and required hard power cycles to recover. The > Linux boxes seemed to recover fastest while the Solaris boxes seemed to > be able to clean themselves up eventually. Until each and every attacked > system had either recovered or was rebooted, the network disruption, due > to the ongoing NIS traffic, would continue to interfere with the entire > network and bog down the NIS servers. > > I would also like to remark about one thing. Solar Designer > quoted one possible action from the advisory. That one point was a > suggestion made by my Sun contacts. It was NOT our recommendation as > the action to be taken. My PERSONAL recommendation is to disable finger > if at all possible. It provides way too much information about accounts and > logins on your system and only provides information to potential attackers > that you really don't need them to have. If you DON'T NEED IT - DON'T > USE IT! Barring that extreme action, limit finger to the minimum number > of systems on which it is required AND limit external access to finger > service to only well control service points (i.e. a few very limited systems). > For some people who want to publish plans and other dynamic information > (Linus Torvald's varying ascii-gram signature was cute), this may still be > too restrictive. The suggestions from Sun included the script suggestion > quoted and/or using a public domain version of finger which does not support > the "wild card" matching feature. > > Someone else mentioned an NIS caching facility in another message. > That sounded to me like yet another possibility for dealing with this nearer > to the root cause and could avoid this problem if yet another service is > uncovered which is capable of committing this kind of mayhem... > > Considering that this is proven capable of "taking out" a > network for hours or even days beyond the end of the attack itself, > I would consider it to be a little more serious than a "ping flood" > or even a "Smurf" attack. :-) > > In a couple of ways, this is related to the same class of attacks > that the "Smurf" attack belongs to. They both involve an issue of > "resource demand amplification". In other words, a limited resource demand > (packets and bandwidth) going into a network, from the attacker, results > in an amplification of that demand within the network. "Smurf" does it > by spoofing the return address of ICMP packets to be the network broadcast > address. Nisnuke does it by generating prodigious levels of NIS traffic in > response to relatively limited levels of finger requests. > > > ------------------------------------------------------------------------------- > > Raymond R Medeiros II email: medeirosat_private > > Junior Systems Administrator www: http://www.eng.usf.edu/~medeiros > > Engineering Computing > > University of South Florida > > > On Fri, 10 Jul 1998, Solar Designer wrote: > > > > Hello, > > > > > > > # mv /usr/bin/finger /usr/bin/finger.exe > > > > # cat > /usr/bin/finger > > > > #!/bin/sh > > > > exec /usr/bin/finger.exe -m $* > > > > ^D > > > > # chmod +x /usr/bin/finger > > > > Hmm, weird, this doesn't look safe to me. Why trust the extra parsing done > > > by the shell? Look at this: > > > > sunny:~$ finger "a -b" > > > finger: a -b: no such user. > > > > sunny:~$ finger a -b > > > finger: illegal option -- b > > > usage: finger [-lmps] [login ...] > > Excellent points. More the reason to either just limit access to > finger or to use one of the public domain implimentations that doesn't > support the wild carding. The script listed in the advisory was the last > of several suggestions for use only if nothing else was practical... > > > > Now, many implementations of fingerd just run finger on data received from > > > the remote, doing some sanity checks first, and splitting the arguments > > > for execv(). These checks often include denying passing of some or all > > > options to finger. If fingerd knows about less word separators than the > > > shell does, then an attacker might be able to pass a forbidden option to > > > finger. For example, if our fingerd didn't know about tabs (which isn't a > > > security hole yet: our fingerd uses execv(), remember?), a remote attacker > > > could send us "user\t-option". > > > > I admit that the problem isn't serious: not all fingerd's are done this > > > way, forbidden finger options are likely to violate someone's privacy > > > only, etc. Still, it's not a good idea to trust the shell, in general. > > > > Signed, > > > Solar Designer > > Mike > -- > Michael H. Warfield | (770) 985-6132 | mhwat_private > (The Mad Wizard) | (770) 925-8248 | http://www.wittsend.com/mhw/ > NIC whois: MHW9 | An optimist believes we live in the best of all > PGP Key: 0xDF1DD471 | possible worlds. A pessimist is sure of it! > > ===== Begin Attachment ===== nisnuke.asc ===== > > -----BEGIN PGP SIGNED MESSAGE----- > > ISS Security Alert Advisory > June 29, 1998 > > > Distributed DoS attack against NIS/NIS+ based networks. > > For purposes of this report, NIS refers to both NIS and NIS+ > as this problem has been observed and reproduced on both services. > > > Synopsis: > > It is possible, through a well orchestrated attack using the finger > service against multiple NIS clients, to disrupt an entire NIS based > network and/or starve the NIS servers for resources. The problem is > in the finger service but the attack causes long duration, network-wide, > congestion and resource exhaustion on NIS servers. > > > Recommended action: > > Disable finger service on any systems connected to any NIS based > network. Disable access to internal finger services at all perimeter > defenses (firewalls, gateways, filtering routers, etc.). If the finger > service is required for some specific purpose, limit it to the minimum > number of restricted hosts or to hosts which are NOT participating > in NIS. > > For those who wish to continue to run the finger service on their > systems, there are other possible actions that could be taken, like using > a public domain fingerd that doesn't do ambiguous lookups. The finger > service can also be protected by even something as simple as wrapping > finger to not do ambiguous lookups like as follows: > > # mv /usr/bin/finger /usr/bin/finger.exe > # cat > /usr/bin/finger > #!/bin/sh > exec /usr/bin/finger.exe -m $* > ^D > # chmod +x /usr/bin/finger > > These recommendations would permit the continued safe use of finger > (or as safe as finger ever gets). > > > Description: > > A finger request results in multiple NIS requests as the responding > daemon attempts to locate all account records matching the finger request. > A request for finger fooat_private will result in one finger daemon searching > incrementally through all of the entries in the passwd map to locate any > accounts with foo in the name. As a consequence, a single finger request > can result in a significantly larger amount of traffic between the NIS > client and the NIS server than the originating traffic to and from the > finger service. The amount of NIS traffic is dependent on the size of > the NIS passwd map. With a passwd map of 10,000 entries, a single finger > request would result in roughly 10,000 NIS requests and 10,000 NIS responses. > This does NOT count retries from packet loss or other failures (a highly > significant factor in this attack). > > By sending a large number of overlapping finger requests to a single > host, it is possible to load that host down with a very significant amount of > traffic just processing the NIS requests. If this is done to multiple hosts, > the network traffic rises dramatically. > > Eventually, a condition arises in which congestion and/or resource > exhaustion on the NIS server begin to cause a significant rise in lost > packets and failed requests. This results in retry attempts from the > NIS clients, adding to the already overloaded network traffic. The > failure / retry / failure cycle becomes an NIS traffic "storm" in which the > retry traffic dominates and little other traffic can squeeze through. > > Network congestion combined with NIS server resource exhaustion > work together to not only deny service to the requesting clients but also > to rapidly clog the network bandwidth and render the network unusable by > anything on the network. > > > Analysis and details: > > In analyzing this attack, a perl script was used to generate finger > traffic attacking a dozen hosts with four finger requests for each of > approximately 100 names (~400 finger requests per host). A demonstration > NIS map of approximately 1000 accounts was used. At an issue rate of > approximately 4 finger requests every two seconds against a given host, > 10's to 100's of lingering finger requests would build up even as some > finger requests would be fufilled. These lingering finger processes > would be attempting to paw their way through the entire NIS password map. > A typical test run attack lasted approximately 30-50 seconds in duration. > > During analysis of this attack, network traffic from even a short > ~30 seconds blast from the perl test script resulted in traffic levels that > caused network disruptions extending for as much as 45 minutes to an hour > after cessation of the attack. During this time, some systems were impacted > to the extent that screen savers froze and systems were unresponsive to the > keyboards. Many systems were left with seemingly hung finger processes. > These stayed on the system for a half an hour or more while the network > congestion cleared. Some systems ran out of swap space because of the > resource demands of the finger processes. On a few of the test runs the > network traffic was observed to have risen to a level which caused a > switched ethernet hub to disable ports due to excesive collisions. > > Finger requests to perform this action have to be distributed and > timed properly. Too many requests, too quickly, seem to result in inetd > disabling the finger service. Too slowly, and the network traffic rises too > slowly and fails to reach the catastrophic level where packet loss and retries > become the dominant traffic input to the network. > > Because the finger requests are TCP based and not dependent on > preauthentication, finger requests can still be delivered by the attacker > to the systems under attack even in the face of increasing network congestion. > By the time the attacking connections are significantly impacted by the > network congestion, the network has been rendered unusable by systems > requiring NIS or other services. > > Timed correctly, an attack of only a few seconds, targeting as few > as a dozen NIS clients on a network with a moderate NIS passwd map can render > even a small network unusable for as long as a half an hour to an hour or > more. Increasing the size of the NIS passwd map, the number of attacked > clients, or the number of requests sent to any given client causes the > recovery time to extend out dramatically and disproportionately to any > particular increases in any particular factor. > > If the NIS server is also one of the attacked systems, it can > rapidly run out of system resources, causing NIS request failures and > accelerating the resulting NIS traffic "storm". When the NIS server > was one of the systems attacked by finger requests, it was not unusual > to see warnings about unable to grow stack, exhausted virtual memory, or > other resource related errors. > > MOST client systems seem to clean themselves up EVENTUALLY. This > can take anywhere from a few moments for some Linux boxes, to a significant > fraction of an hour for some SUN boxes. It was observed that some IRIX > boxes and AIX boxes would become unreachable from the network and > unresponsive to the keyboard, requiring a power cycle to recover. > These last systems may have recovered on their own eventually, but that > time frame appears to be geological. Recovery time seems to also be > dependent on the recovery time of the NIS server for those clients which > were observed to recover. Resetting the targeted systems permits the > network to recover. All tested systems were affected to some extent. > > Because the resulting traffic and congestion is proportional to the > size of the NIS passwd map times the number of attacked hosts times the > number of requests in the attack, large networks are disproportionately > vulnerable to this attack. Even small networks of a few dozen systems > can be disabled by a determined attacker if they have a sufficiently > large NIS passwd map. > > > Conclusion: > > The finger service permits a condition where a limited number > of requests can result in a vastly larger number of internal requests > against a central naming service such as NIS. This permits an attacker > to mount a distributed attack by launching smaller attacks against numerous > hosts. These combine to form a disasterous level of congestion on the > internal systems, disrupting an internal network for an extended period > of time. > > > Afterword: > > It is unknown, at this time, if any other services exhibit similar > characteristics with regard to NIS traffic as does finger. Disabling finger > prevents it from being exploited against a network. It obviously does > not guarantee that some other service might be similarly exploitable. > > > Credits: > > We would like to extend our appreciation to Sun Microsystems, Inc. > for their assistance and consultation with regard to the vulnerability. > > > Michael H. Warfield > Senior Researcher > ISS X-Force > Internet Security Systems, Inc. > > ________ > > Copyright (c) 1998 by Internet Security Systems, Inc. > > Permission is hereby granted for the redistribution of this alert > electronically. It is not to be edited in any way without express > consent of X-Force. If you wish to reprint the whole or any part > of this alert in any other medium excluding electronic medium, please > email xforceat_private for permission. > > Disclaimer > > The information within this paper may change without notice. Use of this > information constitutes acceptance for use in an AS IS condition. There > are NO warranties with regard to this information. In no event shall the > author be liable for any damages whatsoever arising out of or in connection > with the use or spread of this information. Any use of this information is > at the user's own risk. > > X-Force PGP Key available at: http://www.iss.net/xforce/sensitive.html > as well as on MIT's PGP key server and PGP.com's key server. > > X-Force Vulnerability and Threat Database: http://www.iss.net/xforce > > Please send suggestions, updates, and comments to: > X-Force <xforceat_private> of Internet Security Systems, Inc. > > -----BEGIN PGP SIGNATURE----- > Version: 2.6.2 > > iQCVAwUBNZfaJjRfJiV99eG9AQEFDAP/Y5k+8q5lmasrZotzcVw5z43LBUy9AL6l > By8onp07PmIdkQD7vCyiSIHqEvWiGigvhl76gdnkZxUowSYjdI+PfZBbB9S1HoZB > eMRNjd0m5X7aEimIH208TwdegxwLu+A7/FKnl3EImd182Nzma1PhAqXpclPEmJ38 > mUbk7Lm6Yls= > =HXJi > -----END PGP SIGNATURE----- >
This archive was generated by hypermail 2b30 : Fri Apr 13 2001 - 14:04:15 PDT