> What you are describing exactly fits the description of a "midpoint server" > participating in a new form of Distributed Denial of Service attack. We > were on the receiving end of such an attack a little over one month ago. > > Briefly, the idea is that a spoofed source IP SYN flood is gently spread > across a LARGE number of TCP servers. Each of the many servers replies with > SYN/ACK packets ... aimed at the attack's intended target. Since each > unacknowledged SYN/ACK will be repeated (generally three times) this > results in a factor-four bandwidth multiplication. I have also done some research into this for of reflector attack, which is an evolutionary variant on several other forms of reflector attack. A tool was found recently that fits the bill (all it needs is a list of reflectors.) Here is the current writeup I have on this subject: Some Followup Analysis to the 01/11/02 DDoS Attack on grc.com ============================================================= Last modified: Thu Jan 17 10:02:23 PST 2002 On the morning of Friday, January 11 2001, Steve Gibson's web site (grc.com) was hit with a distributed denial of service attack at 2:00 AM PST. This attack used a variation of a TCP based reflection attack that is not widely known to exist in the wild. Steve's early analysis of the attack in included below (Appendix A). While there may be a new (D)DoS program "in the wild" to implement this attack, the risks and methods have been known for two or more years and some simple modifications to existing tools, and a good list of high-capacity routers, switches, and servers, could affect an attack of this type. Reflector attacks ================= Reflector attacks (and their implications for making traceback more difficult) have been know for over a year. Vern Paxson mentioned them at a traceback workshop in Chicago sponsored by CERIAS in September 2000[1]. Vern then went on to give a formal talk on the subject in June of 2001[2] In January of 2001, Register.com was taken out with a DNS reflection attack[3], by a single (or so it seems) system. The attacker used a year+ old list of DNS servers, and used a program to spray out forged (with the victim, register.com's source addresses) DNS requests, making a recursive query for the MX records for the domain "aol.com." If I recall correctly, the request was about 50 bytes, and the answer some 500 bytes (so it also amplified the flood by a factor of 10 -- Steve calculated a "FACTOR FOUR" amplification for his attack.) Since the requests looked somewhat legitimate (very few of these servers should have even been responding to recursive DNS queries in the first place, but were not configured to refuse them), it was very difficult to trace. The attackers, however, only used one IP address for targeting the site, which made it pretty easy to filter in-bound the inbound DNS traffic. It still took a WEEK for the major backbone providers to trace and shut down the host that was originating the flood of requests. Traffic reflection off routers ============================== Back in 1999[4] someone (nobody knows who, or even if there was a someone) appeared to be sending out packets with forged sources addresses within various networks and low TTLs. Routers within the Internet infrastructure would send back ICMP Time Exceeded in Transit packets to sites all over the globe. This one was so low bandwidth that nobody assumed it was an attack (but it looked to me like a possible covert channel for command/control, since it returned some 8 bytes of the original packet within the ICMP Time Exceeded in Transit packet.) Accessibility of Routing Infrastructure Hosts ============================================= Rob Thomas has also been discussing vulnerabilities in the routing infrastructure of the Internet for several months, now, and has pointed out that many routers shouldn't be accessible from outside the owner's network anyway. Analysis and Defenses Proposed by Steve Gibson ============================================== [Steve's early analysis is found in Appendix A] Steve claims that it would take a lot of research to come up with a list of hundreds of routers and servers with open ports that could be exploited. He also claims that he saw both SYN|ACK and RST|ACK packets coming into his network, and the ports being used were for BGP, SSH, Telnet, DNS, and HTTP. Since these are all services with a high probability of being enabled on routers, it is not necessary to first look for them (just assume they are open), and since Steve was also seeing RST|ACK packets, it means that some of them were sent packets to ports that were NOT open. Getting a list of "hundreds" of routers (as Steve claims were used against his site) is not at all difficult. A quickly hacked together Bourne Shell script of a handful of lines, using a list of 10 popular web sites, results in a list of 50 routers and servers. To get "hundreds" of routers/servers, it would only be necessary to add another 30 to 50 web sites. Even easier is to go to a web site that lists BGP route peerings, which gives the addresses of several thousand infrastructure routers. If you are tempted to say, "so let's make the list private instead", you need only fall back on the prior simple script to harvest IP addresses of routers directly. The answer is, of course, do to ingress filtering to protect the routers, not to try to hide their existence in the infrastructure (although backbone providers are quick to point out that they fear CPU overutilization from filtering, so they instead leave their routers exposed to outsider access.) The point that the flooding program is clever enough to send packets out at a small enough rate to fly beneath the radar, so to speak, is not as sophisticated as it seems. The single host that took out Register.com was able to do this with DNS requests, and most DDoS attacks today are done with systems that are sufficiently slow so as to not swamp the high speed networks of today. This may be purely an accident, not a clever ploy or feature. Steve is right about some of the defensive mechanisms, but in his case his "don't allow privileged port traffic" defense works ONLY in the case where your site exclusively has servers making no outbound connections. Of course this will break things like DNS zone transfers, NTP time synchronization, outgoing email, etc. from his site as well. Being able to simply reject all outbound privileged port connections is thus not a widely deployable "solution." Instead, direct ingress filtering that only allows incoming connections from those servers you trust (or stateful filtering as provided by more sophisticated firewalls, filtering bridges, or host based packet filters like "ipchains", "iptables", or "ipf".) He is also correct that detecting this kind of an attack is more difficult (since many sites don't have methods of sniffing traffic within their network infrastructure), but many in the security industry have been saying that for years. Copyright (C) 2002, All rights reserved. David Dittrich <dittrichat_private> References ========== [1] CERIAS Traceback Workshop proceedings http://www.cerias.purdue.edu/traceback/ [2] Vern Paxson talk on reflector attacks http://www.icir.org/vern/papers/reflectors.CCR.01/reflectors.html [3] Thread on Register.com DNS reflector attack http://staff.washington.edu/dittrich/misc/ddos/register.com-unisog.txt [4] Threads on ICMP Time Exceeded in Transit traffic http://lists.insecure.org/incidents/1999/Dec/0005.html http://cert.uni-stuttgart.de/archive/incidents/1999/12/msg00005.html Appendix A: Steve Gibson analysis of 01/11/2002 attack on grc.com ================================================================= This past Friday morning at 2:00 AM PST, we were hit by a new sort of massive Distributed Denial of Service (DDoS) attack. Since this operated in an intriguing new way that I have never seen reported before, I thought you folks might want to know what was going on. Would you please forward this note to anyone else who should learn of it? Thanks. ------------------------------------------------------------------------ Here is the report I wrote and posted to my newsgroups after the attack: ------------------------------------------------------------------------ The "Packet Bounce" Distributed Denial of Service Attack As many of you know, we were the target of a new sort of attack very early this morning. We were restored to operation after four hours of complete outage when I characterized the attack and was able to filter a large part of the nonsense at Verio's upstream routers. However, after 13 hours, well into the day, the attack continues unabated, though blocked. We were initially driven off the Internet by a flood of SYN/ACK and RST/ACK packets -- all having a source port of 179 and apparently random destination ports. These flooding packets were apparently (and confirmed) coming from the Internet's core routers! Our own ISP's (Verio) routers were 'attacking' us, as were the routers of other large ISPs, a few of the main DNS root servers, and many of the web servers belonging to YAHOO.COM. I placed 'attacking' in quotes above, since, as we'll see below, all of these machines were innocent bystanders in the attack. However, they were utilized to swamp our bandwidth and force us off the Net. Since we do not have any need for port 179 traffic -- which is the Border Gateway Protocol (BGP) used for inter-router communications -- we were able to readily block that traffic at Verio's upstream "aggregation" router. But ... the second we did that, another whole class of additional flooding attack traffic (which had been unable to compete with the core routers overwhelming BGP attack) showed itself for the first time. It had been buried "beneath" the main thrust of the attack. With the unneeded BGP protocol now blocked, we were getting SYN/ACK and RST/ACK packet floods coming from ports 22 (SSH), 23 (TELNET), 53 (DNS) and 80 (HTTP). These packets were flooding in from hundreds of actual machines (including many at YAHOO.COM and even one at NSA.GOV) and widely spread across the Internet. This second-tier of traffic was being sourced by well-connected but similarly innocent servers. ------------------------------------------------------------------- Here is what someone has done, and why it's rather clever ... ------------------------------------------------------------------- Someone, or some group of people, have accumulated a large list of well-connected server IPs and corresponding TCP service ports out on the Internet. Among these are several hundred well-connected routers serving the BGP (port 179) protocol. This list also includes many other servers listening to other common service ports including SSH, TELNET, DNS, and HTTP. Using full raw sockets on an unknown platform (presumably UNIX) in order to spoof the source IP and generate isolated SYN packets, the SYN packets are "sprayed" at each machine in their list, at that machine's corresponding listening TCP service port. (BGP for the routers, HTTP for YAHOO. etc.) In each case, the source IP being spoofed is the attack's target. For this attack it was the IP of the grc.com server. The high bandwidth of the attack, crossing our Internet connection, saturated our available bandwidth many MANY times over. The bandwidth of the "spray" is such that no one of these "intermediary" target servers receives a "flooding-level" of SYN packets. In other words, the SYN packet rate to each server is modest so that the server is never placed into DoS by the packet originator, and no alarms are raised on that server locally. However, for every new SYN packet arriving at one of these servers, that server, believing that a TCP connection is being requested and initiated, replies to the apparent sender with a SYN/ACK packet. That SYN/ACK packet is, of course, aimed at the APPARENT source of the SYN ... in this case grc.com. So ... grc.com is consequently subjected to a substantial Distributed Denial of Service (DDoS) attack from many hundreds of well-meaning and innocent Internet servers. And, significantly, none of those servers is being inconvenienced by its role in this attack. ------------------------------------------------------------------- Why is this rather clever? ------------------------------------------------------------------- Unlike traditional DDoS attacks, where a high-bandwidth spoofed-SYN flood can be traced back up the router chain towards its actual bandwidth source (because all packets have a common destination address although a spoofed source IP), the use of a large number of intermediate TCP servers prevents any single "stream" from having enough bandwidth to sound any alarms. And all of these low-bandwidth streams are coming from very different physically isolated servers. Conceivably, only a very few malicious and well-connected SYN-generating machines would be required to "spray" the SYN packets out across a large population of intermediate servers. Each of these malicious well-connected machines may be generating packets at its maximum speed, but since those packets are so widely and immediately distributed -- rather than all being aimed at a single traditional DoS target -- none of the intermediate recipient machines is unduly inconvenienced. Furthermore, since the intermediate machines are widely spread across the Internet, the SYN-source's local flood thins and spreads out quickly, even a few hops away from its source machine. This further impedes successful SYN-source backtracking. This also creates a potentially more manageable architecture than the traditional comparatively difficult to establish many-machine multi-tier DDoS system. And finally, there is a very good chance that the intermediate machine will generate MULTIPLE SYN/ACK packets for every SYN packet received. Since traditional TCP stacks will typically generate four SYN/ACK packets in response to the receipt of a single unanswered SYN packet (believing that their responding SYN/ACK might have been lost in transit) this system has the potential for creating a FACTOR FOUR attack bandwidth multiplication. (If the receiving target system can get a "RST" packet back to the originating server, then further SYN/ACKs will be averted. However, since many commercial enterprise-level firewalls simply drop unsolicited SYN/ACK packets, the bandwidth multiplication effect tends to rein.) Therefore, the use of intermediate TCP servers has the effect of quadrupling the original input bandwidth ... which they are innocently aiming at the attack target. (Note that the fact that we were also seeing some RST/ACK packets amid the SYN/ACK flood tells us that the malicious hacker's IP and Server Port list is not perfect. Those RST/ASK packets were generated in response to a spoofed source IP SYN packet hitting a non-open service port.) However, the list of IPs and Ports was certainly accurate enough, and it is trivial and maintain such a list. Every "TraceRoute" across the Internet yields a list of intermediate, probably very-well-connected routers, and any large well-known web server farm -- such as YAHOO.COM which was used here -- provides plenty of web servers for the "packet bounce". Any site lacking extremely comprehensive and unusually tight upstream filtering will probably be placed into a sustained Denial of Service by the flood of SYN/ACK and RST/ACK traffic pouring into it from many hundreds of well-connected public servers. This attack will NOT appear on the "radar screens" of the organizations monitoring traditional spoofed source IP SYN floods and other similar attacks. There will be no monitorable "backscatter" from this attack since every IP used is for that of a valid machine. For all of these reasons, this new style of attack creates a very potent, highly effective, difficult to filter distributed denial of service. ------------------------------------------------------------------- Defending against the "Packet Bounce" DDoS attack: ------------------------------------------------------------------- The trick to defending against this attack is recognizing the very different roles of an Internet server and client. Since TCP packets are bouncing off an Internet server, the originator and final destination of the traffic are both presumed to be Internet clients. In other words, the packets flooding our network necessarily had a SOURCE port in the "service port" range from 1-1023, since they were "bouncing" off servers handling and answering well known service ports (22, 23, 53, 80 and 179). But our server, when it is acting as a server, NEVER NEEDS to receive packets with a source port less than 1024. Internet clients which access our web, news, pop, and other servers always generate packets with source ports in the client range, typically 1024 to 5000 or more. Only when our server is acting as a client -- when sending eMail via SMTP to a remote port 25, or perhaps when querying external DNS servers, is there any need to receive traffic with source ports in the "service port" range. Specific rules can usually be created to safely allow traffic in those special cases from specific authorized IPs, while denying ALL other traffic with source ports less than 1024. After adding these rules to our upstream router all, malicious "Packet Bounce" attack traffic was filtered and we returned to the Internet with a completely clean connection. ______________________________________________________________________ Steve. ================================================================= -- Dave Dittrich Computing & Communications dittrichat_private University Computing Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 ---------------------------------------------------------------------------- This list is provided by the SecurityFocus ARIS analyzer service. For more information on this free incident handling, management and tracking system please see: http://aris.securityfocus.com
This archive was generated by hypermail 2b30 : Wed Feb 13 2002 - 20:40:06 PST