[Thanks to the folks who responded! The depth of knowledge among Politech subscribers is impressive. Previous Politech message: http://www.politechbot.com/p-04754.html --Declan] --- Date: Thu, 22 May 2003 12:10:26 -0400 From: Brad <bradat_private> To: declanat_private Subject: "Nice" Spam Filtering Respones Declan, I got a ton of responses after asking after a "nice" spam blocklist and a few people asking me to pass on any info I got, so if you'd post the below to the list, I'd appreciate it. ----------- A "Nice" Spam Blocklist I got a huge number of responses, and I don't have time to respond to everyone, and for that I'm sorry, I thank everyone who responded, it was a great help. Quite a number of people sent me what they were using, others pointing to a few different web sources, a couple of commercial offers, and a few people asking for any info I've gleaned, and one person reminding me that "blacklist" has some bad connotations, so I've switched to "blocklist", which makes sense. So here's the lowdown: 1) There is no one blocklist to rule them all. It'd be really nice if there was one big aggregate, but it appears that there is no such beast. Essentially, you need to have three to have any decent hope. One open relay blocker, one proxy blocker, and one manual. A) Of the Open Relay blockers, most people seemed to like ORDB ( http://www.ordb.org ). It scours the net looking for open relays, just like Orbz used to do. B) Of the proxy blockers, there was no clear consensus, but opm.blitzed.org and proxies.relays.monkeys.com seemed to be the favorites. C) Of the manual spam blockers, ones that add known spam sources manually, the Spamhaus SBL ( http://sbl.spamhaus.org ) is by far the most recommend, and probably fits the bill of the "nicest". D) There is actually one aggregate. blackholes.easynet.nl contains both a list of open proxies and the spamhaus sbl, but not an open relay blocker. 2) Additionally, there are two other methods for blocklists, but I'm not so sure they fall under "nice". The first is country blockers. These block all e-mail from the designated country. ( china.blackholes.us korea.blackholes.us nigeria.blackholes.us ) As a business ISP, I'm not so sure I can just go and block whole countries, but I'll wager they would stop a good chunk of spam. The second is blocking "dynamic" and "dialup" IP's. Essentially, these sites try to track IP's that belong to dialup and cable modem users. As someone who runs a home server off his cable modem, I think this is a bad idea, but others might want to consider it. 3) Lastly, everyone seems to love SpamAssassin. One person even sent me a message ten times saying I should use SpamAssassin and probably just didn't know how to use it properly, despite my original message stating SpamAssassin was not what I was looking for. The problem is managing its use for 20,000 people. Different people will want different levels of SpamAssassin. I use it myself, but I have to order it in procmail carefully, otherwise it will mark all of my nightly root-mail and other cron jobs as spam. I'm smart enough to do that. However, I'm not going to be manually set it for a thousand people that get various newsletters or whatnot that spamassassin concludes is spam. The overhead is too high. SpamAssassin is great for individual users who are savvy enough to manage it themselves. However, it isn't a solution for wide-scale enforcement. I'm lucky if my users can find their way around MS Outlook... -Brad Hall Systems Administrator Crisp.Net --- Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? From: Mark Lowes <hamsterat_private> To: declanat_private Date: 17 May 2003 20:48:58 +0100 On Sat, 2003-05-17 at 17:13, Declan McCullagh wrote: > Do you, or anyone on the list know of a "nice" blacklist? One the tries to > avoid collateral damage and quickly unblocks mistakes? I'm responsible for maintaining the mail system at my ISP (SME based ISP in the UK). At the moment I would only recommend the following DNSBLs SBL (http://www.spamhaus.org/): Spamhaus list, low to non-existant collatoral damage, almost entirely the large spam gangs. OPM (http://www.blitzed.org/): List of open proxies, they test IPs on either spam to a spamtrap or on a connection to the blitzed.org irc network. Entries are expired out so it's not as effective as some open proxy lists but the risk of old out of date entries is much lower. ORDB (http://www.ordb.org/): Open relay list, appears to have very little collatoral damage. DSBL (http://www.dsbl.org/): Open relay list, I'd only use this in warn mode as it appears to have some listings I'm not convinced are entirely appropriate for a single-hop relay list. Mark --- From: Todd Meister <toddat_private> To: declanat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? Declan McCullagh writes: >Date: Thu, 15 May 2003 10:33:55 -0400 >From: Brad <bradat_private> >To: declanat_private >Subject: A Nice Spam Blacklist? > >Do you, or anyone on the list know of a "nice" blacklist? One the tries to >avoid collateral damage and quickly unblocks mistakes? > This one is generally very safe,and catches a _lot_ of spam: sbl.spamhaus.org You have to be a naughty, naughty spammer/ISP to get in that list. We've been using this, also, as proxies are the new scourge: proxies.relays.monkeys.com Here's an idea how busy one of our two main mail servers is (this for approximately the last 12 hours): >wc -l /var/log/mail.log 44216 /var/log/mail.log Each connection generates two lines, so that's about 22,000 connections. Here are the number of hits for those two lists: >grep proxies.relays /var/log/mail.log | wc -l 7759 (that's 7,759 blocked connections - 35% of the total connections) >grep sbl.spamhaus /var/log/mail.log | wc -l 1867 (another 8%) The monkeys list is the first one in our sendmail.cf file, so spamhaus, which used to get the most hits (usually around 80%), has many fewer than in the past. That 43% doesn't count the loads of spam that actually gets through, whether to our users, or ultimately to bounce back to the forged account. We've also been manually blocking the biggest offenders at our core router for the last couple days. Spam traffic has increased dramatically in the past week, and our servers have been swamped, though the border blocks nicely snipped them off. In fact, we're in the midst of re-working our mail setup to accomodate all the connections instigated by spammers. Todd Meister Unix Admin LMI.net --- From: Ed Allen Smith <easmithat_private> Date: Sat, 17 May 2003 15:08:45 -0400 Cc: declanat_private To: bradat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? In message <5.2.1.1.0.20030517120419.03a2d570at_private> (on 17 May 2003 12:13:46 -0400), declanat_private (Declan McCullagh) wrote: > >--- > >Date: Thu, 15 May 2003 10:33:55 -0400 >From: Brad <bradat_private> >To: declanat_private >Subject: A Nice Spam Blacklist? > >As a subscriber, I've been following the ongoing problems with spamcop and >have butted heads with a blacklist in the past, but as the SysAdmin of a >regional ISP, I've come to the conclusion that I have no choice but to >begin subscribing to one. The Spam is simply clogging the server. >SpamAssassin is great, but it only increases the load on the server and is >difficult to support for the userbase. > >Do you, or anyone on the list know of a "nice" blacklist? One the tries to >avoid collateral damage and quickly unblocks mistakes? Umm... there are quite a number of different types of anti-spam blacklists. Do you want one that acts vs abused open relays/proxies, for instance? Or do you want one that acts against spam-friendly ISPs? For that matter, there are other categories of blacklists. dsn.rfc-ignorant.org (http://www.rfc-ignorant.org) is a domain-based blacklist that is of use both vs spammers and for postmasters to avoid double-bounces in their inboxes (it's vs domains that refuse to accept bounces with the (RFC-standard) <> return address), although configuring it with some mail programs can be a bit tricky (it's designed for use vs the claimed envelope FROM domain). It works very nicely, IMO, although being one of its administrators, I'm biased... I've actually done some reviews of blacklists vs "known-good" domains/IPs in the past, although it's been a while since I last ran the program for it due to workload, server load, et al. You may also wish to take a look at http://www.sdsc.edu/~jeff/spam/Blacklists_Compared.html. -Allen -- Allen Smith http://cesario.rutgers.edu/easmith/ September 11, 2001 A Day That Shall Live In Infamy II "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin --- To: declanat_private Cc: politechat_private, bradat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? From: Ted Cabeen <tedat_private> Organization: Impulse Internet Services Date: Sat, 17 May 2003 12:13:48 -0700 version=2.53-the_well_w Declan McCullagh <declanat_private> writes: > --- > > Date: Thu, 15 May 2003 10:33:55 -0400 > From: Brad <bradat_private> > To: declanat_private > Subject: A Nice Spam Blacklist? > > As a subscriber, I've been following the ongoing problems with spamcop > and have butted heads with a blacklist in the past, but as the > SysAdmin of a regional ISP, I've come to the conclusion that I have no > choice but to begin subscribing to one. The Spam is simply clogging > the server. SpamAssassin is great, but it only increases the load on > the server and is difficult to support for the userbase. Depending on your mail configuration, using some of the SQL configuration systems available available for SpamAssassin can help a lot in enhancing SA's ease of use. > Do you, or anyone on the list know of a "nice" blacklist? One the > tries to avoid collateral damage and quickly unblocks mistakes? Of the currently available blacklists, ORDB <http://www.ordb.org/> probably fits your description best, but it's somewhat limited in scope. ORDB just blocks open relays, and that's it. The entire system is automated, which eliminates the delays in removal that you often see from blacklists. If ORDB can relay mail through you, you will be listed. If you then repair the problem, you can resubmit for processing and will usually be removed within an hour. I also like the wirehub sender-based blacklist to block the somewhat more honest spammers who don't forge their MAIL FROM addresses. Customers usually don't want to recieve email from anyone at 1stopcyberdeals.biz and similar sites, and the wirehub list is good at blocking those. The most important thing to consider with using a blacklist of any sort is the administrative hassle of maintaining the whitelist that compensates for the problems and mistakes in the list(s) you choose. There's a fine balance between a list that blocks little spam, and one that blocks too much legitimate email, taking up your time and infiruating the customers whose inbound email was blocked. Another good tactic is to use some of the more aggressive blacklists on your secondary mail servers only. Many spammers bypass the primary mail server when sending mail in an attempt to get by local filters that may only be installed on the primary mail server. Making the secondaries more aggressive can help a lot in defending against that, and the collateral damage is minimal because very little legitimate email will use a secondary mail server when the primary is functioning properly. -- Ted Cabeen Systems/Network Administrator Impulse Internet Services --- Date: 17 May 2003 15:17:40 -0400 From: "John R Levine" <johnlat_private> To: "Declan McCullagh" <declanat_private> Cc: bradat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? In-Reply-To: <5.2.1.1.0.20030517120419.03a2d570at_private> > Do you, or anyone on the list know of a "nice" blacklist? One the > tries to avoid collateral damage and quickly unblocks mistakes? Here's the public DNSBLs (DNS blocklists) that I use: sbl.spamhaus.org The Spamhaus SBL is currently the premier anti-spam list. It's very carefully maintained by hand and lists confirmed sources of spam. Very occasionally when an ISP chronically fails to deal with spammers on their network, it'll block the network's administrative servers for a few hours. (It's so widely used, that's all it takes.) It definitely does not do "collateral" blocking of address space adjacent to spammers. blackholes.mail-abuse.org dialups.mail-abuse.org relays.mail-abuse.org The MAPS RBL (manually maintained spam sources and support), dialups and open relay lists. Professionally maintained, costs modest amounts of money to use. See www.mail-abuse.org, the prices are all negotiable if you can't afford list price. opm.blitzed.org The Blitzed open proxies list. Contains insecure open proxies identified by mail and IRC users. proxies.relays.monkeys.com Ron Guilmette's monkeys.com proxy list, also contains insecure open proxies. proxies.blackholes.easynet.nl A third open proxy list, formerly known as the Wirehub list. dynablock.easynet.nl Dynamic IP's, dialup and home users who shouldn't be sending mail directly, similar to the MAPS dialup list. Also formerly Wirehub. korea.services.net My Korean exasperation list, includes most networks in Korea due to the horrible spam problem there. Will have false positives if you have correspondents in Korea, won't if you don't. Regards, John Levine, johnlat_private, Primary Perpetrator of "The Internet for Dummies", Information Superhighwayman wanna-be, http://iecc.com/johnl, Sewer Commissioner "A book is a sneeze." - E.B. White, on the writing of Charlotte's Web --- Date: Sun, 18 May 2003 01:11:33 +0530 From: Devdas Bhagat <dvbat_private> To: Declan McCullagh <declanat_private> Cc: bradat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? On 17/05/03 12:13 -0400, Declan McCullagh wrote: > As a subscriber, I've been following the ongoing problems with spamcop and > have butted heads with a blacklist in the past, but as the SysAdmin of a > regional ISP, I've come to the conclusion that I have no choice but to > begin subscribing to one. The Spam is simply clogging the server. > SpamAssassin is great, but it only increases the load on the server and is > difficult to support for the userbase. Join the club. > Do you, or anyone on the list know of a "nice" blacklist? One the tries to > avoid collateral damage and quickly unblocks mistakes? I would start with spamhaus (Known spam senders): sbl.spamhaus.org. ORDB is a list of open relays (http://www.ordb.org) opm.blitzed.org is a good list of open proxies. These three should take care of a lot of spam. Another good list is the wirehub.nl blacklist (blackholes.wirehub.nl) You could also grab the wirehub access file If this isn't good enough, then move on to spews. This hits hard, has /lots/ of collateral damage, but stops a /lot/ of spam. This is the hardest hitting of the DNSBLs that I dare to use. Looking around on nanae, and recent postfix-users archives should be informative too. Devdas Bhagat --- From: "Clayton, Nik [IT]" <nik.claytonat_private> To: "'declanat_private'" <declanat_private>, "'bradat_private'" <bradat_private> Subject: RE: Request for help from ISP: What is a "nice" anti-spam blackli st? Date: Mon, 19 May 2003 09:38:01 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2655.55) Content-Type: text/plain; charset="ISO-8859-1" X-Scanned-By: MIMEDefang 2.33 (www . roaringpenguin . com / mimedefang) X-Spam-Status: No, hits=-3.3 required=4.0 tests=QUOTED_EMAIL_TEXT version=2.53-the_well_w X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.53-the_well_w (1.174.2.15-2003-03-30-exp) X-UIDL: 263acbdf2ed9a1ed2275ea5fc2a53c86 > Date: Thu, 15 May 2003 10:33:55 -0400 > From: Brad <bradat_private> > To: declanat_private > Subject: A Nice Spam Blacklist? > > As a subscriber, I've been following the ongoing problems with spamcop and > have butted heads with a blacklist in the past, but as the SysAdmin of a > regional ISP, I've come to the conclusion that I have no choice but to > begin subscribing to one. The Spam is simply clogging the server. > SpamAssassin is great, but it only increases the load on the > server and is difficult to support for the userbase. > > Do you, or anyone on the list know of a "nice" blacklist? > One the tries to avoid collateral damage and quickly unblocks mistakes? The SBL (Spamhaus Block List) is very good in this regard. It makes it a bit conservative, but that can be a good thing. http://www.spamhaus.org/ The scores that the various blacklists that SpamAssassin supports can also be a guide as to how reliable they are. Finally, this message from Daniel Quinlan: http://marc.theaimsgroup.com/?l=spamassassin-devel&m=105001264517400&w=2 is the result of him testing a large number of blacklists on the SpamAssassin spam and non-spam collections to see how accurate they are, which might also be useful. N -- 1 1 2 3 4 5 6 7 7 0 0 0 0 0 0 0 5 -- The 75 column-ometer Global Messaging, A: Top posting 120 Cheapside, x83331 Q: What's the most annoying e-mail habit? --- Date: Mon, 19 May 2003 10:52:15 -0500 From: Patty Langasek <harmoneyat_private> To: Declan McCullagh <declanat_private> Cc: bradat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? Message-ID: <20030519155210.GA14941at_private> References: <5.2.1.1.0.20030517120419.03a2d570at_private> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5.2.1.1.0.20030517120419.03a2d570at_private> User-Agent: Mutt/1.3.28i X-message-flag: Outlook: A program to spread viruses that can do email too. X-Spam-Status: No, hits=-4.9 required=4.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,REPLY_WITH_QUOTES, USER_AGENT_MUTT version=2.54-the_well_w X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.54-the_well_w (1.174.2.17-2003-05-11-exp) X-UIDL: 98b0d570aec7742df9ed2437281c8a2c > Date: Thu, 15 May 2003 10:33:55 -0400 > From: Brad <bradat_private> > To: declanat_private > Subject: A Nice Spam Blacklist? > As a subscriber, I've been following the ongoing problems with spamcop and > have butted heads with a blacklist in the past, but as the SysAdmin of a > regional ISP, I've come to the conclusion that I have no choice but to > begin subscribing to one. The Spam is simply clogging the server. > SpamAssassin is great, but it only increases the load on the server and is > difficult to support for the userbase. > Do you, or anyone on the list know of a "nice" blacklist? One the tries to > avoid collateral damage and quickly unblocks mistakes? The ISP I used to work at had similar spam problems. While the spammers are crying out that unsolicited email is costless and good for the environment, ISPs are facing the increasing dilemma of maintaining servers attempting to keep up with the load. My ISP went through a few different spam solutions, but we found that anything done completely automatically and server-side was quickly countered by the spammers' attempts to get around ISPs (who are, in fact, paying for the spammers to send customers unsolicited email). We finally stumbled across Postini <http://www.postini.com/>. It's a mail service that collects all mail going to the ISP's servers, flags and holds potential spam, then sends the unflagged email on to the ISP's server. The ISP's customers then go to a message center provided by Postini to review flagged mail. They then have the choice of sending the flagged mail on to their ISP mailbox, or deleting it directly off the message center. Eventually, their individual message center 'learns' from each individual customer interaction and begins to flag accordingly. The customers have complete control over their individual message center; able to make whitelists, blacklists and choose how vigorous they wish the spam filter to be. As well, last I knew, Postini doesn't maintain a 'blacklist', so there are no concerns about open relay ISPs who have been accidently blocked (yes, my ISP tried one of those services at one point) or overly aggressive companies who have labeled themselves the Internet Police. Naturally, having mail sent to a 3rd party server is putting faith and trust in their ability to maintain their networks, but Postini does have high service level agreements with the ISPs with which it partners. Once the ISP I worked for put Postini in place, the server mail load was cut by over 50%, and *very* few customers complained about the new service. Those who did complain, as I recall, were simply confused and skeptical. I've found that Postini is really a remarkable service. A little cumbersome for the ISP to get going at first, but incredibly easy for even the slowest of customers to learn. And, with the way it works, it's unlikely that spammers are going to find a way to work around it any time soon. Good luck and good hunting! --------------------------------------------------------- Patty Langasek harmoneyat_private --------------------------------------------------------- --- To: declanat_private cc: bradat_private, ausmanat_private Subject: Re: FC: Request for help from ISP: What is a "nice" anti-spam blacklist? In-Reply-To: Message from Declan McCullagh <declanat_private> of "Sat, 17 May 2003 12:13:46 EDT." <5.2.1.1.0.20030517120419.03a2d570at_private> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <3260.1053370884.1at_private> Date: Mon, 19 May 2003 12:01:24 -0700 From: James Ausman <ausmanat_private> X-Spam-Status: No, hits=-0.5 required=4.0 tests=IN_REP_TO version=2.54-the_well_w X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.54-the_well_w (1.174.2.17-2003-05-11-exp) X-UIDL: 944b3259e689e1a0aadb0089e0e52327 Dear Brad, I use the Register of Known Spam Operations where I work, for both corporate and customer email. This is a relatively easy thing to drop into Sendmail and catches about 1/3 of all spam sent to our site. It will increase the load on your mail servers a bit, especially during a spam attack, so don't do it if you are already pushing your services to their limit. http://www.spamhaus.org/rokso/index.lasso You can also block email from sites that do not have PTR DNS entries: http://www.sendmail.org/~ca/email/chk-810.html#810MISCCHECK If you do this, you will block some legitimate email from misconfigured sites. But many large ISPs, including AOL and Earthlink already do this, so your chance of blocking email is quite small. One nice thing about this ruleset is that it comes early enough in the Sendmail process that using it will reduce the load on your mail servers during a spam attack. Unfortunately there is no magic bullet that will catch most or all of your spam painlessly. Cheers, Jim Ausman PS Declan you can forward to FC if you like. --- Date: Sat, 17 May 2003 16:29:29 -0700 To: politechat_private From: Bob K <bkat_private> Subject: A "Nice Blacklist?" No such thing. Mime-Version: 1.0 Content-Type: multipart/mixed; x-avg-checked=avg-ok-4C6E4F3; +boundary="=======54B2FD1=======" X-Envelope-From: bkat_private X-Envelope-To: politechat_private --=======54B2FD1======= Content-Type: text/plain; x-avg-checked=avg-ok-4C6E4F3; charset=us-ascii; +format=flowed Content-Transfer-Encoding: 8bit Declan: This may or may not be appropriate for the list, but I thought I would write it anyway. I've tried to keep it short. Our ISP has been plagued by SPAM. I don't know of any ISP that hasn't been. But we recently shut off our use of Realtime Blackhole systems in favor of in-house SPAM-control. Instead of an effectiveness of 60% or so, we are now trapping more than 90% of unwanted mails, and in a way that our end users have total control. We shut off the RBLs because we incurred a huge costs as a result of our support for them. When the RBLs began to support the blocking of entire Autonomous Systems instead of targeting SPAMmers directly, they lost my support. I had a carrier whose AS numbers were put into the RBL. Like a good anti-SPAMmer I rejected the carrier and moved to another. My ISP is fairly large, and including customer support costs the carrier change cost my company around $35,000. Within 90 days, my new carrier was placed into the RBL system. Again I was mandated by the RBL operators to switch carriers. This time I refused to be extorted by a handful of people with way more power than they deserve. They misuse their power and do so without conscience. I say this because their use of the term "collateral damage" hides a tidal wave of harm to innocent ISPs and their more innocent customers. Some of the RBL operators have become so obsessed with their tools that they have started to create more harm to Internet users than the SPAMmers they want to protect those users from. A SPAMmer clots mailboxes. An agenda operated RBL takes the mailbox away. Plus, I have run out of choices for carriers. My area only has a few that support it, and all of them are AS-wide being blocked. So I tend now to view RBLs with disdain and disrespect. I do so because they don't respect the needs of honest people and honest companies who need unfettered Internet mail. All of the anti-SPAM efforts have created a lost perspective I think. That is, people should control their own mailbox. A system that makes arbitrary decisions about what content should and shouldn't be permitted is a loss of freedom of choice. So our new system affords them as much or as little control over their SPAM as they would like. Plus, it gives them the opportunity to retrieve messages they filtered by mistake. An RBL will drop the communication with a SPAM source. The new system accepts the mail and places it in a temporary holding area where it may be easily retrieved. Yes, this is taking resources to do, but I knew this perspective would entail some costs. Frankly, the cost of some disk space and processing power is a lot less expensive than having to change carriers every time an RBL invokes personal vendetta. It also doesn't leave me wondering if the next carrier will end up in the RBL forcing me to change again. And again... So, there is no such thing as a nice RBL. They are more harmful than helpful, they are less efficient than internal methods, and they take freedom of choice away from ISPs and more importantly, their customers. As long as legitimate RBLs maintain their support for those RBLS that have gone rogue, or are completely inept, they are more criminal or problematic than SPAM. ------------------------------------------------------------------------- POLITECH -- Declan McCullagh's politics and technology mailing list You may redistribute this message freely if you include this notice. ------------------------------------------------------------------------- To subscribe to Politech: http://www.politechbot.com/info/subscribe.html This message is archived at http://www.politechbot.com/ Declan McCullagh's photographs are at http://www.mccullagh.org/ Like Politech? Make a donation here: http://www.politechbot.com/donate/ -------------------------------------------------------------------------
This archive was generated by hypermail 2b30 : Tue May 27 2003 - 07:52:10 PDT