[Politech] Details on how DearAOL.com works, and AOL's junk mail filtering [sp]

From: Declan McCullagh (declan@private)
Date: Fri Apr 14 2006 - 23:33:03 PDT


Apologies to Danny for taking so long to send this out -- it would have 
been a good discussion have during the day on Friday. The Well was 
completely offline until about an hour ago, and email is only now 
trickling in: http://www.salon.com/wellstatus/

Previous Politech messages:
http://www.politechbot.com/2006/04/13/why-was-moveonorg/
http://www.politechbot.com/2006/04/13/aol-blocks-e/

-Declan

-------- Original Message --------
Subject: Re: [Politech] Why was Moveon.org blocked by AOL? Did 
recipients want the email messages? [sp]
Date: Fri, 14 Apr 2006 01:26:00 -0700
From: Danny O'Brien <danny@private>
Reply-To: danny@private
To: Declan McCullagh <declan@private>
CC: Politech <politech@private>, David Farber <dave@private>, 
        Suresh Ramasubramanian <suresh@private>
References: <443F210A.9040701@private>

On 4/13/06, Declan McCullagh <declan@private> wrote:
> I'm sympathetic with many of EFF's positions on spam. But it is
> reasonable to ask: (a) Is each and every address receiving alerts from
> dearaol.com confirmed double-opt in? (b) Did dearaol.com borrow lists
> from some of its member organizations like moveon.org that may have
> less-than pristine list management practices? (c) Did a human at AOL
> intentionally block dearaol.com messages because of the content of the
> mailings or was it entirely automatic because so many AOLers were
> marking the alerts as spam?
> 

I think I can answer these questions.

No-one receives any mail from any dearaol.com address. I run a Mailman list
that sends mails out to the 500+ groups in our coalition, which is run from
the EFF. It is not confirmed double opt-in in the strictest sense: instead I
and three volunteers spent a several days manually whittling down the groups
that had volunteered into 500 that I felt confident had the authority to 
join
the coalition.  There's more to signing up here than just receiving mail:
There's no point having confirmed double-opt-in when one sweet retired lady
from Texas confirms sincerely believing that she represents AARP.  I was
concerned that if we were speaking for all 500 groups, it would be better to
let them know than have them miss the confirm and be left out of the loop.

My first mail had mailman's unsubscribe details, an explanatory note from me
and my personal mobile phone number so that anyone could call if they 
had any
questions. After that, they've received around one or two mails a week,  all
with mailman links and most with my number.

(And I should point out to Suresh that if he believes lists that aren't
confirmed double opt-in are spam, he's going to have serious problems with
Goodmail's acceptable use policy, which permits single opt-in for paying
senders of CertifiedEmail where "At the point of email address collection, a
person has affirmatively requested to be included on an email list to 
receive
email.  No confirmation email is sent and the person is not required to take
further action to be included on the email list." Such single opt-in mail
will, of course, skip AOL's spam filters entirely.  From
<http://www.goodmailsystems.com/aup.pdf>)

We, of course, have no control over who else mentions an URL.  EFF does not,
of course, buy in lists. MoveOn has mentioned the site a few times in its
mailouts but then MoveOn is on AOL's whitelist (and may be on their Enhanced
Whitelist, I'm not sure), so AOL clearly believes their mailing lists are
clean enough. No-one but me has access to DearAOL.com coalition lists, in
accordance with DearAOL.com and EFF's privacy policy.

As to AOL's ban, if you want my opinion, here's what happened. This is a
little long, and much is conjecture, based on the evidence that I've 
collected
so far.

We have about 122 coalition members on the list, which is enough to trip 
AOL's
volume filters, which I understand are set at about 100 mails from a 
single IP
address.  I sent out a mailout to our coalition around noon yesterday. I 
found
out that AOL was bouncing any mail with our URL in it at around 4.45pm - one
of our coalition had mailed a friend at AOL with a note about our site, and
received a bounce.

Playing forensic scientist, I sent a mail today asking our AOL users if 
they'd
received yesterday's mail (carefully avoiding the D*arA*L.com word).  A few
had; the majority had not, which leads me to believe that the ban occurred
somewhere in the middle of the mailing run.

Ploughing through the error logs, I have found one person on the list whose
error message indicates that he does not want to receive mail from my 
address.
Whether he is simply set to only receive mail from friends or whether 
this is
a specific ban is  unclear: but he's the only indication I have that anyone
complained about the mail.

Many AOL users treat the AOL client's "spam" button, rather sensibly, as 
a "I
don't want to receive any more of this mail". I suspect this person was
unsubscribing by hitting this button.

Unfortunately, AOL's semantics are rather different: they take it as meaning
"treat this mail as suspect for everyone else".  (This is one of the 
practical
problems of having intermediaries attempt to make decisions about end-user
email delivery without adequate feedback or transparency. Fixing this 
semantic
gap is one of the ongoing challenges of fighting spam: a consistent standard
for confirm and unsubscribes may well go some way to fixing it.)

Anyway, AOL clearly doesn't view the mail as spam in a strong sense, because
they haven't banned my email address or IP. What they did, it appears, is
check out the mentioned URL.

Somehow - and this is what AOL's tech support folk told me when I called 
them
this morning - they identified www.dearaol.com as a "morpher". This is a 
site
that redirects user clicks to many different sites.

It's true: www.dearaol.com has round-robin DNS.  I plead guilty to
load-balancing of the most heinous kind.

AOL appears to have taken this as a sure-fire indication of a spamming site,
and instantly banned *every email that mentions this URL* from entering the
AOL system.

That includes, incidentally, people mailing themselves the URL. It would 
have
included Suresh's and Declan's mail too, if AOL hadn't fixed the problem
within 30 minutes of reporters calling them for a quote.

AOL's spokesman told reporters variously that that there was a software
glitch, a technical glitch, and finally a hardware glitch that affected 
dozens
of web addresses.

I find all of these hard to believe. The tech support guys I spoke to didn't
seem surprised about the ban; one said that fixing it usually takes 3-5
working days. EFF has received reports of these kind of URL bans before.
Bennett Hasselton, of the free speech group PeaceFire, has documented many
innocent groups who find all mails discussing their URLs removed from
AOLspace.

This appears to be a private AOL ban list. Goodness knows how many URLs 
or how
long they are held. I suspect if I hadn't received that mail from a 
friend, or
put out a press release, www.dearaol.com would still be banned from 20 
million
user's private communications, and would remain so until I made that call.

This is exactly the kind of overreaching, black-and-white anti-spam 
filtering
that goes on all the time among ISPs and is largely unnoticed by their
customers - for the simple reason that nobody notices a mail that never
arrives.

And that's why we're concerned about Goodmail: it rewards ISPs for such bad
filtering, because with such large problems, large companies will pay a 
great
deal to avoid those filters. And no market forces can come into play to fix
this failure to deliver when the symptoms themselves are so hard to detect.

I'm more disturbed that Suresh had a similiar block, which he finally 
deigned
to remove because he believed us to have "legitimate" popularity. Suresh's
company manages filters for over 40 million users. I'm happy that Suresh 
likes
me enough personally to let me escape his blacklists, but when advocacy
campaigns find themselves removed from the inboxes of 60 million users, and
then have to wait to permitted to step back into public debate on the 
whims of
someone judging them "legitimate" speakers, we have some serious 
questions to
ask about our mailbox providers' anti-spam strategies and the feedback 
systems
that keep them in check.

d.
_______________________________________________
Politech mailing list
Archived at http://www.politechbot.com/
Moderated by Declan McCullagh (http://www.mccullagh.org/)



This archive was generated by hypermail 2.1.3 : Fri Apr 14 2006 - 23:56:07 PDT