FC: Group releases "Friendly AI guidelines," Webmind goes bankrupt

From: Declan McCullagh (declanat_private)
Date: Thu Apr 19 2001 - 08:29:12 PDT

  • Next message: Declan McCullagh: "FC: Canning spam through contracts instead of federal regulations"

    http://www.wired.com/news/business/0,1367,43158,00.html
       
       Intelligenesis Faces Dim Future
       By Declan McCullagh (declanat_private)
       2:00 a.m. Apr. 19, 2001 PDT
       
       A pioneering New York company that once hoped to develop the first
       artificial intelligence is preparing to declare bankruptcy.
       
       Intelligenesis Corp., which was creating the Webmind software, has
       been evicted from its Broadway office suite and plans to file for
       Chapter 7 bankruptcy next week.
    
       [...]
    
    **********
    
    http://www.wired.com/news/technology/1,1282,43080,00.html
    
       Making HAL Your Pal
       by Declan McCullagh (declanat_private)
       
       2:00 a.m. Apr. 19, 2001 PDT
       
       Eliezer Yudkowsky has devoted his young life to an undeniably unusual
       pursuit: planning for what happens when computers become far smarter
       than us.
       
       Yudkowsky, a 21-year-old researcher at the Singularity Institute, has
       spent the last eight months writing an essay that's half precaution,
       half thought exercise, and entirely in earnest.
       
       This 750 KB treatise, released Wednesday, is not as much speculative
       as predictive. If a computer becomes sufficiently smart, the argument
       goes, and if it gains the ability to harm humans through
       nanotechnology or some means we don't expect, it may decide it doesn't
       need us or want us around.
       
       One solution: Unconditional "friendliness," built into the AI as
       surely as our genes are coded into us.
       
       "I've devoted my life to this," says Yudkowsky, a self-proclaimed
       "genius" who lives in Atlanta and opted out of attending high school
       and college.
       
       It's not for lack of smarts. He's a skilled, if verbose, writer and an
       avid science-fiction reader who reports he scored 1410 on his SATs,
       not far below the average score for Stanford or MIT students.
       
       Yudkowsky's reason for shunning formal education is that he believes
       the danger of unfriendly AI to be so near -- as early as tomorrow --
       that there was no time for a traditional adolescence. "If you take the
       Singularity seriously, you tend to live out your life on a shorter
       time scale," he said.
       
       Mind you, that's "Singularity" in capital letters. Even so-called
       Singularitians like Yudkowsky admit that the term has no precise
       meaning, but a commonly accepted definition is a point when human
       progress, particularly technological progress, accelerates so
       dramatically that predicting what will happen next is futile.
       
       The term appears to have been coined by John von Neumann, the great
       mathematician and computer scientist who used it not to refer to
       superhuman intelligence, but to the everyday pace of science and
       technology.
       
       Science-fiction author Vernor Vinge popularized the concept in the
       1980s, capitalizing the word and writing about whether mankind would
       approach Singularity by way of machine intelligence alone or through
       augmented mental processes. Predictions vary wildly about what happens
       at the Singularity, but the consensus seems to be that life as
       humanity currently knows it will come to a sudden end.
       
       Vinge is the closest thing Singularitians have to a thought leader,
       spokesman and hero. He offers predictions based on measures of
       technological progress such as Moore's Law, and sees the Singularity
       as arriving between 2005 and 2030 -- though some Vinge aficionados
       hope the possibility of uploading their brains into an immortal
       computer is just around the corner.
       
       One of them is Yudkowsky, who credits Vinge for turning him onto the
       Singularity at age 11. "I read True Names," he said, referring to a
       Vinge novel. "I got to page 47 and found out what I was going to be
       doing for the rest of my life."
    
       Since then, Yudkowsky has become not just someone who predicts the
       Singularity, but a committed activist trying to speed its arrival. "My
       first allegiance is to the Singularity, not humanity," he writes in
       one essay. "I don't know what the Singularity will do with us. I don't
       know whether Singularities upgrade mortal races, or disassemble us for
       spare atoms.... If it comes down to Us or Them, I'm with Them."
    
       [...]
       
       Like a character from science fiction, Yudkowsky sees his efforts as
       humanity's only hope.
       
       In an autobiographical essay, he writes: "I think my efforts could
       spell the difference between life and death for most of humanity, or
       even the difference between a Singularity and a lifeless, sterilized
       planet... I think that I can save the world, not just because I'm the
       one who happens to be making the effort, but because I'm the only one
       who can make the effort."
    
       ###
    
    [Clarification: Yudkowsky just emailed me to say he received a 1600 on his
    SATs when he took them again. --Declan]
    
    
    
    
    
    -------------------------------------------------------------------------
    POLITECH -- Declan McCullagh's politics and technology mailing list
    You may redistribute this message freely if it remains intact.
    To subscribe, visit http://www.politechbot.com/info/subscribe.html
    This message is archived at http://www.politechbot.com/
    -------------------------------------------------------------------------
    



    This archive was generated by hypermail 2b30 : Thu Apr 19 2001 - 08:34:58 PDT