[ISN] The mysteriously persistently exploitable program explained.

From: InfoSec News (isn@private)
Date: Mon Dec 15 2003 - 03:16:10 PST

  • Next message: InfoSec News: "[ISN] Linux Advisory Watch - December 12th 2003"

    |  Linux Security: Tips, Tricks, and Hackery                       |
    |  Published by Onsight, Inc.                                      |
    |                                                                  |
    |  14-December-2003                                                |
    |  http://www.hackinglinuxexposed.com/articles/20031214.html       |
    This issue sponsored by Building Linux VPNs
    Building Linux Virtual Private Networks offers concise, step-by-step
    instructions for building VPNs based on both standard protocols
    (IPSec, SSL, SSH, PPTP) and popular Linux VPN solutions (VTun, cIPe,
    tinc). Through numerous examples and proven practices, you will gain
    important insights into choosing a VPN solution, installing and
    configuring it, setting up routing, configuring firewalls, measuring
    performance, and much more.
    For more information, visit http://www.buildinglinuxvpns.net/
    The mysteriously persistently exploitable program explained.
    By Brian Hatch
    Summary: /bin/rm doesn't mean remove, it means unlink, and it has
    security repercussions.
    In a previous article[1] I described a machine compromise that
    initially would seemed to have been impossible. A vulnerable suid
    root program, /usr/sbin/buggy, was upgraded to a non vulnerable
    version, and yet crackers still were exploiting it. In fact, even
    after the program was removed entirely, it was still being exploited.
    So, how can it be that a program can be accessed even after it's been
    removed? The problem is that it wasn't removed.
    When we access a file on disk, we do so by providing a filename for
    it. The filesystem uses filenames, which are simply entries in
    directories, to look up the file's inode. An inode is a uniq numeric
    identifier, essentially a file number, which identifies the file on
    that filesystem. You can see the inode with the -i option to ls:
    $ ls -il /bin/cat
    32689 -rwxr-xr-x   1 root   root  13912 Sep 17 06:54 /bin/cat
    That first number in the second line is the inode number, here 32689.
    Note also the third field in the line, '1'. That's the link count,
    the number of directory references to the file. Now, compare /bin/cat
    to, say, /bin/gzip:
    $ ls -il /bin/gzip
    32755 -rwxr-xr-x   4 root   root  13912 Apr 19 04:43 /bin/gzip
    Note that link count of 4 in this case. How can we find the other
    filenames that point to this same file? First, let's determine which
    filesystem /bin lives on, and then use find:
    $ cd /bin
    $ df .
    Filesystem          1K-blocks     Used Available Use% Mounted on
    /dev/hda5            10766920  8325216    1894740  82% /
    $ find / -inum 32755 -xdev -ls 2>/dev/null 
     32755   56 -rwxr-xr-x   4 root   root  13912 Apr 19 04:43 /bin/gzip
     32755   56 -rwxr-xr-x   4 root   root  13912 Apr 19 04:43 /bin/gunzip
     32755   56 -rwxr-xr-x   4 root   root  13912 Apr 19 04:43 /bin/zcat
     32755   56 -rwxr-xr-x   4 root   root  13912 Apr 19 04:43 /bin/uncompress
    Here we can see that the four files, /bin/gzip, /bin/gunzip, /bin/
    zcat, and /bin/uncompress are four different filenames, pointing to
    the same actual file on disk, inode number 32755.[2]
    We call these hard links -- when a file on disk is available via
    separate filenames. Compare these to symlinks/softlinks, which are
    separate files that point to a filename, rather than an inode:
    $ ls -il /bin/nc /bin/netcat
    32502 -rwxr-xr-x   1 root   root  19352 Oct 13 15:20 /bin/nc
    51222 lrwxrwxrwx   1 root   root      2 Oct 13 15:20 /bin/netcat -> nc
    Now anyone can make a hard link or symlink, assuming they have write
    permission to a directory. While symlinks can cross filesystem
    boundaries (they point to file names, not inodes) a hard link can
    only be created on the filesystem where the target file resides.
    Thus to create a hard link to /usr/sbin/buggy, the attacker needed to
    have write access to some directory that was on the same filesystem
    as /usr/sbin/buggy. Unfortunately, this machine was created with one
    big huge root (/) filesystem, which contained all the directories,
    including /tmp and /home.
    So what happened? The attacker created a link to /usr/sbin/buggy long
    before anyone knew it had a vulnerability:
    attacker$ cd $HOME
    attacker$ ls -il /usr/sbin/buggy
    18824 -rwsr-xr-x   1 root   root  19352 Feb  5 08:12 /usr/sbin/buggy
    attacker$ ln /usr/sbin/buggy mybuggy.hardlink
    attacker$ ls -il /usr/sbin/buggy mybuggy.hardlink
    18824 -rwsr-xr-x   2 root   root  19352 Feb  5 08:12 /usr/sbin/buggy
    18824 -rwsr-xr-x   2 root   root  19352 Feb  5 08:12 mybuggy.hardlink
    Note how the link count has changed to 2 now -- this file on disk is
    now referenced by two filenames.
    Later on, when the bug in /usr/sbin/buggy came to light, the
    administrators upgraded the package. The upgrade automatically
    deleted the old /usr/sbin/buggy and then copied the new one from the
    binary package into /usr/sbin. But when the upgrade deleted /usr/sbin
    /buggy, the kernel merely decremented the link count (from 2 to 1) of
    that inode, and didn't actually free up the disk space, because the
    file was still referenced by a different filename on the filesystem.
    The original program still existed on disk, with the original suid
    attacker$ ls -il /usr/sbin/buggy mybuggy.hardlink
    45110 -rwsr-xr-x   1 root   root  17889 Nov 29 12:13 /usr/sbin/buggy
    18824 -rwsr-xr-x   1 root   root  19352 Feb  5 08:12 mybuggy.hardlink
    That first file is the new, non-vulnerable /usr/sbin/buggy program.
    The second is the old version, which the attacker had linked, and is
    still hanging around, and which he used to gain root access
    So, how should the administrators have known that this link was still
    hanging around? There are several precautions that would have
    indicated or prevented the problematic situation:
      * Many file integrity tools include link count as one of the
        attributes that they watch for. These would have noticed the link
        count go up by one, and they could have gone searching for the
        new link.
      * If they ran suid program scanners, they would have picked up this
        'new' suid program.
      * If they'd put user-accessible directories on separate partitions,
        then the attacker would not have been able to create a hardlink
        in the first place. I prefer at a minimum to have /tmp, /var/tmp
        and /home on separate partitions for this reason. It's always
        best to have separate partitions for system use vs user use.
    Several readers had other ideas of how this could have been caused.
    I'll cover some of them in following articles, as they are all
    Congrats go to Jim Richardson of Seattle, WA for solving the contest,
    thus winning a signed copy of Hacking Linux Exposed Second Edition.
    [1] See http://www.hackinglinuxexposed.com/articles/20031111.html.
    [2] -xdev was used to make sure we didn't look for files with inode
    number 32755 on other filesystems, such as /home for example. An
    inode is uniq on a filesystem, not across separate filesystems.
    [3] Yeah, it would have made much more sense for the attacker to put
    other backdoors into the system. I'm just concentrating on this one
    hole because anything else would be extraneous.
    Brian Hatch is Chief Hacker at Onsight, Inc and author of Hacking
    Linux Exposed and Building Linux VPNs. He was pleasantly surprised
    that the contest winner was a fellow Seattlite. Besides, then he was
    able to save money on postage. Brian can be reached at
    This newsletter is distributed by Onsight, Inc.
    The list is managed with MailMan (http://www.list.org). You can
    subscribe, unsubscribe, or change your password by visiting
    http://lists.onsight.com/ or by sending email to
    Archives of this and previous newsletters are available at
    Copyright 2003, Brian Hatch.
    ISN is currently hosted by Attrition.org
    To unsubscribe email majordomo@private with 'unsubscribe isn'
    in the BODY of the mail.

    This archive was generated by hypermail 2b30 : Mon Dec 15 2003 - 06:01:40 PST