[ISN] The Upgrade Process: Restarting vs Rebooting.

From: InfoSec News (isnat_private)
Date: Thu Apr 10 2003 - 00:24:44 PDT

  • Next message: InfoSec News: "[ISN] More problems with SpamCop, go figure."

    +------------------------------------------------------------------+
    |  Linux Security: Tips, Tricks, and Hackery                       |
    |  Published by Onsight, Inc.                                      |
    |                                                                  |
    |  09-April-2003                                                   |
    |  http://www.hackinglinuxexposed.com/articles/20030409.html       |
    +------------------------------------------------------------------+
    
    This issue sponsored by Linuxfest Northwest 2003, Bellingham, WA,
    April 26
    
    LFNW is a showcase for what Northwest Linux users are doing with
    Linux and open source software. It's a place for Linux enthusiasts to
    get together to share their passion for what good software can do.
    This year's fest will include boatloads of demonstrations, tutorials,
    and lectures by gurus such as John "Mad Dog" Hall, Randal Schwartz,
    and your very own Linux Security geek, Brian Hatch.
    
    Admission is free. For more information, see http://
    www.linuxnorthwest.com. Come join the fun!
    
    --------------------------------------------------------------------
    
    The Upgrade Process: Restarting vs Rebooting.
    By Brian Hatch
    
    Summary: Upgrading your software is a constant task. But when does it
    require a reboot, and when can you get by without?
    
    Every operating system in the world requires software updates from
    time to time. If you're used to the world of Windows, you have
    probably become accustomed to a simple fact of windows life: you need
    to reboot your machine after any system change, such as upgrading
    software, installing patches, installing new software, moving your
    mouse, and so on. In the Linux world, this is not the case. You can
    keep your machine running for months or years without rebooting as
    long as you properly handle software upgrades.
    
    There are four main types of software upgrades that you could
    require. I'll point out the actions you should take after performing
    an upgrade to be sure that your machine is running smoothly and
    you're not accidentally running old versions.
    
    User-level software
    
    User level software are your standard programs in places like /bin
    and /usr/bin that are executed from the command line interactively,
    from system scripts, cron jobs, you name it. Things like awk, emacs,
    grep, mutt, passwd, perl, ping, vi, and so on. These programs are run
    for short periods of time, rather than hanging around forever like a
    webserver.
    
    These programs, since they are not running continually, do not
    require any reboot of the machine after an upgrade. The next time
    they're called, the new version will automatically be run. On the oft
    chance you have extremely long lived processes (say you've had mutt
    running on a screen session since the beginning of time) then you
    should stop and restart them, but this is rare.
    
    Daemon software
    
    Daemon software is anything that runs continually unattended, such as
    a webserver, mail server, database server, network time server, log
    analyser, etc. Often you can identify these by looking at their
    parent process ID in ps output - it will usually be '1', the init
    process. For example:
    
      $ ps -fC syslog-ng
      UID        PID  PPID  C STIME TTY          TIME CMD
      root       221     1  0 Jun23 ?        00:04:53 /sbin/syslog-ng
    
    These processes are usually controlled by a script in /etc/init.d.
    After you upgrade this sort of software, you should stop and restart
    the daemon. For example
    
      # /etc/init.d/syslog-ng restart
      # /etc/init.d/apache restart
    
      or
    
      # /etc/init.d/ntpd stop
      # /etc/init.d/ntpd start
    
    Doing this after an upgrade assures your daemon is the newly upgraded
    version. The service will be down for a brief moment while it's
    restarting, but that's small change compared to the time required for
    a reboot of the computer.
    
    Inetd-style Daemon software
    
    Some daemon software, particularly network daemons, is started by
    super servers like inetd, xinetd, or Bernstein's tcpserver. Each time
    a client connects to one of these services, the super server will
    accept the inbound connection and launch a program to handle it, for
    example an IMAP or POP server. These programs are only running for
    the time that the connection is active, after which they stop. For
    example, here's a snippet of ps output which shows a POP server
    that's controlled by xinetd:
    
      UID        PID  PPID  C STIME TTY          TIME CMD
      root      1128     1  0 Jun23 ?        00:00:25 xinetd -stayalive -reuse -pidfil
      ryan     10169  1128  1 12:30 ?        00:00:01 [ipop3d]
      reegen   32015  1128  1 12:14 ?        00:00:10 [ipop3d]
      maddie   32015  1128  1 12:31 ?        00:00:05 [ipop3d]
      chris    32015  1128  1 12:28 ?        00:00:03 [ipop3d]
    
    All the [ipop3d] processes were started by xinetd and when they are
    done servicing the request, they will terminate.
    
    If you were to upgrade the ipop3d software, the processes that are
    still running would be the vulnerable version, but all new
    connections will be the bug free version. To stop the buggy versions
    rudely, you could simply
    
      # killall ipop3d
    
    This will stop the old version, and the clients would need to
    reconnect. Again, no reboot is needed.
    
    Libraries
    
    A library is a file that contains compiled code that can be used by
    more than one program. Most Linux programs are compiled to use shared
    libraries, as opposed to being compiled 'statically', which would
    mean that all the code is contained in the binary itself. Using
    existing libraries is good - it means that should a bug be found in
    the implementation of printf, the library needs to be upgraded, but
    the rest of the software doesn't need to be recompiled.
    
    Different Linux distributions may handle library upgrades slightly
    differently, however they all do share one common feature: the
    existing library is deleted, and the new one is installed. The
    problem is that any program that is running has already mapped the
    old buggy library, so it keeps using the old one, even though the new
    one is available.
    
    Let's take a look at this. Say on my Debian system I have slapd (the
    OpenLDAP server) running. Turns out, there's an update for the ldap
    shared libraries -- the libldap2[1] package -- that I need to apply.
    
      $ dpkg -L libldap2 
      /usr/lib/libldap.so.2.0.15
      /usr/lib/libldap_r.so.2.0.15
      /usr/lib/libldap.so.2
      /usr/lib/libldap_r.so.2
    
    So, it looks like the current (buggy) version has files named /usr/
    lib/libldap*. Let's see what programs are using these:
    
      $ lsof /usr/lib/libldap*
      COMMAND   PID  USER  FD TYPE DEVICE   SIZE    NODE NAME
      slapd   29401  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
      slapd   29402  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
      slapd   29403  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
      slapd   29445  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
    
    Ok, slapd is using those libs, unsurprisingly. Let's upgrade and see
    what happens afterwards:
    
      
      # apt-get install libldap2
      ... upgraded package installs ...
    
      $ lsof /usr/lib/libldap*
      COMMAND   PID  USER  FD TYPE DEVICE SIZE   NODE NAME
      slapd   29401  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
      slapd   29402  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
      slapd   29403  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
      slapd   29445  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
    
    What happened here? The way the Debian upgrade worked was to rename
    the old libraries to oldname.dpkg-new, then install the new versions
    and delete the old ones. Slapd was already using the old ones, and
    thus those files are still available to it, even though you can't see
    them in the directory itself -- notice "DEL" in the "TYPE" column,
    and the empty "SIZE" column.[2].
    
    In order to have slapd start using the new libraries you should --
    you guessed it -- restart slapd
    
      # /etc/init.d/slapd restart
      
    
    Again, no reboot necessary, but there will be momentary downtime as
    the program restarts.
    
    The Kernel
    
    So you want to upgrade the kernel to patch the most recent ptrace
    bug, for example? Great idea. Does this require a reboot? You betcha.
    No two ways about it, to run a new kernel you need to reboot. Sorry
    for the bad news.
    
    Summary
    
    So, when do you need to reboot? Technically, the only time you need
    to reboot is if you're installing a new kernel. Any other process can
    be stopped and restarted without much worry.[3] However you may still
    want to schedule a reboot for a convenient time anyway, perhaps at
    3am on a weekend. This allows you to test any new changes and make
    sure that the computer comes up again properly, and have time to fix
    it when it's not much needed. You'd much rather test then, than have
    a power outage take down your machine while you're on vacation and
    find that it won't start up your software properly.
    
    NOTES:
    
    [1] I've taken the liberty of trimming out some of the extraneous
    files to make this explanation cleaner. There are actually more files
    in the libldap2 package.
    
    [2] The Linux kernel won't actually remove a file from disk until all
    files using it close the file, and all hard links to the file on disk
    are removed from the filesystem. You might want to check out a
    previous article about finding deleted files at http://
    www.hackinglinuxexposed.com/articles/20020507.html
    
    [3] Ok, /sbin/init can't really be restarted, but I don't expect
    we'll see an upgrade needed for it, it's pretty rock solid.
    
                                -------------                            
    Brian Hatch is Chief Hacker at Onsight, Inc and author of Hacking
    Linux Exposed and Building Linux VPNs. He changes his bio in these
    newsletters every week, and wonders if anyone notices. Brian can be
    reached at brianat_private
    
    --------------------------------------------------------------------
    This newsletter is distributed by Onsight, Inc.
    
    The list is managed with MailMan (http://www.list.org). You can
    subscribe, unsubscribe, or change your password by visiting
    http://lists.onsight.com/ or by sending email to
    linux_security-requestat_private
    
    Archives of this and previous newsletters are available at
    http://www.hackinglinuxexposed.com/articles/
    
    --------------------------------------------------------------------
    
    Copyright 2003, Brian Hatch.
    
    
    
    -
    ISN is currently hosted by Attrition.org
    
    To unsubscribe email majordomoat_private with 'unsubscribe isn'
    in the BODY of the mail.
    



    This archive was generated by hypermail 2b30 : Thu Apr 10 2003 - 02:51:41 PDT