Using SWATCH for Forensic Analysis of VMware$DD images

From: Ryan Barnett (RCBarnettat_private)
Date: Sat Jan 12 2002 - 11:19:34 PST

  • Next message: Crow, Owen: "RE: Using SWATCH for Forensic Analysis of VMware$DD images"

    
     ('binary' encoding is not supported, stored as-is)
    I originally posted this in the Honeypot mail-list, but 
    many people suggested that I also post it here, due to 
    the relevance.
    
    >>>
    Greetings All:
    
    I took Lance Spitzner's advice to post this to the 
    Honeypot List, I hope it is of some value.
    
    I have been dabbling with using VMware as a 
    honeypot for a couple months now and I must admit, 
    it is a lot of fun.  I thought I would share some of my 
    own info with you all since I know you are all heavily 
    involved with Honeypots, Forensics, VMware, etc.
    
    Many Forensic Analysts are forced with the time 
    comsuming task of combining typical Unix 
    commands such as like "find", "strings" and "grep" to 
    quickly analyze compromised systems.  
    
    Example = "# find . -type f -print | xargs grep -i 
    [PATTERN]" 
    
    I have done this task many times and it is just a plain 
    pain in the A$$ since you can only search for a couple
    patterns at a time.  In order to speed this process up 
    considerably, there is another method that I have 
    used to examine both VMware honeypot images 
    (such as the linux.img file) or normal dd images of 
    compromised systems.  The surprising Forensic tool 
    is SWATCH - http://www.oit.ucsb.edu/~eta/swatch/.  
    SWATCH is well known in the Unix SysAdmin realms 
    to monitor system files in real-time.  Little did I know, 
    that such a powerful Forensic tool was already sitting 
    on my system.  I suggest that you read the SWATCH 
    documentation if you are unfamiliar with how it 
    functions.  If you are already familiar with SWATCH, 
    then read on...  
    
    By using SWATCH, I am able to simultaneously 
    search for many RegularExpressions all at once by 
    putting them within the SWATCH config file.  I have 
    run SWATCH (Version 3.0.4) with the "--
    examine=filename" flag to do a single pass on an 
    entire VMware image to look for suspicious info.  I am 
    still working on tweaking my swatchrc.honeypot 
    config file for malicious keywords.  This tweaking 
    involves both adding common "hacker" words and 
    the appropriate PERL syntax.  This file is used by 
    SWATCH for the alert triggers to search for in a file.  I 
    am trying to compile an extensive list of keywords for 
    the SWATCH file - such as; sniff, IRC, hack, bot, 
    trojan, etc...some of the techniques that I have used 
    for gathering keywords have been to review rootkit 
    files, using the "Hacker Jargon" file - 
    http://www.antionline.com/jargon/ and surprisingly 
    productive, to read the META information within the 
    html pages of hackersites.
    
    Anyways, using SWATCH in this manner allows me 
    to search simultaneously for many keywords that 
    would be realistically unfeasible to do commandline.  
    The only limiting factor that I have run into thus far is 
    that the "context" to which these keyword matches 
    are related is missing- I.E.- What file is this word 
    located in?  Who owns that file? What are the MAC 
    times, etc...  These are questions that must be 
    answered by a more extensive forensic analysis.  It is 
    extremely effective, however, at trowing out a big net 
    and getting some places to start an investigation.  
    This method also is useful to be run as a quick check 
    to see if there is any mischeivious behaviour 
    happening on a live VMware image.  If you get any 
    hits, then you should proceed to a more interactive 
    Forensics analysis.
    
    Here is a quick example -
    
    I setup SWATCH-3.0.4 on a Solaris host and 
    configured it with the following config 
    file "swatchrc.honeypot".  Notice the use of the "pipe" 
    command - where the output is sent into other 
    commands directly through SWATCH.  I pipe each 
    matched line into "strings" and then the output is 
    appended to the swatch_honeypot.txt file.  This is 
    very usefull since many of the hits could be from 
    binary files and this makes sure that the output will be 
    in human readable format. Additionally, the end of the 
    SWATCH REGEXP line has the "i" appended.  This 
    allows for case-insensitivity during the search, which 
    is MUCH needed when dealing with common 
    Blackhat naming conventions - Below is a partial 
    example of my swatchrc.honeypot file -
    
    ###########################################
    # SWATCH configuration file for scanning suspected 
    compromised filesystem  images
    #
    # This SWATCH Forensic Technique and Config file 
    was created by:
    # Ryan C. Barnett
    # SAPHE Security Team Lead
    # SAPHE = System Analysis to Prevent Hacker 
    Exploitation
    # Email: RCBarnettat_private
    #
    
    # The following REGEXP triggers are common 
    keywords associated with suspicious activity.
    # It is assumed that if you are using this technique 
    against a dd imaged copy of a system,
    # chances are you had some reason to suspect that 
    the system was compromised, right?!
    # This technique will help you to quickly scan an 
    image of a system to look for a large
    # number of keywords all at once.  This negates the 
    need for extremely long commandline
    # GREP searches.
    #
    # Additionally, this technique has been most effective 
    when used in conjunction with
    # VMware honeypot images.  You can scan the entire 
    linux.img file for a particular host
    # and get back results.
    #
    # Here are some examples of some keywords to get 
    you started, feel free to add to them.
    # IMPORTANT - Do NOT delete the "i" following the 
    last REGEXP match.  This needs to be
    # in place for case-insensitivity purposes.  This 
    allows for matches of say - "rootkit"
    # and "RootKit".
    #
    # If SWATCH finds a match it will echo the line to the 
    screen in bold and also append
    # the line to the selected honeypot file for analysis.
    
    watchfor   /rootkit|root kit|hack| irc | bot 
    |sniff|backdoor|back 
    door|promisc|knark|hax0r|hide|trojan|virus|TFN2K|ador
    e|LKM|attack|denial-of-service|ddos|brute force|0wn/i
            echo=bold
            pipe `strings | echo $0 
    >> /images/swatch_honeypot.txt`
    ###########################################
    
    I ran SWATCH against one of the Honeynet Projects 
    dd images (From Scan of the Month 15 - Recover a 
    deleted rootkit) - honeypot.hda8.dd.
    http://project.honeynet.org/scans/scan15/
    
    ###########################################
    # ./swatch --config-file=swatchrc.honeypot --
    examine=/images/honeynet/honeypot.hda8.dd
    
    *** swatch-3.0.4 (pid:2278) started at Thu Jan 10 
    16:58:28 EST 2002
    ###########################################
    
    I let SWATCH run for a very short time through the dd 
    image.  After I saw some of the hits that were echo'd 
    to my terminal I exited the SWATCH session and 
    examined the swatch_honeypot.txt file.  This file 
    provided many useful clues of where to begin a live 
    forensice examination!  Here is the actual 
    swatch_honeypot.txt file after running SWATCH 
    against the dd image -
    
    ###########################################
    # more swatch_honeypot.txt
    echo ********* Instalarea Rootkitului A Pornit La Drum 
    *********
    echo 3 linsniffer >>/dev/rpm
    cp linsniffer logclear sense sl2 mkxfs s ssh_host_key 
    ssh_random_seed /dev/ida/.drag-on/
    cp linsniffer logclear sense sl2 mkxfs s ssh_host_key 
    ssh_random_seed /dev/ida/..
    rm -rf linsniffer logclear sense sl2 mkxfs s 
    ssh_host_key ssh_random_seed
    ./linsniffer >> ./tcp.log &
    ./linsniffer >tcp.log &
    ###########################################
    
    As you can see, it identified some quick places to 
    start an investigation -  It looks like there are some 
    new directories ".drag-on", "/dev/ida/.." - and that 
    there was a sniffer installed, etc...  While not perfect, 
    it is still pretty powerfull.  
    
    I am also getting ready to test using SWATCH to 
    monitor VMware GuestOS files while the are acutally 
    in use.  I am not sure if this will work, but we will see.  
    I am going to use this RegExpr syntax in the 
    swatchrc file -
    
    watchfor /.*/
    
    This should catch all changes made to the linux.img 
    file for the GuestOS.  As mentioned in a great 
    VMware paper by Kurt Seifried - 
    http://www.seifried.org/security/ids/20020107-
    honeypot-vmware-basics.html, the size of the 
    VMware image files both contracts and expands 
    during operation.  The test will be if SWATCH is 
    actually able to pick up most of this activity.  I did a 
    trial run and had SWATCH monitor a normal ASCII 
    text file with this config. I then echo'd text into the file 
    and it worked, ofcourse.  I then vi'ed the file and 
    inserted text in the middle of the file.  It caught that as 
    well.  Anyways, I have no idea how this will turn out 
    with the actual VMware image in use, but I will soon 
    find out...I am sure that there will need to be massive 
    changes made to the swatchrc config file using some 
    of the other flags such as "ignore", "throttle", etc...
    
    Anyways, I hope some of my techniques will help you 
    all.  Let me know if you all would be interested in my 
    test results of this scenario, or to get a copy of my 
    entire swatchrc config file with the added Intrusion 
    Keywords.
    
    Goog Luck,
    Ryan
    
    -----------------------------------------------------------------
    This list is provided by the SecurityFocus ARIS analyzer service.
    For more information on this free incident handling, management 
    and tracking system please see: http://aris.securityfocus.com
    



    This archive was generated by hypermail 2b30 : Mon Jan 14 2002 - 03:47:52 PST