Re: Compaq Alpha Bounds Checking

From: Solar Designer (solarat_private)
Date: Wed Oct 20 1999 - 19:51:47 PDT

  • Next message: Antonomasia: "Re: [Re: xmonisdn (isdn4k-utils/Linux) bug report]"

    > In this post below to the Linux security-audit mailing list, Solar was kind
    > enough to fulfill my request for performance data on the Compaq ccc compiler
    > for Linux/Alpha using bounds checking.  Astonishingly, Solar's tests showed
    > virtually no performance overhead for bounds checking.  I found this to be
    > both amazing and depressing for StackGuard, and went away to sulk :-)
    
    Sorry for not answering your last post for that long -- it's still in
    my mailbox, and was going to be answered once I have the time.
    
    > Today, I got my own access to an Linux/Alpha box with ccc, and to a Tru64
    > box.  Both support the "-check_bounds" switch.  I did my own testing, and
    > discovered that as far as I can tell, "-check_bounds" does NOTHING AT ALL.
    > Am I missing something?
    
    Yes, I guess so -- see below.
    
    >      foo() {
    >              char x[50];
    >
    >              gets(x);
    >      }
    
    I would _not_ expect this case to be covered by the compiler's bounds
    checking.  This is in fact the reason I didn't use a strcpy() when
    demonstrating the bounds checking to you in my first post about ccc.
    Their bounds checking only applies to explicit array subscripts:
    
           -[no]check_bounds
               Generates  runtime  code  to check the values of array
               subscripts (and equivalent pointer arithmetic  involv-
               ing pointers produced by converting an array name to a
               pointer) to verify that  the  resulting  address  lies
               within  the  range  for  which the C standard requires
               well-defined behavior.
    
    This was so obvious for me that I forgot to mention this on the list,
    sorry.  Now I realize that when saying "bounds checking" people often
    mean "complete protection", or close to that (with DoS in mind).
    
    Speaking of the usage of gets() and such, even if the compiler was
    able to pass bounds checking information down to functions (which ccc
    doesn't do), it would at least require that you also recompile those
    functions themselves.
    
    > Thus I conclude that Solar's amazing performance results that show no
    > overhead are because the compiler is lying about implementing bounds
    > checking.  There is no overhead because there is no protection.
    
    Well, they could be more verbose in their description, yes.  As for
    the "no protection" -- this wasn't meant as a security feature, but
    there's _some_ protection, it's just far from being complete.
    
    Finally, as this also goes to BugTraq this time, here's a piece of my
    first post on the subject that shows a case where bounds checking can
    work (and does indeed work) --
    
    [ghost@alice tests]$ cat bounds.c
    #include <stdlib.h>
    
    int f(int n, int m)
    {
            char buffer[100];
            int i;
    
            for (i = 0; i < n; i++)
                    buffer[i] = 'x';
    
            return buffer[m];
    }
    
    int main(int argc, char **argv)
    {
            return f(atoi(argv[1]), atoi(argv[2]));
    }
    [ghost@alice tests]$ gcc bounds.c -o bounds -O -s
    [ghost@alice tests]$ ./bounds 1111 33
    Segmentation fault
    [ghost@alice tests]$ ./bounds 99 33
    [ghost@alice tests]$ ccc bounds.c -o bounds -O
    [ghost@alice tests]$ ./bounds 1111 33
    Segmentation fault
    [ghost@alice tests]$ ccc bounds.c -o bounds -O -check_bounds
    [ghost@alice tests]$ ./bounds 1111 33
    Trace/breakpoint trap
    [ghost@alice tests]$ ./bounds 99 3333
    Trace/breakpoint trap
    [ghost@alice tests]$ ./bounds
    Segmentation fault
    [ghost@alice tests]$ ./bounds 99 33
    [ghost@alice tests]$
    
    The first two compiles are with gcc and ccc w/o bounds checking.  We
    get segfaults.  Then the program is recompiled with bounds checking,
    and we're now getting those traps (just like the man page says).  The
    last two tests are to show that the traps are only generated from
    bounds checking and not other errors, and that the program is still
    working.  BTW, here's what the checks look like:
    
            mov     $3, $5
            cmpule  $3, 99, $16
            bne     $16, L$10
            mov     -18, $16
            call_pal 0xAA # gentrap
    L$10:
    [ ... some code skipped: the loop got unrolled and large ... ]
            addq    $sp, $5, $8
            ldq_u   $16, ($8)
    
    I wouldn't say that the option did "nothing at all" to SSH -- it must
    have added quite a few checks, which made the binary 5 KB larger.
    
    Signed,
    Solar Designer
    



    This archive was generated by hypermail 2b30 : Fri Apr 13 2001 - 15:08:19 PDT