Wayne, (I'm ccing this response to bugtraq). Setrlimit will not work to prevent this DoS attack. This attack works because of 2 things: Each process has a fixed amount of data that doesn't get swapped to disk. The process task structure and the page table lists don't get swapped out. Linux maintains a 3 tier page table structure for memory allocation to each process. This table has a single 4 byte integer for each entry. Page tables use a bitfield encoding mechanism of a 32 bit (or 64 bit) pointer. The problem is that when the mmap is done, the page table chain is expanded. When the munmap is done, the page_table isn't collapsed. A usage count could be maintained for the PTEs, since anything that touches them is possibly an "expensive" task. The page tables are used during the swapping process to mark which pages have been swapped out and such. A process maintains (on x86) 1 page table for every 4MB of RAM, then a lower level page table is used to represent every page in a 4MB chunk. I suppose that it's possible to swap out the page table in deep swap mode, but that has a lot of nasty things. So, you could limit the number of processes a person could use, limiting the possible effect of such a DoS. The page tables consume 768*sizeof(pte_t) bytes, assuming the program that's the subject of this discussion. I'm doing some more investigation on this. The source indicates that it should be freeing the PTEs when the munmap is called. --Perry > > > you mentioned something about using setrlimit and such to fix this > problem.. I have been screwing around with it for a while now and cannot > get it to change anything. Could you please explain how you would do it? > -- i tried setting the rlim_cur and max -- don't know what i am doing > wrong. > > Wayne > > > +-=---==- -- -=- ---=------=-=- - -=---------==- -===-- -- - --==--=+ > | NT vs Linux, why Linux: | > | It doesn't matter how big, hot, or easy your *thing* is if it | > | doesn't stay up, perform, or handle it's load. | > +---==-- - --=-- -=---- -=-------=-==-=====-- --=-- -- - --=- -===--+ > > -- Perry Harrington System Software Engineer zelur xuniL () http://www.webcom.com perry.harringtonat_private Think Blue. /\
This archive was generated by hypermail 2b30 : Fri Apr 13 2001 - 13:55:59 PDT