All, I've just caught up on this thread and its been very interesting to read, especially since we just recently started a discussion on the osstmm discussion list with regard to defining the following terms: security test, security assessment, security audit, penetration test, vulnerability assessment. Personally, I feel that these terms are used incorrectly most of the time, and in attempt to capture their nuances I made a crack shot at definitions. Note that my attempt at definitions were without the benefit of several weeks of pen-test list discussion--which I sorely regret not previously absorbing. Granted, there's plenty of room to simply argue that this is all just an exercise in semantics and my definitions posted to the osstmm discussion list may just cloud the issue (that's not my intention I assure you)--they may be considered way off in fact or flat out wrong. I'm willing to hear those opinions (even flames), in the hopes it will better shape the osstmm. Following is the relevant post which I feel speaks to a search for the correct type of testing at the correct time. Sincerely, Don -------- Original Message -------- Subject: Definitions Re: [OSSTMM-Discussion] new tests and stuff Date: Thu, 13 Sep 2001 14:03:53 -0400 From: Don Bailey <baileydlat_private> Organization: The MITRE Corporation To: peteat_private CC: OSSTMM Discussion <ospentest-discussionat_private>,OSSTMM News <ospentest-newsat_private> References: <FEENJDOHIDLOEIFPBFCMMEAICEAA.peteat_private> pete wrote: > <snip side thread on steganography> > I would also like to clarify some definitions for the industry. Perhaps I've lost some sleep over this already, so I'll start taking a crack at it... hang on! > security test > security assessment > security audit First of all, I see these three as supersets of the other two. Also, as these three are listed, I see them as nearly the same but progressively more intense with varying degrees of relevance with regard to policy, officiality, etc. Therefore, it IS important to define each seperately and not just clump them together with some phrase such as "Oh...it's all the same thing.", which is what I hear too often. Security Test: A security test would be a routine and general test of an organization's network security mechanisms, from outside in, to obtain a basic and generally accurate idea of how well the organization has implemented said security mechanisms. No prior warning is given to any employees. The test may be performed "in house" with existing networking personnel. Results from a security test generally would be used to make functional network "tweaks" to remedy any unexpected problems discovered and bring the network back to spec. Security Assessment: A security assessment is an intensified security test in scope and effort, the purpose of which is to obtain an advanced and very accurate idea of how well the organization has implemented network security mechanisms and to some degree policy (such as spot tests of password strengths or acceptable allowed services). No prior warning is given to non-critical employees. Outside technical assistance may be necessary to handle the workload and should be seriously considered. The results of a security assessment may be surprising or unexpected and would be used to make significant changes to both network implementations and policy. Security Audit: A security audit would be an extreme security test, definitely handled by an outside and impartial source, that performs a ground up, and outside in, audit of the organization's network security mechanisms and all pages of security policy. Audit implies finding non-compliance with policy. All employees are informed well ahead of time in order to meet compliance with policy. They are interviewed by auditers with regard to their knowledge of policy and their personal level of compliance. The results of a security audit should NOT be surprising, SHOULD only validate existing implementations with regard to known policy, and violations are to be taken very seriously, to include possible termination of employment of individuals in direct violation of policy or responsible for sections found in direct violation of policy. Extraneous results that are not covered by existing policy should be addressed individually and considered for future policy changes and security assessments. Note that these definitions may incorporate the use of the word "network" and I apologise if I focused on that a bit more than I should have, but that's my realm, so I'm biased. Perhaps these definitions can be sanitized by the removal of the word "network" to be made more general yet accurate and for specific "network" related actions we can say "network security test", "network security assessment", and "network security audit". These latter definitions extremely limit scope... which may actually be preferred in some instances. > penetration test Penetration Test: A penetration test is a no-holds barred, outside to inside, get in any way and as many ways as you can, test of an organizations physical, network, and human facets of security. This is Red Teaming. Non-critical employees may or may not be notified in advance of a pen-test. A Blue Team may or may not be involved for active defense. A White Team may or may not be involved to referee the event. Rules of engagement are defined ahead of time for at least the Red Team and by individuals authorizing the activity. Specific exclusions may be introduced to the pen-test, such as no testing of physical or human facets of the organization's security (i.e. no lock-picking and no social engineering). Variations on this model of testing may be "capture the flag" in which the Red Team has but one type of document to find or alter--a proof of concept crack attempt, if you will--in order to be successful or call and end to the event. Also "cry uncle" is common, in which an onslaught of near-destructive activity to an organisation is so great as to test the limits of the Blue Team's response capabilities, demonstrate for CEOs or corporate-types how bad it could get, until someone in authority simply says, "that's enough... we get the point." Some or all of these variations may occur during a pen-test. Results of a pen-test are almost always surprising and unfair, but are significant in helping to reshape policy, highlight the significance of consistently present security flaws, discover previously unknown weaknesses, as well as testing the resolve and ingenuity of Blue Team members in the organization. Pen-test is commonly but incorrectly used synonymously with "security test", "security audit", "vulnerability assessment", et al. It is closely related to a "security assessment" but a penetration test is a very different and distinct variation of the security assessment and should be recognized as such. > vulnerability assessment Vulnerabiltiy Assessment: A vulnerability assessment is very regulated, controlled, cooperative, and documented evaluation of an organization's network security posture from both outside-in and inside-out, for the purpose of defining or greatly enhancing security policy, and determining the need or removal of security products / implementations. Non-critical employees are not included in a vulnerability assessment. "Defense-in-Depth" will be the phrase du jour during this event, and all types of corporate individuals will be included to participate in discussions with regard to the neccessity of security versus the need for functionality and productivity. The techies are often involved with or responsible for product evaluation, providing their results to analysts that write recommendation reports. Corporate politics are involved and many sub-organizations may prove defensive or outraged with regard to noted deficiencies or recommended security implementations. Outside technical assistance with evaluating the organization's security posture is recommended, but be wary of consultants that bring their product to the table as THE solution to the results of a vulnerability assessment. Tasks may include offline testing or evaluation of existing or anticipated security products, network profiling to determine critical assets or subnets, independent code review of implemented scripts and software, and security budget analysis. A vulnerability assessment may take months to nearly a year to complete, and the focus should be on completing an unbiased and scientific evaluation of the organization's security posture with regard to its current and near-future models of real-world operation. Results of a vulnerability assessment should be the definition or ehancement of an organization's security policy, and implementation plan for new security products & measures to mitigate defined vulnerabilities and/or the removal of ineffective products & measures. A vulnerability assessment is commonly combined, brought on by, or followed by a security assessment but does not require such an activity in order to be conducted, complete, or successful. Crap. That's my first shot at defining these. There's a good bit of the wording I'm still not comfortable with, but I think I got my general ideas on the table. Have fun hacking / editing this. I hope this begins some fruitful dialogue with regard to locking down these definitions. --end-- H C wrote: > <snip> > fruition without being detected. Security consulting > firms should have this as their goal, as well, with > respect to their clients. This being said, what has > been referred to as a "blind pen test" quickly drops > out of the picture all together as a method of > reaching this goal. A vulnerability assessment of the > overall infrastructure examines the configurations of > hosts within that infrastructure, the relationship > between the hosts, and the processes and procedures > used by the admins. The assessment gets into every > nook and cranny and peeks into the deep, dark corners. > Verification testing (ie, "full disclosure pen test") > can be done once recommended changes have been put in > place. > > Attempting to break in blindly using no more > information than a domain name is not something that > can be completed in a week or two for larger > infrastructures, and leaves many items unchecked. > However, a "blind pen test" can be used at a later > date to test the effectiveness of detection, as well > as incident response procedures. At that point, > conducting such a test with full knowledge of the > infrastructure would definitely be very beneficial. > > Thanks for your time. Thoughts/comments appreciated. -- Don Bailey Senior INFOSEC Engineer/Scientist Secure Information Technology The MITRE Corporation ---------------------------------------------------------------------------- This list is provided by the SecurityFocus Security Intelligence Alert (SIA) Service. For more information on SecurityFocus' SIA service which automatically alerts you to the latest security vulnerabilities please see: https://alerts.securityfocus.com/
This archive was generated by hypermail 2b30 : Fri Sep 14 2001 - 12:20:37 PDT