> > >"Computers may produce different results based on different assumptions of >programmers, however, which can only be directly determined by looking at >the source code." > >While looking at source code certainly *can* provide a complete picture of >what a program does, I think we've all (for the set of us who've dealt >with large codebases) seen a lot of code where it doesn't necessarily >help. I'll also throw out another thing- it's perfectly valid to validate >function by giving known input and comparing anticipated output as long as >the scope of the expected behaviour is tested. I am not suggesting that black box testing is valueless when it comes to validating the function reliability, and clearly judicial efficiency would not be served by insisting on an 'Inspector 12' of the computing base, debugger, and object code verification. My advocacy for open code review is based on notions of transparency, repeatability, falsifiability, and error rates - which are standards that courts use to judge the reliability of technical and scientific evidence. The key assumption underlying your comment is: "...as long as the scope of the expected behavior is tested." That said, would you agree that black box testing of complex software, say IDS, given the inability (at least as our methods currently allow us) to test all the possible input/output variables in a network environment, poses a challenge to validating the function ... as compared to verifying with open code? As for testability, can one determine if there are portions of code that have not been executed (thus, potentially affecting the evidentiary outcome) in the case of black box software? >Surely though, there's a threshold of normalcy that, absent other >evidence, should stand as true. For instance, I know that the creation >and last accessed times on my disk drive are changable, but absent >evidence that they have been, each instance of a file timestamp shouldn't >be a mini-challenge waiting to happen. Now, certainly part of this may be >that the analysis and more importantly, standards of analysis might need >to be professionally done or created. That 'threshold of normalcy' goes to the issue of presumptions that are the default settings, if you will, and subject to rebuttal by the opponent to the evidence in question. And, yes, I agree that we can't split hairs on every issue raised by the mutability of digital evidence just because there is a "possibility" that timestamps may have been altered, for instance. If we were to go down that road, the "unknown 3rd party" defense of "isn't it POSSIBLE that, because my client's computer was connected to the Internet, a hacker got on his system and planted everything" would reign supreme and prosecutors/plaintiffs would be forced down a road of having to prove a negative. Your point is well taken, and it boils down to the notion of defining reasonableness in a context that many judges/attorneys/laypersons have little basis upon which to judge as compared to familiar, physical world settings. >If there's a complete and comprehensive explaination- does that provide >enough protection? It seems to me that such an explaination doesn't >affect the integrity of the process, which is the thing that *should* be >the issue, so perhaps you can shed some light on the rationale from a >lawyers perspective of why this is necessary- I certainly don't need to >produce a complete and comprehensive explaination of an internal >combustion engine in a hit and run accident (IOW, is this an artifiact of >the court and the defense just needing to understand the technology, or is >there a fundamental need of the prosecution to understand why a particular >record was generated?) Absolutely, multiple streams of corroborating evidence is the means by which integrity is inferred in many cases. And yes, there is a degree of education and becoming comfortable with the technology that is evolving ...... from a social standpoint, our digital gauges are still forming, so to speak. >It's my guess that a lot of precedent was set before we had farily >widespread computer evidence technicians- can you perhaps enligten us on >where that puts the courts in terms of admissability and veracity? There's actually a very interesting issue brewing along the lines of testimony by forensic examiners and those employing forensic analysis software. That is to say, should these persons be allowed to take the stand as experts or fact witnesses/technicians? The dangerous slope we're heading toward is what I call "automating experts", and it is partially a result of software being engineered to be usable by less-than-computer-literate folks. I'm all for improving human-computer interface and increasing functionality, but don't call a point-and-clicker an expert, especially if the opinion that contributes to whether someone gets sent down the river or has to pay out $20M is a regurgitation of what a software program told him/her, exclusively. >That's only true of centralized log servers- which certainly have some >issues- does that make a forensics friendly environment one which lots >both locally and centrally? Absolutely.... the more, independently and/or corroborating sources, the more easy it is to overcome evidentiary challenges. >I'm not sure this is always true- having done a fair ammount of >investigation in cases where the logs were most of what was there, but the >suspect's admission of performing the act corroberated them, ommission and >decision points aren't all that important compared to actual data points- >the fact that 127.0.0.2 connected to the SSH port at 12:00 EST is a good >indicator even if the IDENT record wasn't captured by a remote log >server because of the SYN flood the attacker started in conjunction with >the attack. > >In other words, while sometimes it's possible that dropping the record >that says "127.0.0.3" connected next, I think we shouldn't then >automatically dismiss the logged evidence as hearsay and not admit it, but >we should rely on expert analysis of the logs to help seperate the >difference between possible, probable and factual. I think you made my point nicely, as it pertains to the importance of context and establishing multiple streams of corroborating evidence. Digital evidence must be taken in context, which includes other digital and non-digital sources of proof. And definitely, the expert opinion is crucial to translating this binary data within the context from which it came into conclusions regarding an ultimate issue.... as I said before, we can do alot with IT, but we have yet to be able to automate expertise. >I'm sure I'll get clubbed down if we're veering too far from the >technical, but I'd appreciate more of your insight, as these issues are >near and dear to one of my roles :) > >I think it'd be nice to see the FRE have a "computer record" section, >rather than trying to lump them all into the buckets we all started out >with. There's a fine balance between falsified evidence and crimnals >escaping justice that needs to be balanced. If we can address even >some of that in the logging technology, we'll all be better off. Agree..... this is an issue worthy of paying attention to. If you get clubbed for veering too far from the technical, I may not have much time left based on my enclosed short response.......... in any case, keep it coming. Thanks- Erin _______________________________________________ LogAnalysis mailing list LogAnalysisat_private http://lists.shmoo.com/mailman/listinfo/loganalysis
This archive was generated by hypermail 2b30 : Thu Dec 12 2002 - 19:30:43 PST