Monday, August 01, 2005

CiscoGate at DEFCON

DEFCON is the annual hackers' conference, a 3 day hacking binge in Las Vegas that follows the more corporate Black Hat Conference. DEFCON has a hard time finding venues, according to Dead Addict (Black Hat's Grand Vizier), because the hackers wreak havoc on a hotel's computer systems. This year, though, they returned to the Alexis Park Hotel, an off-strip property whose campus feel and tolerant GM accommodate the fraternity atmosphere. (The hotel had to drain its pool yesterday after someone poured enough Koolaid powder in to create a massive purple beverage.)

DEFCON, in fact, resembles a giant party, with competing sound systems and widespread intoxication, but in fact there is more going on. The general purpose is to explore unintended avenues by which anyone can "own" someone else's resource through clever hacking. Obviously, such information could be used for Evil, but you have to look at the ends here, not the means. DEFCON exposes vulnerabilities in systems all around us so that, in the long run, we can all live more secure, private lives. How far you can go to expose vulnerability is a matter of widely differing opinion. Most folks at DEFCON go pretty far--supporting the publication of vulnerabilities as well as any stolen though mostly harmless information (some go much further).

At DEFCON yesterday I attended sessions on how to hack your neigbbor's garage door and remote car key, how to slip off handcuffs, and compromise the nation's telecommunications network. I attended a press conference by Dr. Linton Wells (CIO of Defense Dept) and Robert Morris (former NSA Director) regarding cyber terrorism,and I delivered a talk titled Information Security Industry: Billions Blown on Bloopers, Blights and Blunders. (View the Notes Pages for at least some of the narration behind the slides.)

I also visited the war room where teams of hackers compete in the annual Capture the Flag contest. Nearby, the Wall of Shame displays the usernames, passwords, photos and private information like tax returns of the hundreds of "sheep" whose computers were "owned" when they were naive enough to use local WiFi waves to surf. Winn Schwartau describes the ritual in detail. Other tables of the war room focussed on cracking new security products, with cash prizes offered by the forward-thinking vendors.

The parties included the geek-fetish Black and White Ball and the exclusive 9th annual Caesar's Challenge, in which "Caesar" challenges guests to solve a hard problem between chugs. (all the coolest hackers have "handles"--sort of superhero identities--like Mudge, Raven, Agent X and Dead Addict.) Another party favorite was RF-enabled Kegbot.

But the spotlight this week was on CiscoGate. Temperatures held steady in the high-90's all week, but tempers truly flared inside as a result of Cisco's epic faux pas. As widely reported, Cisco didn't want Michael Lynn of ISS to proceed with his planned presentation of flaw in Cisco IOS that would allow hackers to control Cisco routers. So, incredibly, their attorneys descended upon ISS to compel them to supress the talk, and then descended upon Black Hat, insisting on republishing the CD's and binders without the offensive material.

Man, this was the wrong crowd from whom to hide information! Michael Lynn resigned his job minutes before his talk and then, egged on by the crowd, spilled the beans. Cisco then fired off cease-and-desist letters to supress publication of the presentation, but as you can imagine, mirror sites popped up everywhere (there is a rather comprehensive one at the defiant Cisco fired off various explanations along the way that they were simply protecting their proprietary information from being illegally disclosed, and that the vulnerability wasn't so bad after all. But I attended one session by Raven in which she pointed out that Cisco's own explanations accidentally revealed additional information about the vulnerability that facilitated generation of exploits.

Cisco failed to recognize the futility of suppressing data (especially from this group). More important than the black eye, Cisco can no longer expect security researchers to cooperate with the company if provocative findings invite litigation rather than appreciation. The attitude at DEFCON was best summed up by all the CisoGate t-shirts circulating including Raven's which simply read FUCK CISCO.

There were, by the way, lots of fun t-shirts, including my favorite: "Resistance is Futile (when <1 ohm)"

Another highlight of the conference was a dinner Bessemer hosted for the luminaries in attendance: Robert Morris (Cornell, ex-chief scientist NSA), Linton Wells (CIO, DoD), Paul Mockapetris (CSO, Nominum), Mark Maifrett (eEye), Bart Decrem (Flock), Raven, Dead Addict, Agent X, Nico Sell (DEFCON organizer), Brian Krebs (Newsweek), David Mortman (Siebel CSO), Alex Sotirov (Determina), Mike Jacobs (ex-NSA), Paul Proctor of the Bessemer-seeded company Gartner Group (though Paul still likes to say he's with Meta), Adam Shostak, Effugas (aka Dan Kaminsky) and others. Steve Wynn's new restaurant Okada peppered us with a barrage of tasty though not always identifiable morsels. The highlights were the 2001 Robert Sinskey Pinot Noir and the heavy duty air conditioning.

Throughout DEFCON I felt increasingly vulnerable to hacking. But looking back on the weekend, I feel a sense of awe and gratitude for these free-thinking geniuses who, in the long run, make our world safer. Then again, that could just be the sleep deprivation talking.


  1. Anonymous7:33 AM

    Great update on the things going on in Vegas. I also saw your quote in InformationWeek:
    "“We're never going to secure the Net if we don't air and criticize vulnerabilities,” said David Cowan, a managing partner at venture capital firm Bessemer Venture Partners. "
    I am curious: amidst the unlimited number of latent vulns in existence, what is it about the current vulnerability discovery and disclosure process that makes you believe the good guys are always finding the exact same vulnerabilities as the bad guys?
    Security through obscurity is a bad idea; security despite obscurity should be every security professional's objective and is absolutely attainable.

    Pete Lindstrom

  2. Pete,
    If I understand your question, you are asking: why not be more subtle about these vulnerabilities--perhaps the bad guys haven't found them yet, so why not keep them a little more quiet?
    Ideally that would work. But we have seen time and again that Microsoft, Cisco, Oracle and others need the public humilia--er, pressure to fix the bugs. When you tell them quietly, the vulnerabilities persist. The fundamental problem is that the vendors carry no liability and so do not really care about fixing the vulnerability unless customers demand it. If vendors, though, were legally prohibited from carving out liability from their license agreements, perhaps they would care siufficiently to respond to more discreet warnings.

  3. Anonymous6:43 AM


    I agree that there needs to be public pressure for companies such as Cisco and Microsoft to fix vulnerabilities. However, I don't necessarily see how public humiliation is going to work.

    Take this whole Lynn/Ciscogate issue. I'm pretty literate, educated, and I know how to do more with a computer than simply turn it on. However, as this thing was unfolding, I found that there was a lot of misleading and perhaps even contradictory information being published...either as newspaper/magazine articles, or as blogs. The end result is that the true, most factual thread is really hard to find, and there is a great deal of confusion...which really does more to serve the purposes of the Ciscos and the Microsofts.

    I think that the problem right now is that the resources of the loosely-knit community far outweigh the resources of companies such as Cisco and Microsoft. There are a lot more folks out there researching vulnerabilities than there are actually producing the code/products, and fixing bugs. Companies are run based on the bottom line, and they cannot dedicate resources that they do not have. Therefore they have to prioritize their resources, and the managers who do that are often under-skilled...and like the company they work for, are motivated by the bottom line.

    I can see two issues that come out of this is that the "responsible disclosure" process needs to continue to evolve. The security community itself needs to come together and have a single voice to go forward to the Microsofts and the Ciscos and say, "look, there's this vulnerability that you were told about 6 months ago, and you've done nothing to release a patch. Now, systems are being compromised via this vulnerability." This "single voice" needs to be a committee of responsible folks...I can name a few who should be approached for this...but they need to get their information from the community itself. Rather than saying "this could happen", system administrators need to receive the necessary training and develop the skills to be able to determine what happened (the "wipe and reload" mentality needs to die) so that such things can be reported accurately.

    If the companies are going to come back and say, "hey, we're not putting any effort toward fixing this b/c we're not hearing any reports of it happening...", then do what Microsoft refuses to do in their MCSE training courses and teach the sysadmins some real troubleshooting and incident response skills, so that they can go back to Microsoft and say, "yes, you were informed of this issue 6 months ago, you decided not to patch it b/c it wasn't a priority, and now my systems are compromised by a live exploit."

    Second, following on the heels of "responsible disclosure" of vulnerabilities, there needs to be a responsible disclosure of information about situations such as Mr. Lynn's. There are too many sources out there, too many voices, and the real issues are lost in the noise. Something prompted Mr. Lynn to completely disregard legal documents he signed upon starting his employment at ISS, something caused him to put his personal integrity aside for the greater good...I'm guessing that that "something" was pretty serious. For these issues to have the effect that they need to have on large, glacial organizations, they need to be clear and concise...they need to be direct in order to have an impact. Clouding these issues in irrelevant noise about federal law enforcement conspiracies is only going to have a detrimental effect.

    H. Carvey
    "Windows Forensics and Incident Recovery"

  4. Anonymous9:53 AM

    Just chiming in on behalf of the "disclosure is good" crowd. The important thing is for vendors to devote the resources to fast response to every vulnerability disclosure.

    I for one think the organizers of DefCOn blackhat let their constituents down last week. They should have hired security gaurds to exclude Cisco people from the conference all together. Ripping presentations out of conference guides is misplaced effort. Better to address the security issue than try to re-pack Pandora's Box.

  5. Hi there!

    After 2 years passed since this publication, things seems not changed actually. Maybe somebody enjoy, that we have security patches weekly. It looks like most vendors are really aware about vulnerabilities in their products. The industry even invented "vulnerabilities management" packages. Maybe somebody thinks that this is good news. But actually, this is a bad news. This just shows, that attackers are one step in front of us.

    The industry definitely need fresh ideas to combat this.

    I cannot resist the temptation of selfpromotion :-).
    Here is a good news: I think we have a solution, that can deal with all network level vulnerabilities (known and unknown ones), at least on server side.