The Risk I/O Blog

Last week I had the opportunity to present at the Best Practices for Technology Symposium. I have to be honest, I’ve never heard of this event and given the name it’s easily missed. In fact, given my recent post on “best practices” and vanity metrics I would have likely avoided an event with such a name. But that would have been a mistake.

Gene Kim introduced me to Fred Palmer who runs this event which is why I seriously considered it. It turns out it’s nothing like what I thought but rather more than two days of emerging technology companies presenting some of their latest tech. I only wish I knew about this event earlier and I wish that I’d known the format was so open. It’s refreshing to hear an audience that actually wants to talk about whether or not the solution you’re pitching works for them rather than the thinly veiled sales pitch cloaked as thought leadership. Having been a practitioner the majority of my career, I think this format is sorely needed. Fred has done a great job bringing together interesting security technologies and providing open and honest feedback. Much like IANS, it has a great workshop format in an informal setting.

One question during the workshop really resonated with me:

Do you think organizations know what’s important to them?

Of course, the sad truth is “it depends.” We were talking about creating a platform like Risk I/O that was created with flexibility in mind. This allows users to slice the data into views that are important to them in order to get better and faster insight. But this is a valid question. What if the organization isn’t sure where to start or what should be important to them? We like to think our priority and trending along with the Heads Up Display are a great start but we will continue to help our customers by flagging and alerting on issues we see as important while maintaining transparency and flexibility. These go well beyond the standard CVSS calculators and take into account real in-the-wild information.

A common question we get is “What are the metrics others are using to measure themselves against?” We will continue to share important metrics to help teams jump-start their programs. It’s great to see practitioners getting together and sharing information that will benefit and raise the bar, and we’ll continue on our mission in helping you gain visibility into what’s important.

After recently reading a post from Gary McGraw at Cigital arguing for software security training, I became a bit frustrated with cited “evidence” and posted this out on Twitter and received a short follow up from Lindsey Smith over at Tripwire…

Now let me say upfront, I have a lot of respect for Gary and his work AND actually agree with him on the subject of software security training. I’ll get into the why I agree with him in a bit. That said, here’s where my frustration comes in. Gary references the BSIMM as evidence that software security training works. Evidence? I find the BSIMM interesting but it leaves the taste of vanity metrics in my mouth. For those of you not familiar with the term vanity metrics, Eric Ries talks about them a lot as part of The Lean Startup:

“Actionable metrics can lead to informed business decisions and subsequent action. These are in contrast to “vanity metrics” – measurements that give “the rosiest picture possible” but do not accurately reflect the key drivers of a business. Vanity metrics for one company may be actionable metrics for another. For example, a company specializing in creating web-based dashboards for financial markets might view the number of web page views per person as a vanity metric as their revenue is not based on number of page views. However, an online magazine with advertising would view web page views as a key metric as page views as directly correlated to revenue.”

I think the BSIMM and best practices within information security often fall under the definition of vanity metrics. There are things I like about the BSIMM and it’s a great start but only focuses on one half of the data. Telling me what many companies are doing for their security controls becomes a lot more interesting when you also tell me how those controls faired over time. I would love to see the BSIMM and other models like it evolve into an evidence-based set of controls. Today, they certainly should not be cited as evidence that any control within them works as we’re completely missing that side of the picture. This is also not a post to pick on BSIMM but rather an attempt to call out our industry citing best practices without evidence.

I mentioned earlier in this post that I actually agree with Gary on software security training. The reason I can say this is based on evidence, not best practices. At my former employer, we implemented a number of measurements around application defects and specifically security defects. We also did various software security training exercises both internally and with help. As part of this we measured things like defect rates and density within specific groups both before and after. We continued these measurements over time and saw material drops in most categories. Was it completely due to training? No, but we saw a measurable impact each time that correlated with a specific set of training. It’s evidence similar to that I’d like to see combined with a set of “Best Practices.” At best, best practices are a set of things that others *may* be doing; at worst they are meaningless vanity metrics.

The Risk I/O dev team has been developing features at a ridiculous pace with no signs of slowing down. We will be releasing a host of new functionality to our vulnerability intelligence platform over the weeks to come, so stay tuned. Our latest additions will help you identify patches that will reduce the most amount of risk across your environment and quickly push them out to your ticketing system or manage remediation directly within Risk I/O.

We know that identifying remediation can be tedious, so we set out to solve this in a simple way. Patch data is pulled in directly from multiple sources and now made available via patch reports on your Assets tab. Viewing the patch report will give you a quick view of un-patched systems grouped together by patch and sorted in order of total risk score. This view gives you the biggest “bang-for-your-buck,” (in other words reducing the most amount of vulnerabilities with the least amount of work).

View the patches that reduce the most risk via our patch reports.

Creating trouble tickets in Risk I/O has always been fast and simple through our integration with ticketing systems. But we’ve now made this even faster by adding bulk creation of tickets in Risk I/O for both vulnerabilities and assets. With a quick search and select, you can create tickets for hundreds of vulnerabilities and assets in a matter of seconds. Within the vulnerabilities tab you can send multiple vulnerabilities to a single ticket via the bulk ticketing feature, or within the patch report you can create a single ticket to patch thousands of assets at once.

Send multiple vulnerabilities or patch multiple assets with a single ticket.

Log into your account now and try these features for yourself. Of course, if you don’t have a Risk I/O account yet, you can signup for free.

Well the dust has finally begun to settle after another whirlwind week of activity around the RSA Conference. As in years past, my favorite track turned out to be the hallway track, although admittedly I didn’t get to see many of the talks and avoided the show floor most of the time.

One program I was able to not only join but also participate in was e10+ put on by the Securosis team. This is the second time I’ve been and really like the format. Rather than having someone talk at you followed by a short Q&A, it tends to be a more participatory format where all attendees are engaged and contributed throughout. If you’ve been in Security for a while (at least 10 years) I’d definitely recommend it. I enjoyed our panel discussion about the grass being greener and browner running infosec for both small companies and large enterprises.

On Monday afternoon I gave an updated talk on the Security Mendoza line at BSidesSF. Not realizing all the drama that was about to follow my talk, I obliviously enjoyed the conference and hanging out with everyone. I also caught talks by Andrew Hay on cloud forensics and Brett Hardin talking about penetration testing (and why it sucks). I was a bit worried about timing given the handcuff competition running a bit over but was pleasantly surprised at the engaged Q&A following the talk. Clearly a lot of smart people in the room thinking about this problem. I believe BSidesSF will be posting the talks online and some follow up interviews will also be made available via BrightTalk. I’ll update this post once available but I’m also embedding my slides below.

Outside of the many meetings, events, and parties, the week was wrapped up by Metricon. Having attended several in the past I was bummed I wasn’t able to make this one, although admittedly I was spent by Friday. Fortunately our own Michael Roytman attended and took great notes! Metricon was a different format this year including workshops like groups and lightning talks. Michael wrote up a blog post recently on using game theory to solve infosec problems. Within the post he references a paper that does a good job displaying why network topology isn’t nearly as important as you think when prioritizing vulnerability remediation. If you’re relying on firewalls and ACLs as your mitigating controls you might want to take a close look at the referenced research.

Overall we had a very good conference, if for nothing else a red hot hallway track. That said, I’m looking forward to a little conference respite before Thotcon and BSides Chicago.

Visualizing your vulnerability data with our new Heads Up Display.

I’m happy to share our latest enhancement to visualizing your vulnerability data. Today, we are launching a new Heads-Up Display (HUD): a “mini dashboard” if you will,  that allows you to visualize the current state of your vulnerabilities and defects.

Our new Heads-Up Display shows a live presentation of your vulnerabilities. It provides up-to-the-minute information on aspects of your vulnerability management program such as scoring, asset priorities, exploitability and impact calculations, with more metrics on the way. Each metric is interactive: rolling over a graph will show you the actual value of the attribute represented in that graph, while clicking a graph performs a live filter based on that attribute.

A simple use case for quickly finding vulnerabilities in your environment that have a very high likelihood of being exploited may be as follows:

1. Navigate to the Vulnerabilities tab within your instance of Risk I/O where you will find the Heads Up Display.
2. Let’s start by filtering on vulnerabilities that don’t require local access by clicking the Network portion of the Access Vector chart. This filter brings our open vulnerabilities down from 63,060 to 50,970.
3. Now lets drill down further by continuing to look at the simplest of vulnerabilities to exploit. I’ll click the Low value for Access Complexity and None Required for level of Authentication. This brings our list down even further to 16,462.
4. From here I can narrow the number of vulnerabilities to tackle by filtering on their impact subscores. This will give us issues that are not only likely to be exploited but with higher impacts. Lets filter by choosing Complete impacts of Confidentiality, Integrity and Availability. This brings our open list down to 5,709 open vulnerabilities.
5. Next up, we’ll take our current list and narrow it further by only looking at vulnerabilities that have a Known Exploit. Choosing this value to filter on results in 1,111 open vulnerabilities.


The 16 most egregious vulnerabilities via HUD.

6. So far, we’ve been focusing on vulnerabilities that are easy to exploit, could have a higher impact on our environment, and have a publicly available exploit from sources like the Metasploit Framework, ExploitDB, etc. Let’s take this one step further and filter only on vulnerabilities within our DMZ that may be publicly facing. Since I have tagged these assets with DMZ within my instance of Risk I/O, I can simply select the tag ‘DMZ’ to filter on. This gives me a very short list of 16 open vulnerabilities to work with.

As mentioned, this is a simple use case to find the most egregious of vulnerabilities within our environment and I’m certain you will have and find many others. We think HUD will be one of the easiest ways to stay on top of your vulnerability management. We’ll be sprinkling more mini visualizations throughout Risk I/O in the future as we identify specific metrics that would be helpful to see in a more visual fashion. Give it a try and let us know what you think.

Gene Kim was kind enough to provide me with an advanced review copy of The Phoenix Project who is a co-author of the book. Full Disclosure: Gene is an advisor to Risk I/O so you can probably assume there is some bias in that I obviously have a lot of respect for Gene’s opinions and expertise going in. Long before I ever invited Gene to become an advisor, I had read Visible Ops Security which he had also authored. The book is truly a classic and much like his latest work, a “must read.”

The Phoenix Project

A lot of people who have read this are comparing the book to The Goal. Admittedly I haven’t read The Goal but having some understanding of the premise, I could certainly understand the resemblance. For those of you who are practitioners, a fair warning: the first half of this book brought back nails-on-a-chalkboard type memories of dealing with large-scale audits and everything that comes with it. This section reminded me of going through our first SOX 404 audits back in 2004 and the cat herding that went along with it. While this section truly resonates at times, it is so real it’s painful.

The book eventually jumps to the heart of DevOps while applying lean manufacturing techniques to technology operations. It’s a great primer for anyone looking to gain operational efficiency, security and the like from repeatable process, automation and just-in-time production via Kanban.

The book ends up being a quick read, partly because of it’s novel format with interesting story lines. It leaves you thinking there’s probably another book chock full of additional learnings coming in the next few years. I highly recommend picking up a copy which happily has been made available in Kindle format.

Today we’re really excited to announce our latest round of funding and welcome our newest investors to the Risk I/O team. Our new investors include U.S. Venture Partners and Costanoa Venture Capital and I am thrilled to be working directly with Jacques Benkoski, our newest board member, as well as Greg Sands. We also had great participation and validation from our existing investors including Tugboat Ventures and Hyde Park Angels.

Our team has been working incredibly hard on our security intelligence platform, and with this funding we’ll really be able to accelerate the product bringing our vision of data-driven security to the enterprise. We’ve made tremendous progress to our vulnerability intelligence platform over the past couple of years, but I believe we can still do more by bringing in additional contextual data that effects security and remediation decisions. The truth is, there’s a lot of data already out there but  its tremendously difficult to make sense of it.

According to research from analyst firm Enterprise Management Associates, almost half of all enterprises surveyed are overwhelmed by the amount of security data they collect, and even more feel they have insufficient time or expertise to analyze that data. The vast majority of respondents said they would collect more security-related data if they could make sense of it.

Our goal is to get our customers to not rely on gut decisions but evidence based ones.

Our new investors bring a wealth of experience in this area and we couldn’t be happier about the newest talent we have on the team. Watch for future posts here on our progress which may include a few cameos from everyone, including board members.

We’ve always focused on helping our customers fix and remediate the most critical security issues, and now we have more resources to help do just that. Oh and one more thing… we’re hiring.

Mario Mendoza set what would become known as The Mendoza Line

I wanted to thank the folks at SecTor for having me out and allowing me to bring the Security Mendoza Line discussion north of the border. As last year, I had a great time and I’m always impressed with how well the conference is run. This was also the first year we did a sponsorship and we brought along the new t-shirts to share. Watch for the ‘shirt-o-matic’ coming soon.

I was able to meet a ton of people and the kind folks at the Liquid Matrix Security Podcast allowed me to ramble on a bit as part of their ongoing coverage of the con. If you haven’t already, go subscribe to the podcast as they have already been piling up good content. Below is my deck from the talk. It was the first time I gave this talk and after a few follow ups afterwards, I will likely be making some modifications. I also have some additional data to add to it based on real world vulnerabilities and how they relate to the Mendoza line.

After the talk, I do have to agree with both Alex Hutton and Josh Corman that HD Moore’s law resonates more with our audience. Perhaps I could create a hockey analogy for the next time I’m up at SecTor…

We recently added the ability to track publicly available exploits for any vulnerabilities discovered in your environment, regardless of how they were discovered. We viewed this as a step in the right direction and one of many factors that go into prioritizing remediation efforts. Our friend Mike Rothman over at Securosis took notice of this, both acknowledging the need while calling it out as “not enough” and looking for additional data such as attack paths. We couldn’t agree more.

We are continually building more integrations in order to provide as much contextual data as possible to help identify “truly critical” issues inside your environment. This includes available exploits, business processes affected, network connectivity, location and more. With our latest integration, you can now test your mitigating controls to see if they are effective. Think that vulnerability discovered by your scanner is protected by your IPS? Why not test it out? By using your Metasploit connector within Risk I/O, you can attempt to exploit the vulnerability with one click and validate your controls.

Use our Metasploit connector to validate your vulnerabilities.

To start, go to your Connectors tab and create a new Metasploit connector. You’ll need your host location and credentials to set it up. Once created, you can filter by “Known Exploits Exist” within your Vulnerabilities tab to find vulnerabilities that have a publicly available exploit. You can then view any of the vulnerability details, click the Known Exploits tab and initiate your attack via the obviously red attack button. That’s it! From here Risk I/O schedules and performs the attack via Metasploit and reports back and persists the results within the vulnerability record.

Go ahead and give it a try and let us know what you think. If you don’t have an account, sign up for a free one.

By the way, if you haven’t read the Vulnerability Management Evolution paper from Securosis, go do it now. A lot of great content there and best of all, it’s free!

Risk I/O can now be used to identify publicly available exploits to your existing vulnerabilities. Our development team has made it possible for Risk I/O to match attack vectors from databases of quality assured exploits, such as Metasploit and ExploitDB, to applicable vulnerabilities. This information, paired with vulnerability data from assessment tools, allows you to understand how your organization is vulnerable to attacks.

You can now get information on publicly available exploits for a specific vulnerability.

In an earlier post, I wrote about the importance of focusing on data that allows you to “hit above the Security Mendoza line,” or the threats most likely to occur based on the evidence and ease of exploit. Alex Hutton referred to the Security Mendoza Line when talking about vulnerabilities exploitable with MetaSploit modules. Josh Corman expanded on this quite a bit with HD Moore’s Law. We built this feature to weed this out of your environment and allow you to hit above the Mendoza line .

We often talk about “enough security” and Josh frames it correctly by stating this is InfoSec table stakes. By combining data from these publicly available exploits and network accessibility, you can truly identify some low hanging fruit and protect yourself from the Point, Click & Pwn of the most casual and opportunistic attackers.

For the dataheads in the audience, a quick and early glance suggests about 14% of active and open vulnerabilities have publicly available exploits through one of these tools or databases. I think there is more interesting views to be had, but clearly an indicator that we have a long way to go before we can protect ourselves from even the most casual adversaries.

If you are a current customer, you can access this new feature in the Vulnerabilities tab. Don’t have an account? Signup for our free version!