The 2013 scanner benchmark is coming soon!

To all those who are interested in the latest an greatest, I'm currently working on the 2013/2014 web application scanner benchmark, and already I'm seeing some VERY interesting results.

The benchmark will be published soon, and I'm posting many of the results during the assessment process using the comparison twitter account @sectoolmarket, which also publishes news about other information security product comparisons performed around the globe.

This time, I received plenty of help from multiple entities -

Many entities (including the ZAP project and IronWASP project) contributed test cases to wavsep (not included in this benchmark scope, but might be in the next),

Several researchers around the globe offered their help in the assessment process (encouraging me to work on something that will someday make it easier),

And last but not least, I received plenty of help from the wonderful guys at Denim group, which did their best to adjust ThreadFix so I can use it to make the task of comparing and counting results easier (just started checking it - looks great so far) 

Wavsep was already enhanced to v1.5 (with hundreds of additional test cases that will be published after the upcoming benchmark),

The vast majority of commercial vendors already provided me with a valid license and installation, and at least half of the planned open source projects were either tested or currently being tested.

I'm planning to release the information gathered in two or three bulks -

(*) The typical benchmark and analysis (including at least two new vulnerability detection comparison aspects which will remain obscure at the moment - for the sake of the competition).
(*) An analysis of the DAST market status, based on the results and additional information gathered during the test.

I'm also planning to upload the results into a dynamic publication framework (partially implemented), although the first bulk of information will probably be published in the blog and static sectoolmarket website.

In short, stay tuned, results will be published soon .

Security Benchmarks & Comparisons – Plans for 2013

It's kind of hard to admit that your current strategy leads to a dead end… Hard, but liberating.

I initially started this blog because I was searching for a way to sort through an insane amount of tools I collected over the years - so we can all weed out the irrelevant and stick with what works.

Obviously, things got a little complicated, and after doing double shifts and spending half my nights over the past 4 years on comparisons, I realize now that I only covered 60-70 tools.

Sure, I had a good reason to do so - learning curve, comprehensiveness, accuracy, credibility, evolution… but the numbers don't lie.

As much as I like the idea of a one man army, the current rate is NOT what I expected, and to achieve something greater, I'll need to get some resources and some help (yeah yeah, mental too).

Nope, that DOES NOT mean that I'm about to stop any of my planned activities, researches or benchmarks. Giving up is for wusses.

It does mean, however, that I'm going to make some changes that will enable me to cover more, even if I have to make some decisions I was dreading and trying to postpone.

So what I'm planning for 2013 is to branch out and cover additional types of tools & products, in addition to vulnerability scanners.

That means updating WAVSEP with some hybrid issues, becoming less of a control freak, let go the leash I was so inclined on keeping, and probably even creating additional comparison platforms.

Yep… b   a   c   k      t   o      w   o   r   k.