To all those who are interested in the latest an greatest, I'm currently working on the 2013/2014 web application scanner benchmark, and already I'm seeing some VERY interesting results.
The benchmark will be published soon, and I'm posting many of the results during the assessment process using the comparison twitter account @sectoolmarket, which also publishes news about other information security product comparisons performed around the globe.
This time, I received plenty of help from multiple entities -
Many entities (including the ZAP project and IronWASP project) contributed test cases to wavsep (not included in this benchmark scope, but might be in the next),
Several researchers around the globe offered their help in the assessment process (encouraging me to work on something that will someday make it easier),
And last but not least, I received plenty of help from the wonderful guys at Denim group, which did their best to adjust ThreadFix so I can use it to make the task of comparing and counting results easier (just started checking it - looks great so far)
Wavsep was already enhanced to v1.5 (with hundreds of additional test cases that will be published after the upcoming benchmark),
The vast majority of commercial vendors already provided me with a valid license and installation, and at least half of the planned open source projects were either tested or currently being tested.
I'm planning to release the information gathered in two or three bulks -
(*) The typical benchmark and analysis (including at least two new vulnerability detection comparison aspects which will remain obscure at the moment - for the sake of the competition).
(*) An analysis of the DAST market status, based on the results and additional information gathered during the test.
I'm also planning to upload the results into a dynamic publication framework (partially implemented), although the first bulk of information will probably be published in the blog and static sectoolmarket website.
In short, stay tuned, results will be published soon .
Security Tools Benchmarking - A blog dedicated to aiding pen-testers in choosing tools that make a difference.
Hi,
ReplyDeleteI recently found your blog, especially the posts that talk about vulnerability scanners. This study is very valuable and looks like a kind of reference to me. Thank you so much for that.
Right now I am trying to choose a security tool. For that I am collecting many inputs (you blog is definitely one of them) in order to make a pertinent choice.
During my research I found the scanner "Seeker" form "Quotium" could be a solution to study. This was confirmed by several channels.
Since you are are about to release the 2013-14 web application scanner benchmark, I was thinking if it was not to late to include it. Well ... may be you already have it in your list....?
F.
In short - no,
ReplyDeleteSeeker will not be included in the current benchmark, since it's not the same type of tool as the rest of the tested tools.
Seeker belongs to a category called IAST (interactive application security testing) - it's a great tool, but unlike the rest of the *modules* compared in this category, it performs a memory-level "white-box" assessment while making use of an agent installed on the server, and does not contain any "black-box generic vulnerability detection" module, which is the focus the upcoming benchmark.
I'm not sure comparing these different technologies is either right, fair or accurate, especially because in my opinion, they should be used for different phases in the SDLC/Security Testing process.
That being said, I am planning on testing this category of tools in the near future, and will post updates as I progress.