After the benchmark publication, several vendors contacted me with recommended configurations that could enhance their score, and with feature documentation corrections.
After testing the various provided configurations, I was able to update the various charts and data in the benchmark original post, as well as the various charts in sectoolmarket.
Update summary:
The WIVET score of Webinspect was slightly improved from 94% to 96% by selecting the "depth first" mode in the scan wizard (the default configuration still yields 94%), which makes it the official winner of the WIVET category.
The path traversal detection score of arachni was updated from 30.88% to 100% (!!!) by making use of the source code disclosure plugin (as suggested by the vendor, in addition to the path traversal and local file inclusion plugins), which makes it the co-winner in this category, alongside Appscan.
The LFI detection results of Webinspect were likewise improved from 72.06% to 91.18%, by using vendor recommended configuration that included the following plugins: 10287 – Local File Include, 10271 – Local File Inclusion/Reading Vulnerability, 10272 – Possible Local File Inclusion/Reading Vulnerability, 11327 – LFI Tomcat, 11332 – LFI IIS
Finally, the list of supported input vectors was updated after the Appscan team reported support for 4 more vectors, the ZAP project reported support for additional two input vectors, and the arachni project reported support for one additional vector. All updates represent support in the tested versions.
There may be some minor updates to the SQL injection results of one scanner - if the vendor provided configuration will work.
As mentioned earlier, the benchmark charts already reflect the changes, and summarizing content will be published soon.
Did you try "depth first" on IBM AppScan too?
ReplyDeleteNo, assuming such a feature exists in the product.
DeleteHowever, I did perform the test while getting support from the appscan development/research team, and at the time of the test, they did not suggest that such a configuration would provide better results.