NSS Labs' chief executive Vikram Phatak said in a blog post on Wednesday, that the lawsuit was aimed at companies that he claimed "are actively conspiring to prevent independent testing that uncovers those product deficiencies to prevent consumers from finding out about them".
He said his company often came across deficiencies in products during its testing, and this information was conveyed to users of these products.
"As you can imagine, this can hurt a vendor’s sales. So, what is a vendor to do? Some (the good ones) fix their products. Others try to avoid being tested. But being the sole vendor refusing to be tested is bad for sales," Phatak said
Phatak claimed that the actions of the three companies named in the lawsuit were conducted by, and through, their membership of AMTSO.
NSS Labs is suing Crowdstrike (and others) to force them to change EULA's preventing testing by an independent third party. Recall that earlier, Crowdstrike sued NSS Labs to prevent the publication of a test report during @RSAConference. 1/nhttps://t.co/lAmVlRWvyl— Jake Williams (@MalwareJake) September 20, 2018
This organisation "claims its purpose is to establish standards 'for fair and useful testing',” Phatak said.
"What they neglect to tell you is that their version of 'fair and useful' tests are driven by the same security vendors whose products are being tested; not a neutral, independent third-party setting a higher bar for the security vendors and the industry.
"They claim to try to improve testing but what they’re actually doing is actively preventing unbiased testing. Further, vendors are openly exerting control and collectively boycotting testing organisations that don’t comply with their AMTSO standards – even going so far as to block the independent purchase and testing of their products."
Phatak claimed some vendors, including CrowdStrike, had conspired to prevent testing of their products by placing clauses in their end user licensing agreements that made testing of their products subject to their permission.
"This unethical and deceptive behaviour hampers transparency and hinders consumers in their ability to assess whether a product delivers on its promises," he claimed.
Commenting on the NSS Labs action, former NSA hacker Jake Williams pointed out that CrowdStrike had sued NSS Labs to prevent publication of a test report during the RSA security conference.
"If you think this doesn't matter to you, think again. First, how many of you think 'trust us, it works' is a good way to gauge the effectiveness of a product? I for one think that's insane. Vendors lie all the time. All the time," said Williams, who worked with NSA's elite Tailored Access Operations unit before setting up his own infosec outfit, Rendition Infosec.
"Want some examples? An IAM vendor last year proudly explained (while unknowingly misquoting me on a webcast back to me in person) that controlling admin permissions would stop WannaCry," he said. "This was before NotPetya, he was not confusing the two. Bold-faced lie.
"And let's not forget HP telling people at Blackhat a few years ago that they had protection against 'negative day vulnerabilities' even though nobody (including them) knows that the heck that means. Yeah, I asked what a negative day threat was. Nobody could tell me.
Williams said HP also had people on the sales floor telling RSA attendees that they needed printer security because "Stuxnet attacked the print spooler." "I legit lost brain cells listening to that sales pitch," he added.
"I have countless more stories like this, enough to write a book. My point is that in a vacuum, security vendors will tell half truths and outright lies. The industry absolutely needs vendors like NSS Labs to test products and call them on BS."
He said his company regularly violated EULAs during testing. "We evaluate the products to determine how to bypass them. Those results are not shared with the vendors, well, because I don't work for free. But yeah, I'm just as exposed legally as NSS Labs is."
Williams said what differentiated his firm from NSS Labs was that testing products was not Rendition Infosec's core business. "It's more of a side effect. But these abusive EULAs need to stop for the good of the industry and the safety of consumers everywhere. @NSSLabs, you've got a supporter here," he added.
Contacted for comment, a CrowdStrike spokesperson told iTWire: "NSS is a for-profit, pay-to-play testing organisation that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing. We believe their lawsuit is baseless.
"CrowdStrike supports independent and standards-based testing — including public testing — for our products and for the industry. We have undergone independent testing with AV-Comparatives, SE Labs, and MITRE and you can find information on that testing here. We applaud AMTSO’s efforts to promote clear, consistent, and transparent testing standards."
iTWire has also contacted Symantec and ESET for their respective reactions.
Update, 24 October 2020: NSS Labs dismissed its anti-trust complaint without prejudice in December 2019.
The new chief executive, Jason Brvenik, said: "During the past year, AMTSO has made progress to be more fair and balanced in its structure, vendors have shown progress in working with testing organisations, and the market itself has had significant change and notable acquisition activity
“It is said that sunshine is the best disinfectant, and that has been our experience here. We look forward to continued improvement in the security vendor behaviours."