Trend Micro to boycott security tests
Trend's chief technology officer Raimund Genes issued the boycott today and hit out at the test procedures, branding the VB100 "old fashioned" and "totally irrelevant" and claiming the tests are "creating a problem for the industry".
Speaking to SC Magazine, Genes claimed that other security companies felt the same way about the tests, but were afraid to speak out because of the credibility they gained from passing the tests.
"After we failed the last test, we had a huge internal discussion. We informed Virus Bulletin [the Oxfordshire-based organisation behind the tests] that we will not participate anymore," he said.
Trend Micro joins Panda Security as one of only two vendors to shun the tests. Symantec, Microsoft, McAfee and 34 other vendors are still taking part.
"It [the tests] are all old-fashioned, but because the [VB100] label is so valuable, Virus Bulletin is creating a problem for the industry," said Genes. "I hope they are changing the testing method."
He criticised Virus Bulletin for not testing against the prolific Storm Worm nor testing for effectiveness against rootkits.
He also slated the VB100 for not testing against "real-life threats" and for carrying out its testing offline. Virus Bulletin tests against the Wild List from wildlist.org, which is run by ICSA Labs and based on research from many distributed individuals.
Genes insisted that Trend would continue testing with rival labs AVTest.org and AV Comparatives.
SC Magazine asked Genes whether cynics would be right to assume that its withdrawal from the VB100 was sour grapes for failing the test.
He said: "The cynics are slightly right. But everybody knows that the VB100 is totally irrelevant. We have participated in the test for years. Sometimes it has slaughtered us, but we continued with the tests."
John Hawes, technical consultant for Virus Bulletin expressed surprise at Genes' outburst. He said Trend Micro had not told Virus Bulletin that it was withdrawing from the test: "They have not said anything about this to us".
Hawes said that Trend Micro had entered just two out of the last five tests - and had failed them both. The vendor "had issues with polymorphic viruses", he said.
He also defended Virus Bulletin's methodology. "We use the Wild List from reporters all round the world. The test is that they [vendors] constantly cover everything on that list. It would be impossible to test every virus because we don't have it. AVtest.org and AV comparatives have a different style of test. They cover everything out there. That is very useful and informative, but different in purpose," he said.
Hawes conceded that the Wild List is "a little behind the times", but said Virus Bulletin was looking at a way to add to it.
He continued: "The only problem is that some products update from the internet. We operate in a sealed environment and don't allow that," he said.
"That's a slightly odd reason for not taking part. I don't see how they could have anything to be upset about."