The Anti-Malware Testing Standards Organisation (AMTSO) has announced its guidelines for false positive testing.
Just as in its guidelines for the detection of malicious files, AMTSO said that care must be taken to ensure that all samples to be tested are verified, that they are not misclassified and that the vendor has not added detection intentionally because it regards the file as ‘greyware' or ‘possibly unwanted'.
It said that testers should make it clear when false positive testing is performed in conjunction with malware detection testing, as this may bias the results.
The new guidelines suggest a series of criteria for testers to use in determining the magnitude of a false positive: such as the impact of a false positive on the user; how many users would be impacted by a false positive; and recoverability assessments on how difficult it would be to remediate the situation, has data been deleted and does the system need to be taken offline?
Mark Kennedy of Symantec, who introduced the guidelines on behalf of AMTSO in papers for the Virus Bulletin and AVAR Conferences, said: “False positives tend to have a greater visible impact on the customer than on a security product's protection, so it is surprising that more anti-malware tests do not include false positives.
“Recently the introduction of proactive technologies such as behaviour blocking and generic signatures have increased the likelihood of false positives. The problem with current tests is that they are frequently too simplistic in their approach, presuming that all non-malicious files are equally important. However, when you break down a file's specific function it's clear that it this is simply not the case.”