Ellis said it's necessary to align expectations and be consistent with follow through. “Decide what behaviour do you want to attract from researchers? A high level of creative effort goes into (discovering bugs) so encourage them to do it again – and tell friends to do it again.
“Its a market place, so prices are going up – the more you offer the more you get… I believe prices paid will eventually standardise – but we're nowhere near that yet.”
Evans noted that one control to regulate response is the price you set, and this can affect volume, so you should start with a lower price, get some good bugs identified and dealt with, then turn the dial up.
A speaker from the floor added that there is a need for a better model to reward bounty hunters, and there should be a trade off, because companies are paying less than they pay pen-testers and getting better results. It was noted that currently researchers are usually working on something else when they find bugs.
Ellis explained the disparity between what companies pay for vulnerability and their value on the black market, saying that the risk model is 100 percent in favour of the supply side because closing a vulnerability is a one-off action/saving, whereas for criminals, the exploit can be used multiple times so it's worth more. It was accepted that currently it is the ‘hunters' taking on the risk, but there are people making a living relying on this in India and the Phillipines, but not in the West.
“How do you wind down a bounty programme without pissing off researchers?” asked one delegate, and the answer from Jones was that: “You are going to upset someone, either because you are not meeting their expectations or simply because you are winding back the operation. So you need to align expectations – then wind back.”
Kouns, however, questioned why, if you're finding out about a wide range of security issues you wouldn't need to keep on doing that.
While there was no single answer as to what to do to prepare product for test, given the permutations of what's being tested and their risk profile, it was advised that companies do put thought into this aspect.
Jones noted how Facebook runs an entire test-Facebook with test users, and gets people to research with test information rather than real users. And Evans advised that you should have a good idea of how buggy the product is beforehand, based on traditional testing.
How much overlap is there between legal and illegal sale of exploits and is there a grey market where the two overlap or are they separate economies?
Panellists believed that most individuals ‘chose their hat colour' beforehand, rather than making a decision after finding an exploit. One delegate suggested, however, that researchers may go for a different buyer if can't get what they want from the vendor.
He went on to complain that the corporate world drastically underpays in its bug-bounties, saying criminals and governments vastly outbid them “if I have good bugs.”
Ellis agreed that there needs to be some economic parity. But a call for governments to subsidise bug bounties failed to get much traction, and it was noted that many researchers do this for more than just money – including recognition and contributing to the greater good – with the comment: “We decided to use our skill for good, not to rob the bank.”