Online False Positive Reduction via Statistical Process Controls
Brief
Everyone seems to be obsessed with False Positives right now in Security. But this is a solved problem in other industries.. How does a fortune 500 company evaluate thousands of feature changes in their product, across thousands of customers, in real time, continuously… over data that keeps changing..?
Some of them have data scientists that use Statsig, but all of them use some kind of a statistics engine that evaluates performance against costs and risk. How does a Hedge Fund choose to take the best performing prototype investment algorithms?
They give them real data first, and then they use statistical analysis and optimize the results relative to the costs and risks involved, in order to screen for chosen models.