DFIR+AI Primer: How to Combat Hallucinations
Brief
GenAI outputs require verification because of hallucinations. As you integrate AI into your investigations, it is important to understand why hallucinations happen and how to address them based on your risk thresholds.
GenAI can provide value to an investigation, but it’s not for free.
Why does GenAI Hallucinate?
GenAI algorithms use probabilities to generate sequences of words. It can ultimately generate a sequence of words that are probable, but not accurate.
This most often comes into play when you ask GenAI to enrich and reason about your data when it doesn’t actually know much about it. It can generate false information about an artifact and come to conclusions it can’t actually justify.