DFIR+AI Primer: When Not to Use GenAI
Brief
When thinking about where to use GenAI in your investigations, you must consider 3 things:
- How much faster did you get an answer from GenAI?
- How error-prone is the prompt on your GenAI model?
- How much verification will be needed given the stakes of the investigation?
If your verification time is bigger than the time savings of getting the initial answer, don’t use GenAI.
Different Prompts Can Produce Different Errors
GenAI makes errors when the prompt requires it to interpret data, reason about data, or draw conclusions from incomplete data.
Errors commonly happen because the model:
- Makes bad assumptions.
- Uses flawed logic.
- Lacks relevant knowledge or context.
More complicated queries increase the chances of an error.