Why data quality dictates security operations success
Brief
As AI takes on more security operations center (SOC) workflows to automate threat triage, indicator extraction, and incident report generation, security operations leaders face a persistent question: Does SOC performance depend more on the large language model (LLM) deployed or on the underlying quality of the security data it consumes?
Academic studies offer conflicting answers. An original research article from the Frontiers in Artificial Intelligence emphasizes output variation in accuracy, relevance, and clarity across frontier models such as Claude 3. 5 Sonnet, Gemini, ChatGPT 4o, and Mistral Large 2.
Meanwhile, published work in the Information Systems journal argues that “trustworthy AI applications require high-quality training and test data along many quality dimensions, such as accuracy, completeness, and consistency.”