Most high-severity cloud security findings are downgraded once an AI agent checks the environment

Cloud security platforms turn every failed rule into a finding, and those findings pile up in queues sorted by severity. Research from Sola Security tested what happens to that order when each finding gets checked against the environment where it lives.

The researchers, Leon Goldberg and Gal Engelberg, built an AI agent that re-grades findings raised by third-party cloud security platforms. They ran it on 9,967 HIGH findings from two commercial platforms in eight production environments, and the agent changed the grade of 75.1% of them. Most moved down (58.5%), and 16.6% moved up.

The severity on a cloud finding comes from the rule that raised it. Each of 599 vendor rules issued exactly one severity over 99,073 failing findings, regardless of the resource or environment involved. “A static severity reflects the risk of the condition in general,” the researchers wrote, and “a queue ordered by it is ordered by the general case.”

“A rule carries the context its author can encode once for every environment,” they noted. “The context that decides what the condition is worth in a given environment lies elsewhere.” That includes whether the flagged resource is attached to anything and whether a workload runs on it.

Scoring standards reserve a place for this context. CVSS includes environmental metrics that adjust a base score to a particular deployment, and NIST risk assessment guidance ties likelihood to the controls in place. “A static severity leaves that place empty,” the researchers observed.

For each finding, the agent forms a theory of how an attacker would exploit the problem, then collects evidence from the resource’s configuration, the resources connected to it, and data held by other security products.

Under the agent’s evidence policy, “raising a severity needs more evidence than lowering it.” Lowering a grade requires an observed property that offsets the specific way the failure would be exploited. Raising one requires evidence that the condition is worse in that environment, along with a check for firewalls, allowlists or other plausible guardrails in the path. Everything else keeps the vendor grade.

Six recurring patterns accounted for the findings whose deciding facts could be checked through cloud APIs . The largest, at 32.3%, involved inert resources, attached to nothing or orphaned. Downgrades also came from resources unreachable from the internet, missing live services, narrow source restrictions and account-level blocks. One pattern, public exposure confirmed in the environment, mainly raised severity.

Of 138 rules, 62 produced findings that the agent downgraded in some cases and upgraded in others. For 57 of those rules, both outcomes occurred within a single environment. A security group rule, for instance, was lowered when the group fronted only internal load balancers and raised when the environment showed a path from the internet.

About half of the verdicts (49.0%) rested on evidence from outside the flagged resource, from connected resources in the asset graph or from another product’s data. “Outside evidence concentrates in attenuations, where showing a finding safe means ruling out every path to it,” the researchers explained. “The rule-severity gap is large in production, and closing it requires evidence from the whole environment,” they concluded.

To test whether the agent’s verdicts rested on solid ground, the researchers ran read-only probes on eight AWS accounts, re-reading the deciding facts directly from the cloud provider and leaving the agent’s own data out of the loop. For the verdicts the probes could settle, the live environment matched the agent’s facts 99.4% of the time. When every finding the probes could not check was counted as wrong, the figure dropped to 81.8%.

The probes tested the facts behind the verdicts, and whether the revised severity ratings were the right ones is a question the study left open. The authors draw a line between an AI reading findings and an AI acting on them.

“Verified, environment-specific reading of each finding is a precondition for acting on findings autonomously,” Goldberg and Engelberg argued. “Reading and acting differ in blast radius, reversibility, and authorization,” they cautioned, adding that “this study measures reading only.”

The work covers one system’s verdicts on the HIGH queue of two platforms, and 92% of the re-graded findings were on AWS. Live verification covered one platform and five resource types, and it tested the deciding fact behind each verdict, leaving the final grade unchecked.

Findings that started at MEDIUM or LOW fell outside the scope. The study also lacks incident data, so it leaves open whether upgraded findings are more likely to be exploited, or downgraded findings less likely, than their vendor grade implies. Recall, meaning the deciding facts the agent missed, went unmeasured.