Autonomous agentic systems break this paradigm entirely. An agent does not follow hardcoded logic; it interprets, plans, loops, and acts. Faced with an identical prompt, an agent may react completely differently; it might query a database today, call a shell script tomorrow, or hallucinate an entirely nonexistent API endpoint next week. To improve explainability and also, to pretend to know what the hell is going on, we have the agent’s chain-of-thought thrown into this mix too. Basically CoT means the model/agent documents exactly how it reached the result. To fully understand what’s going on we have to use logs for our agent, what it accesses, how it thinks, how it’s authenticating and finally what is it doing.

This also increases the attack surface of an agent as now it isn’t just limited to the assets and data it has access to but also it’s own Chain-of-Thought and Memory.

Some challenges faced in SOC due to the new attack surface:

The sheer volume of data

Just think about it, we have AI Assistants like Claude, AI-enabled Browsers like Perplexity, AI-enabled software like Harvey AI or even Jira and SNOW, MCPs, LLM Endpoints etc. If each person in a 50 person company runs 5 sessions/day it’s almost 250 log events to be monitored across each software, so increasing the amount of logs generated and ingested by almost 150%. By the way, we haven’t even considered OT and IoT devices running manufacturing AI algos or BMS in the mix yet for Enterprise monitoring.

The most difficult question: What to monitor?

OWASP Top 10 for Agentic and LLM covers different vulnerabilities across different Responsible AI tenants. Alerts related to bias or cascading bias, hallucination or agent goal hijack related alerts are required to be sent to the dev team for remediation. If these alerts, reach a SOC team, it’s practically useless because a SOC team manages the incidents for the enterprise and hallucination, bias is specific to the use case only. The dev team responsible for the use case or app can only fix the performance based vulnerabilities. Any prompts where PII data wasn’t properly obfuscated or the outputs show PII/PHI data that it shouldn’t, need Data Privacy Office to review. They determine how to handle the incident.
Prompt Injection and Privilege Escalation etc need the security team to intervene and stop.

At the same time non GenAI or LLM related algos running may have different criteria for monitoring. For example: a team working on an self-driving car, they don’t need to monitor for privacy and security in the traditional sense. The monitoring parameters would be with the perspective of the environment, visibility, detection, control, driving etc. Thus, for general consulting companies or manufacturing companies, the monitoring of an AI system for safety and security is totally different.

Thus, the segregation of events and alerts require a thoughtful framework to be created with each team requiring different data parameters for their review and analysis. Currently no products or even guidelines from any orgs are available in the market to help companies differentiate between log collection and actionable alerting.

The Payload Paradox and Legal challenges

Following up from the previous point we have the payload paradox brought to you by the non-deterministic autonomy of an agent.

Since the agent can take actions on it’s own and their actions may differ for each case, to detect adversarial goal hijacking, developers and security analysts need total forensic visibility into the agent’s chain-of-thought traces, prompt templates, and intermediate tool arguments. Yet indiscriminately dumping all raw execution spans into telemetry pipelines turns the observability stack into a high-risk data liability, aggregating unredacted PII, enterprise secrets, and compliance violations into single, queryable buckets.

Not only does logging everything increase risk but also not logging everything raises several issues in cases of legal hold or court/gov inquiries, once litigation is reasonably anticipated, the duty to preserve potentially relevant evidence extends to AI interaction logs the same way it extends to any other record so “we didn’t think to keep it” is not a defense.

Multimodal inputs

For certain models like LLMs or object-detection (YOLO), the input and payload format received are varied. An LLM accepts video, audio, text, image inputs which requires subsequent calibration for the alert rule. For images, we need to keep in mind about steganography. For audio, we have to consider privacy declarations before even considering ingestion of logs. Other systems with YOLO models will have sensor input which will be processed completely differently from user facing apps providing text or so responses. This overall increases the complexity as there is no common baseline to go from and most integrations requires further nuances to set up credible alerts.

Legitimate vs Malicious events for General Purpose apps

For general purpose AI apps like Claude ort ChatGPT, the use cases that a user can do are countless. One can generate essays, analysis website UI or even do gene computation. Now, how do you determine at a very large scale with just logs what is malicious activity? If a user is leveraging Claude Cowork for editing a video, the agent might create multiple temp image files during processing in the folder and then delete them all. In the logs, it may come across as mass deletion which needs further review on account of it being insider threat. With such general purpose apps, determining use cases that are truly malicious are difficult and may need to be regularly updated.

Trust in CoT

As mentioned earlier in the article, we depend on Chain-of-Though to understand the agent behavior. However, as the agent documents it’s own actions, it raises a questions that if the logs are actually accurate.

Referencing specifically the model system card published by OpenAI for Astra, we see that chain-of-thought faithfulness and accuracy has been chipping away and the picture in 2025–2026 is not encouraging. Models can produce a plausible, coherent-looking rationale that has little to do with the actual computation behind an answer. The stated reasoning and the real decision process can come apart, especially under biased or leading prompts, as Turpin et al.’s original work demonstrated and later studies have replicated. A 2025 study from Anthropic researchers found that reasoning traces verbalize a hint the model actually relied on in fewer than one case in five where the hint was used at all. Anthropic’s own paper on chain-of-thought monitorability is candid about this: CoT is a genuinely valuable safety signal, but a fragile one, subject to exactly the same pressure to look helpful and harmless as any other model output.

There’s a term for the failure mode that’s started circulating in the literature: reasoning theater. A model can already “know” the answer and still generate tokens that perform the appearance of working it out.

This matters enormously for the audit-trail pitch, because an audit log that doesn’t reliably reflect what happened isn’t an audit log. It’s a transcript of a performance. Worse, a security researcher pointed out that the failure mode is exploitable in the other direction too: an adversary can craft prompts that produce a coherent-looking reasoning trace that steers toward an attacker’s preferred conclusion, without ever triggering the safety classifiers built to inspect reasoning for red flags. The very layer you’re trusting to catch misbehavior can be dressed up to pass inspection.

This concludes the challenges a SOC team may face when they try to setup effective monitoring and alerting for AI and agentic systems. Everytime you think of AI systems, make sure it’s not just boxed to GenAI/LLM systems but also systems leveraging other algos like Building Mangement Systems that run ML models or Face scan systems that use Object-detection algos (YOLO).

AI Agents vs Automation: What is the difference

Firstly, what is an AI agent, and why is it different from our plain ol’ automation? Automation follows a set of predefined rules to complete tasks—fast, consistent, and predictable. AI agents, on the other hand, operate with autonomy: they can reason, adapt, and make decisions based on dynamic inputs. For eg: if the goal is to parse a document and then send a mail with the extracted data. If we create an automation script, we would need to first use a parsing algo and then define words that need to be extracted using key-value pairs or regular expressions. After that, we also need the exact template with substitutions defined for the mail to be sent. Whereas if we create an AI agent, we need to use an algo focused on extraction and just define the expectations, like write a mail with a formal tone or make sure the mail is concise. The agent will use its autonomy to draft the mail and send it. Both the AI agent and automation script would still need access to the mailbox, permission to read the doc, permission to send the mail and authentication credentials. Pretty cool to see how far NLP algos have come but also a bit terrifying.

Challenges we may see if we use SOC-AI agents:

1. One of the key challenges AI researchers are working on is Explainability. According to ISO/IEC TR 29119-11:2020, it means the level of understanding how the AI-based system came up with a given result. Let me elaborate with an example: If we consider a decision tree algo or K-means, as a researcher/developer, one can look at the input and give a reasonable guess why a certain output was derived. But what about LLMs or Facial Recognition algos like FaceNet, at any given point a researcher or expert cannot say with certainty why a certain output was derived. You can try it yourself if you ask Claude to write 5 sentences on any topic, like “Tell me about the movie Jurassic Park.” Now, how can you say with certainty what 5 points it will cover? It can talk about actors, directors, movie reviews, CGI or anything.

This is different from interpretability, which is basically the level of understanding how the underlying (AI) technology works, which we can explain, like Claude is an LLM which is based on Transformer architecture. Or FaceNet uses a deep convolutional neural network to learn a mapping/embedding from a set of face images to a 128-dimensional Euclidean space and assesses the similarity between faces based on the square of the Euclidean distance between the images’ corresponding normalized vectors.

So in the context of SOC agents who have more autonomy, tool access, LLM access, etc., how do we account for explainability in a way that is accessible to all the analysts?

2. The idea that the system responsible for monitoring everything and generating alerts uses a mix of probabilistic and deterministic algos to derive answers is a bit scary. The probabilistic algos come into play during the first half, where the agent tries to assess the telemetry for context. They apply behavioral baselines and knowledge graphs to assign risk scores and dynamically select response paths based on asset criticality, rather than following fixed “if X, then Y” rules. As you may know, any system is not 100% accurate, and a probabilistic system may lead to errors that have a huge impact on business. Not a big deal in the grand scheme of things, but has anyone noticed that since we started using AI agents for coding in corps like AWS and Microsoft, there have been more outages? Big organizations may be able to absorb the impact, but smaller orgs will face more challenges.

3. With AI being the IT-girl of the industry, it is also the prime target of hackers. The constant attempt to use AI agents or shared LLMs for attacks and malware propagation is very real. Imagine the system you depend on for alerting you of these incidents and exploits and attacks itself gets compromised because of a shared model or Agent-to-Agent sprawl or even a sci-fi-style rebellion.

4. Lastly, since AI agents have more autonomy, their mistakes are also unpredictable. As per many experts across domains, note that the agent is generally reliable and doesn’t make errors, but when it does, it’s pretty hard to trace the error because the steps taken aren’t logged. In a controlled environment, with limited variables or just 2-3 use cases, it is still manageable, but in prod, where the SOC system handles 1000s of use cases and logs, attack signatures, etc., things become challenging. Also, the butterfly effect, where a small mistake is insignificant in itself, may lead to business outages or legal repercussions.

Then what is the solution? Because analyst fatigue is real, and with fixed security budgets, any help is appreciated. The solution is human-in-the-loop, where the analyst uses SOC for summarization, data extraction, references, and alert generation, but at the end of the day, the analyst makes the final call to dismiss or act on the alert.