Something Risky Happening
Hi everyone :) After a busy, overloaded, and emotionally draining stretch, I am finally getting around to sharing my thoughts on the cyber risks tied to highly capable autonomous harnesses and agentic systems.
Before the technical part, I want to pin down what "capable" actually means. I will pull from a Gemini overview and the Wikipedia definition.
See also the "Slop Free" definition below.
The definitions I have provided mostly describe human attributes ,but shift the context to "intelligence" powered by machines and we can no longer talk about skills or talents the way we do for humans. Agents today can act across network, file, and CRUD operations simultaneously. They can do all of this while you're resting, sleeping, and so on. And the pace of progress has been faster than I expected, even by the start of 2026.
During my senior year project, I kept coming back to a line from the Marvel film Thor. I changed a few words from the original.
“Whosoever holds this agents, if he be worthy, shall possess the power of Coder / SW Dev.”
I did not say "engineer". I simply saying only the trivial part, code generation, not architecture or design patterns.
Anyone can ship whatever they want now. From where I sit, the calculus around building something unique has shifted. Even a fully original SaaS startup or product can be steamrolled by frontier capabilities almost overnight. Companies with an existing client base hold a real edge over newcomers, because frontier models compress the idea-to-MVP distance for everyone at once a unique idea just does not buy the runway it used to. The "startup to riches" era, as it once worked, feels closed. Most AI-era apps also carry a massive number of vulnerabilities, even after agent scanning. Human oversight still wins in these cases. No machine has become as creative as a human yet, even with multi-capable skills like command execution, termination, network operations, and cyber operations (SAST, recon, CTI-related tasks).
Now let's shift to the "risky" part. We've all been paying close attention to AI's own improvements inference rates, context sizes, agentic capabilities, benchmarks, and so on. From an AI red teaming or security perspective, we track a similar corpus, but it's been far too narrow compared to how fast the underlying capability has moved. And compared to traditional digital ecosystems, deterministic software and the like, this progress has now outgrown the old scale entirely.
A couple of references worth reading and thinking through:
Thomas Kwa
METR's core finding is the one that worries me. The length of tasks AI agents can autonomously complete has been doubling roughly every 7 months, consistently, for the past six years. That is a moving target we keep underestimating, not a fixed ceiling. Predicting that forward, within a decade we have been looking at agents that can independently finish a large share of software work that currently takes humans days or weeks.
I have included a capability comparison chart based on METR's evaluation results, human vs. machine. Machine performance is starting to track human-level task completion closely, and the trajectory of improvement stands out most. How do you control this kind of advancement? What happens when the controls themselves are no longer enough? The stronger AI becomes, the more risk seems to compound rather than scale linearly.
Right now, agents can already scan large datasets, traverse entire directory trees including nested subdirectories, and run advanced GNU commands to complete multi-step operations. We can connect agents to our own harnesses, build our own graph nodes, and keep them focused on exactly the scope we define. It is unsettling when I think about this in existential-risk terms.
As a recent graduate, I started feeling this unease after spending real time thinking it through, and it's something I find myself turning over at night before falling asleep. I am mentioning this not for sympathy, but because I think it's diagnostic if a 14-month junior in this field can feel it, senior defenders should probably be paying closer attention too.
Traditional cybersecurity had clear rules, procedures, and methodologies, much clearer than what we're dealing with in the post-AI era. If you put in the attention and sustained it, things were achievable. There was always a boundary you had to cross to accomplish something you hadn't done before. If you enjoyed Active Directory adversarial simulations, scripting, port scanning, and lateral movement, it was fairly clear you'd end up on a network penetration testing path. If you were more drawn to application-level work such as directory fuzzing, finding OWASP Top 10 vulnerabilities, responsible disclosure, or advanced exploit development that path was just as available.
By now, AI has absorbed most of this. Not everything, yet operations have become far more feasible, faster, and easier. Prompt a model and it can spin up a cron job for fuzzing operations using GoBuster, FeroxBuster, FFuf, Dirsearch, or dirb, and report back when the process completes. You take the next action from there. I first noticed this pipeline when the Claude Code CLI was introduced. I gave it a structured prompt designed to solve CTF machines in a single attempt, and doing that I noticed how much prompt engineering itself could be sharpened. On the downside, I gradually lost motivation for solving CTF machines altogether. I became more interested in real targets on GitHub, because I missed doing application security work, full DAST, against real-world applications. As I quickly realized, CTF machines could only ever approximate that experience.
To learn deepen the OWASP categories and vulnerability classes I had not personally encountered, I started an agent-augmented vulnerability discovery process. I follow roughly the same approach every time I go after a CVE:
- Fuzz the entire application via the harness.
- Threat model the application, which produces a large schema of the most probable vulnerability candidates.
- Prioritize based on the threat model. I pick scoped vulnerabilities like drawing cards in a Yu-Gi-Oh match resembling draw a card, move toward that vulnerability.
- Report in CNA-supported format, validating through the harness with both exploit generation and manual triggering.
Following this methodology, I ended up doing source code reviews far more often than I expected. Every agent iteration kept surfacing vulnerable code blocks. I'd look at the code and the endpoint, and go after it.
These are operations carried out by someone with 14 months of professional cybersecurity experience. Meanwhile, progress in mechanistic interpretability, refusal-direction ablation specifically, has pushed things into even more concerning territory. Uncensored AI access is now possible through API providers too. I have seen ablated Qwen versions running through routers like Orca. Even when providers ask for permission or legal approval upfront, how do they ensure that only people with benign, responsible intent are the ones using it? How do labs or providers control against threat actors at all? A year ago, refusal-ablated small models existed, but API-level access to that capability didn't. We could at least count on people without significant compute being unable to cause large-scale harm. That is what keeps pulling my thinking back to cyber defense and monitoring operations.
Picture an L1 analyst already struggling against obfuscated, chained, or silent attack attempts today. What happens when the attacker's agent starts generating payloads more sophisticated than what a human could produce alone? I do not have a clean answer. I am not convinced models are more capable on the defensive side than on the offensive side right now, and agents can still deceive, hallucinate, or fail outright, especially across longer task chains.
However, I keep circling back to that hammer. Whoever holds these agents does not need to be worthy anymore unlike Mjölnir, the hammer does not check and more than any benchmark, is what I lose sleep over.