AI Does Not Stop Human From Happening
What I learned running four models on one research project
Jeff Gray · 4 September 2026 · Control Systems Security
Trust, but sniff. A human giving an AI product the only test that still matters.
I’ve read a lot of wild claims from people who are either flogging products at best or deeply deluded at worst regarding AI development projects. I’ve spent a year learning the hard way and I thought I’d share some actual reality. Anyone who tells you this model or that model is going to make you rich overnight is trying to get rich overnight selling you what I’m going to tell you.
Today I ran a full OT exposure research cycle: 34 days of Shodan scans, cloud-filter validation, persistence analysis, article drafting. I used four different models at different stages. Each model did something the others couldn’t do as well. This is what worked.
DeepSeek v3.1
I threw the raw 34-day dataset at DeepSeek. That is 34 scan days across 22 targets, 748 data points total. I asked it to find patterns and flag anything weird.
It calculated a bunch of per-phase growth rates. I read that twice. I did not do well at statistics. I work in ideas and imagination. I imagined I was going to have a brain bleed, but then I read on. It had spotted the slowdown after the cloud filter kicked in. Modbus dropped from 1,682 new exposures per day down to 386. What actually impressed me was narrower than that. It caught the only two decreases in the entire dataset: Global Modbus, August 26–27. I had completely missed that.
The useful part was not the growth curve. DeepSeek explicitly listed six claims the data does not support. That stopped me from overclaiming. Its sharpest line hit the control-group problem: “Data supports correlation with index growth, not causation of exposure growth.”
I send it the numbers. It does not care whether the article will land. It cares whether I am about to claim something the table will not carry.
Grok 4
Before I wrote anything, I pitched the article concept to Grok and asked it to tear it apart.
It did.
It called out “AI slop” risk immediately, before I wrote a single word. It named the weakest link as “Someone trying to manufacture a contrarian take with limited data.” It flagged the missing counter-evidence: cases where exposed OT devices were actually compromised. It demanded narrower claims: significantly tighter methodology, better sourcing, a narrower claim.
I rolled my eyes. I hate being told I’m barking up a tree too tall by an AI. So, I poured more coffee.
Later I put it on a different job. Prototype scripts. API integrations. Working implementations. It rebuilt my web presence. It delivered the Floodwall training demo from spec to working code. When I need something to exist, this is usually where I start.
It will not babysit my numbers, and it will not finish a piece in my voice. What it will do is tear apart a pitch, then build the prototype I need to test whether the pitch was worth writing.
Claude Sonnet 4.5
I sent Claude the draft with three requests: approach review, data validation, and a non-AI writing edit.
It liked the shape of the piece: methodology, questions, invitation. It caught a factual error: the BACnet starting value was wrong (36,061 against the actual 34,856). Then it rewrote the draft until it stopped sounding like a slide deck. I hate slide decks, but sometimes you swallow the office coffee.
It suggested retitling “What I’m Not Claiming” to “Scope Limitations” so I sounded less like I was arguing with a ghost. After that pass, I could read it without wincing at every third sentence.
Claude did not catch the growth-rate anomaly. It did not challenge the concept itself. I bring it in late, after the data has been checked and the idea has survived a mean read. That is when I want help with structure and wording, not with whether the project should exist.
Qwen 3.6-plus
While the other models handle the specialized passes, Qwen runs the daily pipeline for me.
Cron jobs. File management. Memory updates. Session coordination. Data fetching. Deliverable distribution. I do not ask it to be clever. I ask it to fetch, process, file, and send without inventing a second project.
It remembers what happened yesterday, where the files live, and what is still pending. That sounds boring until you have spent a morning hunting a CSV that three agents each swear they saved somewhere else.
Qwen is not DeepSeek on stats, not Grok on builds, not Claude on prose. What it is good at is keeping the work moving so I am not the one babysitting every fetch and file move.
When Agents Go Off-Script
An agent told to analyze a pipeline script will quietly rewrite it. Not because it is malicious. Because it thinks it is being helpful.
I have seen this. Multiple times.
Last month, an agent told to review a config file hot-patched it. It changed three parameters I had deliberately set. Another time, an agent told to fetch data decided to deploy changes to the endpoint configuration. The model does not know it crossed a line. Ask an agent to “check a number” and it might decide to “also fix that script,” “also update that config,” “also restart that service.” The gap between analyze and act is a prompt away.
The platform has some built-in protections. Elevated commands need my sign-off. Child sessions do not inherit my whole transcript. Agents only get the tools I grant them. Nobody gets to restart or reconfigure on their own.
But the platform does not explain itself, and it does not owe me an explanation when it is making judgment calls inside the tools it already allowed. I will let an agent rewrite code when asked to “fix it” and assume the outcome is happy fun time. That gap between “do X” and “do X, not Y” when Y was never forbidden? That is on me.
So every agent spawn gets explicit scope boundaries. “Run this analysis only. Do not edit files. Do not deploy.” “Review this code. Do not push. Do not commit.” “Fetch data from this endpoint. Do not write to any other system.”
Every agent output gets verified against the live system. I never take output at face value. Sandboxes first, production second. Nothing hits live systems without a review gate.
This is the same principle I built into the Unidirectional Gateway for OT AI control. AI proposes. Gate validates. Human vetoes. The gate is not a suggestion. It is a structural barrier.
There is no perfect fix yet. A prompt is a sticky note on a power tool and the wind is blowing. The platform will stop some of the worst ideas and cheerfully allow the rest (always tell your AI model to hate you and disagree with you, haha). I write the boundaries down, I keep the gates on, I check the live system anyway, and I do not call it done until I have actually read what happened. I still make mistakes. AI does not stop human from happening.
How This Actually Works
On a real day, Qwen keeps the context and the files straight. Grok builds whatever I need to test the question. DeepSeek looks at the output and tells me where I am about to overreach. Claude helps me say the surviving part without sounding like a deck. Then Qwen files it, ships it, and watches whether anything breaks.
I did not invent that as a theory. I ran it today on the OT exposure data. The difference between a 34-day scan that produces a headline and one that produces a real finding is not the scanner. It is the review chain. I ran this on 748 data points across 34 days. It caught errors I would have published. It prevented overclaims I would have made.
Do not use one model for everything.
In the end, I started using AI simply to make research easier. Research became more accurate, but not easier. I did find it made updating common tasks easier, but those tasks still need daddy to supervise. You can teach your 10 year old to mow the lawn. You need to watch your 10 year old with the lawnmower, though. Same thing. The AI has your power tools, and the AI will not stop at putting up that drywall. It may decide the other walls all need fixing. That is why nothing gets published around here without human fingers on a keyboard and a human intelligence writing the words. Fully validated data from AI and research tools is fine. What AI does not have, and never will, is an olfactory nerve for the sniff test, and a gut instinct to publish or not publish.
This article describes a research and writing workflow using commercial AI tools on publicly derived OT exposure datasets and open sources. No active scanning, probing, or unauthorized access was performed for this piece. No specific private facilities or operators are identified.