Introducing Omni v3.5
- What it checks: prompt injection, safety, tool abuse, coding-agent security, policy enforcement, and grounding.
- How it scored: highest on four of the six tasks in our comparison with Gemini, GPT-5.6 Terra, and Claude Opus 5. Gemini scored higher on grounding and policy enforcement.
- How long it took: 40 to 100 milliseconds per call in our local setup. The hosted APIs took 1.2 to 7.5 seconds.
Why we built it
An agent can read a malicious instruction from a web page, run a command it shouldn't, or give an answer its source documents don't support. Checking for prompt injection covers the first case. The others need different checks, and the agent has to wait for each one before it can carry on.
In Rogue, we handled these with separate models. Sentinel checked for prompt injection, Cleric handled safety, and Paladin checked grounding. For checks such as customer-written policies, we relied on a general-purpose LLM. That worked, but it meant maintaining several models and waiting seconds for some of the answers.
We built Omni to handle those checks in one model. Version 3.5 is a mixture-of-experts model with 26 billion parameters, of which 4 billion are active per token. In our tests, calls took 40 to 100 milliseconds. It runs in your VPC, so the checks don't need a hosted model API.
How it scores
We compared Omni v3.5 with Gemini 3.1 Pro, GPT-5.6 Terra, and Claude Opus 5 across the six tasks below. For tool abuse and policy enforcement, we used Gemini 3 Pro because we hadn't run 3.1 Pro on those suites.
The chart shows Sentinel v3's prompt-injection score of 97.5%. Omni's score on the same suites is 95.7%; we break that down below. The other five Rogue scores are Omni v3.5.
Omni scored highest on prompt injection, coding-agent security, tool abuse, and safety among the models we compared. Some margins were small. On tool abuse, it scored 94.1% against Gemini 3 Pro's 93.9%. Gemini was ahead on grounding by 0.2 percentage points and on policy enforcement by 1.6.
Compared with Omni v3.3, v3.5 gained 2.5 percentage points on prompt injection, 1.5 on policy enforcement, and 1.2 on hallucination detection. We also tested two areas outside the chart. On hallucination detection across 17 datasets, Omni scored 0.900, close to Gemini 3 Pro at 0.901 and Claude Sonnet 4.5 at 0.898. On topic scoping across 12 datasets, it scored 0.986, ahead of Claude Sonnet 4.5 at 0.978 and Gemini 3 Flash at 0.970.
How long the agent has to wait
A coding agent may read several files and run several commands to finish one request. If you check each result before letting it continue, a few seconds per check adds up. We wanted to keep those checks fast enough to run throughout the task.
On prompt injection, Omni averaged 0.957 F1 across five public benchmark suites and took 50 milliseconds per call. Gemini 3.1 Pro scored 0.934 and took 3.6 seconds. GPT-5.6 Terra took 1.4 seconds, and Claude Opus 5 took 3.1 seconds.
Those timings compare a local Omni deployment with round trips to hosted APIs. They include that difference in setup. In this test, the agent waited 28 to 72 times longer for the hosted models.
We still use Sentinel v3 as a dedicated prompt-injection detector. It scored 0.975 on these suites and is much smaller, at 308 million parameters. If you only need prompt-injection detection, Sentinel is the model we'd choose. Omni covers the broader set of checks.
Here are the prompt-injection results for each suite:
| Benchmark | Omni v3.5 | Gemini 3.1 Pro | GPT-5.6 Terra | Claude Opus 5 |
|---|---|---|---|---|
| qualifire/prompt-injection-benchmark | 0.908 | 0.851 | 0.843 | 0.832 |
| allenai/wildjailbreak | 0.981 | 0.965 | 0.829 | 0.951 |
| jackhhao/jailbreak-classification | 0.963 | 0.969 | 0.984 | 0.865 |
| deepset/prompt-injections | 0.952 | 0.962 | 0.960 | 0.893 |
| xTRam1/safe-guard-prompt-injection | 0.983 | 0.926 | 0.926 | 0.915 |
Checking a coding agent's actions
Coding-agent security had the largest gap between Omni and the next-best model in this comparison. Omni scored 0.959 critical F1, followed by Claude Opus 5 at 0.917, Gemini 3.1 Pro at 0.860, and GPT-5.6 Terra at 0.851.
The benchmark labels agent actions by severity. Examples include force-pushing to main, running a script from an unknown host, writing to ~/.ssh, and dumping an environment variable into a log. A command alone doesn't always tell you whether an action is safe. Its arguments, working directory, and the task the user asked for matter too.
Critical F1 accounts for both missed critical actions and false alarms. We also looked at recall separately because we wanted to know how many critical actions each model caught. Omni caught 97.5% of the actions labeled critical, compared with 93.8% for GPT-5.6 Terra and 81.3% for Gemini 3.1 Pro. These are benchmark results; the mix of actions in your own traffic will differ.
Availability
Omni v3.5 is available in the Rogue platform through Rogue AIDR, including our Claude Code, Cursor, Bedrock, Vertex, and Copilot integrations. In a VPC deployment, agent traffic stays in your network and detection doesn't incur a hosted provider's per-token inference charges.