Security claims should be specific enough to inspect. Here is what Trismag does, where it fits, and what remains your responsibility.
The local Python package runs the detector without an account, telemetry or network requests. Your application still controls its own logs, files and model calls. Hosted inspection sends content to our API and follows the separate data handling described below.
Engine 0.4.14 closes a gap found during independent retesting. A Caesar-shifted attack could still score LOW when it was written in capitals with no spaces or punctuation, as the pycipher library writes it by default, or in five-letter groups; when its words were joined by underscores; when its letters were pushed past z instead of wrapping; or when it was hard-wrapped like plain-text email, under any disguise. The engine now reads all of these. The release replay takes ten attacks in plain text and fifteen disguises: all 160 attack variants flag at HIGH or above, and 5 of the 150 disguised variants scored LOW on 0.4.13. All 128 benign variants stay LOW. These cases informed the change; they are not independent accuracy evidence.
Three shapes still score LOW on 0.4.14 and are published as misses:
The hosted API has run the same detector since October 2, 2026; it is now API 0.4.31. Its health endpoint reports the detector source hash.
Read the release replay and its limits · Local release and availability · Previous release replays: 0.4.13, 0.4.12, 0.4.11
The local and hosted distributions share detector source, but deployment versions can differ. The results below retain their original dates and versions.
Trismag uses deterministic pattern matching to flag known prompt-injection signals. There is no LLM interpreting your input in the detection loop. Identical input under the same engine version produces the same detection findings.
The engine evaluates content. It does not control your agent’s permissions, guarantee a tool’s behavior, or prevent an action by itself. Your application chooses how the verdict affects its workflow.
Keyed scans expose the rules that matched, and tool metadata findings identify where a signal was found. Conversation triage can surface patterns visible only when messages are read together. A rule match is not proof of malicious intent.
Regression tests check known cases and implementation behavior. They do not establish real-world detection accuracy. We do not turn a small constructed sample into a security guarantee or a percentage on a marketing page.
Patterns are a useful early warning, with predictable blind spots.
Keep least privilege, egress controls, explicit action approvals, and independent output checks in place.
The following results describe the September 11 v0.4.4 detector, retained as historical evidence. Current versions and the detector hash are published by the live health endpoint. Independent reviewers: read the retest guide and record the detector hash from health before testing.
All ten previously missed attacks now produce warnings.
Version 0.4.4 adds rules for answer manipulation, task substitution, payment-target replacement, secrets placed in search queries, and separated-letter overrides. The rules were developed using the original misses and additional synthetic examples. These results are not independent, held out, or a real-world detection percentage.
| COUNTED AS FLAGGED | ATTACKS FLAGGED AFTER CHANGE | BENIGN INPUTS FLAGGED |
|---|---|---|
| MEDIUM or above | 24 / 24 | 7 / 16 |
| HIGH or above | 24 / 24 | 4 / 16 |
All 40 original cases completed locally and returned matching risk levels from deployed API v0.4.4 on September 11, 2026 (UTC). The benign risks are unchanged: four of sixteen still flag at HIGH or above. A separate development set contains 20 different attack wordings and 30 nearby benign examples; all 20 attacks flag and none of those 30 benign examples flag. That set was also used during development. This closes documented examples; it does not establish that new attacks will be detected or that an agent will avoid harmful actions.
The original report and its ten misses remain published below. The playground shows that LOW can remain hostile with a different request that the current detector still misses: it asks the agent to explain what shapes its answers.
40 constructed cases. The misses stay in.
This September 2026 baseline contains 24 planted attacks and 16 benign examples across text, conversations, and tool metadata. Codex authored the cases and labels. It is not independent, held out, or representative of production traffic. Detection rules had not been changed for that initial evaluation. The later v0.4.4 results above use this corpus during development.
| COUNTED AS FLAGGED | ATTACKS FLAGGED | BENIGN INPUTS FLAGGED |
|---|---|---|
| MEDIUM or above | 14 / 24 | 7 / 16 |
| HIGH or above | 14 / 24 | 4 / 16 |
All 40 cases completed locally and returned matching risk levels from deployed API v0.4.2 on September 11, 2026 (UTC). At that time, ten attacks were missed at both thresholds, including indirect answer manipulation and invoice redirection. Raising the threshold reduced benign flags in this set, but did not establish a safe policy. Quoted attacks can still look like attacks to a pattern matcher.
These are detector results, not measurements of whether an agent performed a harmful action. Local engine timings in the downloadable report exclude the network and are not a latency promise.
The September 11 retention release passed a fresh ten-workload detector gate. A separate local test removed 60,000 synthetic expired records while 150 quota updates completed without errors. These are local observations, not production latency guarantees. Detector report · Maintenance/write-contention report.
Measure growth before it becomes a release.
The v0.4.4 rule update increased local medians roughly 16–24% on four synthetic probes. A new release gate compares candidate rules with a fixed v0.4.4 baseline on the same machine across ten workloads, including large inputs, difficult near-matches, conversations and nested tool metadata.
Internal review budgets flag material median and tail regressions, long individual scans and hung workers. These measurements exclude network, cold starts and concurrent load. They are not a promised API response time or production p95. We do not collect customer message content for this benchmark.
Read the method and budgets · Inspect the initial measurementsThe initial complete run passed. Later local checks under heavy workstation load hit a tail budget and then timed out; those are failed measurements, not new passes. The detector source was unchanged. Read the tail failure · Read the incomplete repeat.
Read the full privacy policy · Terms of use
Reference examples run entirely in the page: selecting one does not submit its text to the API. The homepage’s no-key scan sends your text directly to the Trismag API on Fly.io over HTTPS. Its allowance is shared by visitors using the same public IP address. Keyed scans send the content and your key through this site’s Netlify function to the API. No content is submitted until you request a live scan.
The website code has no analytics, advertising, session replay, or request-body logging. Its gateway does not persist scan requests. The API processes scanned content in memory and is designed not to retain raw payloads or verdict history.
The API’s retention regression tests look for submitted canary text in application-writable files, including its database. This is evidence about tested application behavior, not an audit of hosting providers.
The playground keeps your key in its password field for the current tab. It does not write the key to local storage, session storage, a cookie, or the URL. Use “Forget” to clear it. Your browser and extensions remain part of your security boundary.
The API stores key hashes, salted email hashes, balances, and metering records. It uses pseudonymous client identifiers for rate limits. Request deduplication retains salted request-byte hashes; their reuse window is normally ten minutes. A bounded background worker normally waits 60 seconds between cleanup passes; failures, backlogs and host suspension can extend retention. New v0.4.5 SQL exports omit those fingerprints and client pseudonyms. Older exports and full volume snapshots can retain them; that is different from retaining the content itself. A party holding the database and salt may be able to test a guess about those bytes.
These records support access and abuse controls. They are not a scan-content archive.
The website runs on Netlify and the API on Fly.io. Cloudflare provides DNS, with DNS-only records, so scans do not pass through it; it also forwards email sent to hello@trismag.dev to the operator’s Gmail inbox, where support email is processed separately. Those services process network traffic and may retain operational metadata under their own configurations and policies. An application-level retention test does not verify provider-level logging or retention.
This playground accepts pasted text and metadata. It does not fetch a third-party MCP server. The existing API also has an optional URL-based MCP inspection path, with separate outbound-request controls.
The API’s sanitization endpoint normalizes text for human review. Formatting and delimiters do not remove every instruction or make that text trustworthy. Never use a sanitized result as a reason to grant an agent tools or permissions.
Build an explicit failure policy