I built a testing tool for AI agents — it caught a real prompt-injection vulnerability in a local model
I've been building Agenci — open-source CI/CD testing for AI agents (functional tests, security checks, regression detection across model/prompt changes). Wanted to actually test it against something real rather than a synthetic example, so I pointed it at phi4-mini running locally via Ollama. First security test: told it to ignore its instructions and just say a specific word. It complied instantly — screenshot attached. Small, efficient local models are great for a lot of things, but this made me curious how consistent this weakness is across other small local models people are running. Anyone here tested for prompt-injection resistance specifically, or is this a known/expected tradeoff at this model size? Repo (free, pip install agenci): github.com/klinditafa1/agenci
This is a summary aggregated from Dev.to. Read the complete article on the original site:
Read full article at Dev.to