LLM security gateway for prompts and responses

The challenge

Enterprises adopting large language models expose themselves to prompt injection, jailbreaks and leakage of personal or regulated data in both directions. Security teams also lack a record of which models are being used, by whom, and under which controls.

What we built

The platform places a filter in front of the model. Each prompt is scored by a rules and pattern engine (injection, PII, jailbreak, toxicity, data extraction) and then allowed, flagged or blocked against configurable thresholds. Model responses pass through a separate outbound rule set that can redact or block. Requests are served through scoped API keys with per-key rate limits, routed to Gemini or AWS Bedrock, and every decision is written to an audit trail. Incidents carry SLA deadlines and escalation, and an AI inventory records the assets discovered from live API calls.

Key capabilities

• Inbound prompt scoring with allow, flag and block thresholds
• Outbound response filtering using organisation-defined security rules
• API keys with granular permissions, rate limits and IP restrictions
• Incident management with SLA deadlines, assignment and escalation
• Auto-discovery inventory of AI models and assets from observed API calls
• Optional compliance-framework rule packs per organisation, with signed audit log entries

What it delivers

• Prompts and responses are evaluated against explicit, reviewable rules before reaching users or models
• Every filter decision is recorded with threat score and detected threats for later review
• AI model usage across an organisation becomes visible in a single inventory
• Security incidents are tracked against severity-based response deadlines with escalation