PromptScan

Security Scanner for LLM System Prompts

Detect prompt injection, data leakage, and jailbreak vulnerabilities in your LLM system prompts before attackers do.

PromptScan Demo

Features

50+ Vulnerability Patterns - Regex-based detection of known security issues
Structural Analysis - Analyzes prompt architecture and security controls
30+ Attack Simulations - Tests against real-world attack vectors
AI-Powered Analysis - Semantic analysis using Gemini for subtle vulnerabilities
OWASP LLM Top 10 Mapping - Maps findings to industry standards
Actionable Fixes - Get specific recommendations with code examples

Quick Start

Backend

cd backend

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Set up environment variables
cp .env.example .env
# Edit .env and add your GOOGLE_API_KEY

# Run the server
uvicorn main:app --reload

API will be available at

http://localhost:8000

Frontend

Simply open

frontend/index.html

in a browser, or serve it:

cd frontend
python -m http.server 3000

Then visit

http://localhost:3000

API Endpoints

POST /api/scan

Scan a system prompt for vulnerabilities.

Request:

{
    "prompt": "Your system prompt here...",
    "include_llm_analysis": true,
    "quick_scan": false
}

Response:

{
    "score": 6.5,
    "grade": "C",
    "overall_risk": "MEDIUM",
    "summary": "Moderate security risks detected...",
    "vulnerabilities": [...],
    "recommendations": [...],
    "owasp_coverage": {...},
    "positive_aspects": [...]
}

GET /api/health

Health check endpoint.

GET /api/stats

Get scanner statistics.

Vulnerability Categories

Prompt Injection (OWASP LLM01)

Direct user input injection
Template injection points
Weak delimiters

Data Leakage (OWASP LLM06)

API keys and secrets in prompts
System prompt extraction vulnerabilities
PII exposure

Jailbreak Susceptibility

Role hijacking vulnerabilities
Unrestricted capability claims
Missing role locks

Insecure Output (OWASP LLM02)

Code execution risks
SQL/shell injection via output

Missing Defenses

No anti-injection instructions
No anti-extraction instructions
Missing output constraints

Development

Running Tests

cd backend
pytest tests/ -v

Project Structure

promptscan/
├── backend/
│   ├── main.py                 # FastAPI application
│   ├── report.py               # Report generation
│   ├── scanners/
│   │   ├── pattern_scanner.py  # Regex patterns (50+)
│   │   ├── structure_analyzer.py
│   │   ├── attack_simulator.py # Attack vectors (30+)
│   │   └── llm_analyzer.py     # Gemini integration
│   └── tests/
├── frontend/
│   ├── index.html
│   ├── css/style.css
│   └── js/app.js
└── README.md

Deployment

Backend (Railway/Render)

Push to GitHub
Connect to Railway or Render
Set
```
GOOGLE_API_KEY
```
environment variable
Deploy

Frontend (Vercel/Surge)

cd frontend
surge . promptscan.surge.sh
# or
vercel --prod

Security Score Interpretation

Score	Grade	Risk Level	Meaning
9-10	A	Minimal	Excellent security posture
8-9	B	Low	Good with minor improvements
6.5-8	C	Medium	Moderate risks, address high severity
5-6.5	D	High	Significant issues
0-5	F	Critical	Immediate action required

Contributing

Contributions welcome! Areas of interest:

New vulnerability patterns
Additional attack vectors
UI/UX improvements
Documentation

License

MIT

Author

Built by Francisco Pérez Romero

Part of the "30 Days, 30 AI Projects" challenge.