This skill provides an intelligent configuration generator for the vLLM Semantic Router project. It helps users create proper configuration files for semantic routing based on their specific requirements and use cases.
Sign in to like and favorite skills
This skill provides an intelligent configuration generator for the vLLM Semantic Router project. It helps users create proper configuration files for semantic routing based on their specific requirements and use cases.
The Semantic Router Configurator Skill enables agents to:
Run the skill as a Docker container:
docker build -t semantic-router-configurator . docker run -p 8080:8080 semantic-router-configurator
POST /generate
Generate a semantic router configuration based on provided parameters.
Request body:
{ "endpoints": [ { "name": "default", "url": "http://localhost:8000/v1/chat/completions" } ], "plugins": [ { "type": "semantic-cache", "enabled": true, "similarity_threshold": 0.92 } ], "port": 8888 }
Response:
# Generated semantic router configuration version: "1.0" listeners: - port: 8888 endpoints: - name: default url: http://localhost:8000/v1/chat/completions decisions: - name: default_route endpoint: default plugins: - type: semantic-cache configuration: enabled: true similarity_threshold: 0.92
POST /validate
Validate a semantic router configuration file.
Request body:
{ "config": "version: \"1.0\"\nlisteners:\n - port: 8888" }
Response:
{ "valid": true, "errors": [] }
version: "1.0" listeners: - port: 8888 endpoints: - name: default url: http://localhost:8000/v1/chat/completions decisions: - name: default_route endpoint: default
version: "1.0" listeners: - port: 8888 endpoints: - name: default url: http://localhost:8000/v1/chat/completions decisions: - name: cached_route endpoint: default plugins: - type: semantic-cache configuration: enabled: true similarity_threshold: 0.92 ttl_seconds: 3600
version: "1.0" listeners: - port: 8888 endpoints: - name: gpt4 url: http://localhost:8001/v1/chat/completions - name: gpt3 url: http://localhost:8002/v1/chat/completions decisions: - name: secure_route endpoint: gpt4 plugins: - type: jailbreak configuration: enabled: true threshold: 0.8 - type: pii configuration: enabled: true threshold: 0.7 - type: semantic-cache configuration: enabled: true similarity_threshold: 0.92
This skill is designed to be used by agents through the kagent framework. The skill exposes a RESTful API that agents can call to generate and validate semantic router configurations.
See the main repository for development guidelines and contribution instructions.