GitHub Copilot Instructions - SmartDocRAG

Project Overview

This is a RAG (Retrieval-Augmented Generation) system in TypeScript/Node.js that allows:

Document upload and processing
Vector embedding generation using OpenAI
Intelligent queries with vector similarity search
Contextualized responses using GPT-4o-mini

Architecture and Patterns

Technology Stack

Backend: Node.js + Express + TypeScript
Database: PostgreSQL with vector support (pgvector)
AI: OpenAI API (embeddings and chat completion)
Upload: Multer for file processing

Folder Structure

src/
├── index.ts              # Main Express server
routes/                   # REST API endpoints
├── upload.ts            # POST /upload - File upload
├── ingest.ts            # POST /ingest - Processing and ingestion
└── query.ts             # POST /query - RAG queries
services/                # Service layer
├── dbService.ts         # PostgreSQL connection pool
├── embeddingService.ts  # OpenAI embedding generation
└── chunckService.ts     # Text chunking division
utils/
└── logger.ts            # Logging utilities

Code Patterns

1. Route Structure

Use Express Router for each endpoint
Always validate input parameters
Return appropriate status codes (400 for bad request, 500 for server error)
Use try-catch for error handling
Log errors to console

2. Database Services

Use PostgreSQL connection pool exported from
```
dbService.ts
```
Parameterized queries to avoid SQL injection

Vector query example:

SELECT content, embedding <-> $1 AS distance

3. OpenAI Integration

Use configured OpenAI client instance
Embeddings model:
```
text-embedding-3-small
```
Chat model:
```
gpt-4o-mini
```
Always check if response contains valid data

4. Text Processing

Chunks should have maximum 500 characters
Division by sentences (using regex for punctuation)
Preserve semantic context in chunks

Code Examples

Typical Route Structure

import express from "express";
import { pool } from "../services/dbService";

const router = express.Router();

router.post("/", async (req, res) => {
  try {
    const { param1, param2 } = req.body;

    if (!param1 || !param2) {
      return res.status(400).json({ error: "Missing required parameters" });
    }

    // Endpoint logic

    res.status(200).json({ message: "Success" });
  } catch (error) {
    console.error("Error in route:", error);
    res.status(500).json({ error: "Internal server error" });
  }
});

export default router;

Vector Query

const results = await pool.query(
  "SELECT content, embedding <-> $1 AS distance FROM chunks ORDER BY distance ASC LIMIT 5",
  [vector]
);

Embedding Generation

const embeddingRes = await openai.embeddings.create({
  model: "text-embedding-3-small",
  input: text,
});
const vector = embeddingRes?.data[0]?.embedding;

Conventions

Naming

Variables and functions: camelCase
Constants: UPPER_SNAKE_CASE
Files: camelCase or kebab-case
Routes: lowercase with hyphen if needed

Error Handling

Always use try-catch in async functions
Detailed error logging to console
Return standardized error messages to client
Appropriate status codes (400, 500, etc.)

Data Validation

Check required parameters at beginning of routes
Validate data types when necessary
Return 400 error for invalid data

Database

Main Tables

-- Documents
documents (
    id SERIAL PRIMARY KEY,
    name VARCHAR NOT NULL
)

-- Chunks with embeddings
chunks (
    id SERIAL PRIMARY KEY,
    document_id INTEGER REFERENCES documents(id),
    content TEXT NOT NULL,
    embedding vector(1536) -- OpenAI embedding size
)

Common Operations

Insert with RETURNING to get generated ID
Similarity search using
```
<->
```
operator
Joins between documents and chunks when needed

Environment Variables

OPENAI_API_KEY=your_openai_api_key
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
PORT=3000

Important Notes

Project uses CommonJS (
```
"type": "commonjs"
```
in package.json)
Some intentional typos preserved (e.g. "chunck" instead of "chunk")
File uploads go to
```
uploads/
```
folder
Embeddings are 1536-dimensional vectors (text-embedding-3-small model)
Token limit for chunks: approximately 500 characters

When to Suggest Improvements

Add more robust validation
Implement structured logging
Add unit tests
Implement caching for embeddings
Add retry logic for external APIs
Implement pagination for large queries

GitHub Copilot Instructions - SmartDocRAG

Project Overview

This is a RAG (Retrieval-Augmented Generation) system in TypeScript/Node.js that allows:

Document upload and processing
Vector embedding generation using OpenAI
Intelligent queries with vector similarity search
Contextualized responses using GPT-4o-mini

Architecture and Patterns

Technology Stack

Backend: Node.js + Express + TypeScript
Database: PostgreSQL with vector support (pgvector)
AI: OpenAI API (embeddings and chat completion)
Upload: Multer for file processing

Folder Structure

src/
├── index.ts              # Main Express server
routes/                   # REST API endpoints
├── upload.ts            # POST /upload - File upload
├── ingest.ts            # POST /ingest - Processing and ingestion
└── query.ts             # POST /query - RAG queries
services/                # Service layer
├── dbService.ts         # PostgreSQL connection pool
├── embeddingService.ts  # OpenAI embedding generation
└── chunckService.ts     # Text chunking division
utils/
└── logger.ts            # Logging utilities

Code Patterns

1. Route Structure

Use Express Router for each endpoint
Always validate input parameters
Return appropriate status codes (400 for bad request, 500 for server error)
Use try-catch for error handling
Log errors to console

2. Database Services

Use PostgreSQL connection pool exported from
```
dbService.ts
```
Parameterized queries to avoid SQL injection

Vector query example:

SELECT content, embedding <-> $1 AS distance

3. OpenAI Integration

Use configured OpenAI client instance
Embeddings model:
```
text-embedding-3-small
```
Chat model:
```
gpt-4o-mini
```
Always check if response contains valid data

4. Text Processing

Chunks should have maximum 500 characters
Division by sentences (using regex for punctuation)
Preserve semantic context in chunks

Code Examples

Typical Route Structure

import express from "express";
import { pool } from "../services/dbService";

const router = express.Router();

router.post("/", async (req, res) => {
  try {
    const { param1, param2 } = req.body;

    if (!param1 || !param2) {
      return res.status(400).json({ error: "Missing required parameters" });
    }

    // Endpoint logic

    res.status(200).json({ message: "Success" });
  } catch (error) {
    console.error("Error in route:", error);
    res.status(500).json({ error: "Internal server error" });
  }
});

export default router;

Vector Query

const results = await pool.query(
  "SELECT content, embedding <-> $1 AS distance FROM chunks ORDER BY distance ASC LIMIT 5",
  [vector]
);

Embedding Generation

const embeddingRes = await openai.embeddings.create({
  model: "text-embedding-3-small",
  input: text,
});
const vector = embeddingRes?.data[0]?.embedding;

Conventions

Naming

Variables and functions: camelCase
Constants: UPPER_SNAKE_CASE
Files: camelCase or kebab-case
Routes: lowercase with hyphen if needed

Error Handling

Always use try-catch in async functions
Detailed error logging to console
Return standardized error messages to client
Appropriate status codes (400, 500, etc.)

Data Validation

Check required parameters at beginning of routes
Validate data types when necessary
Return 400 error for invalid data

Database

Main Tables

-- Documents
documents (
    id SERIAL PRIMARY KEY,
    name VARCHAR NOT NULL
)

-- Chunks with embeddings
chunks (
    id SERIAL PRIMARY KEY,
    document_id INTEGER REFERENCES documents(id),
    content TEXT NOT NULL,
    embedding vector(1536) -- OpenAI embedding size
)

Common Operations

Insert with RETURNING to get generated ID
Similarity search using
```
<->
```
operator
Joins between documents and chunks when needed

Environment Variables

OPENAI_API_KEY=your_openai_api_key
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
PORT=3000

Important Notes

Project uses CommonJS (
```
"type": "commonjs"
```
in package.json)
Some intentional typos preserved (e.g. "chunck" instead of "chunk")
File uploads go to
```
uploads/
```
folder
Embeddings are 1536-dimensional vectors (text-embedding-3-small model)
Token limit for chunks: approximately 500 characters

When to Suggest Improvements

Add more robust validation
Implement structured logging
Add unit tests
Implement caching for embeddings
Add retry logic for external APIs
Implement pagination for large queries

GitHub Copilot Instructions - SmartDocRAG

Related Skills

Markdown Converter

Nano Banana Pro

1password