LMQL

A programming language for large language models.
Documentation »

Explore Examples · Playground IDE · Report Bug

LMQL is a programming language for large language models (LLMs) based on a superset of Python. LMQL offers a novel way of interweaving traditional programming with the ability to call LLMs in your code. It goes beyond traditional templating languages by integrating LLM interaction natively at the level of your program code.

Help us shape the next major version of LMQL by filling out the LMQL developer survey: https://forms.gle/pGvAicNpUhS1rAkK9

Explore LMQL

An LMQL program reads like standard Python, but top-level strings are interpreted as query strings: They are passed to an LLM, where template variables like

[GREETINGS]

are automatically completed by the model:

"Greet LMQL:[GREETINGS]\n" where stops_at(GREETINGS, ".") and not "\n" in GREETINGS

if "Hi there" in GREETINGS:
    "Can you reformulate your greeting in the speech of \
     victorian-era English: [VIC_GREETINGS]\n" where stops_at(VIC_GREETINGS, ".")

"Analyse what part of this response makes it typically victorian:\n"

for i in range(4):
    "-[THOUGHT]\n" where stops_at(THOUGHT, ".")

"To summarize:[SUMMARY]"

Program Output:

LMQL allows you to express programs that contain both, traditional algorithmic logic, and LLM calls. At any point during execution, you can prompt an LLM on program variables in combination with standard natural language prompting, to leverage model reasoning capabilities in the context of your program.

To better control LLM behavior, you can use the

where

keyword to specify constraints and data types of the generated text. This enables guidance of the model's reasoning process, and constraining of intermediate outputs using an expressive constraint language.

Beyond this linear form of scripting, LMQL also supports a number of decoding algorithms to execute your program, such as

argmax

sample

or even advanced branching decoders like beam search and

best_k

Learn more about LMQL by exploring thne Example Showcase, by running your own programs in our browser-based Playground IDE or by reading the documentation.

Feature Overview

LMQL is designed to make working with language models like OpenAI and 🤗 Transformers more efficient and powerful through its advanced functionality, including multi-variable templates, conditional distributions, constraints, datatypes and control flow.

Getting Started

To install the latest version of LMQL run the following command with Python ==3.10 installed.

pip install lmql

Local GPU Support: If you want to run models on a local GPU, make sure to install LMQL in an environment with a GPU-enabled installation of PyTorch >= 1.11 (cf. https://pytorch.org/get-started/locally/) and install via

pip install lmql[hf]

Running LMQL Programs

After installation, you can launch the LMQL playground IDE with the following command:

lmql playground

Using the LMQL playground requires an installation of Node.js. If you are in a conda-managed environment you can install node.js via
conda install nodejs=14.20 -c conda-forge
. Otherwise, please see the official Node.js website https://nodejs.org/en/download/ for instructions how to install it on your system.

This launches a browser-based playground IDE, including a showcase of many exemplary LMQL programs. If the IDE does not launch automatically, go to

http://localhost:3000

Alternatively,

lmql run

can be used to execute local

.lmql

files. Note that when using local HuggingFace Transformers models in the Playground IDE or via

lmql run

, you have to first launch an instance of the LMQL Inference API for the corresponding model via the command

lmql serve-model

Configuring OpenAI API Credentials

If you want to use OpenAI models, you have to configure your API credentials. To do so you can either define the

OPENAI_API_KEY

environment variable or create a file

api.env

in the active working directory, with the following contents:

openai-org: <org identifier>
openai-secret: <api secret>

For system-wide configuration, you can also create an

api.env

file at

$HOME/.lmql/api.env

or at the project root of your LMQL distribution (e.g.

src/

in a development copy).

Alternatively, you can use LMQL-specific env variables

LMQL_OPENAI_SECRET

and

LMQL_OPENAI_ORG

Installing the Latest Development Version

To install the latest (bleeding-edge) version of LMQL, you can also run the following command:

pip install git+https://github.com/eth-sri/lmql

This will install the

lmql

package directly from the

main

branch of this repository. We do not continously test the

main

version, so it may be less stable than the latest PyPI release.

Contributing

LMQL is a community-centric project. If you are interested in contributing to LMQL, please see the contributing guidelines for more information, and reach out to us via Discord. We are looking forward to your contributions!

Setting Up a Development Environment

To setup a

conda

environment for local LMQL development with GPU support, run the following commands:

# prepare conda environment
conda env create -f scripts/conda/requirements.yml -n lmql
conda activate lmql

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh

Operating System: The GPU-enabled version of LMQL was tested to work on Ubuntu 22.04 with CUDA 12.0 and Windows 10 via WSL2 and CUDA 11.7. The no-GPU version (see below) was tested to work on Ubuntu 22.04 and macOS 13.2 Ventura or Windows 10 via WSL2.

Development without GPU

This section outlines how to setup an LMQL development environment without local GPU support. Note that LMQL without local GPU support only supports the use of API-integrated models like

openai/text-davinci-003

. Please see the OpenAI API documentation (https://platform.openai.com/docs/models/gpt-3-5) to learn more about the set of available models.

To setup a

conda

environment for LMQL with no GPU support, run the following commands:

# prepare conda environment
conda env create -f scripts/conda/requirements-no-gpu.yml -n lmql-no-gpu
conda activate lmql-no-gpu

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh

LMQL

A programming language for large language models.
Documentation »

Explore Examples · Playground IDE · Report Bug

Help us shape the next major version of LMQL by filling out the LMQL developer survey: https://forms.gle/pGvAicNpUhS1rAkK9

Explore LMQL

An LMQL program reads like standard Python, but top-level strings are interpreted as query strings: They are passed to an LLM, where template variables like

[GREETINGS]

are automatically completed by the model:

"Greet LMQL:[GREETINGS]\n" where stops_at(GREETINGS, ".") and not "\n" in GREETINGS

if "Hi there" in GREETINGS:
    "Can you reformulate your greeting in the speech of \
     victorian-era English: [VIC_GREETINGS]\n" where stops_at(VIC_GREETINGS, ".")

"Analyse what part of this response makes it typically victorian:\n"

for i in range(4):
    "-[THOUGHT]\n" where stops_at(THOUGHT, ".")

"To summarize:[SUMMARY]"

Program Output:

To better control LLM behavior, you can use the

where

Beyond this linear form of scripting, LMQL also supports a number of decoding algorithms to execute your program, such as

argmax

sample

or even advanced branching decoders like beam search and

best_k

Learn more about LMQL by exploring thne Example Showcase, by running your own programs in our browser-based Playground IDE or by reading the documentation.

Feature Overview

Getting Started

To install the latest version of LMQL run the following command with Python ==3.10 installed.

pip install lmql

pip install lmql[hf]

Running LMQL Programs

After installation, you can launch the LMQL playground IDE with the following command:

lmql playground

Using the LMQL playground requires an installation of Node.js. If you are in a conda-managed environment you can install node.js via
conda install nodejs=14.20 -c conda-forge
. Otherwise, please see the official Node.js website https://nodejs.org/en/download/ for instructions how to install it on your system.

This launches a browser-based playground IDE, including a showcase of many exemplary LMQL programs. If the IDE does not launch automatically, go to

http://localhost:3000

Alternatively,

lmql run

can be used to execute local

.lmql

files. Note that when using local HuggingFace Transformers models in the Playground IDE or via

lmql run

, you have to first launch an instance of the LMQL Inference API for the corresponding model via the command

lmql serve-model

Configuring OpenAI API Credentials

If you want to use OpenAI models, you have to configure your API credentials. To do so you can either define the

OPENAI_API_KEY

environment variable or create a file

api.env

in the active working directory, with the following contents:

openai-org: <org identifier>
openai-secret: <api secret>

For system-wide configuration, you can also create an

api.env

file at

$HOME/.lmql/api.env

or at the project root of your LMQL distribution (e.g.

src/

in a development copy).

Alternatively, you can use LMQL-specific env variables

LMQL_OPENAI_SECRET

and

LMQL_OPENAI_ORG

Installing the Latest Development Version

To install the latest (bleeding-edge) version of LMQL, you can also run the following command:

pip install git+https://github.com/eth-sri/lmql

This will install the

lmql

package directly from the

main

branch of this repository. We do not continously test the

main

version, so it may be less stable than the latest PyPI release.

Contributing

Setting Up a Development Environment

To setup a

conda

environment for local LMQL development with GPU support, run the following commands:

# prepare conda environment
conda env create -f scripts/conda/requirements.yml -n lmql
conda activate lmql

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh

Operating System: The GPU-enabled version of LMQL was tested to work on Ubuntu 22.04 with CUDA 12.0 and Windows 10 via WSL2 and CUDA 11.7. The no-GPU version (see below) was tested to work on Ubuntu 22.04 and macOS 13.2 Ventura or Windows 10 via WSL2.

Development without GPU

This section outlines how to setup an LMQL development environment without local GPU support. Note that LMQL without local GPU support only supports the use of API-integrated models like

openai/text-davinci-003

. Please see the OpenAI API documentation (https://platform.openai.com/docs/models/gpt-3-5) to learn more about the set of available models.

To setup a

conda

environment for LMQL with no GPU support, run the following commands:

# prepare conda environment
conda env create -f scripts/conda/requirements-no-gpu.yml -n lmql-no-gpu
conda activate lmql-no-gpu

# registers the `lmql` command in the current shell
source scripts/activate-dev.sh

<div align="center">

LMQL

Explore LMQL

Feature Overview

Getting Started

Running LMQL Programs

Configuring OpenAI API Credentials

Installing the Latest Development Version

Contributing

Setting Up a Development Environment

Development without GPU

Related Skills

<h1 align="center">

- Identify gaps

2. Apply Deepthink Protocol (reason about dependencies

LMQL

Explore LMQL

Feature Overview

Getting Started

Running LMQL Programs

Configuring OpenAI API Credentials

Installing the Latest Development Version

Contributing

Setting Up a Development Environment

Development without GPU