LLM inference engine

[stvowi]

PasLLM is a LLM inference engine in Object Pascal.

This makes it possible to create Unleashed Pascal programmes with AI capabilities without the usual need to install endless dependencies.
I therefore propose that PasLLM be pre-installed in the IDE.

Hey everyone, please give this project as many stars as possible to encourage the developer to keep working on it.

@see also:

• What is LLM inference?
• PasLLM - AI LLM inference engine in Object Pascal (1)
• PasLLM - AI LLM inference engine in Object Pascal (2)

[stvowi]

You can download PasLLM models from:

• Pre-converted and quantized models: [PasLLM model folder on MEGA](https://mega.nz/folder/krcgHCpZ#0tjLqup_Hc4THWC9itDrTg)
• PasLLM and conversion tools: [BeRo1985/pasllm on GitHub](https://github.com/BeRo1985/pasllm)
• Original models: [Hugging Face](https://huggingface.co/models)

Place downloaded PasLLM models in:

pasllm/bin/models/

Example:

./bin/pasllmcli \
  -model=bin/models/qwen2.5_0.5b_instruct_q40nl.safetensors

PasLLM does not directly load arbitrary GGUF files. It uses its own Safetensors-based format with quantization types such as Q40NL, Q40, Q80, FP8, and FP16. Models from Hugging Face must be converted using tools/convert.py, as documented in the [PasLLM README](https://github.com/BeRo1985/pasllm).

Good models for initial testing include:

• Qwen 2.5 0.5B Instruct
• Qwen 3 0.6B
• SmolLM2 360M Instruct
• TinyLlama 1.1B Chat

Remember to check the license of each original Hugging Face model.

[stvowi]

For Lazarus/Free Pascal development, the best practical model is Qwen3-Coder 30B—especially if you want a capable local assistant through Ollama. It is code-specialized, supports a 256K context window, and its MoE design activates only 3.3B parameters per request, making it unusually efficient for its class.[ollama]

For the highest code quality on difficult multi-unit refactors, debugging, and architecture, use a frontier cloud coding model alongside it; but for your 32 GB RAM / 12 GB VRAM local workflow, Qwen3-Coder 30B is the best starting point.

Recommended setup

Use case | Recommended model | Why -- | -- | -- Best local coding assistant | qwen3-coder:30b | Purpose-built for agentic software engineering and coding; 19 GB download with 256K context. [ollama] Larger local/server setup | qwen3-coder-next | Coding-focused 80B MoE model, but the Q4 build alone is 52 GB, so it is not a good fit for your present machine. [ollama] General Pascal reasoning | qwen3:14b | A useful lighter fallback for explanations, API design, and reviewing compact units. [ollama] Maximum quality, cloud | Qwen3-Coder 480B Cloud | Stronger for repository-scale work, but the local model requires about 290 GB. [ollama]

Why this fits Pascal

No leading model is specifically trained or benchmarked primarily on Free Pascal/Lazarus, so expect occasional Delphi-centric code, incompatible syntax, or assumptions about VCL rather than LCL. The decisive advantage comes from supplying the model with your target details: FPC version, Lazarus/LCL widgetset, operating system, compiler switches, and the involved unit interfaces.

Qwen3-Coder is explicitly positioned for long-context, agentic coding workflows and can be launched through Ollama with coding-agent tooling. That is particularly useful for your likely tasks: tracing dependencies across units, improving PostgreSQL access code, generating tests, and diagnosing compiler messages.[ollama]

Start locally

ollama pull qwen3-coder:30b
ollama run qwen3-coder:30b

The official Ollama catalogue lists this 30B model at approximately 19 GB and with a 256K context window.[ollama]

If responsiveness or Intel Arc support becomes a limiting factor, use:

ollama pull qwen3:14b
ollama run qwen3:14b

It will be less reliable on multi-file refactoring, but is a sensible interactive fallback.[ollama]

Prompt template

Use a stable project instruction such as:

You are an expert Free Pascal and Lazarus developer.
Target:

Free Pascal: 3.2.x
Lazarus: 3.x
GUI: LCL, not Delphi VCL
Platforms: Windows, Linux, macOS
Database: PostgreSQL via [specify library]
Compatibility: avoid Delphi-only language and RTL features

Rules:

Return compilable Object Pascal.
State required units in the uses clauses.
Preserve existing public interfaces unless explicitly asked to change them.
Prefer LCL and FPC-compatible APIs.
Identify ownership, lifetime, thread-safety, exception handling, and SQL-parameter issues.
If uncertain about an API, say so instead of inventing it.

Task:

[paste compiler error, unit interface, implementation, and expected behavior]

For best results, give it a small, coherent slice of the codebase: the failing unit, related interfaces, exact compiler output, and expected behavior. Ask it first to analyse and propose a patch, then ask it to emit a complete revised unit; this reduces accidental API changes.

For Lazarus/Free Pascal development, **the best practical model is Qwen3-Coder 30B**—especially if you want a capable local assistant through Ollama. It is code-specialized, supports a 256K context window, and its MoE design activates only 3.3B parameters per request, making it unusually efficient for its class. [[ollama](https://ollama.com/library/qwen3-coder)](https://ollama.com/library/qwen3-coder)

For the highest code quality on difficult multi-unit refactors, debugging, and architecture, use a frontier cloud coding model alongside it; but for your 32 GB RAM / 12 GB VRAM local workflow, Qwen3-Coder 30B is the best starting point.

Recommended setup

Use case	Recommended model	Why
Best local coding assistant	qwen3-coder:30b	Purpose-built for agentic software engineering and coding; 19 GB download with 256K context. [ollama](https://ollama.com/library/qwen3-coder)
Larger local/server setup	qwen3-coder-next	Coding-focused 80B MoE model, but the Q4 build alone is 52 GB, so it is not a good fit for your present machine. [ollama](https://ollama.com/library/qwen3-coder-next)
General Pascal reasoning	qwen3:14b	A useful lighter fallback for explanations, API design, and reviewing compact units. [ollama](https://ollama.com/library/qwen3:14b)
Maximum quality, cloud	Qwen3-Coder 480B Cloud	Stronger for repository-scale work, but the local model requires about 290 GB. [ollama](https://ollama.com/library/qwen3-coder)

Why this fits Pascal

No leading model is specifically trained or benchmarked primarily on Free Pascal/Lazarus, so expect occasional Delphi-centric code, incompatible syntax, or assumptions about VCL rather than LCL. The decisive advantage comes from supplying the model with your target details: FPC version, Lazarus/LCL widgetset, operating system, compiler switches, and the involved unit interfaces.

Qwen3-Coder is explicitly positioned for long-context, agentic coding workflows and can be launched through Ollama with coding-agent tooling. That is particularly useful for your likely tasks: tracing dependencies across units, improving PostgreSQL access code, generating tests, and diagnosing compiler messages. [ollama](https://ollama.com/library/qwen3-coder)

Start locally

ollama pull qwen3-coder:30b
ollama run qwen3-coder:30b

The official Ollama catalogue lists this 30B model at approximately 19 GB and with a 256K context window. [ollama](https://ollama.com/library/qwen3-coder)

If responsiveness or Intel Arc support becomes a limiting factor, use:

ollama pull qwen3:14b
ollama run qwen3:14b

It will be less reliable on multi-file refactoring, but is a sensible interactive fallback. [ollama](https://ollama.com/library/qwen3:14b)

Prompt template

Use a stable project instruction such as:

You are an expert Free Pascal and Lazarus developer.

Target:
- Free Pascal: 3.2.x
- Lazarus: 3.x
- GUI: LCL, not Delphi VCL
- Platforms: Windows, Linux, macOS
- Database: PostgreSQL via [specify library]
- Compatibility: avoid Delphi-only language and RTL features

Rules:
- Return compilable Object Pascal.
- State required units in the uses clauses.
- Preserve existing public interfaces unless explicitly asked to change them.
- Prefer LCL and FPC-compatible APIs.
- Identify ownership, lifetime, thread-safety, exception handling, and SQL-parameter issues.
- If uncertain about an API, say so instead of inventing it.

Task:
[paste compiler error, unit interface, implementation, and expected behavior]

For best results, give it a small, coherent slice of the codebase: the failing unit, related interfaces, exact compiler output, and expected behavior. Ask it first to analyse and propose a patch, then ask it to emit a complete revised unit; this reduces accidental API changes.

[stvowi]

Sorry, there is a error in the text, Install Qwen with: ollama run qwen3-coder !

[stvowi]

If you want play with PasLLM-Models you can download a Windows-App .

[stvowi]

By the way, the preferred language for prompts is English. Apparently, this gives the best results.

Have fun!