Local LLM

This sample serves as a simple starting point for TX Text Control AI. Load a local language model, enter a question, and print the answer. In this tutorial, we will build a small .NET C# console chat that streams replies and remembers previous messages.

No OpenAI account. No API key to configure. No per-token API charges. The model runs on infrastructure you control.

What Is TX Text Control AI?

TX Text Control AI is a toolkit that adds local generative AI and document-aware capabilities to .NET applications. The core TXTextControl.AI package loads GGUF models owned by the application and exposes chat through Microsoft.Extensions.AI.IChatClient . You select the model and design the user interface, while the library handles model loading and the inference runtime.

In this example, we use the model and chat APIs from the core package. The TXTextControl.AI.LlamaServer dependency supplies integration with the native llama.cpp engine. The NuGet packages do not contain model weights or GPU drivers.

From a GGUF File to a .NET Chat Client

GGUF → llama.cpp → TXTextControl.AI → IChatClient → your application

GGUF: The file on disk containing model weights and metadata. llama.cpp: The native inference engine that loads those weights and generates tokens in a local llama-server process. TXTextControl.AI: The .NET API that loads the model, creates chat sessions, and manages the owned process. IChatClient: The standard .NET abstraction used to send messages and receive streaming responses. Your application: A console loop that reads input and writes the generated answer.

The native server uses a locally compatible OpenAI protocol. This protocol does not require an OpenAI account and does not send the prompts from this demo to OpenAI.

Create the Console Project

Use the .NET 10 SDK for this demo. The example works with the managed runtime on Windows x64 and Linux x64 with glibc; Linux also needs libgomp.so.1 . Create a project and install the preview package:

dotnet new console --name LocalChat --framework net10.0 cd LocalChat dotnet add package TXTextControl.AI --version 0.1.0-beta.1 The APIs shown here are for preview version 0.1.0-beta.1 . The TX Text Control AI introduction provides context on the product and licensing. Appropriately licensed TX Text Control products are required.

Download and Copy the Model

This demo uses Qwen_Qwen3.5-4B-Q4_K_M.gguf , a quantized GGUF conversion of Qwen3.5-4B. Download that individual file from the model's file page on Hugging Face using its Download button. The publisher's model card describes the available quantizations and license.

Create a Models directory beside the .csproj file and copy the downloaded GGUF into it. Keep the filename unchanged:

LocalChat/ ├── LocalChat.csproj ├── Program.cs └── Models/ └── Qwen_Qwen3.5-4B-Q4_K_M.gguf

The file size is about 3 GB. Allow for extra memory for the context and runtime buffers. Note that the download size is not the total RAM requirement. Note that downloading the model is a separate step from installing the NuGet package.

Build the Chat in 30 Lines of C#

Replace Program.cs with the following code. It loads the model once, creates an IChatClient , and keeps reading messages until you enter /exit :

using Microsoft.Extensions.AI; using TXTextControl.AI;

var options = new LocalModelOptions { HardwareBackend = HardwareBackend.Cpu, RuntimeAcquisition = RuntimeAcquisitionPolicy.DownloadIfMissing, ContextSize = 2048, MaxOutputTokens = 512 };

var path = Path.GetFullPath(args.Length &gt; 0 ? args[0] : "Models/Qwen_Qwen3.5-4B-Q4_K_M.gguf");

await using var model = await LocalLanguageModel.LoadAsync(path, options); using IChatClient chat = model.CreateSession(); var history = new List&lt;ChatMessage&gt;();

Console.WriteLine("Type /exit to quit.");

while (true) { Console.Write("User: "); var input = Console.ReadLine(); if (input is null || input.Equals("/exit", StringComparison.OrdinalIgnoreCase)) break; if (string.IsNullOrWhiteSpace(input)) continue; history.Add(new ChatMessage(ChatRole.User, input)); Console.Write("Assistant: "); var updates = new List&lt;ChatResponseUpdate&gt;(); await foreach (var update in chat.GetStreamingResponseAsync(history)) { Console.Write(update.Text); updates.Add(update); } history.AddMessages(updates); Console.WriteLine(); } LocalChat is the sample name. The library entry point is LocalLanguageModel.LoadAsync ; CreateSession() returns the chat client. The using declarations release the session and model, including the owned inference process, when the program finishes.

GetStreamingResponseAsync returns response updates as the model generates them. Printing update.Text displays the answer incrementally. Collecting those updates and calling history.AddMessages(updates) retains the assistant's reply for the next request.

The session itself does not store the conversation history. Instead, the application sends the accumulated user and assistant messages with each turn. This compact example uses a 2,048-token context and allows up to 512 output tokens per reply. Therefore, keep the conversation short, or restart it when the context is full. The full console demo includes a missing-file check and a clear command that calls history.Clear() .

Run and Ask a Question

From the directory containing LocalChat.csproj , run:

dotnet run You can also keep the model elsewhere and pass its absolute path:

dotnet run -- "C:\Models\Qwen_Qwen3.5-4B-Q4_K_M.gguf" Wait for the model to load, then enter your message. For example, a conversation could look like this. The exact wording depends on the model and generation settings.

The second question works because the application includes the first exchange in its history.

Runtime Downloads and GPU Acceleration

RuntimeAcquisitionPolicy.DownloadIfMissing explicitly allows the library to download and verify its pinned llama.cpp runtime when a suitable engine is missing. The first load can therefore take longer and require internet access. Later runs reuse the installed runtime. Model weights are never downloaded by this setting.

Once the packages, model, and runtime are provisioned, inference can run offline. For an explicitly provisioned engine, set LlamaServerExecutablePath to its absolute path and use RuntimeAcquisitionPolicy.NeverDownload .

The article uses HardwareBackend.Cpu so a GPU is optional. On Windows x64 with a compatible NVIDIA GPU and driver, select it by changing the backend setting:

HardwareBackend = HardwareBackend.Cuda, To run the built executable from its output directory, also copy the model there. In Visual Studio, set the GGUF file's Copy to Output Directory property to Copy if newer , or add this item to the project file:

PreserveNewest Run from either the project or output directory, as described above. Alternatively, pass an absolute model path to avoid dependence on the current working directory.

Frequently Asked Questions

What is TX Text Control AI? TX Text Control AI is a .NET toolkit for local generative AI and document-aware applications. The core TXTextControl.AI package loads local GGUF models and exposes chat through Microsoft.Extensions.AI.IChatClient.

Does this local chat require an OpenAI account or API key? No. This demo runs inference in a local llama.cpp process using a model file you supply. You do not configure an OpenAI account or API key, and there are no hosted per-token API charges.

Where do I download and copy the GGUF model? Download Qwen_Qwen3.5-4B-Q4_K_M.gguf from the bartowski/Qwen_Qwen3.5-4B-GGUF repository on Hugging Face. Copy it into a Models directory beside the project file, or pass its absolute path as the first command-line argument.

Can the demo run without a GPU or internet connection? The article uses HardwareBackend.Cpu, so a GPU is optional. Initial package, model, and runtime downloads need internet access. After provisioning those files, inference can run offline. Use NeverDownload with an explicitly provisioned llama-server executable for offline deployment.

How does the chat remember previous messages? The application retains user and assistant messages in a List of ChatMessage objects and submits that history with each request. ChatSession does not persist history itself. The compact article example retains history until exit; the full demo also provides /clear to reset it.

Featured Product

AI Development

Text Control MCP Server

Text Control Agent Skills

Server and Web Document Processing

TX Text Control .NET Server

Client-Side Packages - works together for web apps

DS Server

ReportingCloud

Desktop Document Processing

Windows Forms

TX Text Control .NET

TX Spell .NET

WPF

TX Text Control .NET

TX Spell .NET

ActiveX

TX Text Control ActiveX

Featured Projects

SignFabric

AI-Powered Documents

HEDBridge

Core Technologies

​

Latest Posts

​

Support

Getting Started

​

Other Resources

Company

Contact

Newsletter

Legal

Text Control Account

​

​

Local LLM

Post tagged with Local LLM in 2026 ​

September 2026

Post tagged with Local LLM in 2026