Build a Production-Oriented AI Chat Agent in .NET

1. Start with the Right Mental Model

An embedding model does not write the final answer. It converts text into vectors so your application can compare semantic similarity. A chat model can write an answer, while a deterministic local composer can format retrieved content when no chat model is available.

Website HTML
  -> extract meaningful content
  -> create parent sections and focused child chunks
  -> generate child embeddings
  -> store structured JSON

Visitor question
  -> guard unsupported factual claims
  -> detect known entities
  -> exact metadata search
  -> vector search when exact matching is insufficient
  -> keyword fallback if embeddings fail
  -> resolve child matches to parent context
  -> OpenAI answer, otherwise Ollama chat model, otherwise local answer composer
  -> citations, lead prompts, and contact handoff

2. Organize the .NET Application by Responsibility

Services/
  ChatKnowledge/
    HtmlContentExtractor.cs
    ChatKnowledgeGenerator.cs
    ChatKnowledgeModels.cs
    IEmbeddingProvider.cs
    OllamaEmbeddingProvider.cs
    OpenAiEmbeddingProvider.cs
    NoEmbeddingProvider.cs
  Chat/
    ChatAnswerComposer.cs
    ChatAnalyticsService.cs
  Admin/
    AdminPasswordHasher.cs

wwwroot/
  assets/data/chat-knowledge.json
  assets/js/main.js

Program.cs

This separation keeps extraction, chunk generation, embeddings, retrieval, answer formatting, analytics, authentication, and frontend behavior independently understandable.

3. Store More Than Text and Embeddings

A useful knowledge record should carry retrieval and answer context. Rich metadata gives exact search and reranking something reliable to work with before vector search.

public sealed class ChatKnowledgeChunk
{
    public string Id { get; init; } = "";
    public string Type { get; init; } = "";
    public string Page { get; init; } = "";
    public string Url { get; init; } = "";
    public string Category { get; init; } = "";
    public string Section { get; init; } = "";
    public string Intent { get; init; } = "";
    public string Entity { get; init; } = "";
    public string Audience { get; init; } = "business buyer";
    public string AnswerType { get; init; } = "topic-explanation";
    public string[] Keywords { get; init; } = [];
    public int SourcePriority { get; init; }
    public string ParentId { get; init; } = "";
    public string ParentTitle { get; init; } = "";
    public bool IsParent { get; init; }
    public string Title { get; init; } = "";
    public string Description { get; init; } = "";
    public string Text { get; init; } = "";
    public float[] Embedding { get; set; } = [];
}

Useful metadata includes likely visitor intent, a recognized product entity, expected audience, answer type, authority weight, and parent relationship. Store embeddings only for searchable child chunks; parent chunks exist to provide complete answer context.

4. Implement True Parent-Child Chunking

Searching large sections reduces retrieval precision. Answering directly from tiny fragments loses context. Parent-child retrieval solves both problems.

Page parent: AI Chat Agent Tutorial
  Section parent: Hybrid Retrieval
    Child 1: guarded factual questions
    Child 2: exact metadata matching
    Child 3: vector semantic matching
    Child 4: keyword fallback

Embed and search the children. Once a child matches, replace it with its parent before composing the answer:

static ChatChunk[] ResolveParentChunks(
    IEnumerable<ChatChunk> matches,
    IReadOnlyDictionary<string, ChatChunk> byId)
{
    return matches
        .Select(chunk =>
            !string.IsNullOrWhiteSpace(chunk.ParentId) &&
            byId.TryGetValue(chunk.ParentId, out var parent)
                ? parent
                : chunk)
        .GroupBy(chunk => chunk.Id)
        .Select(group => group.First())
        .ToArray();
}

5. Make Embedding Providers Replaceable

public interface IEmbeddingProvider
{
    string ProviderName { get; }
    string ModelName { get; }
    int Dimensions { get; }

    Task<float[]> CreateEmbeddingAsync(
        string text,
        CancellationToken cancellationToken);

    Task<float[][]> CreateEmbeddingsAsync(
        IReadOnlyList<string> texts,
        CancellationToken cancellationToken);
}

Local Ollama configuration

{
  "Embeddings": {
    "Enabled": true,
    "Provider": "Ollama",
    "Model": "nomic-embed-text",
    "Endpoint": "http://localhost:11434/api/embed",
    "Dimensions": 768,
    "BatchSize": 24,
    "TimeoutSeconds": 60
  },
  "OllamaGeneration": {
    "Enabled": true,
    "Endpoint": "http://localhost:11434/api/chat",
    "Model": "llama3.2",
    "MaxOutputTokens": 350,
    "TimeoutSeconds": 60,
    "Temperature": 0.2
  }
}

Install both model types: ollama pull nomic-embed-text for vectors and ollama pull llama3.2 for final answer generation.

OpenAI configuration

{
  "Embeddings": {
    "Enabled": true,
    "Provider": "OpenAI",
    "Model": "text-embedding-3-small",
    "Endpoint": "https://api.openai.com/v1/embeddings",
    "Dimensions": 256
  }
}

Keep keys in server configuration, user secrets, environment variables, or protected hosting settings. Never expose them in JavaScript.

6. Use Hybrid Retrieval in a Deliberate Order

A reliable business assistant should not send every question directly to vector search.

1. Guarded factual: awards, clients, certifications, metrics, guarantees, and exact prices.
2. Direct entity: known products, projects, and aliases.
3. Exact metadata: titles, headings, URLs, keywords, intent, entity, and source priority.
4. Vector: broad natural-language questions whose wording differs from the source.
5. Keyword fallback: dependable retrieval when embeddings are missing or unavailable.

public sealed record RetrievalResult(
    ChatChunk[] Chunks,
    string Provider,
    string Strategy);

var exact = MatchExactChunks(question, chunks).Take(8).ToArray();
if (exact.Length > 0)
    return new RetrievalResult(exact, "local-metadata", "exact-metadata");

var questionVector = await embeddingProvider.CreateEmbeddingAsync(question, ct);
var vectorMatches = chunks
    .Where(c => !c.IsParent && c.Embedding.Length == questionVector.Length)
    .Select(c => new { Chunk = c, Score = CosineSimilarity(questionVector, c.Embedding) })
    .OrderByDescending(x => x.Score)
    .Take(8)
    .Select(x => x.Chunk)
    .ToArray();

if (vectorMatches.Length > 0)
    return new RetrievalResult(vectorMatches, embeddingProvider.ProviderName, "vector");

return new RetrievalResult(
    MatchKeywords(question, chunks).Take(8).ToArray(),
    "local-keyword",
    "keyword-fallback");

7. Ask a Question That Actually Reaches Ollama

Questions containing exact product names or page headings are intentionally answered before vector search. To exercise semantic retrieval, describe the need without using known labels:

People arrive unsure of what they need and explain their situation conversationally.
How can the system interpret that description and surface the most relevant material?

Other useful semantic tests:

Our staff spend too much time looking through scattered internal information.
How could they ask naturally worded questions and receive grounded guidance?

We need to understand a visitor's situation, guide them toward the right offering,
and pass useful context to our team. What kind of system supports that workflow?

A successful vector result should include:

{
  "answerProvider": "local-composer",
  "retrievalProvider": "ollama",
  "retrievalStrategy": "vector"
}

If the strategy says exact-metadata or direct-entity, the system found a stronger deterministic match and correctly skipped Ollama.

8. Separate Retrieval from Answer Generation

After resolving parents, choose an answer provider. Try OpenAI when configured, then an enabled local Ollama generative model, then the deterministic composer. Guarded unsupported claims and direct named-product answers should remain deterministic before this provider chain.

OpenAI configured and successful
  -> OpenAI grounded answer
otherwise Ollama llama3.2 enabled and available
  -> Ollama grounded answer
otherwise
  -> deterministic ChatAnswerComposer

public string ComposeFromChunks(
    string question,
    IReadOnlyCollection<ChatChunk> chunks)
{
    var lines = chunks
        .SelectMany(chunk => ExtractRelevantLines(question, chunk.Text))
        .Distinct(StringComparer.OrdinalIgnoreCase)
        .Take(4)
        .ToArray();

    return $"""
    Short answer
    {lines.FirstOrDefault() ?? "No exact answer was found in approved content."}

    Key points
    {string.Join(Environment.NewLine, lines.Skip(1).Select(line => $"- {line}"))}

    Best next step
    Share the workflow, users, integrations, timeline, and budget range.
    """;
}

The composer is extractive: it selects, cleans, deduplicates, and formats approved content. It should not invent claims.

9. Make API Operations Transparent

Do not overload one field named mode. Report answer generation and retrieval separately.

public sealed record ChatResponse(
    bool Ok,
    string Answer,
    string Source,
    string? AnswerProvider = null,
    string? RetrievalProvider = null,
    string? RetrievalStrategy = null);

With Ollama embeddings and Llama 3.2 generation running, and no OpenAI chat key, health should look similar to:

{
  "mode": "ollama-answer-with-grounded-retrieval",
  "answerProvider": "ollama",
  "answerProviderOrder": "openai-if-configured > ollama-if-enabled-and-available > local-composer",
  "retrievalProvider": "ollama",
  "retrievalStrategy": "per-question: guard > direct-entity > exact-metadata > vector > keyword-fallback",
  "embeddingProvider": "Ollama",
  "embeddingModel": "nomic-embed-text",
  "ollamaGenerationModel": "llama3.2",
  "ollamaGenerationModelReady": true,
  "embeddingsEnabled": true,
  "embeddingEndpointReachable": true,
  "embeddingModelReady": true,
  "embeddingProbe": "reachable-model-ready"
}

For Ollama, query /api/tags to confirm endpoint reachability and model availability. For OpenAI, a non-billed GET can confirm network reachability, but it cannot prove model access; report model readiness as unknown unless you intentionally perform an authenticated embedding request.

10. Build a Dependable Frontend Experience

The browser interface should remain useful without the backend:

• Open only after the visitor selects the AI Copilot control.
• Show a Thinking state while waiting.
• Display citations and helpful qualification prompts.
• Keep lightweight intent and entity context in sessionStorage.
• Prefill a visitor-reviewable contact summary.
• Load the same JSON knowledge and use guards, direct matching, exact scoring, parent resolution, and keyword fallback when /api/chat is unavailable.

try {
    const response = await fetch("/api/chat", requestOptions);
    const result = await response.json();
    renderAnswer(result);
} catch {
    const knowledge = await fetch("assets/data/chat-knowledge.json").then(r => r.json());
    renderAnswer(await localBrowserAnswer(question, knowledge.chunks));
}

11. Add Operations, Security, and Privacy

Authenticated administration

Use a PBKDF2 password hash, rate-limited sign-in, an HTTP-only same-site cookie, short session expiration, and persisted ASP.NET Core data-protection keys. Protect knowledge health, retrieval traces, analytics export, and analytics deletion.

Anonymous analytics

Useful events include question asked, intent detected, citation selected, lead prompt selected, and contact handoff. Avoid storing question text, answer text, names, email addresses, IP addresses, and contact-form messages in chat analytics.

Retention

Rotate JSON Lines files monthly, enforce a maximum file size, delete files after a defined retention period, and let authenticated administrators export or clear retained data.

Privacy disclosure

Explain the random session identifier, browser session storage, contact-form handoff, external AI processing when configured, analytics exclusions, and retention period. Ask visitors not to submit passwords, API keys, patient data, financial credentials, or other sensitive production information.

12. Deploy Without Breaking Fallbacks

dotnet build -c Release
dotnet publish -c Release

Deploy the complete ASP.NET Core publish output. Configure secrets as hosting environment settings:

ASPNETCORE_ENVIRONMENT=Production
OpenAI__ApiKey=...                 # optional
Embeddings__Enabled=false         # or configure a production provider
Email__SmtpPassword=...
Admin__Username=...
Admin__PasswordHash=...

If production cannot run Ollama and stored vectors were generated by Ollama, disable vector retrieval or regenerate the knowledge base with the production embedding model. Never compare vectors from different models.

Conclusion

A trustworthy AI chat agent is not simply a prompt attached to a vector database. It is a layered system: deterministic safety, entity recognition, exact metadata, semantic retrieval, keyword fallback, parent context, controlled answer generation, citations, useful handoff, operational visibility, and privacy-aware retention.