Skip to main content
Zalt Logo
Back to Projects
AI Knowledge Studio - Screenshot 1 - AI Utility, Analytics project

AI Knowledge Studio

AI Utility, Analytics

Project Summary

A natural-language analytics layer that lets teams ask questions over massive metrics data and get accurate, explainable answers in seconds.

The system queries both real-time event streams and relational metadata, generates safe queries with guardrails, and returns results with full context and summaries.

All inference runs on self-hosted fine-tuned large language models with tracing and evaluation built in. Data and models stay inside private infrastructure with no external data egress.

Case Study

Overview

Built a chat-first analytics assistant that answers natural-language questions over real-time metrics while keeping data and inference inside private infrastructure.

Problem

Teams were exporting telemetry and KPI data into ad-hoc spreadsheets. Answers were slow, inconsistent, and security requirements ruled out sending data to external LLM APIs.

Solution

A lightweight chat UI backed by a query service that routes questions to the appropriate data source, executes safe queries, and returns summaries with computed results, all within private infrastructure.

Goals

  • 1Answer common analytics questions in under 10 seconds end to end.
  • 2Support averages, rankings, and grouped comparisons across billions of events.
  • 3Keep data fresh with sub-2 minute lag from ingestion to query.
  • 4Provide explainable results with the exact query used.
  • 5Keep data and inference inside private infrastructure (no external data egress).
  • 6Capture traces and feedback to improve accuracy over time.

Approach

  • Used a natural-language query engine to generate SQL over both real-time event streams and relational metadata without duplicating data.
  • Separated high-cardinality real-time metrics from reference data and access control into dedicated storage layers.
  • Deployed all data stores inside a private cloud VPC with strict network boundaries.
  • Self-hosted inference with a fine-tuned 70B-parameter model on dedicated GPU servers to control cost and latency.
  • Built an observability pipeline to log prompts, generated queries, and outcomes for evaluation and continuous tuning.

Results & Impact

Outcomes

  • Delivered near real-time analytics without a separate warehouse or external LLM APIs.
  • Reduced analysis time from hours of exports to minutes of Q&A.
  • Improved trust by exposing the exact query and result context.

Key Metrics

Median response time
6-10 sec
End-to-end question to answer.
Query success rate
90-95%
Valid queries on first pass.
Data freshness
<2 min
Ingestion to availability.
Data scale
5-8B events
Rolling 12-month window.
Model size
70B params
Fine-tuned for analytics NLQ.

Timeline

1
Schema mappingJan 2026

Metrics, dimensions, and examples.

2
Dual-source NLQJan 2026

Multi-source query routing.

3
Private inferenceJan 2026

Self-hosted model deployment on local GPUs.

4
Evaluation loopJan 2026–Present

Observability traces + tuning.

Challenges

  • Preventing unsafe or expensive queries from untrusted prompts.
  • Handling ambiguous questions across two data sources.
  • Balancing explainability with concise answers in a chat UI.

Project Info

Start:December 2025
End:
January 2026
Duration:1 month
Tech:15 (private)
Images:1 available

Get AI analytics built over your data.

I built a natural-language analytics layer over billions of events with self-hosted models. Let me help you build yours.

Book a Technical ConsultationSee How I Build AI Systems

Technologies Used

Private stack – contact for info

NLQ or self-hosted AI question?

Get practical advice on fine-tuning, deploying private models, or building natural-language query systems.

Ask a Question
or book directly
End-to-End Development
Modern Tech Stack
Scalable Architecture