Q: What is RAG?

Retrieval-Augmented Generation (RAG) combines two building blocks: a search step that pulls in trusted information and a generation step that writes the response. Instead of asking an LLM to invent an answer from memory, you give it the exact context it needs. How it works Retrieve. Query a vector database or search index to find the most relevant documents, transcripts, or knowledge base entries. Augment. Package the best snippets into a compact prompt—often with citations or metadata. Generate. Ask the model to answer using only the supplied context, discouraging speculation. Why teams use RAG Grounded answers. Responses reference your content, so accuracy and tone stay on brand. Fresh knowledge. You can update the index without retraining the model. Smaller prompts. Retrieval keeps prompts focused, lowering token cost. When it’s the right fit You have evolving documentation (support portals, policy manuals, research notes). You need provenance—every response should show where it came from. Your content mix is structured enough to index (markdown, HTML, transcripts, PDFs). Use RAG when “check the docs” is part of your team’s workflow. It gives operators and customers fast answers while keeping the model firmly tethered to verified knowledge.

Question 1

How does this chatbot work?

Accepted Answer

Direct Answer

Aria is a Retrieval-Augmented Generation (RAG) chatbot that answers questions by searching through more than 200 content sources on nicolalazzari.ai—including articles, guides, case studies, Q&A entries, pricing information, and consulting pages. Instead of relying on pre-written responses, it retrieves relevant content in real-time and generates answers that cite their sources, ensuring accuracy and staying up-to-date as the site evolves. For a detailed technical deep-dive into the architecture, implementation, and results, see the Aria chatbot RAG case study.

Architecture Overview

The chatbot uses a lightweight, modular architecture where each component handles one responsibility. This makes the system observable, easy to extend, and cost-effective to run.

Content Sources

Aria draws from a unified corpus that includes:

Markdown articles — Technical blog posts and tutorials
Structured guides — Experimentation playbooks, AI implementation guides, and frameworks
Case studies — Detailed project narratives with results and technical details
Pricing and consulting pages — Service descriptions, engagement models, and rate information
Q&A entries — Both database-backed and static fallback entries covering common questions
External signals — Live data from APIs like Last.fm for personal context

Content Ingestion Pipeline

Every night, automated workers crawl all content sources and prepare them for embedding:

Normalization. Content is converted to clean HTML, removing formatting noise while preserving structure.
Metadata enrichment. Each document gets canonical URLs, breadcrumbs, and content type tags.
Fingerprinting. A SHA-256 hash is computed for each document to detect changes.
Change detection. If a document's hash hasn't changed since the last run, it's skipped entirely—this keeps embedding costs predictable.

Embedding Generation

Only changed or new documents are sent to OpenAI's text-embedding-3-small model, which converts text into high-dimensional vectors (embeddings). These vectors capture semantic meaning, so similar concepts cluster together in vector space. All embeddings are stored in a PostgreSQL database with pgvector extension, creating a searchable knowledge base.

The embedding process is incremental and cost-optimized. Hash comparison prevents re-embedding unchanged content, and the unified corpus means adding a new article automatically feeds the chatbot without manual configuration.

Retrieval System

When you ask a question, Aria uses a hybrid retrieval approach that combines two search methods:

Semantic search (cosine similarity). Your question is converted to an embedding, then compared against all stored content embeddings. Documents with similar meaning score higher, even if they don't contain exact keywords.
Keyword search (BM25). BM25 (Best Matching 25) is a ranking algorithm that boosts documents containing exact keyword matches. This ensures precise terms like "Calendly" or "experimentation" get proper relevance.

The hybrid approach ensures both semantic understanding and precise keyword matching work together. Documents below a relevance threshold are discarded, which is why hallucinations stay below 3%.

Response Generation

Once the most relevant content is retrieved, it's packaged into a context prompt and sent to OpenAI's gpt-4o-mini model, which:

Generates a natural-language answer using only the retrieved context
Streams the response token-by-token for fast perceived performance (1.9s average first token)
Includes citations linking back to source pages
Adapts the tone to match the site's voice
Surfaces context-aware calls-to-action based on conversation topics

If there's a risk of truncation or the response quality drops, the system falls back to gpt-4 automatically.

Call-to-Action Intelligence

Aria tracks conversation context, scroll depth, and CTA performance to surface relevant next steps:

When someone asks about pricing, it suggests booking a strategy call
When discussing experimentation, it links to the experimentation playbooks
When exploring AI workflows, it surfaces relevant case studies
It integrates with Calendly to show live availability from Google Calendar

This context-aware approach has increased CTA conversion by 2.4× compared to static prompts.

Performance & Results

By the numbers:

1.9 seconds average first-token latency (68% faster than the previous version)
92% grounded answers — every response cites an internal resource
Under 3% hallucinations — thanks to strict relevance filtering
35% longer sessions — visitors who engage for three turns stay meaningfully longer
2.4× CTA conversion — context-aware prompts outperform static flows
Zero manual upkeep — content ingestion, hashing, and embedding run automatically

Privacy & Data Handling

Conversations are stored locally in your browser (localStorage) and are not sent to external analytics unless you've given consent. The chatbot only accesses publicly available content on nicolalazzari.ai—it doesn't have access to private data or user accounts.

Continuous Improvement

The system is designed to improve automatically as content is added. When new articles, guides, or Q&A entries are published, they're automatically included in the next embedding run. The hash-based change detection ensures only new or updated content triggers re-embedding, keeping costs low while maintaining freshness.

Technical Stack

Frontend: Next.js with React, streaming UI with Server-Sent Events (SSE)
Backend: Next.js API routes, PostgreSQL with pgvector
Embeddings: OpenAI text-embedding-3-small
Generation: OpenAI gpt-4o-mini (with gpt-4 fallback)
Deployment: Vercel with automatic deployments

Takeaway & Related Content

Aria demonstrates how RAG can transform a basic chatbot into a precision assistant that stays accurate, up-to-date, and helpful. The architecture prioritizes cost efficiency, accuracy, and maintainability—making it a practical blueprint for production RAG systems.

Question 2

Choosing the Right LLM

Accepted Answer

Selecting a large language model starts with your constraints: cost, latency, brand voice, and compliance. The goal is to match the model’s strengths to the job, not chase the newest release.

Evaluation rubric

Use case clarity. List the tasks you expect the model to handle (summaries, reasoning, code, generation). Score each model on first-party benchmarks or quick pilots.
Latency & throughput. Measure response time under realistic loads. If you need sub-second replies or streamability, tighter models or hosted fine-tunes may win.
Cost profile. Estimate spend per request and at peak usage. Consider context window size—larger windows help reasoning but increase token cost.
Brand & tone. Evaluate how well the model mirrors your voice. Few-shot prompting, style guides, or lightweight fine-tunes can close gaps.
Risk & compliance. Check data residency, logging policies, and available guardrails (moderation, redaction). Some verticals require SOC 2 / HIPAA-ready vendors.

Decision tips

Run head-to-head trials with the same prompt sets and judge outputs blindly.
Balance a primary model with a backup to avoid vendor lock-in.
Automate evaluations—track accuracy, hallucination rate, and tone alignment over time.

The “best” LLM is the one that meets your service-level expectations at a sustainable cost. Treat the selection as a product decision: define the spec, test options against real workloads, and revisit quarterly.

Question 3

What is RAG?

Accepted Answer

Retrieval-Augmented Generation (RAG) combines two building blocks: a search step that pulls in trusted information and a generation step that writes the response. Instead of asking an LLM to invent an answer from memory, you give it the exact context it needs.

How it works

Retrieve. Query a vector database or search index to find the most relevant documents, transcripts, or knowledge base entries.
Augment. Package the best snippets into a compact prompt—often with citations or metadata.
Generate. Ask the model to answer using only the supplied context, discouraging speculation.

Why teams use RAG

Grounded answers. Responses reference your content, so accuracy and tone stay on brand.
Fresh knowledge. You can update the index without retraining the model.
Smaller prompts. Retrieval keeps prompts focused, lowering token cost.

When it’s the right fit

You have evolving documentation (support portals, policy manuals, research notes).
You need provenance—every response should show where it came from.
Your content mix is structured enough to index (markdown, HTML, transcripts, PDFs).

Use RAG when “check the docs” is part of your team’s workflow. It gives operators and customers fast answers while keeping the model firmly tethered to verified knowledge.

ai workflows

How does this chatbot work?

Direct Answer

Choosing the Right LLM

Evaluation rubric

What is RAG?