Home All Chapters Previous Next

Chapter 13: Using LLMs in Business Analytics

The integration of Large Language Models (LLMs) into business analytics represents one of the most significant shifts in how organizations extract insights from data. This chapter explores how LLMs complement traditional analytics methods, when to use them, and how to prompt them effectively to generate reliable, business-ready outputs.

13.1 Why LLMs Matter for Business Analytics

Traditional Business Intelligence (BI) tools and Machine Learning (ML) models excel at working with structured, numerical data—think dashboards, SQL queries, and predictive models trained on historical datasets. Large Language Models, by contrast, bring a fundamentally different capability: they understand and generate human language .

According to   Intellias , while conventional data analytics software operates on structured and numeric data, LLMs can interpret human language and extract sentiments, speech patterns, and specific topics from unstructured textual data. By fusing LLMs with data analytics, businesses can use more data points and create a conversational interface to explore them .

Key advantages of LLMs in analytics:

Where traditional BI/ML still wins:

The sweet spot : Combining both approaches. Use LLMs to interpret unstructured data, generate SQL queries, and explain results—then validate outputs with traditional analytics methods.

Typical Business Problems LLMs Solve Well

LLMs have proven particularly effective for several categories of business analytics tasks:

1. Customer Sentiment Analysis

Intellias reports  that companies like Colgate-Palmolive use generative AI to synthesize consumer and shopper insights and better capture consumer sentiment. LLMs can detect nuances in textual data and interpret the semantics of written content at massive scale, distinguishing between "terrible" and "terribly awesome" with high accuracy.

2. Sales Analytics and CRM Querying

Instead of relying on dashboards and SQL queries, business analysts can interact with CRM, ERP, and other data sources via a conversational interface. Morgan Stanley, for example, has launched an AI workforce assistant that can handle research inquiries like "What's the projected interest rate increase in April 2024?" and general admin queries.

3. Market Intelligence

By combining textual and numerical data, business analysts can identify nascent trends, patterns, and potential growth opportunities that might be missed by traditional methods alone.

4. Document Analysis and Summarization

LLMs excel at extracting key information from lengthy reports, contracts, financial statements, and regulatory filings—tasks that would take analysts hours or days.

5. SQL Generation and Query Explanation

According to   Tinybird's research , modern LLMs can generate SQL queries from natural language with increasing accuracy, though success rates vary by model and query complexity.

6. Diagnostic Analysis

When KPIs change unexpectedly, LLMs can help analysts explore potential causes by generating hypotheses, suggesting relevant data to examine, and framing diagnostic questions.

Limits You Must Understand Upfront

Despite their impressive capabilities, LLMs have significant limitations that analysts must understand:

1. Hallucinations and Fabricated Data

LLMs can generate plausible-sounding but entirely incorrect information. They may invent statistics, misinterpret data relationships, or confidently state falsehoods. Never trust LLM outputs without verification .

2. Inconsistent Results

The same prompt can produce different outputs across runs due to the probabilistic nature of LLMs. This variability makes them unsuitable for processes requiring deterministic, reproducible results.

3. Limited Mathematical Reasoning

While improving, LLMs still struggle with complex calculations, multi-step mathematical reasoning, and precise numerical operations. They're better at explaining calculations than performing them.

4. Context Window Limitations

Even with extended context windows (up to 200K tokens in some models), LLMs can lose track of information in very large datasets or lengthy conversations.

5. Training Data Cutoff

LLMs don't have access to real-time data unless explicitly connected to external sources. Their knowledge is frozen at their training cutoff date.

6. Bias and Fairness Issues

LLMs can perpetuate biases present in their training data, potentially leading to unfair or discriminatory insights in sensitive business contexts.

7. Cost and Latency

API calls to powerful LLMs can be expensive at scale, and response times may not meet requirements for real-time analytics applications.

8. Security and Privacy Risks

Sending sensitive business data to external LLM APIs raises data privacy concerns and potential compliance issues.

13.2 How LLMs Work

At a fundamental level, LLMs operate on a simple principle: text in, text out . But understanding the nuances of this process helps analysts use them more effectively.

The Input (Prompt)

Everything you send to an LLM—your question, instructions, examples, and any data—constitutes the "prompt." The quality and structure of your prompt directly determine the quality of the output.

The Context Window

LLMs process inputs within a "context window"—the maximum amount of text they can consider at once. Modern models offer context windows ranging from 4K tokens (roughly 3,000 words) to 200K tokens or more. Once you exceed this limit, the model either truncates early information or refuses to process the request.

The Output (Completion)

The LLM generates a response by predicting the most likely next tokens (words or word fragments) based on patterns learned during training. This process is probabilistic, not deterministic.

Key Concept: Tokens

LLMs don't process whole words—they work with "tokens," which can be words, parts of words, or punctuation. Understanding token limits matters for cost (many APIs charge per token) and for staying within context windows.

Variability, Reliability, and Cost

Why Results Change Between Prompts

LLMs use a parameter called "temperature" to control randomness:

Even at temperature 0, slight variations can occur due to the model's architecture and sampling methods.

Reliability Considerations

According to   research on LLMs in business analytics , while LLMs demonstrate high performance in many data preparation tasks, they may sometimes produce incorrect outputs, especially when input data is noisy or ambiguous. They may also struggle with tasks requiring complex reasoning or domain-specific knowledge.

Cost Structure

Most commercial LLMs charge based on:

For example (approximate 2025 pricing):

Cost Management Strategies:

13.3 Prompting Fundamentals for Analysts

Task, Context, and Constraints

Effective prompts for business analytics typically include three core elements:

1. Task : What you want the LLM to do

"Analyze the following customer feedback and identify the top 3 themes."

2. Context : Background information the LLM needs

"This feedback comes from our Q4 2025 customer satisfaction survey for our SaaS product.

We recently launched a new dashboard feature and changed our pricing structure."

3. Constraints : Boundaries and requirements

"Focus only on feedback related to usability and pricing.

Provide your answer as a bulleted list with supporting quotes."

Complete Example:

Task: Analyze the following customer feedback and identify the top 3 themes.

Context: This feedback comes from our Q4 2025 customer satisfaction survey for our SaaS product.

We recently launched a new dashboard feature and changed our pricing structure.

Constraints: Focus only on feedback related to usability and pricing.

Provide your answer as a bulleted list with supporting quotes.

Data:

[Customer feedback text here]

Specifying Output Format

LLMs can generate outputs in various formats. Being explicit about format dramatically improves usability:

For Structured Data:

"Return your analysis as a JSON object with this structure:

{

  "themes": [

    {"theme": "string", "frequency": number, "sentiment": "positive|negative|neutral"}

  ],

  "summary": "string"

}"

For Tables:

"Present your findings in a markdown table with columns: Theme, Count, Example Quote"

For Executive Summaries:

"Provide a 3-paragraph executive summary suitable for a C-level audience,

avoiding technical jargon."

Clear vs. Vague Prompts (Quick Examples)

❌ Vague Prompt:

"Tell me about sales."

Problems : No context, unclear task, no constraints, ambiguous scope.

✅ Clear Prompt:

"Analyze our Q4 2025 sales data for the North America region.

Identify the top 3 performing product categories by revenue and explain

what factors might have contributed to their performance.

Format your response as a brief report with sections for each category."

❌ Vague Prompt:

"Why did revenue drop?"

✅ Clear Prompt:

"Our monthly recurring revenue (MRR) dropped 12% from October to November 2025.

Based on the following data [data here], generate 5 hypotheses that could explain this decline.

For each hypothesis, suggest specific data we should examine to validate or refute it."

❌ Vague Prompt:

"Write SQL for customer data."

✅ Clear Prompt:

"Write a SQL query for our PostgreSQL database that returns:

- Customer ID

- Customer name

- Total purchase amount in 2025

- Number of orders in 2025

Only include customers with more than 3 orders. Sort by total purchase amount descending. Our customer table is named 'customers' and orders table is 'orders'."

13.4 Iterative Prompting in Practice

In business analytics, the first LLM output is almost never production-ready. This isn't a failure—it's the nature of working with probabilistic models on complex, ambiguous business problems.

Common issues with first outputs:

The key mindset shift : Think of LLMs as collaborative partners in an iterative conversation, not as one-shot answer machines.

Prompt → Output → Refine Loop

Effective LLM-assisted analysis follows a cycle:

1. Initial Prompt  → 2. Review Output  → 3. Refine Prompt  → 4. Improved Output  → Repeat


Example Iteration:

Iteration 1:

Prompt: "Summarize this customer feedback data."

Output: "The feedback is generally positive with some concerns about pricing."

Problem : Too vague, no actionable insights.

Iteration 2:

Prompt: "Summarize this customer feedback data. Specifically:

- What are the top 3 most mentioned issues?

- What percentage of feedback is positive vs. negative?

- Are there any emerging trends compared to last quarter?"

Output: [More detailed analysis with specific themes and percentages]

Problem : Better, but still missing business context.

Iteration 3:

Prompt: "Summarize this customer feedback data. Specifically:

- What are the top 3 most mentioned issues?

- What percentage of feedback is positive vs. negative?

- Are there any emerging trends compared to last quarter?

Context: We launched a new pricing tier in Q3 and redesigned our onboarding flow in Q4.

Focus particularly on feedback related to these changes.

Format: Provide a 2-page executive summary with:

1. Key findings (bullet points)

2. Detailed analysis by theme

3. Recommended actions"

Output: [Comprehensive, contextualized analysis ready for stakeholder review]

Adding Constraints and Corrections

As you iterate, add constraints to guide the LLM toward better outputs:


Constraint Types:

Scope Constraints:

"Focus only on enterprise customers with >$100K annual contract value."

Format Constraints:

"Limit your response to 250 words."

"Use only bullet points, no paragraphs."

Tone Constraints:

"Write in a formal, technical tone suitable for a board presentation."

Accuracy Constraints:

"Only cite statistics that appear directly in the provided data.

Do not make assumptions or extrapolate."

Correction Example:

Initial Output: "Revenue increased significantly in Q4."

Correction Prompt: "Your previous response was too vague.

Provide the exact percentage increase and compare it to the previous 3 quarters.

Also specify which product lines drove this growth."

Asking the Model to Self-Check

One powerful technique is prompting the LLM to validate its own work:

Self-Check Prompts:

"Review your analysis above and identify any assumptions you made that might not be valid.

List them explicitly."

"Double-check your SQL query for syntax errors and logical issues.

Explain your reasoning for each JOIN and WHERE clause."

"Critique your own summary: What important information might be missing?

What questions would a skeptical executive ask?"

Chain-of-Thought Prompting:

Asking the LLM to "think step-by-step" often improves accuracy:

"Before providing your final analysis, think through this step-by-step:

1. What are the key data points?

2. What patterns do you observe?

3. What alternative explanations exist?

4. Which explanation is best supported by the data?

Then provide your final analysis."

Worked Example: From Rough Prompt to Business-Ready Output

Scenario : You're analyzing why customer churn increased in November 2025.


Iteration 1 (Rough Prompt):

Prompt: "Why did churn increase?"

Output: "Churn could increase for many reasons including poor customer service,

better competitor offerings, pricing issues, or product quality problems."

Assessment : Generic, not actionable, no data analysis.

Iteration 2 (Adding Data):

Prompt: "Analyze this churn data and explain why churn increased in November 2025.

Data:

- October churn rate: 3.2%

- November churn rate: 5.1%

- Churned customer segments: 60% SMB, 30% Mid-market, 10% Enterprise

- Average customer tenure of churned users: 8 months

- Top cancellation reasons: 'Too expensive' (45%), 'Missing features' (30%), 'Other' (25%)"

Output: "Churn increased from 3.2% to 5.1%, a 59% relative increase.

The majority of churned customers were SMBs who cited pricing as the primary reason.

The average tenure of 8 months suggests issues during the renewal period."

Assessment : Better, but still lacks business context and actionable recommendations.

Iteration 3 (Adding Context and Constraints):

Prompt: "Analyze this churn data and explain why churn increased in November 2025.

Context:

- We increased prices for our SMB tier by 20% in October 2025

- A major competitor launched a lower-priced alternative in September 2025

- We deprecated a legacy reporting feature in October that some SMB customers used

Data:

- October churn rate: 3.2%

- November churn rate: 5.1%

- Churned customer segments: 60% SMB, 30% Mid-market, 10% Enterprise

- Average customer tenure of churned users: 8 months

- Top cancellation reasons: 'Too expensive' (45%), 'Missing features' (30%), 'Other' (25%)

Requirements:

1. Identify the most likely primary cause of increased churn

2. Quantify the impact of each potential factor

3. Provide 3 specific, actionable recommendations

4. Format as a 1-page executive brief"

Output:

"EXECUTIVE BRIEF: November 2025 Churn Analysis

KEY FINDING:

November churn increased 59% (3.2% → 5.1%), driven primarily by SMB segment price sensitivity following our October 20% price increase.

ROOT CAUSE ANALYSIS:

1. Price Increase (Primary Factor - Est. 60-70% of impact)

   - 45% of churned customers cited 'too expensive'

   - 60% of churn concentrated in SMB segment (most price-sensitive)

   - Timing: Churn spike occurred immediately after October price change

2. Competitive Pressure (Secondary Factor - Est. 20-30% of impact)

   - Competitor's September launch provided lower-cost alternative

   - Likely amplified price sensitivity from our increase

3. Feature Deprecation (Tertiary Factor - Est. 10-20% of impact)

   - 30% cited 'missing features'

   - Legacy reporting feature removal affected SMB segment disproportionately

RECOMMENDED ACTIONS:

1. IMMEDIATE: Implement SMB retention offer

   - Provide 3-month 15% discount to at-risk SMB customers (tenure 6-12 months)

   - Estimated cost: $X, potential churn prevention: Y customers

2. SHORT-TERM (30 days): Restore critical reporting functionality

   - Rebuild deprecated feature or provide alternative solution

   - Target SMB segment specifically

3. MEDIUM-TERM (90 days): Restructure SMB pricing

   - Consider usage-based pricing tier for price-sensitive segment

   - Conduct competitive pricing analysis

   - Test pricing with new customer cohort before rolling out to existing base"

Assessment : Business-ready. Specific, actionable, contextualized, and formatted appropriately.

Iteration 4 (Self-Check):

Prompt: "Review your analysis above. What assumptions did you make?

What additional data would strengthen your conclusions?"

Output:

"ASSUMPTIONS MADE:

1. Assumed price increase was the primary driver based on timing correlation,

   but haven't verified if churned customers actually saw the price increase

   (some may have been grandfathered)

2. Assumed 'too expensive' feedback relates to our price increase specifically,

   not general budget constraints

3. Estimated impact percentages (60-70%, 20-30%, 10-20%) based on qualitative assessment,

   not statistical analysis

ADDITIONAL DATA NEEDED:

1. Cohort analysis: Churn rate of customers who saw price increase vs. grandfathered customers

2. Exit interview details: Specific competitor mentions, budget constraint details

3. Feature usage data: How many churned customers actively used deprecated reporting feature

4. Historical churn patterns: Typical churn rate variation to determine if 5.1% is truly anomalous

5. Win-back data: Success rate of retention offers in past quarters"

Assessment : Excellent. The LLM has identified gaps in its own analysis, making the output more trustworthy and providing a clear roadmap for deeper investigation.

13.5 Common Business Analytics Use Cases


KPI Explanations and Executive Summaries

Use Case : Translating complex metrics into clear, executive-friendly narratives.

Example Prompt:

"Create an executive summary of our Q4 2025 performance metrics:

Metrics:

- Revenue: $12.3M (↑ 18% YoY, ↑ 5% QoQ)

- Customer Acquisition Cost (CAC): $450 (↑ 12% QoQ)

- Customer Lifetime Value (LTV): $3,200 (↓ 3% QoQ)

- LTV:CAC Ratio: 7.1 (↓ from 8.2 previous quarter)

- Net Revenue Retention: 112% (↓ from 118%)

- Gross Margin: 72% (stable)

Context: We expanded into European markets in Q4 and increased marketing spend by 40%.

Format: 3-paragraph executive summary highlighting:

1. Overall performance

2. Key concerns

3. Strategic implications

Tone: Balanced, data-driven, suitable for board presentation."

Sample Output:

Q4 2025 delivered strong top-line growth with revenue reaching $12.3M, representing 18% year-over-year  growth and 5% sequential growth. This performance was driven by our European market expansion and increased marketing investment, demonstrating successful execution of our geographic diversification strategy. Gross margins remained stable at 72%, indicating healthy unit economics despite expansion costs.

However, several efficiency metrics warrant attention. Customer Acquisition Cost increased 12% to $450, while Customer Lifetime Value declined 3% to $3,200, compressing our LTV:CAC ratio from 8.2 to 7.1. Additionally, Net Revenue Retention decreased from 118% to 112%, suggesting either increased churn or reduced expansion revenue within our existing customer base. While these metrics remain within healthy ranges, the directional trends require monitoring.

Strategically, these results suggest our European expansion is generating growth but at higher acquisition costs than our mature North American market. We should investigate whether the LTV decline is temporary (due to newer, less mature European customers) or structural (indicating product-market fit challenges). The NRR decrease merits immediate attention to understand if it reflects expansion-related resource constraints or deeper customer success issues. Recommended next steps: cohort analysis by geography, customer success capacity review, and European pricing strategy assessment."

Diagnostic Analysis ("Why Did This Change?")

Use Case : Generating hypotheses when metrics move unexpectedly.

Example Prompt:

"Our website conversion rate dropped from 3.2% to 2.1% between October and November 2025. Generate 8 hypotheses that could explain this decline, organized by likelihood.

Context:

- We redesigned our pricing page in late October

- Launched a new ad campaign targeting a broader audience in November

- Experienced a site outage for 4 hours on November 15th

- Competitor launched a major promotion in November

- Traffic increased 40% month-over-month

For each hypothesis:

1. Explain the mechanism

2. Suggest specific data to examine

3. Rate likelihood (High/Medium/Low)

Format as a numbered list."

Sample Output:

HYPOTHESIS 1 (High Likelihood): Audience Quality Dilution

Mechanism: The new ad campaign targeting a broader audience brought in less qualified traffic,

reducing overall conversion rate despite higher volume.

Data to examine:

- Conversion rate by traffic source (new campaign vs. existing channels)

- Bounce rate and time-on-site by source

- Conversion rate trend: did it decline gradually starting with campaign launch?

Likelihood: HIGH - 40% traffic increase with conversion rate decline is classic quality-vs-quantity tradeoff

HYPOTHESIS 2 (High Likelihood): Pricing Page Redesign Impact

Mechanism: The new pricing page design may have introduced friction, confusion, or removed

persuasive elements that previously drove conversions.

Data to examine:

- Conversion rate before/after pricing page redesign (specific date)

- Heatmaps and session recordings of pricing page interactions

- Exit rate from pricing page (before vs. after)

- A/B test data if available

Likelihood: HIGH - Pricing page is critical conversion point; redesigns often have unintended consequences

[... continues with 6 more hypotheses ...]

Scenario Framing and Assumption Testing

Use Case : Exploring "what-if" scenarios and stress-testing business assumptions.

Example Prompt:

"We're considering raising prices by 15% for our Pro tier. Model three scenarios:

Current State:

- Pro tier: $99/month

- 5,000 Pro customers

- Monthly churn rate: 4%

- Estimated price elasticity: -0.8 (based on past increases)

Scenarios to model:

1. Best case: Lower churn than expected (3% churn rate)

2. Base case: Expected churn based on elasticity

3. Worst case: Higher churn than expected (6% churn rate)

For each scenario, calculate:

- New monthly recurring revenue (MRR)

- Net MRR change vs. current state

- Break-even point (how many customers can we lose before MRR declines)

Format as a comparison table with clear recommendations."

SQL Generation and Query Explanation

Use Case : Translating business questions into SQL queries and explaining existing queries.

SQL Generation Example:

"Write a SQL query for our PostgreSQL database to answer this business question:

Question: Which customers have decreased their monthly spending by more than 20%

in the last 3 months compared to their average spending in the previous 6 months?

Database schema:

- customers table: customer_id, customer_name, signup_date, segment

- orders table: order_id, customer_id, order_date, order_amount

Requirements:

- Only include active customers (at least one order in last 30 days)

- Return: customer_id, customer_name, segment, previous_avg_monthly_spend,

  recent_avg_monthly_spend, percent_change

- Sort by percent_change ascending (biggest decreases first)

Include comments explaining each step."

Sample Output:

-- Calculate average monthly spending for two time periods and identify customers with >20% decline

WITH

-- Define time periods

time_periods AS (

  SELECT

    CURRENT_DATE AS today,

    CURRENT_DATE - INTERVAL '3 months' AS recent_period_start,

    CURRENT_DATE - INTERVAL '9 months' AS previous_period_start,

    CURRENT_DATE - INTERVAL '3 months' AS previous_period_end

),

-- Calculate average monthly spending for previous 6-month period (9-3 months ago)

previous_spending AS (

  SELECT

    o.customer_id,

    AVG(monthly_total) AS previous_avg_monthly_spend

  FROM (

    SELECT

      customer_id,

      DATE_TRUNC('month', order_date) AS month,

      SUM(order_amount) AS monthly_total

    FROM orders

    WHERE order_date >= (SELECT previous_period_start FROM time_periods)

      AND order_date < (SELECT previous_period_end FROM time_periods)

    GROUP BY customer_id, DATE_TRUNC('month', order_date)

  ) o

  GROUP BY o.customer_id

),

-- Calculate average monthly spending for recent 3-month period

recent_spending AS (

  SELECT

    o.customer_id,

    AVG(monthly_total) AS recent_avg_monthly_spend

  FROM (

    SELECT

      customer_id,

      DATE_TRUNC('month', order_date) AS month,

      SUM(order_amount) AS monthly_total

    FROM orders

    WHERE order_date >= (SELECT recent_period_start FROM time_periods)

    GROUP BY customer_id, DATE_TRUNC('month', order_date)

  ) o

  GROUP BY o.customer_id

),

-- Identify active customers (order in last 30 days)

active_customers AS (

  SELECT DISTINCT customer_id

  FROM orders

  WHERE order_date >= CURRENT_DATE - INTERVAL '30 days'

)

-- Final query: Join everything together and calculate percent change

SELECT

  c.customer_id,

  c.customer_name,

  c.segment,

  ROUND(ps.previous_avg_monthly_spend, 2) AS previous_avg_monthly_spend,

  ROUND(rs.recent_avg_monthly_spend, 2) AS recent_avg_monthly_spend,

  ROUND(

    ((rs.recent_avg_monthly_spend - ps.previous_avg_monthly_spend) / ps.previous_avg_monthly_spend) * 100,

    1

  ) AS percent_change

FROM customers c

INNER JOIN previous_spending ps ON c.customer_id = ps.customer_id

INNER JOIN recent_spending rs ON c.customer_id = rs.customer_id

INNER JOIN active_customers ac ON c.customer_id = ac.customer_id

WHERE

  -- Filter for >20% decline

  ((rs.recent_avg_monthly_spend - ps.previous_avg_monthly_spend) / ps.previous_avg_monthly_spend) < -0.20

ORDER BY percent_change ASC;  -- Biggest decreases first

Query Explanation Example:

"Explain what this SQL query does in plain English, suitable for a non-technical stakeholder:

[Complex SQL query here]"

13.6 Tools Analysts Actually Use

Gemini: Strengths for Document-Heavy and Spreadsheet Workflows

Overview : Google's Gemini models excel at processing long documents and multimodal inputs (text + images + spreadsheets).

Key Strengths:

Best Use Cases:

Example Workflow:

1. Upload Q4 financial report (PDF) to Gemini

2. Prompt: "Analyze this financial report and create a summary table of:

   - Revenue by segment

   - YoY growth rates

   - Key risks mentioned

   - Management's forward guidance

    Then compare these figures to our internal forecast spreadsheet [link to Google Sheet]"  

3. Gemini processes both documents and generates comparative analysis

Limitations:

When to use : Document-heavy analysis, Google Workspace environments, multimodal data interpretation.

GitHub Copilot: Coding, SQL, and Analytics Notebooks

Overview : GitHub Copilot is an AI pair programmer that assists with code generation, including SQL queries, Python data analysis scripts, and R code.

Key Strengths:

Best Use Cases:

Example Workflow:

# In Jupyter notebook, start typing a comment describing what you want:

# Load customer data from PostgreSQL, calculate monthly retention rate by cohort,

# and create a cohort retention heatmap

# Copilot suggests complete code:

import pandas as pd

import psycopg2

import seaborn as sns

import matplotlib.pyplot as plt

# Connect to database

conn = psycopg2.connect(

    host="your_host",

    database="your_db",

    user="your_user",

    password="your_password"

)

# Query customer data

query = """

SELECT

    customer_id,

    DATE_TRUNC('month', signup_date) AS cohort_month,

    DATE_TRUNC('month', order_date) AS order_month

FROM customers c

LEFT JOIN orders o ON c.customer_id = o.customer_id

"""

df = pd.read_sql(query, conn)

# [... continues with retention calculation and visualization code ...]

Limitations:

When to use : Code-heavy analytics workflows, SQL development, Python/R data analysis, notebook-based exploration.

Cursor: LLM-First IDE for Data and Analytics Code

Overview : Cursor is a code editor built from the ground up around LLM assistance, offering more sophisticated AI features than Copilot.

Key Strengths:

Best Use Cases:

Example Workflow:

1. Open your analytics project in Cursor

2. Use Cmd+K to open AI chat

3. Prompt: "This ETL script is running slowly on large datasets.

   Analyze the code and suggest optimizations for processing 10M+ rows."

4. Cursor analyzes your entire script, identifies bottlenecks, and suggests:

   - Vectorized pandas operations instead of loops

   - Chunked processing for memory efficiency

   - Parallel processing opportunities

5. Accept suggestions with one click, AI updates code across multiple functions

Limitations:

When to use : Complex analytics projects, code refactoring, learning new techniques, AI-assisted debugging.

Windsurf: AI-Assisted Analytics and Data Workflows

Overview : Windsurf is an emerging platform focused specifically on data analytics workflows, combining code generation with data exploration and visualization.

Key Strengths:

Best Use Cases:

Example Workflow:

1. Connect Windsurf to your data warehouse

2. Ask: "Show me monthly revenue trends by product category for 2025"

3. Windsurf:

   - Generates SQL query

   - Executes query

   - Creates visualization

   - Provides statistical summary

4. Follow-up: "Now break this down by customer segment and highlight anomalies"

5. Windsurf updates analysis, adds segmentation, flags outliers

Limitations:

When to use : Quick exploratory analysis, collaborative analytics, learning data analysis, rapid prototyping.

When to Use Each Tool, and When Not To

Tool

Best For

Avoid When

Gemini

Long documents, Google Workspace, multimodal analysis

Need deterministic outputs, real-time processing, non-Google ecosystem

GitHub Copilot

Code completion in existing IDE, SQL generation, notebook work

Need full codebase understanding, complex refactoring, business context

Cursor

Complex projects, multi-file refactoring, learning new techniques

Simple scripts, one-off analyses, budget constraints

Windsurf

Quick EDA, collaborative analysis, ad-hoc questions

Production pipelines, highly customized workflows, strict governance

General Guidance:

Use LLM tools when:

Don't use LLM tools when:

13.7 Bad vs. Good Prompting Practices

Overloaded Prompts

❌ Bad Practice: Cramming Too Much Into One Prompt

"Analyze our customer data and tell me about revenue trends, churn patterns,

customer segmentation, product performance, marketing channel effectiveness,

seasonal patterns, geographic distribution, customer lifetime value,

acquisition costs, and competitive positioning. Also generate SQL queries

for each analysis and create visualizations."

Problems:

✅ Good Practice: Break Into Focused Prompts

Prompt 1: "Analyze our customer churn patterns for Q4 2025.

Identify the top 3 customer segments with highest churn rates and

suggest potential causes based on the data."

[Review output, then...]

Prompt 2: "Based on your churn analysis, generate a SQL query that

identifies at-risk customers in the [specific segment] who match

the churn pattern you identified."

[Review output, then...]

Prompt 3: "Create a Python script using matplotlib to visualize

the churn rate trends by segment over the past 12 months."

Why it's better:

Missing Business Context

❌ Bad Practice: Providing Data Without Context

"Here's our sales data:

Q1: $2.3M

Q2: $2.1M

Q3: $2.8M

Q4: $2.5M

What does this tell us?"

Problems:

✅ Good Practice: Provide Rich Business Context

"Analyze our quarterly sales data for 2025:

Business Context:

- We're a B2B SaaS company selling project management software

- Typical deal size: $50K-200K annually

- Sales cycle: 3-6 months

- We launched a new Enterprise tier in Q2

- Hired 5 new sales reps in Q3 (ramp time: 3 months)

- Major competitor acquired in Q4, causing market uncertainty

Data:

Q1: $2.3M (baseline)

Q2: $2.1M (↓ 9%)

Q3: $2.8M (↑ 33%)

Q4: $2.5M (↓ 11%)

Questions:

1. Is the Q2 dip concerning given the Enterprise tier launch?

2. Did the Q3 spike come from new reps or other factors?

3. What might explain the Q4 decline?

4. What should we investigate further?"

Why it's better:

Blind Trust in Confident Outputs

❌ Bad Practice: Accepting LLM Outputs Without Verification

Prompt: "Calculate the ROI of our marketing campaigns."

LLM Output: "Based on the data, your email marketing ROI is 450%,

paid search is 280%, and social media is 190%."

Analyst: [Copies numbers directly into executive report]

Problems:

✅ Good Practice: Verify, Validate, and Cross-Check

Prompt: "Calculate the ROI of our marketing campaigns using this data:

[data provided]

Show your calculation methodology step-by-step.

Include any assumptions you're making about attribution,

time windows, and cost allocation."

LLM Output: [Provides calculations with methodology]

Analyst Actions:

1. Review calculation methodology for correctness

2. Verify a few calculations manually

3. Cross-check against historical ROI patterns

4. Ask LLM: "What are the limitations of this ROI calculation?

   What factors might make these numbers misleading?"

5. Only then use in report, with appropriate caveats

Why it's better:

Side-by-Side Bad → Improved Prompts

Example 1: Customer Segmentation

❌ Bad:

"Segment our customers."

✅ Improved:

"Create a customer segmentation strategy for our B2B SaaS business.

Current situation:

- 2,500 customers across SMB, Mid-market, and Enterprise

- Available data: company size, industry, product usage, revenue, tenure, support tickets

Business goal: Identify segments for targeted retention campaigns

Requirements:

1. Propose 3-5 meaningful segments based on behavior and value

2. For each segment, describe:

   - Defining characteristics

   - Estimated size

   - Key needs/pain points

   - Retention risk level

3. Suggest 2-3 retention strategies per segment

Format: Table with segment profiles, then detailed recommendations"

Example 2: SQL Query Generation

❌ Bad:

"Write SQL to get customer info."

✅ Improved:

"Write a PostgreSQL query to retrieve customer information for our monthly business review.

Database schema:

- customers: customer_id, name, signup_date, plan_type, mrr

- usage_logs: customer_id, feature_used, usage_date, duration_minutes

- support_tickets: customer_id, ticket_date, status, priority

Requirements:

- Include customers active in the last 90 days

- Calculate for each customer:

  * Total MRR

  * Days since signup

  * Number of features used in last 30 days

  * Total usage minutes in last 30 days

  * Number of open support tickets

- Filter to customers with MRR > $500

- Sort by MRR descending

Include comments explaining each section of the query."

Example 3: Trend Analysis

❌ Bad:

"Why are sales down?"

✅ Improved:

"Analyze why our sales declined 15% in November 2025 compared to October.

Context:

- We sell B2B marketing software

- Typical seasonality: Q4 is usually our strongest quarter

- Recent changes:

  * Raised prices 10% on Nov 1

  * Launched new competitor in October

  * Reduced marketing spend 20% in November

  * Lost 2 senior sales reps in October

Data available:

- Monthly sales by product line

- Lead volume and conversion rates

- Average deal size

- Sales cycle length

- Win/loss reasons

Task:

1. Generate 5 hypotheses ranked by likelihood

2. For each hypothesis, specify what data would confirm/refute it

3. Recommend immediate actions to investigate

4. Suggest leading indicators to monitor going forward

Format: Structured analysis suitable for sales leadership meeting"

13.8 Making LLMs Reliable in Business Settings

Validation and Human-in-the-Loop

LLMs should augment human judgment, not replace it. Effective validation requires structured processes:

Three-Layer Validation Framework:

Layer 1: Automated Checks

Example:

def validate_llm_analysis(output):

    """Automated validation of LLM-generated analysis"""

    checks = {

        'has_summary': 'summary' in output,

        'has_recommendations': 'recommendations' in output,

        'recommendation_count': len(output.get('recommendations', [])) >= 3,

        'metrics_in_range': all(0 <= m <= 100 for m in output.get('percentages', []))

    }

    return all(checks.values()), checks

Layer 2: Spot Validation

Layer 3: Continuous Monitoring

Human-in-the-Loop Patterns:

Pattern 1: Review-Before-Use

LLM generates output → Human reviews → Human approves/rejects → Use in decision

Best for: High-stakes decisions, regulatory contexts, unfamiliar use cases

Pattern 2: Use-Then-Audit

LLM generates output → Automated validation → Use in decision → Periodic human audit

Best for: High-volume, lower-stakes tasks with good validation rules

Pattern 3: Collaborative Refinement

LLM generates draft → Human refines → LLM improves → Human finalizes

Best for: Complex analyses, reports, strategic recommendations

Reusable Prompt Templates

Creating standardized prompt templates improves consistency and reduces errors:

Template Structure:

# Template Name: [Descriptive name]

# Purpose: [What this template accomplishes]

# When to use: [Appropriate use cases]

# Validation required: [How to verify outputs]

## Prompt Template:

[Task description]

Context:

- [Context variable 1]: {VARIABLE_1}

- [Context variable 2]: {VARIABLE_2}

Data:

{DATA_INPUT}

Requirements:

1. [Requirement 1]

2. [Requirement 2]

3. [Requirement 3]

Output format:

{FORMAT_SPECIFICATION}

Constraints:

- [Constraint 1]

- [Constraint 2]

Example Template: Monthly KPI Summary

# Template: Monthly KPI Summary

# Purpose: Generate executive summary of monthly performance metrics

# When to use: Monthly business reviews, board reports

# Validation: Verify all metrics against source data, check YoY/MoM calculations

## Prompt:

Create an executive summary of {MONTH} {YEAR} performance metrics.

Context:

- Company: {COMPANY_NAME}

- Industry: {INDUSTRY}

- Key strategic initiatives this period: {INITIATIVES}

Metrics:

{METRICS_DATA}

Requirements:

1. 3-paragraph summary (Overall performance, Key concerns, Strategic implications)

2. Highlight metrics that moved >10% vs. previous period

3. Compare to same period last year

4. Identify 2-3 actionable insights

Output format:

- Paragraph 1: Overall performance (3-4 sentences)

- Paragraph 2: Key concerns (3-4 sentences)

- Paragraph 3: Strategic implications (3-4 sentences)

- Bullet list: Top 3 actionable insights

Tone: Professional, data-driven, balanced (acknowledge both positives and concerns)

Constraints:

- Maximum 300 words total

- Cite specific metrics with numbers

- No speculation beyond what data supports

Template Library Organization:

/prompt-templates/

  /kpi-analysis/

    - monthly-summary.md

    - quarterly-deep-dive.md

    - yoy-comparison.md

  /diagnostic/

    - metric-decline-analysis.md

    - anomaly-investigation.md

    - cohort-comparison.md

  /sql-generation/

    - customer-segmentation.md

    - revenue-analysis.md

    - churn-prediction.md

  /reporting/

    - executive-brief.md

    - technical-deep-dive.md

    - stakeholder-update.md

Team Standards and Governance

Establishing clear standards ensures consistent, reliable LLM use across your analytics team:

LLM Usage Policy Framework:

1. Approved Use Cases

✅ Approved:

- Exploratory data analysis and hypothesis generation

- SQL query generation (with review)

- Report drafting and summarization

- Code documentation and explanation

- Data interpretation and insight generation

⚠️ Requires Additional Review:

- Financial calculations for external reporting

- Customer-facing communications

- Regulatory or compliance-related analysis

- Pricing or contract decisions

❌ Prohibited:

- Sending PII or sensitive customer data to external APIs

- Final decision-making without human review

- Automated actions based solely on LLM outputs

2. Data Handling Standards

- Never send customer PII to external LLM APIs

- Anonymize data before using with cloud-based LLMs

- Use on-premise or private LLM deployments for sensitive data

- Log all LLM interactions with business data

- Implement data retention policies for LLM inputs/outputs

3. Quality Standards

- All LLM-generated SQL must be reviewed before execution

- Statistical claims must be verified against source data

- LLM-generated insights require human validation before sharing with stakeholders

- Document prompt templates and validation procedures

- Track accuracy metrics for common use cases

4. Documentation Requirements

For any LLM-assisted analysis shared with stakeholders:

- Disclose that LLM was used

- Document the prompt(s) used

- Describe validation steps taken

- Note any limitations or caveats

- Provide source data references

Governance Structure:

LLM Governance Committee

├── Data Privacy Officer: Ensures compliance with data policies

├── Analytics Lead: Defines approved use cases and quality standards

├── IT Security: Manages API access and data security

└── Domain Experts: Validate outputs in specialized areas

Responsibilities:

- Quarterly review of LLM usage patterns

- Incident investigation and response

- Template and standard updates

- Training and onboarding for new team members

Productivity vs. Risk Trade-Offs

Using LLMs in business analytics involves balancing speed and innovation against accuracy and risk:

Risk Assessment Matrix:

Use Case

Productivity Gain

Risk Level

Mitigation Strategy

Exploratory data analysis

High (5x faster)

Low

Minimal validation needed

SQL query generation

High (3-4x faster)

Medium

Mandatory code review

Executive report drafting

Medium (2x faster)

Medium

Human editing + fact-checking

Financial calculations

Low (1.5x faster)

High

Full manual verification

Customer-facing insights

Medium (2x faster)

High

Multi-layer review process

Regulatory reporting

Low (1.2x faster)

Very High

Traditional methods preferred

Decision Framework:

When to prioritize productivity (use LLMs more freely):

When to prioritize risk mitigation (use LLMs cautiously):

Balanced Approach:

1. Start with LLM assistance for speed

2. Apply appropriate validation based on risk level

3. Document LLM usage and validation steps

4. Build confidence through repeated successful use

5. Gradually expand to higher-stakes use cases as trust builds

13.9 Practical Prompt Templates (Quick Reference)

KPI Summary

Generate a KPI summary for {TIME_PERIOD}.

Metrics:

- {METRIC_1}: {VALUE} ({CHANGE} vs. {COMPARISON_PERIOD})

- {METRIC_2}: {VALUE} ({CHANGE} vs. {COMPARISON_PERIOD})

- {METRIC_3}: {VALUE} ({CHANGE} vs. {COMPARISON_PERIOD})

Context: {BUSINESS_CONTEXT}

Format:

1. Overall assessment (2-3 sentences)

2. Key highlights (bullet points)

3. Areas of concern (bullet points)

4. Recommended actions (numbered list)

Tone: {AUDIENCE_APPROPRIATE_TONE}

Length: {WORD_LIMIT} words maximum

Executive Briefing

Create an executive briefing on {TOPIC} for {AUDIENCE}.

Background:

{SITUATION_DESCRIPTION}

Key Data Points:

{DATA_SUMMARY}

Requirements:

1. Executive summary (3-4 sentences)

2. Situation analysis (1 paragraph)

3. Implications (1 paragraph)

4. Recommendations (3-5 specific actions)

Format: Professional memo style

Length: 1 page (approximately 400-500 words)

Tone: Direct, data-driven, action-oriented

Constraints:

- Avoid jargon

- Lead with conclusions

- Support claims with specific data

- Focus on decisions, not just information

Diagnostic Analysis

Analyze why {METRIC} changed from {BASELINE_VALUE} to {CURRENT_VALUE}

between {START_DATE} and {END_DATE}.

Context:

- Business: {BUSINESS_DESCRIPTION}

- Recent changes: {RELEVANT_CHANGES}

- Historical patterns: {SEASONALITY_OR_TRENDS}

Available data:

{DATA_SOURCES}

Task:

1. Generate 5-7 hypotheses ranked by likelihood

2. For each hypothesis:

   - Explain the mechanism

   - Specify data needed to validate

   - Rate likelihood (High/Medium/Low)

3. Recommend immediate investigation steps

4. Suggest leading indicators to monitor

Format: Structured analysis with clear sections

SQL Generation

Write a {DATABASE_TYPE} SQL query to answer this business question:

Question: {BUSINESS_QUESTION}

Database schema:

{TABLE_DEFINITIONS}

Requirements:

- {REQUIREMENT_1}

- {REQUIREMENT_2}

- {REQUIREMENT_3}

Output columns: {DESIRED_COLUMNS}

Filters: {FILTER_CONDITIONS}

Sort order: {SORT_SPECIFICATION}

Include:

- Comments explaining each major section

- Proper formatting and indentation

- Handling of NULL values where relevant

Output Validation Prompt

Review the analysis you just provided and perform a self-check:

1. Accuracy Check:

   - Are all calculations correct?

   - Are statistics properly interpreted?

   - Are there any logical inconsistencies?

2. Completeness Check:

   - Did you address all parts of the question?

   - Are there important caveats or limitations to mention?

   - What additional context would strengthen the analysis?

3. Assumption Check:

   - What assumptions did you make?

   - Which assumptions are most likely to be wrong?

   - How would incorrect assumptions change your conclusions?

4. Alternative Explanations:

   - What alternative interpretations of the data exist?

   - What evidence would support or refute each interpretation?

Provide your self-assessment, then revise your original analysis if needed.

Key Takeaways

  1. LLMs complement, not replace, traditional analytics : Use them for unstructured data, natural language interfaces, and rapid exploration—but validate outputs with traditional methods.
  2. Iteration is essential : The first LLM output is rarely usable. Plan for multiple refinement cycles.
  3. Context is everything : Provide rich business context, clear constraints, and specific requirements to get useful outputs.
  4. Validate rigorously : Never trust LLM outputs blindly. Implement multi-layer validation, especially for high-stakes decisions.
  5. Build reusable templates : Standardized prompts improve consistency, reduce errors, and accelerate team adoption.
  6. Balance productivity and risk : Use LLMs freely for exploration and drafting, but apply strict validation for financial, regulatory, or customer-facing work.
  7. Choose the right tool : Different LLM tools excel at different tasks—match the tool to your workflow.
  8. Establish governance : Clear policies, documentation standards, and quality controls ensure reliable, responsible LLM use.

Exercises

Exercise 1: Prompt Refinement Practice

Take this vague prompt and refine it into an effective business analytics prompt:

"Analyze our customer data and tell me what's important."

Your refined prompt should include:

Exercise 2: Iterative Prompting Workflow

You're analyzing why website conversion rates dropped 25% last month. Design a 4-step iterative prompting workflow:

  1. Initial exploratory prompt
  2. Hypothesis generation prompt (based on initial findings)
  3. Data validation prompt
  4. Recommendation generation prompt

For each step, write the complete prompt and describe what you'd look for in the output before proceeding.

Exercise 3: Validation Framework Design

Design a validation framework for LLM-generated SQL queries in your organization. Your framework should include:

Exercise 4: Prompt Template Creation

Create a reusable prompt template for a common analytics task in your organization (e.g., monthly churn analysis, campaign performance review, product usage trends). Your template should:

Further Reading

This chapter has equipped you with practical frameworks for integrating LLMs into your business analytics workflow. The key to success lies not in treating LLMs as magic solutions, but in understanding their strengths, limitations, and appropriate use cases—then applying rigorous validation and iteration to generate reliable, business-ready insights.