A large language model (LLM) is trained on vast amounts of text to interpret and generate human-like writing across many tasks. Many LLMs are foundation models: their broad pretraining lets people request useful output without first training a model for a particular domain. A prompt supplies the immediate request. section 12.5 section 12.4.1
In analytics, an LLM can help overcome the blank page by drafting a document or template. Chatbots and assistants built into analytics tools can also support everyday workflows. They do not replace an analyst’s judgment about the business question, the data, or what a stakeholder needs to know. section 12.4.1
Make the question specific
Asked only to analyze an uploaded customer-login dataset, a chatbot described its columns and asked what insights were wanted. A follow-up request for the mean and median number of login days, plus regional means, produced those statistics. The useful step was specifying the measures and grouping rather than relying on “analyze” alone. section 12.4.1
Table 12.4 Updated prompt and LLM response using ChatGPT 4.0
|
Prompt
|
Response
|
|---|---|
|
Provide the mean and median for login_days, and mean login_days values by region.
|
Here are the statistics for login_days:
Mean Login Days (Overall): 15.73 days
Median Login Days (Overall): 16.0 days
Mean Login Days by Region:
Asia: 21.37 days
Europe: 14.49 days
North America: 17.53 days
South America: 9.51 days
These metrics highlight how often customers log in per month, both overall and within different regions. If there’s more you’d like to explore or another aspect of the data you’re interested in, just let me know!
|
The revised prompt and response show how naming the desired statistics produces a more targeted answer. section 12.4.1
Prompt engineering means revising and testing instructions for more consistent, useful output. One way to assess an LLM as an analytics assistant is to ask it to document a predictive model, report performance metrics, provide its Python code, and compare its work with an existing analysis. A plausible response is not, by itself, evidence that the analysis is sound. section 12.5 section 12.4.4
Use the output critically. LLMs can reproduce bias present in training data and generate convincing misinformation. Training and using them also consumes substantial resources. Review claims and recommendations before they inform decisions, particularly where errors or bias could affect people. section 12.4.2
