Pro Skill Packs logo PRO SKILL PACKS

Guides / Data and spreadsheets

How do I summarise survey results without overclaiming?

A prompt that turns a survey export into counts with the base size, themed open-text answers with quotes, and a plain list of what the data cannot show. Copy it, paste in your rows, check the counts.

You ran a survey. Now you have a spreadsheet, a pile of free-text answers and a meeting on Thursday. The risk is not that you cannot summarise it. The risk is that you summarise it too well: "customers want faster shipping" when twelve of three hundred people said so.

The fix is a habit, not a tool. Counts first, then percentages. The base size next to every number. Quotes for themes, and a plain statement of what the data cannot tell you.

What you need

  • The export: a CSV or the rows pasted with the header. If it is large, paste the header and a sample, and say how many rows there are in total. Counts from a sample are not counts from the whole file, so a tool that can run code is better for big files.
  • How people were recruited: an email list, a social post, a pop-up on your site, customers only. This decides what you can say.
  • The decision the survey should inform, and any groups you want to compare.

The prompt

You summarise survey results with the numbers behind them, and you stay inside what the data can support.

Survey about: [topic]
Recruited how: [email list / social post / customers / unknown]
Total rows: [n]
Decision it should inform (optional): [ ]
Groups to compare (optional): [ ]

Data:
[paste header and rows]

Do this:
1. Data check: number of rows, blank rate per question, duplicate rows, obvious test or invalid rows. Show the base (n) for every question, because blanks differ.
2. Closed questions: counts first, then percentages rounded to whole numbers, with n. For scales show the distribution, and give an average only for true numeric scales. If a group has fewer than 20 responses, show the raw count next to any percentage.
3. Open text: read all answers (or say how many you read). Build 4 to 8 themes from the answers themselves. For each theme give a name, how many responses it appears in (one response can fit more than one), two exact quotes, and whether it is a complaint, a request or praise. Count "Other or unclear". Redact names, emails and other personal details in quotes.
4. Comparisons: only for groups I named or that clearly matter, and only if each group has at least 10 responses. Show both groups' numbers. Do not say "significant" unless you ran a named test and say which one.
5. Limits: sample size, how people were recruited, self-selection, non-response, and any leading or double-barrelled questions.

Output: Headline findings (3 to 5 bullets, each with its number and n), Data check, Results by question, Themes from open text, Comparisons, What this data cannot tell you, Suggested follow-up questions, and a Method note saying what you counted by hand and whether you worked from a sample.
Never invent respondents, quotes or causes. Write "12 of 300", not "customers want".

How to check the result

Pick three numbers from the summary and recount them in your spreadsheet with a filter or a pivot table. If the base size is wrong in one, assume it is wrong in others.

Read the themes against the raw answers. Look for answers the assistant put in the wrong theme, and for a theme that is really two. Check that every quote is exact and has no names left in it.

Look at the "cannot tell you" section. If it is short, the summary is probably overclaiming. A survey of people who joined your newsletter does not describe your market.

A note on small groups

If only nine people answered an open question, nine is the finding. Say "9 of 120 people wrote anything" and treat the themes as a list of things to ask about next time, not as a ranking. A good summary says when there is not enough data to compare groups, and does not do it anyway.

Limits

  • It reports what is in the file. It cannot know whether respondents are representative of anyone.
  • Free-text fields such as country or job title may need merging by hand. The summary should list what it merged, and you should check it.
  • It does not run statistical tests unless it can run code and tells you which one. Without that, it will not call a difference significant.
  • Do not paste personal data into a chatbot you do not control. Remove names and emails first.
  • A good summary tells you what to ask next. It does not replace asking.

Browse all skills in the catalog