--- title: Getting “under the hood” of a sample size calculator, Part I: Margin of error, population variability, and confidence levels description: "Explore the key factors influencing sample size calculation: margin of error, population variability, and confidence levels. Understand their interconnections to strengthen your research approach." image: https://info.keylimeinteractive.com/hubfs/two%20colleagues%20interior%20designer%20discussing%20data%20and%20digital%20tablet%20and%20computer%20laptop%20with%20sample%20material%20on%20wooden%20desk%20as%20concept.jpeg --- [![Key Lime Logo](https://info.keylimeinteractive.com/hs-fs/hubfs/Key%20Lime%20Logo.png?width=200&height=39&name=Key%20Lime%20Logo.png "Key Lime Logo")](https://keylimeinteractive.com/) - SERVICES - [All Services](https://keylimeinteractive.com/services) - [UX Strategy](https://keylimeinteractive.com/ux-strategy-design/) - [User Research](https://keylimeinteractive.com/user-research/) - [UX/UI Design](https://keylimeinteractive.com/ux-design-strategy/) - [EmTech Strategy](https://keylimeinteractive.com/emerging-technology/) - OUR WORK - [Our Approach](https://dev.keylimeinteractive.com/our-approach/) - [Enterprise Clients](https://keylimeinteractive.com/work/) - [Public Sector](https://keylimeinteractive.com/public-sector-gsa-schedule/) - [ABOUT US](https://keylimeinteractive.com/about-us/) - [Who We Are](https://keylimeinteractive.com/about-us/) - [People](https://keylimeinteractive.com/our-team/) - [Careers](https://workforcenow.adp.com/mascsr/default/mdf/recruitment/recruitment.html?cid=1eb079d3-bb50-438a-a731-eaa18a982193&ccId=19000101_000001&lang=en_US) - INSIGHTS - [CX Blog](https://info.keylimeinteractive.com) - [Events](https://keylimeinteractive.com/upcoming-events/) - [Competitive Indexes](https://keylimeinteractive.com/reports/#rankings) - [White Papers](https://keylimeinteractive.com/reports/) - [CONTACT US](https://keylimeinteractive.com/contact/) [![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg)](https://info.keylimeinteractive.com/) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) [Contact us](https://info.keylimeinteractive.com/contact) [![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg)](https://info.keylimeinteractive.com/) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Blog](https://info.keylimeinteractive.com/) - Resources - [Case Studies](https://keylimeinteractive.com/case-studies/) - [Events](https://info.keylimeinteractive.com/events) [Contact us](https://info.keylimeinteractive.com/contact) - - - # Getting “under the hood” of a sample size calculator, Part I: Margin of error, population variability, and confidence levels [By Peter Caprariello](https://info.keylimeinteractive.com/author/peter-caprariello) March 05, 2025 [UX Research](https://info.keylimeinteractive.com/topic/ux-research), [Unmoderated](https://info.keylimeinteractive.com/topic/unmoderated), [sample size calculation](https://info.keylimeinteractive.com/topic/sample-size-calculation) - [Tweet](https://twitter.com/share) ![sample size calculation](https://info.keylimeinteractive.com/hubfs/two%20colleagues%20interior%20designer%20discussing%20data%20and%20digital%20tablet%20and%20computer%20laptop%20with%20sample%20material%20on%20wooden%20desk%20as%20concept.jpeg) This is Part I in a series about sample size. Part II extends the concepts discussed below. To read Part II, please click here.    For research professionals in behavioral fields like UXR, consumer behavior, and social psychology, sample size is often the most expensive line item in the budget. Participants exchange their time and energy for your research needs, and the price tag for this exchange adds up quickly. Survey research in particular requires many data points to draw definitive conclusions, so it does not pay to cut corners with the research budget when planning a survey study. But how do we know how many participants is *enough*?   # **Peeking under the hood**   Many of the sample size calculators available online are handy (here [is one](https://www.statskingdom.com/50_ci_sample_size.html), and here [is another](https://www.appinio.com/en/blog/market-research/margin-of-error-and-sample-size)). You feed the calculator some inputs, it spits out a minimum sample size, and – voila! – you’ve done your due diligence. The research budget is now justified to stakeholders.    Except, further diligence may be due if you’re asked to explain *how* the sample size calculator arrived at its answer. Like driving a car without understanding how an engine works, it’s easy to calculate sample size without a full grasp of the online calculator’s mathematical guts. Let’s change that.   If we take a peek “underneath the hood” of a sample size calculator, you may find, as I did, that understanding the mechanics of sample size builds your strengths as a communicator. Better communication with stakeholders justifies not only the integrity of the research but also your seat at the table.   This post will review three important inputs to sample size: margins of error, confidence levels, and population variability. Each is like the “lever” of the sample size calculator. These levers are interconnected, resulting in **two dependencies**, discussed below, that are key to unlocking the sample size algorithm.   # **Margins of error**   No sample perfectly estimates the population average. The margin of error is a metric of precision, defining how narrowly the sample approximates the population’s “true value.” The smaller the margin of error, the more precise the sample’s estimates. For example, a study reporting margins of error of +/- 2% has more narrow boundaries, and therefore more precise estimates, than a study reporting margins of error +/- 5%.    A survey reporting a narrow margin of error is like a powerful magnifying glass, zooming in on an increasingly precise sample estimate. Added precision provides clarity for decision-makers in determining an appropriate course of action from your results.   # **Population variability **   Population variability refers to a population’s heterogeneity or diversity. The more diverse the population, the more likely there are extreme or outlying cases not well represented by the population’s average. In other words, the more heterogeneous the population, the more responses within a sample will spread widely around the average.   Populations with high variability require larger samples than populations with low variability. Gardens provide a nice analogy. A garden that grows one type of flower is like a population with low variability. You can pick a few bunches and each bunch will resemble the one before it. It will take only a few bunches to summarize the garden’s average color. A garden with high variability grows many kinds of flowers, and you will probably have to pick many bunches before you can determine the garden’s average color.   # **Dependency #1: High variability means wide margins of error**   Population variability and margin of error are naturally linked. *All else being equal,* as population variability increases, so does the margin of error. Quite literally, there is more diversity to account for. When population variability is low, the margin of error shrinks, because samples from these populations will contain fewer deviations from the average.    Visualize this relationship in the garden analogy as the number of flowers within each bunch that are of average color. In a garden with low variability, the margin of error is low, so there will be fewer flowers of extreme or unique colors. In a garden with high variability, the margin of error is high, and each bunch will contain flowers of many colors, some average and some far from average.    # **Confidence levels**   Confidence levels account for the fact that sampling is inherently probabilistic: There is always a risk of seeing something that’s not there or of missing something that is. Confidence levels therefore answer the question: What is the probability that your study produces “true” results and not “false positive” results?    Confidence levels are set in advance, based on how much risk we are willing to take. Setting confidence levels to 90% means accepting the risk that studies identical to ours capture the true value of the population nine times out of 10.    In the garden analogy, this would mean that one out of 10 bunches will contain zero flowers of average color. If this happened to be the bunch that you picked, you would end up making false inferences about the garden. If you cannot tolerate being wrong 10% of the time, then maybe you set confidence levels to 95%, allowing the possibility that the average color exists within 19 bunches out of 20. Setting confidence levels higher reduces the risk of false positives even more.    # **Dependency #2: High confidence means wide margins of error**   Confidence levels and margins of error are directly related. *All else being equal*, the tradeoff for having higher confidence is accepting greater uncertainty about the precise “location” of the population’s true average value.    This can be counterintuitive. High confidence levels are good, and low margins of error are good, so why should setting high confidence levels lead to something bad?    Theoretically, imagine having 100% confidence in a sample. This would mean the margins of error must encompass every response you collected. We wouldn’t know “where” the true average is, but we would have 100% confidence that it’s *somewhere* in the sample. By this logic, we could as easily claim that a few outliers reflect the average as any of the many responses clustered near the center. Outliers are by definition rare, so this can’t be true.    So we must give up some confidence - reducing to let’s say 99% - in exchange for the ability to exclude the edge cases. We are now 99% certain we’ve captured the true population average, by narrowing the margin of error.   This process could continue. If it did, the range of values within which we claim the true average lies would continue to shrink. But we would make this claim with increasingly less confidence than if the margins of error were wider.    # **“Wait, I thought this post was about sample size!”**   Margins of error, population variability, and confidence levels are interconnected. It’s not just the “levers” themselves that affect sample size, it’s the interconnections between the levers. A foundational understanding of these dependencies is central to understanding the dynamics of sample size estimation.    With this foundation in hand, please see Part II of this blog, which will discuss how sample size materializes from these inputs and their dependencies.  To learn more about how [Key Lime Interactive](https://keylimeinteractive.com/) can improve customer experiences at every stage—from discovery to development to validation, [contact us](https://keylimeinteractive.com/contact/).    ###### More by this Author [Peter Caprariello](https://info.keylimeinteractive.com/author/peter-caprariello) ##### You might also like ![Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/hubfs/Multiethnic%20group%20of%20young%20people%20sitting%20in%20conference%20room%20and%20brainstorming%20on%20business%20meeting.jpeg) [Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/upselling-in-a-saas-environment-with-a-user-centric-approach) March 05, 2025 ![5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/hubfs/Attractive%20businessman%20speaking%20with%20fellow%20coworkers.jpeg) [5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/5-ways-to-get-the-most-out-of-working-with-a-ux-research-consultant) March 05, 2025 ![Parrot Talk, Brain Dumps, and Other Tips & Tricks to be a Better Active Listener in UX Research Sessions](https://info.keylimeinteractive.com/hubfs/Concentrated%20bearded%20young%20man%20using%20laptop%20while%20his%20friends%20studying%20together.jpeg) [Parrot Talk, Brain Dumps, and Other Tips & Tricks to be a Better Active Listener in UX Research Sessions](https://info.keylimeinteractive.com/parrot-talk-brain-dumps-and-other-tips-tricks-to-be-a-better-active-listener-in-ux-research-sessions) March 05, 2025 #### Comments [Add Comment](https://info.keylimeinteractive.com/getting-under-the-hood-of-a-sample-size-calculator-part-i-margin-of-error-population-variability-and-confidence-levels#comment-form) ABOUT KLI [Key Lime Interactive](https://keylimeinteractive.com/about-us)is a user experience research and service design agency, with a sweet spot for emerging technology. As UX experts, our goal is to make your life easier, optimize user experiences, and make the world a better place. - [Recent](https://info.keylimeinteractive.com/getting-under-the-hood-of-a-sample-size-calculator-part-i-margin-of-error-population-variability-and-confidence-levels#tabs-0) - [Popular](https://info.keylimeinteractive.com/getting-under-the-hood-of-a-sample-size-calculator-part-i-margin-of-error-population-variability-and-confidence-levels#tabs-1) ![Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/hubfs/Multiethnic%20group%20of%20young%20people%20sitting%20in%20conference%20room%20and%20brainstorming%20on%20business%20meeting.jpeg) [Upsell with Purpose: Fueling SaaS Growth Through User Insight](https://info.keylimeinteractive.com/upselling-in-a-saas-environment-with-a-user-centric-approach) Mar 5, 2025 ![5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/hubfs/Attractive%20businessman%20speaking%20with%20fellow%20coworkers.jpeg) [5 Ways to Get the Most Out of Working with a UX Research Consultant](https://info.keylimeinteractive.com/5-ways-to-get-the-most-out-of-working-with-a-ux-research-consultant) Mar 5, 2025 ![Designing Engagement Through Micro-Interactions](https://info.keylimeinteractive.com/hubfs/AI-Generated%20Media/Images/A%20closeup%20shot%20of%20a%20finger%20gently%20tapping%20a%20digital%20button%20on%20a%20sleek%2c%20modern%20interface.jpeg) [Designing Engagement Through Micro-Interactions](https://info.keylimeinteractive.com/the-significance-of-micro-interactions-in-shaping-user-engagement-and-experience) Mar 5, 2025 ![What is the Keystroke-Level Model?](https://info.keylimeinteractive.com/hubfs/Keystroke.jpg) [What is the Keystroke-Level Model?](https://info.keylimeinteractive.com/what-is-the-keystroke-level-model) Mar 5, 2025 ![Current-State vs. Future-State Journey Mapping](https://info.keylimeinteractive.com/hubfs/destination-2386255_1280.jpg) [Current-State vs. Future-State Journey Mapping](https://info.keylimeinteractive.com/current-state-vs.-future-state-journey-mapping) Mar 5, 2025 ![A History of Voice Technology](https://info.keylimeinteractive.com/hubfs/Screen%20Shot%202020-08-11%20at%2012.18.34%20PM.png) [A History of Voice Technology](https://info.keylimeinteractive.com/history-of-voice-technology) Mar 5, 2025 Create a UX Strategy [![key-lime-interactive-ux](https://info.keylimeinteractive.com/hs-fs/hubfs/key-lime-interactive-ux.png?width=635&height=634&name=key-lime-interactive-ux.png)](https://keylimeinteractive.com/ux-strategy-design) ### Post By Topics - [Research Methods](https://info.keylimeinteractive.com/topic/research-methods) - [UX Strategy](https://info.keylimeinteractive.com/topic/ux-strategy) - [Events & News](https://info.keylimeinteractive.com/topic/events-news) - [UX Research](https://info.keylimeinteractive.com/topic/ux-research) - [User Experience](https://info.keylimeinteractive.com/topic/user-experience) - [User Interface](https://info.keylimeinteractive.com/topic/user-interface) - [UX](https://info.keylimeinteractive.com/topic/ux) - [News](https://info.keylimeinteractive.com/topic/news) - [Mobile UX](https://info.keylimeinteractive.com/topic/mobile-ux) - [Usability Testing](https://info.keylimeinteractive.com/topic/usability-testing) - [Competitive Research](https://info.keylimeinteractive.com/topic/competitive-research) - [UX Design](https://info.keylimeinteractive.com/topic/ux-design) - [Eye tracking & Biometrics](https://info.keylimeinteractive.com/topic/eye-tracking-biometrics) - [Qualitative Research](https://info.keylimeinteractive.com/topic/qualitative-research) - [Customer Experience](https://info.keylimeinteractive.com/topic/customer-experience) - [Journey Mapping](https://info.keylimeinteractive.com/topic/journey-mapping) - [DEI](https://info.keylimeinteractive.com/topic/dei) - [Remote Testing](https://info.keylimeinteractive.com/topic/remote-testing) - [User Experience Consultant](https://info.keylimeinteractive.com/topic/user-experience-consultant) - [VR](https://info.keylimeinteractive.com/topic/vr) - [XR](https://info.keylimeinteractive.com/topic/xr) - [Multicultural UX](https://info.keylimeinteractive.com/topic/multicultural-ux) - [Mobile Banking](https://info.keylimeinteractive.com/topic/mobile-banking) - [Quantitative Research](https://info.keylimeinteractive.com/topic/quantitative-research) - [App Development](https://info.keylimeinteractive.com/topic/app-development) - [User Experience Research Studies](https://info.keylimeinteractive.com/topic/user-experience-research-studies) - [Diversity Equity Inclusion](https://info.keylimeinteractive.com/topic/diversity-equity-inclusion) - [UI](https://info.keylimeinteractive.com/topic/ui) - [Voice Technology](https://info.keylimeinteractive.com/topic/voice-technology) - [Accessibility](https://info.keylimeinteractive.com/topic/accessibility) - [Behavioral Personas](https://info.keylimeinteractive.com/topic/behavioral-personas) - [Diary Study & Ethnography](https://info.keylimeinteractive.com/topic/diary-study-ethnography) - [Global Research](https://info.keylimeinteractive.com/topic/global-research) - [Inclusivity Index](https://info.keylimeinteractive.com/topic/inclusivity-index) - [User Testing](https://info.keylimeinteractive.com/topic/user-testing) - [Card Sort & Tree Test](https://info.keylimeinteractive.com/topic/card-sort-tree-test) - [Human-first Approach](https://info.keylimeinteractive.com/topic/human-first-approach) - [Personalization](https://info.keylimeinteractive.com/topic/personalization) - [CX](https://info.keylimeinteractive.com/topic/cx) - [Diary Study](https://info.keylimeinteractive.com/topic/diary-study) - [In-Depth Interviews](https://info.keylimeinteractive.com/topic/in-depth-interviews) - [Inclusivity](https://info.keylimeinteractive.com/topic/inclusivity) - [A/B Testing](https://info.keylimeinteractive.com/topic/a-b-testing) - [CX Research](https://info.keylimeinteractive.com/topic/cx-research) - [IDIs](https://info.keylimeinteractive.com/topic/idis) - [Bias](https://info.keylimeinteractive.com/topic/bias) - [Personas](https://info.keylimeinteractive.com/topic/personas) - [Research Participant Recruitment](https://info.keylimeinteractive.com/topic/research-participant-recruitment) - [Usability](https://info.keylimeinteractive.com/topic/usability) - [Women in UX](https://info.keylimeinteractive.com/topic/women-in-ux) - [Workshops](https://info.keylimeinteractive.com/topic/workshops) - [Customer Journey Mapping](https://info.keylimeinteractive.com/topic/customer-journey-mapping) - [Remote Research](https://info.keylimeinteractive.com/topic/remote-research) - [Research Analysis](https://info.keylimeinteractive.com/topic/research-analysis) - [UX Methods](https://info.keylimeinteractive.com/topic/ux-methods) - [Webinars](https://info.keylimeinteractive.com/topic/webinars) - [Anthropology](https://info.keylimeinteractive.com/topic/anthropology) - [Competitive Benchmarking](https://info.keylimeinteractive.com/topic/competitive-benchmarking) - [Competitive Insights](https://info.keylimeinteractive.com/topic/competitive-insights) - [Competitive Intelligence](https://info.keylimeinteractive.com/topic/competitive-intelligence) - [Confirmation Bias](https://info.keylimeinteractive.com/topic/confirmation-bias) - [Ethnography](https://info.keylimeinteractive.com/topic/ethnography) - [Gaming](https://info.keylimeinteractive.com/topic/gaming) - [Healthcare CX Design](https://info.keylimeinteractive.com/topic/healthcare-cx-design) - [Healthcare Customer Experience Design](https://info.keylimeinteractive.com/topic/healthcare-customer-experience-design) - [Heuristics](https://info.keylimeinteractive.com/topic/heuristics) - [Unmoderated](https://info.keylimeinteractive.com/topic/unmoderated) - [sample size calculation](https://info.keylimeinteractive.com/topic/sample-size-calculation) - [AI development](https://info.keylimeinteractive.com/topic/ai-development) - [Biometrics](https://info.keylimeinteractive.com/topic/biometrics) - [Communication](https://info.keylimeinteractive.com/topic/communication) - [Competitive Intelligence Analysis](https://info.keylimeinteractive.com/topic/competitive-intelligence-analysis) - [Concept Testing](https://info.keylimeinteractive.com/topic/concept-testing) - [Consultant](https://info.keylimeinteractive.com/topic/consultant) - [Custom Projects](https://info.keylimeinteractive.com/topic/custom-projects) - [Dedicated Services](https://info.keylimeinteractive.com/topic/dedicated-services) - [Digital Divide](https://info.keylimeinteractive.com/topic/digital-divide) - [Impact](https://info.keylimeinteractive.com/topic/impact) - [Keystroke Level Model](https://info.keylimeinteractive.com/topic/keystroke-level-model) - [Literature Review](https://info.keylimeinteractive.com/topic/literature-review) - [Negotiation in UX](https://info.keylimeinteractive.com/topic/negotiation-in-ux) - [Nonprofit UX](https://info.keylimeinteractive.com/topic/nonprofit-ux) - [Presentation](https://info.keylimeinteractive.com/topic/presentation) - [Rapid Research](https://info.keylimeinteractive.com/topic/rapid-research) - [Report](https://info.keylimeinteractive.com/topic/report) - [SaaS](https://info.keylimeinteractive.com/topic/saas) - [UX Templates](https://info.keylimeinteractive.com/topic/ux-templates) - [World Usability Day](https://info.keylimeinteractive.com/topic/world-usability-day) - [digital design](https://info.keylimeinteractive.com/topic/digital-design) - [persona](https://info.keylimeinteractive.com/topic/persona) - [user engagement](https://info.keylimeinteractive.com/topic/user-engagement) ![ ](https://keylimeinteractive.com/wp-content/uploads/2024/05/aicpa-soc-certified.png) ![Logo of the National Minority Supplier Development Council with colorful abstract shapes and text.](https://keylimeinteractive.com/wp-content/uploads/2024/05/NMSDC-Logo-NATIONAL-1024x716.jpg) ![Logo with the text "Insights Association" and colorful squares forming a pattern above the text.](https://keylimeinteractive.com/wp-content/uploads/2024/02/insights-association-logo-v2.png) ![](https://keylimeinteractive.com/wp-content/uploads/2024/02/key-lime-interactive.svg) Your Trusted Partner for UX Research Key Lime Interactive - [What We Do](https://keylimeinteractive.com/ux-research-services/) - [Proven Process](https://keylimeinteractive.com/proven-process/) Resources - [Case Studies](https://info.keylimeinteractive.com/case-studies) - [Events + Workshops](https://info.keylimeinteractive.com/events) - [Blog](https://info.keylimeinteractive.com/) Company - [About Us](https://keylimeinteractive.com/about-us/) - [Meet Our Team](https://keylimeinteractive.com/our-team/) - [Careers](https://keylimeinteractive.com/careers/) - [News](https://keylimeinteractive.com/news/) - [Security & Compliance](https://app.vanta.com/keylimeinteractive.com/trust/cylnxu23wx6pzvej7bloh9) - [Public Sector GSA](https://keylimeinteractive.com/public-sector-gsa-schedule/) Get in touch - [Contact Us](https://keylimeinteractive.com/contact) © 2025 All Rights Reserved. - [Privacy Policy](https://keylimeinteractive.com/privacy-policy/) - [Terms of Use](https://keylimeinteractive.com/terms/)