Why IR Matters More Than You Think
When conducting market research, the concept of incidence rate (IR) seems simple: it’s the percentage of respondents within a specific target population who meet the criteria for participation in a study. However, understanding IR is more than just applying a formula—it’s about ensuring your targeting strategy aligns with your research goals. Misunderstood IRs can drive up costs, reduce data quality, and compromise research outcomes, making it essential for clients to grasp what truly affects their project outcomes.
Navigating IR Complexities: Four Approaches for Smarter Sampling
Simplified IR: The most straightforward way to calculate IR is by using this basic formula:
Completes / (Completes + Terminates)
This method works well in theory. For example, if your study is looking for parents of children, and the general population data shows that 40% of adults are parents, your IR should be around 40%. However, in modern sampling, panels can directly target parents, allowing you to reach this audience more effectively. This targeted approach effectively increases your IR closer to 100% for this specific group since you’re contacting only the relevant demographic. However, this simple approach doesn’t account for the impact of mis-targeting, un-targetable quotas, or quality losses.
Conversion Rate: The panel companies’ perspective is often to focus on conversion rates, calculated as:
Completes / Starts
While efficient from a panel management perspective, this formula overlooks essential factors like poor project management, quality losses, or incorrect targeting efforts. Many panels use this metric because their internal systems are designed around yield management. Even if they don’t charge you directly for a lower conversion rate, these systems often deprioritize studies with lower conversions. This means programs with lower conversion rates are less likely to receive high-priority sample allocations, causing slower fills and extended field times. As a result, vendors may request higher cost per interview (CPI) rates to boost sampling priority, which can increase your overall project costs. Although this approach doesn’t affect the true IR calculation, it fails to account for how improved targeting and higher-quality sample provision could enhance conversion rates. If left unchecked, poor targeting or low-quality respondents can drive down conversion rates, leading to higher costs and reduced data quality for clients—often without them realizing the root cause.
Closer to the True IR: Adjusting for targeting errors is a more accurate approach which refines the formula:
Completes / (Starts – Partials – Targetable Terminates – Targetable Overquotas)
This method protects clients from being penalized for supplier targeting errors or incomplete responses (partials). For instance, if your study focuses on tea drinkers, panel companies may be able to directly target tea drinkers but may not accurately target specific brands or preferences (e.g., decaffeinated or Twining’s drinkers). By filtering out non-targetable terms, this calculation reflects the real effectiveness of your targeting strategy. Additionally, the impact of un-targetable quotas can significantly influence IR. For instance, setting quotas around specific brands of tea—where brand usage is not directly targetable—can drive down IR due to the oversampling required to fill those quotas. It’s important to understand which quotas can be controlled through targeting and which cannot. This understanding is critical when scoping projects, as it ensures an awareness of the cost implications of quota and design choices.
The True Incidence Rate: Accounting for quality control, the most comprehensive method accounts for poor-quality respondents and targeting issues by using this formula:
Completes / (Starts – Partials – Targetable Terminates – Targetable Overquotas – Quality Control Terminates)
This method incorporates quality control by filtering out respondents flagged for speeding, straight-lining, or other low-quality behaviors. In some studies, a significant portion of respondents can be removed for poor quality—without proper adjustments, these low-quality responses can distort your IR and inflate costs. Additionally, the structure of terminations and quotas within the program can further complicate accurate IR calculations. For example, having terminations and quota checks occur during initial screening may distort the actual IR for a specific audience. If a non-targetable termination occurs before a demographic check, a panel sending excessive sample to a closed demographic group can also negatively impact the apparent IR for the overall audience. We help clients structure their checks and quotas in a way that informs the most accurate IR calculations possible, ensuring better targeting and cost efficiency.
How Misunderstood IR Impacts Data Quality and Costs
Misunderstanding or misusing incidence rates can have significant effects on both the quality of your data and the overall cost of your research. Inaccurate IR calculations can lead to unnecessary oversampling, misallocated resources, and higher expenses, all while compromising the reliability of your findings.
- Poor Targeting Leads to Skewed Samples and Higher Costs
If IR is miscalculated, targeting criteria may be improperly applied, resulting in irrelevant or unqualified respondents entering the study. This skews the sample, compromises data integrity, and requires additional sampling to meet quotas—ultimately driving up costs. - Compromised Representativeness
A low IR that doesn’t adjust for targeting errors or poor-quality respondents can lead to over-sampling of certain groups. This reduces the representativeness of your data and distorts the final analysis. - Increased Risk of Poor-Quality Responses
Artificially low IRs can prompt panels to broaden their sampling, increasing the chance of capturing disengaged or low-quality participants who reduce the reliability of your results. - Higher Costs Without Improved Data Quality
Clients may overspend on broader audiences with little relevance, increasing costs without enhancing data quality. Additionally, poor IR calculations can inflate project budgets due to longer field times, higher cost per interview (CPI) rates, and unnecessary oversampling. Accurate IR calculations help ensure every respondent contributes value while optimizing your research budget.
Ensuring Accurate IR: Best Practices
- Refine Your Targeting Strategy: Start with clear, specific targeting criteria and adjust as needed, but don’t compromise your IR by expanding to a general population prematurely.
- Incorporate Quality Control Measures: Use redirects and other tools to filter low-quality respondents early in the process to maintain data integrity.
- Evaluate Panel Performance: Not all panels are created equal—track panel partners’ ability to meet targeting specifications and deliver high-quality respondents consistently.
Maximize Your Research Value with Accurate IR Calculations
Understanding and accurately calculating incidence rates isn’t just a technical exercise—it’s the foundation for meaningful insights, better targeting, and cost-effective research outcomes. Missteps in IR calculations can silently inflate costs, skew results, and dilute the effectiveness of your research.
At Research Results, we specialize in helping clients navigate these complexities by improving targeting strategies, optimizing sample quality, and ensuring every respondent adds value to your research goals. Don’t let misunderstood metrics undermine your project’s success.
Let’s connect to discuss how we can refine your incidence rate strategy and drive better outcomes for your next study. Contact Ellen Pieper, Chief Client Officer, Ellen_Pieper@researchresults.com, or 919-368-5819 today.