Research Rounds #3 and Data Governance Engagement Sessions Video Series Premiere!

7/15/2016

FNHA-Regional-Data-Governance-Engagement-Sessions.JPG

We are pleased to share the third installment of FNHA Research Rounds with this post on the subjects of Precision and Accuracy. 

As well, we are premiering the first of a handful of videos documenting the FNHA Data Governance Engagement Sessions that took place over the last six months.

Watch the first Data Governance Engagement Session video here! 

 


Precision and Accuracy

By Anya Smith, PhD

Statistical Coordinator – Regional Surveys

Before introducing Precision and Accuracy, it will be helpful to understand the concept of 'sampling'. Ideally, researchers could collect health information from everyone and calculate percentages and averages based on the entire population. However, this is often not possible due to time and funding constraints, and so instead, a selection of individuals is 'sampled' to represent the entire population. Of course, the estimates generated from a sample will not reflect the population exactly - there is an inherent level of error involved in using a sample of people to represent everyone. The amount of error can vary widely due to factors like sample size and demographic representation, which are reflected in the concepts of precision and accuracy.

Precision

Precision relates to how close the values of a measurement would be to each other if random samples were repeatedly generated from the same population. The goal is to maximize the precision of an estimate by producing values that are as close as possible to one another. Precision is affected by sample size: the larger the sample size, the higher the precision. Some readers may have heard the term "confidence interval", which is a way to express the error inherent in estimates derived from a sample. The aim for researchers is to make this interval as narrow as possible and therefore maximize confidence and precision. For example, imagine it is reported that 20% of women have been pregnant at some point in their lives. If the sample size is large, the estimate might look something like: 20% (95% confidence interval: 18-22%). In this case, we can be 95% confident that the true population value falls within the range 18-22%, which indicates good precision. On the other hand, if our sample is small, the estimate might look something like: 20% (95% confidence interval: 2-38%). The 20% estimate, in this case, is not very useful because the true population value could lie anywhere between 2% and 38%. The RHS and REEES use large sample sizes at the provincial level to maximize precision for as many indicators as possible. Results from the RHS and REEES will include confidence intervals together with estimates so the reader is aware of the precision, and therefore reliability, of each and every estimate.

Accuracy

Where precision relates to the repeatability of a measurement, accuracy is related to how representative the sample is of the population. The closer the estimate is to the true population value, the more accurate it is. One way to improve accuracy is to ensure that major demographic groups, such as age and gender, are included in the sample in a way that reflects the population age and gender distributions as closely as possible. If this is not accounted for, the estimates may deviate widely from the true population value. The sampling strategy for the REEES and RHS was designed to account for population age and gender distributions to increase accuracy of estimates.

Precision versus Accuracy

Precision and accuracy, although both important, are independent of one another. It is possible to achieve high precision and low accuracy, for example. Consider a population with 50% males and 50% females that it is sampled and surveyed. Imagine this sample is large (5000 people), but a flawed sampling design resulted in 4500 females to 500 males surveyed. Although the precision might be high due to the large sample size, the accuracy of the estimates would be low if the results claimed to represent both males and females. The diagram below illustrates how the levels of precision and accuracy can differ using a classic dartboard analogy.

Accuracy-Image.gif

Image source: http://www.yorku.ca/psycho/en/postscript.asp

 

Questions about research? Please email RHS@fnha.ca or FNREEES@fnha.ca