Standardised scales are a commonly used tool in quantitative research. CLOSER’s Metadata Officer, Becky Oldroyd, highlights the benefits and challenges of using these scales, along with CLOSER’s plan to create and disseminate standardised scale metadata on the UK’s most detailed search engine for longitudinal population studies: CLOSER Discovery.
CLOSER Discovery
CLOSER Discovery, our flagship resource, was developed to meet the complex needs of modern research where depositing data in an archive is simply no longer enough. It allows researchers to search, explore, and assess questionnaire and dataset metadata from 11 of the UK’s leading longitudinal population studies. Users can filter by study, life stage, and topic to discover the dataset that fulfils their research requirements before investing time in accessing the data.
A research tool like CLOSER Discovery is a living thing – always growing, improving, and taking in more information as new data are collected, or as historic records are made available. Our new project will add standardised scale metadata to CLOSER Discovery to help address some of the challenges that researchers face and support world-class research.
What are standardised scales?
Standardised scales are a group of related questions designed to measure an underlying concept (e.g., depression, parenting quality, smoking behaviour). These scales are standardised, validated, and re-used by the research community. Standardised scales are often interchangeably referred to as summated scales or validated instruments, but should be distinguished from questionnaires or surveys which cover a range of topics rather than one underlying concept. You might find standardised scales within questionnaires or surveys (e.g., you might find a standardised scale which measures anxiety within a questionnaire or survey that covers multiple topics).
Unlike questionnaires or surveys, standardised scales are developed using statistical analyses (e.g., factor analysis, Item Response Theory, Cronbach’s alpha) to ensure that the questions are related and collectively measure the desired underlying concept. Standardised scales can also be piloted with a sample that represents the intended audience to check that the questions are understandable and to identify problems. For example, all respondents might provide extreme responses to a question (i.e., the lowest or highest response option), which could indicate unusual question wording.
The benefits of standardised scales
Standardised scales are commonly used in quantitative research because they are cost-effective and easy to implement. They also allow researchers to gather data on large groups of individuals in a timely manner, in comparison to other methods of data collection such as interviews or observations.
Due to the standardisation process involved in creating these scales, standardised scales provide researchers with measurement tools that are both valid and reliable. This means that the scale measures what it sets out to measure (validity), and it can consistently produce the same results over time and across samples (reliability).
When using a standardised scale, it is important to include the full set of questions; adding or removing questions jeopardises the reliability and validity of the scale because the questions have been designed to collectively measure the underlying concept. Some scales have short forms (i.e., a shortened version of the questionnaire which includes a subset of questions) which can be a useful alternative if the respondents have a limited amount of time to answer the questions, or, when included as part of a questionnaire or survey, if you have a limited amount of space. It is also important to consider the sample you are investigating, because standardised scales are developed in a specific time, country, and culture and so may not be valid to use across all populations.
The challenges of using standardised scales in research
Despite the many benefits, there are several challenges of using standardised scales. Metadata for standardised scales (e.g., title, author, questions) can be difficult to obtain as information is often scattered across multiple sources, and permission is sometimes required to access the scale. Some scales have multiple versions (e.g., short forms), and information about the original version of the scale may be hard to locate.
It can also be challenging to find information about where standardised scales have been used in other research studies, and even more so to determine what questions within a study’s questionnaire are from a standardised scale. This is because studies sometimes do not document when they have used a standardised scale within their questionnaires, and if they do, this information is not always publicly available. Even when this information is publicly available, it would be incredibly time-consuming for researchers to gather this information from individual studies.
CLOSER’s mission to scale up Discovery
To address these challenges, CLOSER aims to make this information more easily accessible to researchers by adding standardised scale metadata to the UK’s most detailed search engine for longitudinal population studies: CLOSER Discovery.
The first step in this process includes gathering metadata for several of the standardised scales used by the CLOSER studies and displaying these on the CLOSER Technical Wiki. The final stage of the project is to add the metadata to CLOSER Discovery. This will enable researchers to identify whether questions from the studies’ questionnaires belong to a standardised scale and see where the standardised scales are used across studies and waves of data collection, thus increasing the visibility and use of standardised scales.
Further information
This is the first in a four part CLOSER blog series exploring standardised scales. Read parts 2 and 3 below:
- Related blog: Part 2: In search of standardised scales
- Related blog: Part 3 – Empowering social and biomedical researchers with a new tool
- Related: CLOSER Discovery
- Related: CLOSER Technical Wiki
Becky Oldroyd is a Metadata Officer for CLOSER. Follow Becky on Twitter: @Becky_Oldroyd
Suggested citation:
Oldroyd, B. (2022). ‘Scaling up CLOSER Discovery’. CLOSER. 26 September 2022. Available at: https://closer.ac.uk/news-opinion/blog/scaling-up-closer-discovery/