In February 2020, CLOSER and the UK Data Service (UKDS), ran an online workshop series designed to help students and newcomers to longitudinal research get to grips with analysing real-world study data.
The course was developed by CLOSER Research Fellows Neil Kaye and Dara O’Neill, with Valerija Kolbas and Deborah Wiltshire, Senior User Support and Training Officers from the UKDS.
Within a few weeks, as a result of the Covid-19 public health emergency, all university campuses in the UK went into lockdown and all education efforts went online. The chance synchronicity of these events provides us with reason to reflect on the opportunities and obstacles of the delivery of online learning using our recent experience, and how these experiences can inform our training efforts going forward.
Background: A workshop to deal with ‘real-world’ data
Following a CLOSER review of quantitative analytical training needs for users of longitudinal studies undertaken in summer/autumn 2019, a gap was identified for training users in how to access, clean and handle actual study datasets, like those made available to researchers and students by the CLOSER partner studies.
Whilst many institutions provide introductory courses on data analysis at undergraduate level, few offer modules specifically in relation to longitudinal data that are aimed at non-statisticians. Moreover, it is common for introductory statistics courses to use simulated datasets that have been ‘pre-cleaned’ to provide streamlined and more simplistic examples.
Real-world study data, by contrast, commonly pose additional, sometimes unexpected, challenges, such as missing data or the need to link together data from different waves of a study before they can be used in answering a longitudinal research question. Providing hands-on training that uses actual study data can therefore better equip students for finding and using relevant real-wold data in their research, particularly in advance of their final year undergraduate projects.
Our review of training needs clearly highlighted that there was a demand for more guidance on how to deal with the ‘messiness’ of actual datasets to give new users a realistic view of data structures and the processes involved in cleaning and preparing datasets for analysis. By joining forces with UKDS, we were able to address this training gap, bringing together two training providers with established expertise in and access to a huge number of ‘real-world’ datasets.
The workshop series we subsequently developed comprised both live teaching sessions and resources for self-paced learning. At the heart of the series were two 90-minute interactive webinars, providing guided examples, interactive quizzes and a live Q&A. These sessions were run on alternate weeks, with students given additional hands-on analysis tasks and supplementary reading suggestions to complete between and after the sessions. The focus was on increasing participants’ awareness of how to access datasets, how to manage data obtained from existing studies, and how to begin to explore, analyse and interpret longitudinal study data.
What we learnt through training others: experiences and wider lessons for online teaching
Delivering teaching online
In comparison to in-person training, online teaching inevitably creates an element of distance between teacher and student, making it harder to promote interaction and assess engagement. Opportunities to evaluate learning and facilitate questions are important to the learning process. However, virtual learning environments are now increasingly better equipped to help facilitate such engagement efforts and monitoring.
Our workshop series was designed to be as interactive as possible in order to maximise learner engagement. It was important to enable participants to ask questions, give feedback and share ideas. We embedded multiple assessments into each webinar and provided homework tasks to help continued learning between and after the live sessions. Participants showed a keenness to utilise these engagement opportunities. Our course site also had a discussion forum which was not used by participants, suggesting that students preferred being able to ask questions directly to the course team rather than in a more open forum. None of these technical solutions can as yet adequately replace the ease of in-person communication, but some solutions may be more effective than others.
Global reach
Online courses by contrast to in-person training are not constrained by situational factors, and therefore enable us to have a wider reach than more traditional styles of workshop delivery. This was the case for our course. The 175 participants that took part comprised a wide geographical spread, with representation from every continent bar Antarctica!
Map shows geographical spread of participants.
This nonetheless poses challenges simultaneously around language and cultural factors that necessitate consideration in the design and delivery of training, as well as logistical concerns related to having course participants in different time zones.
Our course was also free to access, and was not restricted to anyone from particular disciplinary backgrounds or experience. As captured by our pre-course questionnaire, the participation was consequently quite diverse:
Our course was aimed at Stata users specifically, which is one of the most commonly-used software packages amongst social and biomedical science university courses. However, as proprietary software, this may have limited the learning benefits for some users who did not have access to this particular analysis tool. Using open source software (e.g. R or Python) in the delivery of such training may help widen access and usability of courses like this.
Feedback and future delivery
Feedback from the participants in this course was very positive with 80% of respondents rating the course ‘excellent’ or ‘good’ overall. A majority of participants said that the workshop had increased their knowledge of statistics and improved their confidence in carrying out statistical analysis. There was consensus that the level of interactivity was very good and participant comments included praise for the quizzes, learning assignments and how clearly information was presented in the webinars.
However, there is clearly room for improvement and expansion. Participant feedback following the course underlined the participants’ strong motivation to further develop their knowledge and skills on using and analysing longitudinal data, with many stating they would be ‘quite likely’ or ‘very likely’ to attend further sessions covering more advanced techniques. At the same there is a need to ensure that the content we deliver is even more accessible and diverse. A benefit of online course delivery is that the training materials and format is adaptable and re-usable. Ultimately, a multi-modal, multi-platform delivery format will maximise benefits to the wider community of nascent longitudinal researchers.
As uncertainty continues around campus access and the nature of university education over the next academic year, online teaching will inevitably play a central role in helping new researchers to access training and continue their professional development. As educators, there is an onus on us to continue our own professional development, to explore and implement effective new ways of enhancing our online training efforts, facilitating student engagement, and providing a supportive and stimulating arena for learning. The added benefits of such efforts lie in their ability to contribute to a wider, more inclusive form of education, and in collectively building and sharing best practice on the use of longitudinal data.
Building on our success
CLOSER plans to build upon the success of this workshop in collaboration with UKDS through future training efforts, expanding our teaching resources on real-world datasets to address more diverse analysis methods and data management considerations. In so doing, we will also seek to further explore, evaluate and expand our use of innovative methods for content delivery and improving the accessibility and utility of the content amongst wider range of participants.
Content from the workshop is already being incorporated into the CLOSER Learning Hub. Our Learning Hub is an important resource for undergraduates and newcomers to longitudinal research, providing online support and guidance for those will little or no prior knowledge or experience in longitudinal study design, analysis and dissemination.
The Learning Hub is the central resource for CLOSER’s commitment to delivering online training in an engaging, accessible format. Clearly defined modules guide learners through the key processes involved in understanding how and why to undertake effective longitudinal research, and example research case studies covering a broad range of social science and biomedical disciplines, including education, social mobility, employment, mental health, obesity, health behaviours and child development help to build further understanding. The Learning Hub also contains teacher-focused resources, including the CLOSER teaching dataset, accompanying exercises and downloadable resources.
Acknowledgements
We would like to acknowledge and thank our collaborators at UKDS: Valerija Kolbas (University of Essex, UK), Deborah Wiltshire (University of Essex, UK) and Patty Doran (University of Manchester, UK).
Further information
- Explore the CLOSER Learning Hub
- Sign up to CLOSER’s email newsletters for the latest longitudinal news and future training events.
Neil Kaye is a Research Fellow at CLOSER. Follow Neil on Twitter: @NeilKaye_phd
Dara O’Neill is a Senior Research Fellow at CLOSER.
Suggested citation:
Kaye, N. & O’Neill, D. (2020). ‘Delivering training in a rapidly changing educational landscape: Experiences from a CLOSER/UKDS course on real-world data’. CLOSER. 16 June 2020. Available at: https://www.closer.ac.uk/news-opinion/blog/delivering-training-rapidly-changing-educational-landscape-experiences-joint/