Bringing together further and higher education, the University of the Highlands and Islands is one of a new style of integrated regional universities emerging across Europe. A partnership of 13 colleges and research institutions, it serves over 40,000 students across more than 70 learning centres.
The problem: The three annual surveys produced a vast quantity of information – which could be immensely valuable to faculty and staff across the institution – if they could easily access and interpret it.
The solution: Student data project setting up data collation processes and customisable reporting, plus Natural Language Processing meant that rather than days spent collating and analysing results, full results were available within hours.
The importance of student feedback in decision-making
The University of the Highlands and Islands is unique in Scotland with its offer of a blend of college and university level study. Serving more than 40,000 students on campus and online, the university recognises that student feedback is an important indicator of quality in education and is increasingly used in evaluation and decision-making. In addition to small-scale feedback mechanisms at individual module level, and legacy mechanisms from the organisations, which came together to form the institution, the university gathers vital student feedback from three major annual surveys.
Two of these three major surveys are commissioned on behalf of the higher and further education funding bodies: the National Student Survey (NSS), which gathers feedback from final year students; and the Student Satisfaction and Engagement Survey (SSES), which seeks the same information from students at other points in their course of study.
The third major annual survey undertaken among students is the Early Student Experience Survey (ESES), an internal study which examines students’ experience of applications, inductions and their introduction to study.
The data challenge: vast quantity of information that wasn’t easy to access and interpret
With so much student feedback available, the challenge was less in gathering data and more in ensuring it could be used to its full potential, explains Dr Heather Fotheringham, the university’s Evidence Based Enhancement Lead.
Not only are these surveys important in maintaining our standing as a degree-awarding institution, but we also use this information in our annual quality monitoring at module, programme and faculty level. The responses we receive help us plan for changes to our curriculum and how it’s delivered, to reflect on past academic years and to plan for the future.
The three annual surveys produced a vast quantity of information – which could be immensely valuable to faculty and staff across the institution – if they could easily access and interpret it. Dr Fotheringham and her team would spend weeks analysing the results of each survey, extracting and manipulating data to provide tailored insights to relevant departments. The team were keen to investigate a data based solution that would reduce the amount of time they spent manually analysing and processing results, while maintaining ease of access and understanding for colleagues.
Natural Language Processing vastly improved a previously time consuming process
When a university committee highlighted the opportunity to become a host organisation in The Data Lab’s MSc Placement Programme, Dr Fotheringham prepared a brief overview of the desired results of any project. Covering data collation and customisable reporting, the business intelligence elements were familiar to MSc student Karen Jewell, who was returning to study after a decade of workplace experience in business and data analysis roles. “I knew I’d need more data science elements in order to use this project for my MSc,” explains Karen, “so Heather and I discussed what data was available, what business or operational problems it might be able to help solve, and how we could work to include that in the project.”
Together, they identified the potential to improve the processing of qualitative data through Natural Language Processing (NLP). Each of the three major surveys offered the opportunity for students to enter free text responses. While these responses often contained valuable insights, processing them required significant time from the team. Each comment would be reviewed manually, coded to identify which topic or topics were being referenced, and analysed to reveal the sentiments being conveyed on each topic.
This process was quite time consuming, as the team had to trawl through each and every response. Using NLP, we were able to build a solution, which automatically coded comments against topics and correctly identified the sentiment of what was being said. This included making allowance for counter-intuitive sentiment tagging, for example when the answer is ‘nothing’, but the questions is ‘what could we improve?’, then that’s a positive sentiment! Filtering out these basic responses also meant that the team could spend more time focusing on analysis of the more substantive comments.
MSc student Karen Jewell
With the University of the Highlands and Islands being spread over such a large geographical area, the team were well set up for remote collaboration, so when the covid-19 pandemic put paid to plans to have Karen join the team on-site for part of her placement, the project was unaffected.
Rather than having to spend days collating and analysing results, full results were available within hours
The team were able to apply Karen’s model to a fresh dataset for the first time when the annual NSS results were released towards the end of the placement. Karen explains:
Previously, Heather’s team would release headline data on the day NSS results were published. They would then have to spend weeks preparing bespoke analyses for various different departments and individuals. With our new model, full results were available to everyone within hours on the day of publication.
While Karen used specialist data programmes such as Microsoft Power BI to design her solutions, the model was also made available in Microsoft Excel, ensuring that colleagues across the university could access it.
“Our goal was to have data that was more usable and more used,” explains Dr Fotheringham, “and to reduce the time we spent processing. With the introduction of NLP and accessible tools, Karen’s project ticked both boxes for us.” In future years, the keyword trend analysis incorporated within the NLP element will identify hidden patterns and themes in qualitative answers which may have gone unnoticed in human analysis, making the data ever more valuable as more results from all three major surveys are added to the dataset.
With more powerful data available, Dr Fotheringham is looking forward to expanding the role of student feedback in decision making and ongoing quality monitoring, as well as the opportunity to undertake deeper analyses having freed up so much time previously spent processing.
Karen, too, has benefited from the project:
I went back to uni to look at how data can be used to make better decisions and solve practical problems, rather than just providing historical benchmarks, and this project demonstrates that. Moving into a completely new type of organisation meant I also gained confidence in my own ability to adapt in new environments.
With support from the Data Lab, both student and host had a positive experience with the MSc Placement Programme.
It was surprisingly easy to set up. I’d expected a lot of bureaucracy but it all went through very quickly. I would recommend to other organisations that they consider hosting an MSc student – there is power in data and this is a chance to harness it.