The hypothesis being tested by Sounds Good is:
• Digital audio can be used to save assessors time AND give students better feedback.
Most of the data is in now, so do we have a verdict? Cautiously, yes. It’s hedged around with ‘health warnings’ and the detail may change, but this is the picture at present.
On the Sounds Good project sixteen Leeds Met staff have been giving audio feedback to students on some of their coursework. The circumstances have varied widely. One of the differences has been the number of students to whom each member of the staff team has given audio feedback – for some it has been dozens, for others the total is in single figures.
So, bearing in mind the widely differing experiences, has using audio instead of their usual methods saved assessors time? Of the twelve who have commented on this issue so far, six reported that giving digital audio feedback took more time than their usual methods, four thought that it took about the same time and two reported that it took less time.
On the face of it, the fact that only a small minority of staff reported saving time might be regarded as disappointing and a negation of the hypothesis. However, most expressed satisfaction with their efforts and noted that students received more, and higher-quality, feedback than they otherwise would have done. Some team members acknowledged that they got quicker as they became familiar with the technology and techniques of audio feedback. Some who found audio feedback took them more time only used it with quite a small number of students so, arguably, they have not yet achieved full familiarity with it.
The module leader who probably spent the most time giving and sending audio feedback to students was the leader of a small team of assessors. He said that whilst it was “incredibly time-consuming” he had “got the brief wrong” and found himself doing far more on behalf of his colleagues than had been intended. Another who reported spending more time on audio feedback even after becoming used to the tools and techniques noted two factors which are probably significant: a) her feedback was very brief; b) she was quick at typing and handwriting.
Both of the assessors who did report saving time by using digital audio feedback said that they managed to do so without compromising the quality of the feedback they provided. Indeed one (who was giving formative feedback to dissertation students) said, “I spent a little less time but gave much more detailed feedback and expanded on explanations.” It is worth noting that of all the members of the Sounds Good team, these two assessors are probably the most experienced users of technology.
So the indications are that it is possible, in some circumstances, to use digital audio to save time and not compromise on the amount and quality of feedback to students. The most favourable circumstances would appear to be:
• The assessor is comfortable with the technology.
• The assessor writes slowly but manages to record his/her speech relatively quickly.
• A substantial amount of feedback is given.
We might expect that some people will take longer than others to become familiar with the technology and the techniques of recording their comments. Perhaps more of the Sounds Good team would have managed to save time without compromising quantity and quality if they had had more practice with giving feedback via digital audio. They’ll get their chance to do that next academic year in ‘Sounds Good 2’, an extension to the project which JISC has recently decided to fund. More on that later.
(Posted by Bob Rotheram, Project Manager, Sounds Good)