Why do we keep a Sounds Good blog? Because we have to, that’s the simple answer. For projects in JISC’s Users and Innovation (U&I) programme, maintaining a ‘reflective blog’ is part of the deal. Would Sounds Good do it if it weren’t expected? I’m not sure.

The U&I programme manager hasn’t been prescriptive about project blogs, thank goodness. So there have been no diktats about the frequency, length or nature of postings. I’ve felt our project team has had the freedom to post (or not) what it wanted, provided it put something into the public domain now and again. Fine, but from my vantage point as project manager, the main problem has been what to publish. Do I encourage the team each to upload a daily bare-your-soul, ‘Dear diary’ stream of consciousness? Hmm, complete frankness might cause a few difficulties! Much easier, and safer, would be an occasional, short, highly-sanitised communiqué. But what a yawn that would be.

Not wanting to make enemies, create hostages to fortune, or send readers to sleep, I’ve gone for an approach somewhere between the extremes. I arrogated the role of editor and encouraged all 17 members of the Sounds Good team to add to the blog, via me. My intention with others’ contributions has been to be ‘light touch’. With my own, I’ve tried to publish something every couple of weeks on average. Not just anything, mind. The aim has been to make the postings worth reading, with a discernible element of reflection, not a whitewash but not warts-and-all either.

What’s happened? The bald facts are that, in the five months it has been running, the blog has 19 postings, not including this one. I have written 16 of them. These items have prompted 10 comments, two of which have been from me. The blog is also syndicated to the JISC ‘Emerge’ community at http://elgg.jiscemerge.org.uk/leedsmetbob/weblog, where it has produced 11 further comments, including two from me. Most of my blog postings on the Sounds Good site have a parallel existence as podcasts. None of these has prompted any comments.

So it’s fair to say that most of the input has been from me. As for the response, there hasn’t been much, has there? I have no way of knowing how many people have been reading the blog or listening to the podcasts. I do know that the counter on the Sounds Good home page is showing 1043 hits at the time of writing but, if memory serves, 987 of them are down to me and my mother. [Joke] All in all, I’m very much the main contributor and there isn’t much evidence that we/I have been engaging with the public.

So is it worth it? Speaking personally and (for once) completely frankly, I’ve been quite enjoying writing, for several reasons. First, it has allowed me to do a bit of self-indulgent vanity publishing, something for which I criticise other bloggers but, hey, I’m a hypocrite! Second, the requirement to be reflective has made me think about various aspects of the project, which is no bad thing. Third, as it has turned out, I haven’t had to trouble my conscience by using buckets of whitewash to tell a positive story; the project really has been going pretty well.

On the other hand, who cares? Part of me suspects the readership is about the same as that of a typical academic paper: a dozen people and a dog or two. And what if the project hadn’t gone well? In particular, what if the ‘blogger-conscript’ truly had been a conscript, with all that usually implies – sullen, foot-dragging, minimal compliance with a bullying sergeant-major? What if the project had been struggling, or worse? What would have appeared in the blogosphere then?

JISC asked for a reflective blog. Why? What did they expect? Would they do it again? I think we should be told.

(Posted by Bob Rotheram, Project Manager, Sounds Good)

Focus groups are popular with researchers. Everyone is holding them! Personally, having been in a few myself, I’ve got reservations – mainly about what can happen when the group contains a dominant, highly-opinionated and apparently knowledgeable individual. Yet I could see a potential benefit of a student focus group on Sounds Good: a way of getting beyond the routine and rather superficial questionnaires we were issuing, asking students about their experience of receiving audio feedback on their coursework. It seemed worth a try, so I built it into the project design.

I consulted the staff team who were actually giving students the audio feedback. When would be a convenient time to hold a focus group? How should it be publicised? Inevitably the answers varied, but I went for a compromise on the timing: 4pm on a Monday in a week when classes were winding down but exams had not started. The publicity – clear and friendly, I thought, with assurances about confidentiality – was issued several weeks in advance via the staff team, inviting interested students to get in touch with me. The incentives – about which, looking back, I didn’t consult – were a £30 book token and light refreshment, for an hour and a half of their time. That would do it, surely. I would be spoilt for choice!

Er, no. Of the 400+ students who received audio feedback about their work, only one contacted me to say he would come to the focus group. So I cancelled it.

Does this only happen to me? I’ve since spoken to several people about it and drawn some comfort from their comments that it is difficult to lure students to focus groups these days. But were they just being nice to me? Have you succeeded where I failed? If so, how did you do it?

(Posted by Bob Rotheram, Project Manager, Sounds Good)

Can you help?

The Sounds Good project has been about lecturers using digital audio to give students feedback on their coursework. Right from the beginning, one of my concerns has been to make things as simple, quick and convenient as possible for everyone – staff and students. Any obstacle is likely to put some people off, I reckon. So as we were getting ready for the project I assembled some technical tips and guidelines on digital audio, and we’ve been gathering experience as we’ve gone along. However, I know I’m an amateur when it comes to technical stuff. Can you help improve the advice that Sounds Good offers on using digital audio for feedback? Here’s the current list of bullet points.

• A handheld digital audio recorder will probably be more convenient than using a microphone connected to a computer.

• If you’re buying a handheld recorder, make sure it:

    - can record direct to MP3 (many can’t);

    - has a USB port, for easy upload to a computer.

• Two recorders meeting these specifications are the Edirol R-09HR and the Sony ICD-UX70. The Edirol is excellent – over-the-top for our purposes – but it’s not cheap (about £250). The Sony is adequate – with a cheap clip-on mike to avoid recording the noise of your hand on the controls – and costs a lot less (maybe £85 total).

• Check that your students can receive audio files. They will need access to a computer equipped with a sound card, speakers or headphones, and suitable software (e.g. Windows Media Player or QuickTime). Other potential obstacles include: firewalls (blocking certain types of file or email attachments); file size limits; full email inboxes; spam filters.

• To avoid some of these obstacles, upload the audio files to a place from where they can be retrieved easily. Then send links, not the files, to students. Maybe your university’s virtual learning environment is not the best place to store audio feedback files. [ANY SUGGESTIONS?]

• Try uploading a test file for each student before you attempt to give them real feedback. Ask for help if you meet problems you can’t solve.

• On the basis of this experience, give students further guidance (perhaps in writing) on how to receive and listen to audio feedback.

• Make sure your recordings are loud enough to be heard easily on a variety of equipment. The general advice is to set the recorder’s input level as high as possible without introducing distortion. Some experimentation may help.

• Make your audio files as small as possible, so they can be sent quickly and stored economically.

• Mono recording – giving files typically half the size of stereo – is likely to be adequate if only one person is speaking.

• MP3 is a very useful format: compact and widely-playable. An MP3 file is typically less than 10% of the size of a WAV file of similar quality and can be played on a broad range of devices, including the portable music players owned by many students.

• Aim for the minimum acceptable sound quality for the particular purpose. Speaking to an individual student will probably not require as high quality as feedback to a group or a podcast on a public website.

• 32kbps MP3 – which translates to about four minutes per megabyte – will probably be good enough for giving individual feedback to a student.

• Back up your files. Copy them all to at least one other device.

Comments and suggestions welcome!

(Posted by Bob Rotheram, Project Manager, Sounds Good)

Students taking part in Sounds Good were overwhelmingly positive about receiving audio feedback on their coursework. They frequently remarked approvingly about its personal nature and the detail provided, saying it showed that the lecturer had carefully considered their work. There was particular approval if the feedback was received quickly. Some students appreciated the advantage of replaying the feedback. Others noted that audio made it easier to grasp what the lecturer felt was most important, or that it helped them to understand better why they had received a particular mark. Students whose first language is not English were pleased that it gave additional practice with their listening skills. A dyslexic student said that it was easier to listen than to read.

On the other hand, some students had reservations. A minority said that they preferred written feedback; a few asked for both audio and written comments on their work. Some noted that it was quicker to skim-read a piece of text than to listen to an audio recording to find the passage of particular interest. Understandably, the reservations were more common when the only feedback provided was audio to the whole group, i.e. when students did not receive any feedback – audio or written – on their individual work. However, in one of the two instances where this happened, the module leader noted that if he had not supplied the audio feedback for the group, the students would have had to rely on their own notes of what he said in class – the audio feedback was extra.

A few words of caution seem justified. First, although students were overwhelmingly positive about audio feedback, some of the positive responses may, in part, be due to the novelty of audio. Second, how seriously should we take student reservations about it being harder to skim an audio recording than a piece of text? Many lecturers complain that students pay little attention to written feedback. Perhaps, therefore, it may be seen as an advantage that students find it harder to skim audio feedback! Third, if staff make more use of recorded audio, it should not be at the expense of face-to-face time with students – a concern expressed by some Sounds Good students who received assessment guidance as well as feedback via digital audio.

(Posted by Bob Rotheram, Project Manager, Sounds Good)

I’ve been looking at the comments sent in by the Sounds Good team about their experience of using audio for feedback. I haven’t got all the information yet, but the picture is fairly clear. What do they think of it?

The staff team are, on balance, strongly in favour of audio feedback; most have clearly said that they intend to continue using it. Several people commented that they were able to give more, and higher-quality, feedback using audio. The extra feedback might include examples to illustrate the point being made or to show how the work might be improved. A language tutor said, “it’s an interesting and personal way to do the feedback for language students.” Some remarked positively on the fact that they used more natural language when speaking, rather than writing, their feedback. One noted that students found her voice and tone reassuring and comforting. Another, citing widening participation initiatives, thought that audio feedback is “an ideal medium to assist in the development of skills and confidence of students.” Yet another now plans to extend his use of audio beyond feedback, to provide more detailed guidance for students, for example on assessment tasks. In only one instance – where the marker gave brief feedback, and was in any case a quick typist and writer – does it seem likely that the project member will not continue with audio feedback.

The great majority of the team found it either ‘very easy’ or ‘fairly easy’ to record their audio feedback and prepare it for sending to students. Everyone in the project team was supplied with an Edirol R-09 handheld recorder and nine out of eleven who commented explicitly found it ‘very easy’ to use. A few also tried recording via various combinations of Audacity, WIMBA Voice Tools and (one person) on an Apple Mac computer. Opinions on these tools suggest that, apart from the Apple Mac, they were not quite so easy to use.

Staff reservations about audio feedback were mainly to do with the practical difficulties they encountered. Some of the problems were: learning to use the recording devices; finding a quiet place in which to record; getting used to hearing their own voices; the time needed to rename audio files; feedback sometimes not reaching students; students having difficulty in accessing the audio files. Three module leaders expressed doubts about scaling up audio feedback to larger groups of students. It is no doubt significant that these three were among those who had not managed to save time by giving audio feedback.

Even though the team’s experience has been very positive overall, it should not surprise us that they were not unanimous about the use of audio for feedback. It was, after all, a new type of activity, performed under pressure. However, the reservations could mainly be regarded as resulting from ‘teething problems’.

Perhaps the last word, for now, should go to the module leader who acknowledged that at first he “over-egged the pudding” by opting for a very labour-intensive approach to providing audio comments for his large group. Despite his unnecessarily hard work, he said:

“I think the experience has been really positive. … I’ve got plans to roll this out to other modules now, providing we can get round the technological and logistical demands it places on us. … It’s a trial that has worked well.”

(Posted by Bob Rotheram, Project Manager, Sounds Good)

The hypothesis being tested by Sounds Good is:

• Digital audio can be used to save assessors time AND give students better feedback.

Most of the data is in now, so do we have a verdict? Cautiously, yes. It’s hedged around with ‘health warnings’ and the detail may change, but this is the picture at present.

On the Sounds Good project sixteen Leeds Met staff have been giving audio feedback to students on some of their coursework. The circumstances have varied widely. One of the differences has been the number of students to whom each member of the staff team has given audio feedback – for some it has been dozens, for others the total is in single figures.

So, bearing in mind the widely differing experiences, has using audio instead of their usual methods saved assessors time? Of the twelve who have commented on this issue so far, six reported that giving digital audio feedback took more time than their usual methods, four thought that it took about the same time and two reported that it took less time.

On the face of it, the fact that only a small minority of staff reported saving time might be regarded as disappointing and a negation of the hypothesis. However, most expressed satisfaction with their efforts and noted that students received more, and higher-quality, feedback than they otherwise would have done. Some team members acknowledged that they got quicker as they became familiar with the technology and techniques of audio feedback. Some who found audio feedback took them more time only used it with quite a small number of students so, arguably, they have not yet achieved full familiarity with it.

The module leader who probably spent the most time giving and sending audio feedback to students was the leader of a small team of assessors. He said that whilst it was “incredibly time-consuming” he had “got the brief wrong” and found himself doing far more on behalf of his colleagues than had been intended. Another who reported spending more time on audio feedback even after becoming used to the tools and techniques noted two factors which are probably significant: a) her feedback was very brief; b) she was quick at typing and handwriting.

Both of the assessors who did report saving time by using digital audio feedback said that they managed to do so without compromising the quality of the feedback they provided. Indeed one (who was giving formative feedback to dissertation students) said, “I spent a little less time but gave much more detailed feedback and expanded on explanations.” It is worth noting that of all the members of the Sounds Good team, these two assessors are probably the most experienced users of technology.

So the indications are that it is possible, in some circumstances, to use digital audio to save time and not compromise on the amount and quality of feedback to students. The most favourable circumstances would appear to be:

• The assessor is comfortable with the technology.

• The assessor writes slowly but manages to record his/her speech relatively quickly.

• A substantial amount of feedback is given.

We might expect that some people will take longer than others to become familiar with the technology and the techniques of recording their comments. Perhaps more of the Sounds Good team would have managed to save time without compromising quantity and quality if they had had more practice with giving feedback via digital audio. They’ll get their chance to do that next academic year in ‘Sounds Good 2’, an extension to the project which JISC has recently decided to fund. More on that later.

(Posted by Bob Rotheram, Project Manager, Sounds Good)

Emerge projects are required to disseminate, so I’ve been doing my bit on behalf of Sounds Good. Not that this has been a burden, you understand. A conference in the US, followed by one in Canada sounds good, I thought. And so it has proved.

At both the International Consortium for Educational Development (ICED) conference in Salt Lake City, Utah, and that of the Society of Teaching and Learning in Higher Education (STLHE) in Windsor, Ontario, I’ve led a roundtable discussion related to Sounds Good. Each session has attracted a healthy number of people who are interested in the idea of using digital audio for feedback. I’ve also been using the informal opportunities afforded by conferences of inflicting your obsessions on strangers. Some of them have shown (or feigned) interest in the project and the work going on at Leeds Met.

A pleasing feature of my conference sessions has been that on both occasions, several participants have expressed a wish either to try using digital audio for feedback themselves or to encourage colleagues to do so. One guy I met has been doing it for years, and delighting his students writing theses, so he clearly has more experience than I do. However, he has been unable to interest his workmates. Maybe we can learn from each other.

Best of all, some of the folks responded to my offer to facilitate an online forum where we can keep in touch and share materials and ideas. Still away from home and very tired, I haven’t yet decided how to do this. Maybe I’ll set up a Google group, but I’m open to suggestions. Please write in if you can think of a better way or want to join.

Sounds Good is going international!

(Posted by Bob Rotheram, Project Manager, Sounds Good)

The presenter is Mike Atkinson, Associate Professor of Psychology at the University of Western Ontario. He is a '3M Fellow', the Canadian equivalent of a UK National Teaching Fellow. His email address is:

atkinson@uwo.ca

Hope this leads to some fruitful communications - in First or Second Life.

Bob 

Last weekend, my son brought me a new toy, a cheap MP3 recorder. How cheap? Forty quid. Compared with the gadgets we’re using on Sounds Good which cost £250 a throw, that’s cheap! The most frustrating thing has been waiting a week before I could begin to try it out. Life has been exceptionally busy – blame my daughter for getting married.

This morning, though, I’ve indulged myself and spent a happy couple of hours playing with the new toy. Yes, it does what it says on the tin: records direct to MP3 and allows easy uploading via a USB port. Better, it can record at lower bit rates than its ‘Rolls-Royce’ cousin, resulting in smaller, yet still-intelligible audio files. On the other hand, it needs to be held in the hand. The snag with this is that the noise of manipulation is recorded too, which is highly distracting for the listener. Fortunately, there’s a simple workaround: plug in an external mike. I already had one, costing only a few pounds, and used it to record Podcast 08 at 32kbps MP3 format. Check it out. Does the result sound good enough?

I’m not done with the evaluation yet. The nerd in me will play happily for a few more hours, when I get the chance. However, I think I’ll be buying and recommending the cheaper recorder if Sounds Good receives any further funding. (Watch this space for news on that!)

Nerds and bargain hunters will be keen to know which gadgets I’m talking about. The ‘Rolls-Royce’ recorder is the Edirol R-09, recently superseded by the even better Edirol R-09HR. The newer version is a lovely piece of kit, aimed mainly at musicians and broadcast journalists. Very easy to use, it can produce excellent quality recordings. Even on the minimum setting, 64kbps MP3 format, it’s fine for our main purposes on Sounds Good: lecturer speaking to student; podcasts for the website. In truth, it’s rather over the top for our needs. But there wasn’t a lot of choice at the time, he says defensively.

The cheapo machine I’m still playing with is the Sony ICD-UX70. Will you be able to get it for the £40 I mentioned? Probably not. My son lives in the US and is an awesomely expert shopper. He knows all the tricks for getting a good deal! Even so, here in rip-off Britain, I’ve recently seen the Sony device advertised online for £82, and the trend is downwards. That’s still a big saving on £250. Of course, you get what you pay for, and the ICD-UX70 cuts some corners. That’s economy for you.

But let’s not get carried away with technology and economy. As a little voice keeps reminding me, ‘it’s the education, stupid!’

(Posted by Bob Rotheram, Project Manager, Sounds Good)

“How’s it going?” That’s a common question, one which is easy to ask and often hard to answer. Why might it be difficult? Well, first, what’s being enquired about? My health or some combination of the various activities in my life? Second, does the questioner really want to know? Can I unburden myself for the next twenty minutes, or is the limit a cheery “fine, thanks”?

So how’s it going? Er, given the context, let’s assume you’re asking about Sounds Good. That’s the relatively simple bit. But what do you really want to know, how far does the data entitle me to go, how frank should I be and how long have I got? They are much tougher questions, especially when my answers will be pitched at a largely anonymous audience ‘out there’ somewhere. For better or worse, I’ve decided:

• You want an answer to the central question for Sounds Good: can digital audio be used to give students quicker, better feedback on their work?
• The data doesn’t entitle me to go far at present.
• I should be reasonably open and make sure I’m fair in presenting the evidence.
• I’ve not got much space or time: a few hundred words or five minutes of speech.

Student comments
How helpful was the audio feedback?
• “Very, very helpful as I felt like we were face to face. Really good.”
• “It is very helpful. I can listen to it any number of times and giving feedback verbally is far easier than writing, so we get more in [the same time].”
• “Initially I found it disjointed, mainly because I didn’t have my assignment beside me when listening. However, on re-listening with the assignment, I found it very helpful.”

How does audio feedback compare with written feedback?
• “Better because the tone of the person’s voice helps put emphasis on the areas they thought most important, whereas this can be lost in written feedback.”
• “I like it a lot.”
• “More effective.”
• “As I am dyslexic this has helped me because I find it easier to listen than to read.”
• “I think sound files can be helpful, but sometimes it could be better just to have it in writing.”

Staff comments
Once you were used to it, how long did creating and sending audio feedback to a student typically take compared with your usual methods?
• “As this was the first time, I felt it took the same amount of time as if I was using feedback sheets BUT I can say a lot more by audio than by feedback sheets. … I think with practice this will get quicker as I get more used to things.”
• “I got better and better at using the digital recorder. It is much quicker to speak than to write, although I had to make notes and speak them otherwise I forgot what I wanted to say. Overall [it is] quicker.”
• “It took slightly longer for me to prepare the feedback. But I feel this was worthwhile.”
• “The efficiency of giving audio versus written feedback would depend very much on what kind of assessment it was – the feedback I gave was quite short and very repetitive and it was incredibly boring doing the audio feedback. Also I am used to writing and typing, so that would be much quicker for me.”

Do you have any other comments about giving audio feedback?
• “The students really liked it. I think it was because it was new and one said they found my voice and positive tone reassuring and comforting and could tell I wasn’t cross, just being constructive. Sometimes this doesn’t come across in written words on feedback sheets.”
• “One of the down sides of using just audio is that you have to guide the student by saying, for example, third paragraph on page 2. … I used audio in conjunction with an assessment criteria sheet, which I think helps, and I then made comment relating to the criteria as well.”
• “I think there are many students who would take in the information better if it’s audio rather than written.”
• “I thoroughly enjoyed it. I was able to increase both the quantity and depth of feedback. I now plan to roll this out across all of the modules that I lead next year and to involve my team members in provision of audio feedback.”

Straws in the wind? I hope so.

(Posted by Bob Rotheram, Project Manager, Sounds Good)