If you can spare 56 minutes, you might be interested to watch a video of my presentation on Sounds Good at a Lancaster University staff development event on 15 May 2009. Here's the link: http://www.lancs.ac.uk/celt/celtweb/02rotheram. Thanks to Lancaster University for recording the session and agreeing to a link from here.
I think most of the session is coherent but it includes a passage where I take people through a simulation of a book review exercise and receiving audio feedback. This bit is likely to be more understandable if you also study the book review exercise and then listen to the audio feedback. The materials for this are in the Downloads section of the Sounds Good website, at: http://sites.google.com/site/soundsgooduk/downloads.
Friday, 4 September 2009
Wednesday, 1 July 2009
Still breathing
There’s life in the old dog yet. The trigger for this posting is the publication of JISC’s ‘Effective Practice in a Digital Age’ www.jisc.ac.uk/practice, which includes two versions of a case study on Sounds Good. Both are entitled ‘Making assessment a learning experience’. The shorter version is on pages 26-7 of the PDF file available here. The longer version, in Word format, is here.
Whilst I’m online, it’s worth saying that Sounds Good continues to arouse interest. Since the end of March I’ve given a further six presentations, making a total of 30 so far. Several more are lined up in the next few months. If you’d like me to run a staff development event on audio feedback, or speak at a conference, please get in touch and let’s see if we can arrange something.
Whilst I’m online, it’s worth saying that Sounds Good continues to arouse interest. Since the end of March I’ve given a further six presentations, making a total of 30 so far. Several more are lined up in the next few months. If you’d like me to run a staff development event on audio feedback, or speak at a conference, please get in touch and let’s see if we can arrange something.
Wednesday, 25 March 2009
Thanks, team!
Now that the formal part of the project is over and the final report is pretty much ready to go, it's time to acknowledge that it's been a team effort and thank everyone.
Sounds Good has involved many people, staff and students, all of whom helped the project to succeed in one way or another. Thanks are particularly due to JISC, for the funding and also for consistent encouragement and support, most obviously from Lawrie Phipps, Programme Manager for the Users and Innovation Programme. My line manager, Prof Sally Brown, Pro-Vice-Chancellor for Assessment, Learning and Teaching at Leeds Met, deserves an accolade for allowing me to run the project as I wished and to give it more time than budgeted for. Simon Thomson, Sounds Good’s Deputy Project Manager, has been a valuable ally, sounding board and source of advice. I am also grateful to my main contacts at the three partner institutions for Sounds Good 2: Bob Ridge-Stearn at Newman University College, Caroline Stainton and Katie Jackson at the University of Northampton and Simon Sweeney at York St John University. Peter Chatterton, the project’s ‘critical friend’, provided reassurance and an extra forum for discussion as well as provoking productive thought. Isobel Falconer, our external evaluator, negotiated sensitively on how to review the project and then worked with colleagues before producing her helpful insights and perspectives. It has also been fun to get to know Will Stewart, leader of the ASEL project at the University of Bradford, and to share experiences with him. To all these people, and to others too numerous to mention, I much appreciate your contributions.
What a team!
Sounds Good has involved many people, staff and students, all of whom helped the project to succeed in one way or another. Thanks are particularly due to JISC, for the funding and also for consistent encouragement and support, most obviously from Lawrie Phipps, Programme Manager for the Users and Innovation Programme. My line manager, Prof Sally Brown, Pro-Vice-Chancellor for Assessment, Learning and Teaching at Leeds Met, deserves an accolade for allowing me to run the project as I wished and to give it more time than budgeted for. Simon Thomson, Sounds Good’s Deputy Project Manager, has been a valuable ally, sounding board and source of advice. I am also grateful to my main contacts at the three partner institutions for Sounds Good 2: Bob Ridge-Stearn at Newman University College, Caroline Stainton and Katie Jackson at the University of Northampton and Simon Sweeney at York St John University. Peter Chatterton, the project’s ‘critical friend’, provided reassurance and an extra forum for discussion as well as provoking productive thought. Isobel Falconer, our external evaluator, negotiated sensitively on how to review the project and then worked with colleagues before producing her helpful insights and perspectives. It has also been fun to get to know Will Stewart, leader of the ASEL project at the University of Bradford, and to share experiences with him. To all these people, and to others too numerous to mention, I much appreciate your contributions.
What a team!
Monday, 23 March 2009
So where are we?
At the end of February I called ‘time’ on Sounds Good: audio feedback after that date would not be taken into account when writing up the project. Since then, I’ve been gathering and analysing information and drafting the reports for JISC. Several months ago I asked on the blog ‘Are we nearly there yet?’ Only now, after a journey of a year, can we answer ‘yes’.
So where are we? Regular readers will know the main aim of Sounds Good was to test the hypothesis that using digital audio for feedback can benefit staff and students by:
Overlapping with the second phase, two HE Academy subject centres – Engineering and Geography, Environmental and Earth Sciences (GEES) – were funded to introduce audio feedback to their constituencies as part of JISC’s ‘Widening Stakeholder Engagement’ initiative. I’ve been helping the subject centres with this work.
Sounds Good has mainly been a qualitative study. Even so, it has produced a few statistics. Taking the two phases of Sounds Good together, 38 teachers in four institutions have supplied audio feedback to at least 1,201 students at all educational levels from foundation degree and first-year undergraduate to doctoral. The staff were located as follows: Leeds Met 23, Newman University College 8, University of Northampton 4, York St John University 3. In the first phase the numbers on the various modules ranged from six to 151, with at least 463 students receiving one or more items of audio or video feedback. In Sounds Good 2 the numbers on modules ranged from three to 150 and at least 738 students received one or more items of audio feedback.
The project has operated in widely differing circumstances, which has been a mixed blessing. The main advantage of this diversity is that has enabled a worthwhile preliminary exploration of the potential of digital audio for assessment feedback. On the other hand, the differing circumstances have led to a suite of case studies rather than one large, standardised experiment.
Sounds Good has worked very well overall. In the first phase it ran almost entirely to plan. In the second phase it generally went well in all four institutions, but there were a few minor problems, including:
No doubt some staff are encouraged by the fact that the great majority of students were positive about receiving audio feedback on their coursework. Students particularly appreciate the personal nature of individual audio feedback, as well as the detail they often received.
As for the central question tackled by Sounds Good:
There is much yet to explore in the field of audio feedback. There is plenty of scope for larger trials, attempting to tease out the variables and studying the effectiveness of audio feedback (i.e. whether it enables students to learn more). However, a particularly pressing problem – one which might be solved quickly by a programmer – is to automate the process of sending feedback to students. Audio feedback is already an attractive proposition, yet if assessors could be confident that – regardless of cohort size – it would take them little or no time to let students have their audio feedback, even more would probably find audio feedback worth adopting.
Sounds Good has broadly achieved what it set out to achieve. It has done some valuable exploration and produced useful practice guidelines. All in all, it has delivered an excellent return on JISC’s modest investment and, most of the time, it’s been fun.
So where are we? Regular readers will know the main aim of Sounds Good was to test the hypothesis that using digital audio for feedback can benefit staff and students by:
- saving assessors’ time (speaking the feedback rather than writing it)
- providing richer feedback to students (speech is a richer medium than written text).
Overlapping with the second phase, two HE Academy subject centres – Engineering and Geography, Environmental and Earth Sciences (GEES) – were funded to introduce audio feedback to their constituencies as part of JISC’s ‘Widening Stakeholder Engagement’ initiative. I’ve been helping the subject centres with this work.
Sounds Good has mainly been a qualitative study. Even so, it has produced a few statistics. Taking the two phases of Sounds Good together, 38 teachers in four institutions have supplied audio feedback to at least 1,201 students at all educational levels from foundation degree and first-year undergraduate to doctoral. The staff were located as follows: Leeds Met 23, Newman University College 8, University of Northampton 4, York St John University 3. In the first phase the numbers on the various modules ranged from six to 151, with at least 463 students receiving one or more items of audio or video feedback. In Sounds Good 2 the numbers on modules ranged from three to 150 and at least 738 students received one or more items of audio feedback.
The project has operated in widely differing circumstances, which has been a mixed blessing. The main advantage of this diversity is that has enabled a worthwhile preliminary exploration of the potential of digital audio for assessment feedback. On the other hand, the differing circumstances have led to a suite of case studies rather than one large, standardised experiment.
Sounds Good has worked very well overall. In the first phase it ran almost entirely to plan. In the second phase it generally went well in all four institutions, but there were a few minor problems, including:
- Only four of the six Leeds Met mentors managed to engage with mentees.
- Only seven, rather than the planned 12, mentees were recruited at Leeds Met.
- The extended communication channels between me and some team members led, occasionally, to staff not being entirely clear what was expected and me being less well-informed than previously as to what was happening.
- The data returned were somewhat less complete and even more varied in nature than in the first phase.
- I found it difficult to give the project sufficient time in January-February 2009.
No doubt some staff are encouraged by the fact that the great majority of students were positive about receiving audio feedback on their coursework. Students particularly appreciate the personal nature of individual audio feedback, as well as the detail they often received.
As for the central question tackled by Sounds Good:
- Can digital audio be used to give students quicker, better feedback on their work?
- The assessor is comfortable with the technology.
- The assessor writes or types slowly but records his/her speech quickly.
- A substantial amount of feedback is given.
- A quick and easy method of delivering the audio file to the student is available.
There is much yet to explore in the field of audio feedback. There is plenty of scope for larger trials, attempting to tease out the variables and studying the effectiveness of audio feedback (i.e. whether it enables students to learn more). However, a particularly pressing problem – one which might be solved quickly by a programmer – is to automate the process of sending feedback to students. Audio feedback is already an attractive proposition, yet if assessors could be confident that – regardless of cohort size – it would take them little or no time to let students have their audio feedback, even more would probably find audio feedback worth adopting.
Sounds Good has broadly achieved what it set out to achieve. It has done some valuable exploration and produced useful practice guidelines. All in all, it has delivered an excellent return on JISC’s modest investment and, most of the time, it’s been fun.
Sunday, 22 February 2009
Too little too late?
A thousand apologies for being so tardy with this: I've been running around trying to get to grips with my new job, but I'm now in a position to start if it is not too late.
Below are a few points that have occurred to me. They relate to my context of teaching 'English as a Foreign Language' to international students on a Foundation programme.
First of all, because one of our aims is to improve the level of language knowledge and skills, with written assignments we usually show some of the main erros we find as well as comment on the content. This is not correction, but an indication of the types of error, so that the students have to think for themselves in order to correct their own work. This would be far too tedious to attempt in a spoken format, as it would be so difficult indicating which line of which paragraph the problem was in and so on. So we will continue to need to give a form of written annotation as well as audio. On the face of it, this might suggest duplication, although in reality adio feedback will be additional feedback such that we don't normally give - perhaps a sentence or two only is the norm. However, the potential benefits are great, so I'm keen to give it a go.
The second issue is that our students are doing a wide range of assignments, from group presentations, webquests, online listening to the more traditional grammar tests and written essays (the latter mostly not very long). Typically, they have two or three short assignments each week, as well as longer pieces for the end of semester. Feedback is therefore regular and quite varied. I did a trial run with MP3 files after a recent speaking test, and needed help to sort out the technical issues. Once I'd got the hang of it though, it was relatively straightforward. We'll see if I remember what to do next time. But with so many pieces of work coming in all the time, I'm concerned about the implications for time of doing additional recordings.
Finally, for now, there is the question of how much some of the students understand spoken rather than written feedback. I can imagine certain students thinking 'what did she say?' With no written word there is not the same option to check in the dictionary, especially when you have no idea how to spell a spoken word. This suggests the need for regular dialogue with the students as to how they are finding oral feedback, with further impications for time ... aaaaggghhh!
Anyway, as I said, I'm still keen to give it a go, so let me know if I'm too late or not.
Below are a few points that have occurred to me. They relate to my context of teaching 'English as a Foreign Language' to international students on a Foundation programme.
First of all, because one of our aims is to improve the level of language knowledge and skills, with written assignments we usually show some of the main erros we find as well as comment on the content. This is not correction, but an indication of the types of error, so that the students have to think for themselves in order to correct their own work. This would be far too tedious to attempt in a spoken format, as it would be so difficult indicating which line of which paragraph the problem was in and so on. So we will continue to need to give a form of written annotation as well as audio. On the face of it, this might suggest duplication, although in reality adio feedback will be additional feedback such that we don't normally give - perhaps a sentence or two only is the norm. However, the potential benefits are great, so I'm keen to give it a go.
The second issue is that our students are doing a wide range of assignments, from group presentations, webquests, online listening to the more traditional grammar tests and written essays (the latter mostly not very long). Typically, they have two or three short assignments each week, as well as longer pieces for the end of semester. Feedback is therefore regular and quite varied. I did a trial run with MP3 files after a recent speaking test, and needed help to sort out the technical issues. Once I'd got the hang of it though, it was relatively straightforward. We'll see if I remember what to do next time. But with so many pieces of work coming in all the time, I'm concerned about the implications for time of doing additional recordings.
Finally, for now, there is the question of how much some of the students understand spoken rather than written feedback. I can imagine certain students thinking 'what did she say?' With no written word there is not the same option to check in the dictionary, especially when you have no idea how to spell a spoken word. This suggests the need for regular dialogue with the students as to how they are finding oral feedback, with further impications for time ... aaaaggghhh!
Anyway, as I said, I'm still keen to give it a go, so let me know if I'm too late or not.
Wednesday, 11 February 2009
Simon (York)
HELLLLLLLOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!
Finally managed to get back in to the Blog. The trouble is if you use three computers, three email addresses and forty seven and three quarters passwords you make mistakes. However, with the help of my pet whippet I now have an ENTIRELY MEMORABLE blog password!!!!!!!!
OK, Sounds Good!!!???
This has been - as Bob said it would be - an exciting journey (although frustrating at times). I will be putting a full report together...but talk about a learning curve!!! I found that wherever there was a choice to make I made the wrong choice, so everything took hugely longer than I anticipated. The worst thing has been getting the feedback to the students and getting an acknowledgement from them and getting the questionnaires back. This has been marked by a truly gargantuan level of difficulty.
A friend who runs an educational support organisation for creative writing told me last year his network (NAWE) had 1000 members and usually about 100 manage to attend the annual conference. 1 in 10 struck me as very good. Likewise with a survey, 1 in 10 response isn't too bad (though that means to get a response from 50 needs 500 surveys posting out which is a lot of work. Anyway, on Sounds Good I am indeed running at about 1 in 10. Actually I think this is low, and feel sure Bob will be disappointed. The worst thing though is I cannot be sure that the students have even got the Audio feedback at all because I am not receiving acknowledgements!!
So what went wrong? It seems to me that unless you set up the Assignment delivery electronically inside the VLE you are pretty well unable to post out the feedback through the VLE - and in any case once the module is to all intents and purposes over, the students never go near the VLE for that module again anyway!!!
Mark and I set about sending files through the VLE but it was HUGELY timeconsuming for Mark (and he's a VLE expert). So instead I sent the files out using my private email and the students' private emails if I had them, if not University email accounts. Here there's always the problem of file size and not knowing whether they have opened their email account - even once in recent times!!! Or ever!!!?? IN ADDITION - and Mark will be surprised by this - it took hours to do this and therefore probably didn't save (me) time over the 'attaching to the VLE approach'....
So... and that....has been ONLY SOME of the problems....
So do I have anything positive to say??? YES. Both my colleague Bev Geesin and I have been delighted with the Audio Feedback in terms of what it can bring to the process and the students who HAVE responded have all been very positive (except one who wanted written feedback AS WELL!)...so yes, I think Audio Feedback is the future. I would always want to use it (even though I think with all the technical problems resolved it WILL NOT SAVE TIME. It is though a much more sophisticated, detailed and personal feedback service. OK...full report at the end.
For the moment I am still stuck in a quandary. What should I do for those students who have not responded at all? I may have to go through the laborious process of attaching the files into the VLE...but even then, will they visit Blackboard weeks after they handed in their work???
OK...that's it. More detail will be provided later....Happy Blogging!
Simon signing off (and taking the whippet for a walk).
Finally managed to get back in to the Blog. The trouble is if you use three computers, three email addresses and forty seven and three quarters passwords you make mistakes. However, with the help of my pet whippet I now have an ENTIRELY MEMORABLE blog password!!!!!!!!
OK, Sounds Good!!!???
This has been - as Bob said it would be - an exciting journey (although frustrating at times). I will be putting a full report together...but talk about a learning curve!!! I found that wherever there was a choice to make I made the wrong choice, so everything took hugely longer than I anticipated. The worst thing has been getting the feedback to the students and getting an acknowledgement from them and getting the questionnaires back. This has been marked by a truly gargantuan level of difficulty.
A friend who runs an educational support organisation for creative writing told me last year his network (NAWE) had 1000 members and usually about 100 manage to attend the annual conference. 1 in 10 struck me as very good. Likewise with a survey, 1 in 10 response isn't too bad (though that means to get a response from 50 needs 500 surveys posting out which is a lot of work. Anyway, on Sounds Good I am indeed running at about 1 in 10. Actually I think this is low, and feel sure Bob will be disappointed. The worst thing though is I cannot be sure that the students have even got the Audio feedback at all because I am not receiving acknowledgements!!
So what went wrong? It seems to me that unless you set up the Assignment delivery electronically inside the VLE you are pretty well unable to post out the feedback through the VLE - and in any case once the module is to all intents and purposes over, the students never go near the VLE for that module again anyway!!!
Mark and I set about sending files through the VLE but it was HUGELY timeconsuming for Mark (and he's a VLE expert). So instead I sent the files out using my private email and the students' private emails if I had them, if not University email accounts. Here there's always the problem of file size and not knowing whether they have opened their email account - even once in recent times!!! Or ever!!!?? IN ADDITION - and Mark will be surprised by this - it took hours to do this and therefore probably didn't save (me) time over the 'attaching to the VLE approach'....
So... and that....has been ONLY SOME of the problems....
So do I have anything positive to say??? YES. Both my colleague Bev Geesin and I have been delighted with the Audio Feedback in terms of what it can bring to the process and the students who HAVE responded have all been very positive (except one who wanted written feedback AS WELL!)...so yes, I think Audio Feedback is the future. I would always want to use it (even though I think with all the technical problems resolved it WILL NOT SAVE TIME. It is though a much more sophisticated, detailed and personal feedback service. OK...full report at the end.
For the moment I am still stuck in a quandary. What should I do for those students who have not responded at all? I may have to go through the laborious process of attaching the files into the VLE...but even then, will they visit Blackboard weeks after they handed in their work???
OK...that's it. More detail will be provided later....Happy Blogging!
Simon signing off (and taking the whippet for a walk).
Subscribe to:
Posts (Atom)
Posts
