 |
|||
| THIS OLD OPEN SOURCE VERSION OF SPARK IS NO LONGER AVAILABLE A NEW NON-OPENSOURCE VERSION (SPARK PLUS) IS BEING DEVELOPED BY UTS IN ASSOCIATION WITH THE UNIVERSITY OF SYDNEY. THE TEXT AND LINKS BELOW ARE STILL VALID EDUCATIONALLY BUT THIS SITE WILL BE UPGRADED WHEN THE NEW VERSION IS AVAILABLE |
|||
 |
|
|
|
Contents | What & When | Why | How | Outcomes
Abstract“Group projects aren't fair” is a frequent student response in higher education. Group work is used to facilitate peer learning and encourage students to develop collaboration, a crucial graduate attribute. Since assessment strongly influences learning, any course objective to improve peer learning and/or collaboration must have assessment that promotes it. Self and peer assessment is a valid solution for promoting these objectives and overcoming potential inequities of equal marks for unequal contributions. Group members are responsible for negotiating and managing the balance of contributions and then assessing whether the balance has been achieved. SPARK is an 'open source' web-based self and peer assessment kit solving major problems in current paper-based approaches. It enables confidentiality to students rating their own and their peers' contributions. A range of criteria related to team tasks and maintenance promotes fair acknowledgment of individuals' contributions. SPARK automates significant data collection, collation and calculation problems that academics would otherwise face. The factors produced by SPARK are used to change group marks to individual marks. Without this automation, academics with large classes simply could not consider self and peer assessment. As with all educational technology the essential caveat applies: Careful and thoughtful student-centred integration is vital for success! |
Contents
What & When? ^
 |
SPARK WAS an academic open source project freely available according to the rules of the CUTSD funding IN 1999. It has been appreciated that case studies/academic papers and other interesting research using SPARK were sent to Darrall Thompson at darrall.thompson@uts.edu.au SPARK was developed by a multidisciplinary team at UTS over a five year period with funding received in 1999: Project management - Mark Freeman (Business), Jo McKenzie (Academic developer), Darrall Thompson (Design) Programming SPARK engine - Duncan Ford (Business) Programming interface and logo design - Darrall Thompson (Design) Academic developers - Jo McKenzie (Interactive Media & Learning), Simon Housego (Interactive Media & Learning) Reference group and pilot participants - Mark Freeman (Business), Rob McLaughlan (Engineering), Bob Spencer (Engineering), Michael Adams (Law) Darrall Thompson (Design)
|
SPARK is designed to improve students' learning from assessed group tasks and improve both students' and lecturers' confidence in and satisfaction with the process of assigning group marks. Specific ways in which SPARK can contribute to the curriculum include:
Why? ^
Click here for more details.
How? ^
* Final and prompting criteria can be distinguished in that the former are used to prompt students to think about the team process and do not affect the calculation of the SPA factors. Final criteria do affect the calculations. If no criteria are set as 'final', no SPA factors can be calcualted! ** Rating scale
on each criterion example: 3=contribution above group average, 2=average
contribution (to rest of group), 1=contribution below group average, 0=no
contribution, -1=hindrance to group
|
Sample Screenshots: (these will all open in a new window) ^
Instructor
screen (with help panel) for modifying a subject which is similar
to the one for creating a subject.
Instructor screen
for selections regarding Self and Peer assessment rating results
Instructor screen
for viewing rating results for the subject ASCILITE 111. It will be noticed
that TheCharacters team has not yet submitted ratings and ascii.stud may rate
as part of the ‘Try It’ test by the judgement panel.
Student screen to
select the Self and Peer Assessment rating screen.
Student screen to
rate team members against categorised criteria.
Outcomes ^
Table 1: Sample of student responses to SPARK in one of the pilot subjects (n=48)
Feedback
Question |
SA/Agree | SD/Disagree |
| The system was accessible | 79% | 8% |
| The system was easy to use | 70% | 13% |
| The process helped me learn more about teamwork | 40% | 24% |
| Identified aspects of teamwork I hadn't thought about before | 41% | 27% |
| Items were appropriate for assessing contributions | 69% | 9% |
| Encouraged greater effort | 40% | 33% |
| Able to give an honest assessment | 78% | 11% |
| Fair way of assessing team contributions | 69% | 18% |
The percentage of students who reported that the process had helped them to learn more about teamwork was encouraging, considering that most students had encountered team tasks in previous subjects. It was also interesting that 40% felt it encouraged them to make more effort whereas 33% disagreed, the latter often commenting that they were self motivated to contribute or wanted to do well and did not need the external incentive to make an effort.
| Sample students comments: |
| “ ... she has been a role model for me in this project and possibly my future career as a designer. I hope that I can still have the chance to express my gratefulness through SPARK.” (Spring 2000) |
| “I feel that the online ratings is a fair system and that i can be honest about my views on other group members. I also think that at the present this sort of online service is still a 'token' one, because it is only a small aspect of the course that tries to use the internet in a relevant way.” (Spring 2001 online survey) |
| “Having been through this appeal because I thought SPARK was unfair I now realise it works, we should be using it for all our group projects” (Appeals interview 2001) |
| “I just felt that the group would have worked better without ***** and his premenstrual mood swings every day. Ok, he was a decision maker, but in terms of a contribution, I gave a below average.” (Spring 2000) |
| Sample staff comments: |
| “My
second year students asked me if they could use SPARK for their group research
project... I said what’s SPARK?... they said phone Darrall! It’s
set up now and really helping them settle down to the project because they
feel it’s a fair system.” (Design Tutor, Sept, 2002) |
| “I was down in the labs after a class and watched students from Darrall’s class using SPARK. One student was unsure how to do it and a friend from his team said ‘I’ll show you and you can give me a good mark’. I saw the student come back into the lab later and resubmit different ratings. This is what convinced me I wanted to use it.” (Architecture lecturer 2001) |
Extract from Freeman and McKenzie (2002) ^
“Group and team work are commonly used in higher education to facilitate peer learning and encourage students to develop their capacity to work as part of a team. There seems little argument about the value of teamwork, but its assessment has proved considerably more problematic (Conway, Kember, Sivan & Wu, 1993; Lejk, Wyvill & Farrow, 1996). One author has likened group assessment to a game, maintaining that the rules of the game advantage some students and disadvantage others, and that factors such as teamwork and contribution to a team are “essentially impossible to assess fairly” (Pitt, 2000, p. 240). However, assessment strongly influences students’ learning (Ramsden, 1992; Biggs, 1999). If courses include objectives about students’ capacity to work as part of a team, and we value peer learning then we need some means of assessing teamwork in a fair and meaningful way which promotes peer collaboration (Sampson, Cohen, Boud and Anderson, 1999).
Peer assessment of individuals’ contributions to assessed teamwork isn't a new idea, although the addition of self assessment is relatively innovative. While there is some debate about the inclusion of self assessment (Lejk et al 1996), we believe it encourages students to reflect on their own contributions and capabilities. In fact, Boud, Cohen and Sampson (1999) favour self-assessment informed by peer feedback on specific criteria, in preference to peer assessment per se.
SPARK was intended to have benefits for both students and staff. It was intended to encourage students to negotiate the way they will work in the team to achieve the best task result with equal contributions by all students. Using self and peer assessment encourages students to develop the capacity to reflect on and evaluate their own and others’ contributions, develop awareness of their own strengths and needs as a team member and develop their teamwork skills. For staff, the intention was that they would gain satisfaction from seeing improvements in learning and have fewer problems with complaints about the fairness of team based assessment tasks.^
SPARK is based on a well-designed and evaluated paper-based peer assessment system in which students rated each other's contributions and the lecturer used the ratings to calculate adjustments to individual marks (Goldfinch 1994). While Goldfinch's system was reasonably effective in adjusting team marks to reflect individual contributions, it involved a series of time consuming calculations to generate adjustment factors. This created a disincentive for lecturers and delayed the provision of feedback to students, particularly in large classes.
The SPARK software deals with this problem by automating the processes of collecting the student assessments and completing the calculations. This was a major efficiency benefit, in addition to the learning benefits. Compared with paper-based systems SPARK was also intended to improve student confidentiality and reduce data entry and calculation errors.
A further intention was to enable dissemination. SPARK was created to be a relatively generic template which can be easily adapted to any learning context where group or team work and/or self and peer assessment are used.”
