Archive for December, 2011

Likert scale survey results. Presenting your findings is always tricky. In my example the children I taught filled in forms with question that are statements about which they can strongly agree, agree, be netural, disagree or strongly disagree. So ordinal options, the 5-point Likert scale.

I need to provide an overview of the results. I’m thinking of a stacked bar like this:

(Click to embiggenatte.)

Trouble is there are many good reasons why analysis of ordinal data like this is a bad thing. Furthermore I’ve taken some lierties myself.

1) Interval data (e.g. an absolute measures of, say, temperature over time) can be plotted like this. Ordinal data is arbitrary. E.g. who is to say that Person A’s “strongly agree” is the same as Person B’s strongly agree? Therefore mapping their results together is flawed.

2) The same is true between questions answered by the same person. Statement 1 might be “strongly agree” as might Statement 2, but is it the same strength of feeling? What if Statement 1 was “Murder is bad” and Statement 2 is “Mars Bars taste nice”. I strongly agree with both, but the strength of feeling is obviously different.

3) Similar issues exist between all the options in a single question. E.g. is the gap between “agree” and “strongly agree” the same as between “neutral” and “agree”?

4) I’ve taken the liberty of opposing the “positive” and “negative” responses so as to give instant visual feedback on how the responses balance each other.

5) I’ve also taken the liberty of equally distributing “neutral” responses across the +ve and -ve axis. A bit naughty. I could ‘zero’ the neutral responses, but then they drop off the chart completely. But doing this would lead to a fairer comparison between the explicitly +ve and -ve responses.

Having said all this, the stacked bar gives a good overall representation of the survey results.

I could present the combined data as clustered bars. This lessens the cross-question and inter-question-answers comparison inference, but it’s basically the same data. Like this:

(Click for enlargification.)

Question is: is this misleading and/or can you think of any other way I could present the data?

Development: complete!

I’ve been quiet for the past few weeks because I’ve been frantically fising those last-minute bugs that one invariably finds in a development project like this. But it was all worth it because I spent three days this week using my software to teach 9 and 10 year olds about astronomy.

Shirley Community Nursery & Primary School in Cambridge were kind enough to let me teach a few lessons. I had an excellent time at Shirley. The students and staff were all very helpful, friendly and enthusiastic. This made my job both pleasant and easier!

The sessions were well received by the children. It got a few ‘oohs’, ‘aahs’ and ‘awesomes’, so the biggest challenge – which was to engage the children in the activity – was met. I’m studying a computing degree, so my main focus is to assess the technical implementation of my work. This naturally means ‘did it work?’, but there is no point in producing something that is technically proficient unless it achieves its purpose well. In this case the purpose was to impart learning, and feedback indicates that this aim was met well too.

I asked the children to complete a Likert-scale survey after the intervention. The ‘enjoyment factor’ shown in the results is very positive. Similarly the children say that they learned about astronomy because of the lesson. Good news! Some of the discursive feedback was really helpful and very insightful too. Commments such as “I think the kinect should be able to track which person is controlling it so it doesn’t get confused’ and ‘the information text should also be read out by the computer so that blind people can hear it’ show a depth of thought that I perhaps foolishly wasn’t expecting.

Technically, the software worked well. I included three different control types:┬ádeictic gestures (pointing and hovering to control a cursor), symbolic gestures (rotate, zoom, pan) and voice commands. Probably the most highly developed and accurate method (from a technical perspective) is my deictic control method. This is highly tuned, highly developed and ‘just works’. Interestingly, the children pretty much ignored this control method. For them it was just an expected behaviour, requiring very little to understand and use proficiently.

My symbolic control method was arguably the least successful technically. The success rate of recognising gestures was around 60% (as compared with around 95% for the deictic method). The children commented on this, were occasionally frustrated and had a long list of ideas to improve it. Great!

And finally the voice control method was very popular. Of the three, it was voice controls that prompted the most ‘wow’s, and the children really enjoyed shouting at the computer to make it work. It technically performed well. I told Kinect to listen only to audio coming from an angle of 0 radians and also to suppress background noise and echoes. It did a sterling job of listening only to the audio we wanted it to, ignoring the noise. Microsoft’s Speech Recognition Engine could do with being a little faster, but overall this part of my project was remarkably stable.

I hope to run these sessions in another school in January, so it will be interesting to see if I get similar results. In the mean time I have enough information to begin writing up my findings. Here’s hoping I can articulate what I’ve done in such a way as to yield marks. I fear a discrepancy between the amount of effort I’ve put into this and the nature of the marking scheme which will define my overall grade.