SuraYaUkimwi: difficulty starting and feedbacks

Ruaha Secondary School (20th April)

On Wednesday 15th April me and Markus visited the secondary school, located in Ipogolo 5 km from Iringa and met with the Principal, Mwageni (Student Counselor) and Grace (IT-lab responsible) and Silas(IT staff). Principle welcomed us and said we were allowed to conduct test sessions at the Computer Lab. Mwageni promised to organize a group of 10 students. We said any students could join only preference was not to be first year(grade) students. Grace suggested Mondays or Tuesdays for the test sessions. We decided to have first session Monday 20th. Today at lunch time I called Mwageni and no group was yet organized but he promised to organize a group. When we arrived at 15:30 half of the group organized had to be excused due to other obligations at the school. We briefly met with the five students and asked if they could find five more students to join. Grace and Silas in the beginning raised concerns that they would like to leave at already at 4 and someone needed to be present and also lock the lab after we were done with our work. Grace made an exception for this time but told that there will be a problem for coming sessions. I suggested that any of the volunteers at the school could stay and lock but Grace did not seem to like that solution. Marcus is thinking maybe we can ask how much she would charge for extra time for next sessions??

Finally at appr. 16:00 students had got organized and we started with 9 students that soon had expanded to 12 (5boys and 7girls) all in form four(Grade-4). We introduced ourselves and explained our objectives and that we are in need of the students’ feedback on the material that we have prepared. 12 Cd's had been prepared with the material, of which one was given to Mwageni, one to Grace the remaining 10 was distributed to the students. We began by asking students to find a computer and trying to open the material by themselves. To our surprise only one computer had a functional CD-player. We then tried placing material on server so that it could be accessed from other computers and eventually two more computers could view the material. With the use of My and Markus’s laptop we had a total of 5 computers for them where 3 of them had sound.

All the technical problems made it difficult to get a good idea of how easy they found the material to use but it was observed that all the groups got a hang on how to navigate the stories and extra material quite easy without the need of asking for assistance. When questioning them about the user friendliness they all claimed that it was easy to use and user friendly interface. I made several attempts to get feedback on what might not have been so intuitive but they all answered it was easy and no problems.

On the material itself there was several comments:

1. Limitations in additional material, the risk of transmission while in accident where you might get someone’s blood on your wounds. The importance not only about education as pointed out that different education needed for different ages as some things could be harmful knowledge for younger students

2. Why not making use of video and making material available on more accessible VHS video format as few students have access to computers, students also welcomed printed materials. The VHS option they suggested were for the one who might not have access to computers.

3. Some found cartoons to static and said it is a bit boring

4. Requested more and better animations and to make people walking properly and to make mouth move while they speak, to make them look more alive.

5. Speech bubbles not synchronized enough but students found them important and wanted to see more speech bubbles in the material. They also pointed out the importance for deaf people. (Or when no sound)

6. In Kisamvu story there is apparently some problem with voices telling something about the mothers asking her husband to inform relatives while already in funeral, which was found a bit strange

7. Concerns about breasts showing which was not considered appropriate by some

8. The use of Swahili and English was considered very good but they also wanted to have English voice available for foreign students and to follow school principles of the use of English

9. The stories were found very practical and look relevant and easy to relate to by the students they did not find material foreign or out of context

10. Sounds were considered excellent and they enjoyed that there was also some singing.

11. Some of the students did not like the idea of cartoons because they think they are not small enough to watch cartoon characters and would have been better if the real people were involved in making the stories.

12. There were also some comments about the presented stories being very summarized, and things were happening too fast.

13. Some of them suggested that the story should go automatic, ie stories coming one after another without doing anything.

14. The suggestions were made to synchronize the sound with picture.

15. There is a necessity to improve the pictures.

16. Most of the picture did not had the feeling in what the actions were being done. For example, in Mina's Story, she was being beaten but there were no expressions of being in pain.

17. More story for other causes(other than sexual intercourse) of the HIV disease were found necessary.

18. Most of the story presented a risk factor focused on males and there was a necessity to provide suggestions to avoid the contamination.

19. There was a bit of discussion about the speech bubbles. Some of them thought that there should be a lot of speech bubbles with all the things said in dialog while some of them thinks they were too many of them already.

20. At last, they were asked “If you are given this material and asked to provide education to the other students about HIV/AIDS, do you think you can do that? Why?”. Most of them were very positive that they can definitely go easily with them and the reason was that most of the stories were very simple and practical to say that these could happen easily to them as well.

After about an hour of going through material we had approximately a 15 min discussion about the material

SuraYaUkimwi: Iringa Girls and information sections

Second Session with the Iringa Girls:

Participants: 13 students from 'O' level distributed across the 9 computers.

Observations: Students who don't have much knowledge about the computers started with the information section and got stuck with how to get back. Most of them thought it only contained information.

Questions Raised:

• Does AIDS always get infected into person when they have wounds in the sexual parts?(in ABC)

Suggestions:

• When a doctor is telling about a person getting infected, they did not give any counsellings or advice and the boy don't have any feeling of being surprised in the speech.

• There are only condoms used by males. It would be good to have some stories that shows the use of female contraceptives.

• In Meena story, the story says that she went to the village to her relatives, but then it shows that she is dancing in disco the next moment and with modern dress (and dancing with boys). We don't have any villages in Tanzania with such discos in villages.

• People getting sick but not shown (except in one stories)

• Use of real person rather than cartoon (but clarified with the fact that they are not suitable for Internet versions)

• In questions for reflection, there is a mistake like:

  • unawe zakuhr : unaweza kuhu

  • ungefanga je : ungefanyaje

Design Considerations: Minimizing the chances of confounding variables

Confounding variables are the one which are varied between experiments and which affects the measured values of dependent variable. While testing an interface with the different people, there might be case that we could ask participants to enter different messages for different interfaces. This has effect on how long it takes to enter a message, regardless of the interface. This message length is the confounding variable. Others might be complexity of entering certain text compared to other words. The simplest solution is to make sure that same message is entered on all interfaces. Thus, even if the message takes different times, over the whole experiment, the effect of different messages would be evenly spread out across all interfaces. This is called counter balancing.

The aim is to vary independent variable in a known manner, to measure dependent variable(s) and to minimize the effect of confounds on outcome of study

How was this performed during the tests session?

To reduce the confounding between the different feedbacks, a series of questions were prepared. Although the questions were random, appropriate questionnaire was not difficult due to factors like:

• Time allocation constraints for students during the test sessions: usually we only had a chance to test the platform on a short time period, as the testing was usually conducted after school hours, which usually lasts for 2-2.5 hours per sessions on average. Most of these times students spend their time in activities like being comfortable with the platform, learning about the information given in the information box, enjoying the stories provided and discussing with the peers about it.

• Language barrier: Some of the participants from the lower form were not very comfortable with the English. Due to this reason, most of them just use simple way of communication to say that everything is good and they don’t have anything to say. Also due to the observer basic skills in Swahili, participants’ feedbacks were based on some other students’ interpretations (which again depend on their English deviation). This might loose out the possibility of the participants real comment and evaluation from feedback.

• Writing over was difficult part rather than comfortable talking: As it was a known that students appear in the test session after their classes in the schools, and they have just been out of possible stressful classes throughout the day, it was not sensible to ask them to fill up the questionnaire. Although to give out a simple evaluation, a simple fillout-form was prepared and handed over to students during one of the sessions. It was observed that most of them had a feeling of an official writing and felt difficult to write as most of them thought it would be recorded and given off to the teachers for their evaluation. Even after consenting that it is not the case and the feedback will not affect any of the educational activities, they emphasized on oral feedback to the questionnaire. Possible cause were being tired after a school day, being afraid of the writing being recorded, thoughts about their language(grammar) not being quality enough.

Design Considerations: Which method should I use?

Which method should I use?

One way to think about which method I should use can be made by answering following questions (refer to as ‘PRET A Reporter’ framework)

• Purpose of the evaluation: what are the goals of testing, or the detailed questions to be answered in the study? (this will find need and requirement that method was developed to address)

• Resources and constraints: what resources are available for conducting the study and conversely what constraints must the study work within? this will address the challenges that might come during testing

• Ethics: what ethical considerations need to be addresses? In method testing, the main ethical concerns are likely to relate to privacy of participants information.

• Techniques for gathering data must be identified, such as controlled experimentation, questionnaire(design, length, structure, wording, types and scales), interviews (structure, style, settings, and recording) and focus groups (break into groups and facilitate discussion) and eye tracking

• Analysis techniques must be selected such as GOMS [2], KLM [3],

• Reporting of findings.

Many of the tests of methods that have been conducted are comparative

--

[2] John, B. E. and Kieras, D.E 1996a, ’The GOMS family of user interface analysis technique: comparison and contrast.’ ACM Transaction on Computer-Human Interaction 3(4): 321-55

[3] Kieras, D.E 2001, ‘Using the keystroke-level model of estimate execution times.’ Unpublished manuscript, University of Michigan

Design Considerations: Dependent and Independent Variables

Designing an experiment is another very important aspect to think about before jumping right away into the possible available ones. As most of the aspects of HCI design is not based on quantitative evaluation but rather concise qualitative variables that represent corresponding measurement parameters.

One possible technique can be the use of dependent and independent variables. During the experiment one can change and see the effect of these variables on the result found out. The variable is intentionally varied is referred to as independent variable and that which is measured is dependent variable. In HCI, these changes might be to interaction design, interface features, participant knowledge and so on. [1] The value of dependent variable depends on value of independent variable. There might be multiple dependent variable (e.g. time to complete the task, error rate etc) within one experiment, but there should normally be only one independent variable (this is applicable to only for simple experiments).

The failure to prove the null hypothesis provides evidence that there is a casual relationship between independent and dependent variable.

--

[1] Card, S., Moran, T. and Newell, A. 1983: The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates