SuraYaUkimwi: interface and cultural tests

Iringa Girls:

Observations:

• Out of 10 participant groups, 5 started the tool with the information slides.

• It was observed that some of them had a difficulty with information mode only with going back to previous page (confused with the use of X button)

• Some of the participants who were not used to with the computers were using the right click buttons on the mouse.

Feedback:

• Pictures need to be realistic, for example the university student must be wearing smart dress, must not be wearing funny cap when the person is in tie, the dress

• When a person is infected or dying, his/her health must be seen as if it is so, for example he is lean, thin, with rashes and must be seen ugly in the picture.

• It was suggested that there must be some way of indication about the start and end of the conversation would be provided. Too many information at the same time can be sometimes a bit of problem with the reader. One possible suggestion was to put text at first and then the pictures. And some more additional work needed to be done with the bubble and texts.

Suggestions:

• Improve the Swahili language in the texts. (Grammar and more suitable words)

• In the HIV/AIDS statistics, there is a mistake with the percentage of infection in Zanzibar as 0,7% (it is supposed to be 0.7%?)

• Side text (information tab) can be put horizontal at the bottom reducing the size of the logo. This will have more space for the pictures.

Cultural Aspects: another new excerpts.

Event: 4^th testing of tool SuraYaUkimwi with a secondary level school children at Iringa Girls.

Date: 7^th May,2009.

Session title: Testing SuraYaUkimwi to ‘A’ level students at Iringa Girls

Participants: 29 ‘A’ Level students divided into 10 groups.

Mr Loupenza informed about the popularity of the test session going on in their school and there are lots of students who want to participate and volunteer in what they called “teaching class of computer”. I was given three days (tuesday, thursday and friday) suitable for them. Tuesday I had a class on Tumaini so could not possibly make it so I inform him that I will be there on Thursday (I was leaving for a trip on Friday early morning).

On Thurday, 7^th May, I reached computer lab and was really amazed to see so many number of students at the same session as compared to previous ones. There were 29 students (3 of them were from previous sessions) all from the ‘A’ level and they were eagerly waiting for the session to start. Most of them already heard from their friends who have participated in other sessions and knew what they were supposed to do before I gave instructions. At first it was bit of difficult with finding enough space for such large number of students accommodating and the place was much crowded.

Observations:

• Starting a program was a main issue in this session of testing. Most of them were struggling with how to start the program. Some of them had a program started but even did not noticed as they were also using some other programs and our tool was not visible. It would be useful to start program in full-screen mode and also run in front of any other running windows.

• 4 out of 10 groups started with the information section and when asked didn’t knew whether it contains stories as well and felt difficult to go back (needed some additional work on how to get them back to the place they were before).

• 6 groups didn’t knew that there is an English version of the text along with Swahili. (need some highlight features to indicate this)

• 2 groups were struggling with the information; when shown how to go back, they again found difficult to go back to main menus and were trying to use “back” button rather than the “start” button

Suggestions:

• 2 other groups didnt liked the placement and shape of next and previous button told that they were struggling with them for some time until they really get the idea of their use.

• Horizon in the sky are usually white(or light blue), but not dark blue.

• One of the important issues raised was with the idea of making characters similar in all pictures. Some of the places the person was difficult to identify and same person was presented differently (eg, character Mark)

• No reaction of feeling pain (eg Mina beaten by her step mother)

• 18 participants didnt liked the idea of showing breast and other parts of women. They think it is against the culture and thus strictly not suitable for children to show in schools. There was a deep discussion about the issue and it was eventually advised that the best way to show a couple in bed is their clothes lying around in bed in floor.

• There is no culture in Tanzania for girls to go to parents for talking about the marriage with the boys family. Boys are responsible to do that, except that they have finished the university (but not at least when they are in A level).

• No one in the session were clear about how Rahema got HIV in the story.

• Finally there were lots of suggestions about the necessity of improving the language used.

How it could the session be improved?

inform in advance to the concern people about the number of participants required so that I will prepare small questionnaire enough for all of them. Because of more number of students, I was not able to cover all of the students view individually. Also if there are more number of students, most of them were not able to use the tool by themselves and could not get their own hands on the tool. It is found that sound are really making difference in the way students were reacting and giving feedback on different aspects of the tool. It was taking them more deep onto the issues that students without sound were not able to go through it and find out about those aspects.

What was this session intended to achieve?

When a work is being done by people who have never been to the culture that he/she is working on, the gap between the developer and the real user is not only the case of being expert and novice but also difference between the cultural traditions the both of them belong to. The basic idea behind this session was to study the coverage of the different cultural aspects and the student view about the material covered in the tool.

Why did I attend the session?

See particular research log (requires login) for this.

SuraYaUkimwi: difficulty starting and feedbacks

Ruaha Secondary School (20th April)

On Wednesday 15th April me and Markus visited the secondary school, located in Ipogolo 5 km from Iringa and met with the Principal, Mwageni (Student Counselor) and Grace (IT-lab responsible) and Silas(IT staff). Principle welcomed us and said we were allowed to conduct test sessions at the Computer Lab. Mwageni promised to organize a group of 10 students. We said any students could join only preference was not to be first year(grade) students. Grace suggested Mondays or Tuesdays for the test sessions. We decided to have first session Monday 20th. Today at lunch time I called Mwageni and no group was yet organized but he promised to organize a group. When we arrived at 15:30 half of the group organized had to be excused due to other obligations at the school. We briefly met with the five students and asked if they could find five more students to join. Grace and Silas in the beginning raised concerns that they would like to leave at already at 4 and someone needed to be present and also lock the lab after we were done with our work. Grace made an exception for this time but told that there will be a problem for coming sessions. I suggested that any of the volunteers at the school could stay and lock but Grace did not seem to like that solution. Marcus is thinking maybe we can ask how much she would charge for extra time for next sessions??

Finally at appr. 16:00 students had got organized and we started with 9 students that soon had expanded to 12 (5boys and 7girls) all in form four(Grade-4). We introduced ourselves and explained our objectives and that we are in need of the students’ feedback on the material that we have prepared. 12 Cd's had been prepared with the material, of which one was given to Mwageni, one to Grace the remaining 10 was distributed to the students. We began by asking students to find a computer and trying to open the material by themselves. To our surprise only one computer had a functional CD-player. We then tried placing material on server so that it could be accessed from other computers and eventually two more computers could view the material. With the use of My and Markus’s laptop we had a total of 5 computers for them where 3 of them had sound.

All the technical problems made it difficult to get a good idea of how easy they found the material to use but it was observed that all the groups got a hang on how to navigate the stories and extra material quite easy without the need of asking for assistance. When questioning them about the user friendliness they all claimed that it was easy to use and user friendly interface. I made several attempts to get feedback on what might not have been so intuitive but they all answered it was easy and no problems.

On the material itself there was several comments:

1. Limitations in additional material, the risk of transmission while in accident where you might get someone’s blood on your wounds. The importance not only about education as pointed out that different education needed for different ages as some things could be harmful knowledge for younger students

2. Why not making use of video and making material available on more accessible VHS video format as few students have access to computers, students also welcomed printed materials. The VHS option they suggested were for the one who might not have access to computers.

3. Some found cartoons to static and said it is a bit boring

4. Requested more and better animations and to make people walking properly and to make mouth move while they speak, to make them look more alive.

5. Speech bubbles not synchronized enough but students found them important and wanted to see more speech bubbles in the material. They also pointed out the importance for deaf people. (Or when no sound)

6. In Kisamvu story there is apparently some problem with voices telling something about the mothers asking her husband to inform relatives while already in funeral, which was found a bit strange

7. Concerns about breasts showing which was not considered appropriate by some

8. The use of Swahili and English was considered very good but they also wanted to have English voice available for foreign students and to follow school principles of the use of English

9. The stories were found very practical and look relevant and easy to relate to by the students they did not find material foreign or out of context

10. Sounds were considered excellent and they enjoyed that there was also some singing.

11. Some of the students did not like the idea of cartoons because they think they are not small enough to watch cartoon characters and would have been better if the real people were involved in making the stories.

12. There were also some comments about the presented stories being very summarized, and things were happening too fast.

13. Some of them suggested that the story should go automatic, ie stories coming one after another without doing anything.

14. The suggestions were made to synchronize the sound with picture.

15. There is a necessity to improve the pictures.

16. Most of the picture did not had the feeling in what the actions were being done. For example, in Mina's Story, she was being beaten but there were no expressions of being in pain.

17. More story for other causes(other than sexual intercourse) of the HIV disease were found necessary.

18. Most of the story presented a risk factor focused on males and there was a necessity to provide suggestions to avoid the contamination.

19. There was a bit of discussion about the speech bubbles. Some of them thought that there should be a lot of speech bubbles with all the things said in dialog while some of them thinks they were too many of them already.

20. At last, they were asked “If you are given this material and asked to provide education to the other students about HIV/AIDS, do you think you can do that? Why?”. Most of them were very positive that they can definitely go easily with them and the reason was that most of the stories were very simple and practical to say that these could happen easily to them as well.

After about an hour of going through material we had approximately a 15 min discussion about the material

SuraYaUkimwi: Iringa Girls and information sections

Second Session with the Iringa Girls:

Participants: 13 students from 'O' level distributed across the 9 computers.

Observations: Students who don't have much knowledge about the computers started with the information section and got stuck with how to get back. Most of them thought it only contained information.

Questions Raised:

• Does AIDS always get infected into person when they have wounds in the sexual parts?(in ABC)

Suggestions:

• When a doctor is telling about a person getting infected, they did not give any counsellings or advice and the boy don't have any feeling of being surprised in the speech.

• There are only condoms used by males. It would be good to have some stories that shows the use of female contraceptives.

• In Meena story, the story says that she went to the village to her relatives, but then it shows that she is dancing in disco the next moment and with modern dress (and dancing with boys). We don't have any villages in Tanzania with such discos in villages.

• People getting sick but not shown (except in one stories)

• Use of real person rather than cartoon (but clarified with the fact that they are not suitable for Internet versions)

• In questions for reflection, there is a mistake like:

  • unawe zakuhr : unaweza kuhu

  • ungefanga je : ungefanyaje

Design Considerations: Minimizing the chances of confounding variables

Confounding variables are the one which are varied between experiments and which affects the measured values of dependent variable. While testing an interface with the different people, there might be case that we could ask participants to enter different messages for different interfaces. This has effect on how long it takes to enter a message, regardless of the interface. This message length is the confounding variable. Others might be complexity of entering certain text compared to other words. The simplest solution is to make sure that same message is entered on all interfaces. Thus, even if the message takes different times, over the whole experiment, the effect of different messages would be evenly spread out across all interfaces. This is called counter balancing.

The aim is to vary independent variable in a known manner, to measure dependent variable(s) and to minimize the effect of confounds on outcome of study

How was this performed during the tests session?

To reduce the confounding between the different feedbacks, a series of questions were prepared. Although the questions were random, appropriate questionnaire was not difficult due to factors like:

• Time allocation constraints for students during the test sessions: usually we only had a chance to test the platform on a short time period, as the testing was usually conducted after school hours, which usually lasts for 2-2.5 hours per sessions on average. Most of these times students spend their time in activities like being comfortable with the platform, learning about the information given in the information box, enjoying the stories provided and discussing with the peers about it.

• Language barrier: Some of the participants from the lower form were not very comfortable with the English. Due to this reason, most of them just use simple way of communication to say that everything is good and they don’t have anything to say. Also due to the observer basic skills in Swahili, participants’ feedbacks were based on some other students’ interpretations (which again depend on their English deviation). This might loose out the possibility of the participants real comment and evaluation from feedback.

• Writing over was difficult part rather than comfortable talking: As it was a known that students appear in the test session after their classes in the schools, and they have just been out of possible stressful classes throughout the day, it was not sensible to ask them to fill up the questionnaire. Although to give out a simple evaluation, a simple fillout-form was prepared and handed over to students during one of the sessions. It was observed that most of them had a feeling of an official writing and felt difficult to write as most of them thought it would be recorded and given off to the teachers for their evaluation. Even after consenting that it is not the case and the feedback will not affect any of the educational activities, they emphasized on oral feedback to the questionnaire. Possible cause were being tired after a school day, being afraid of the writing being recorded, thoughts about their language(grammar) not being quality enough.

Design Considerations: Which method should I use?

Which method should I use?

One way to think about which method I should use can be made by answering following questions (refer to as ‘PRET A Reporter’ framework)

• Purpose of the evaluation: what are the goals of testing, or the detailed questions to be answered in the study? (this will find need and requirement that method was developed to address)

• Resources and constraints: what resources are available for conducting the study and conversely what constraints must the study work within? this will address the challenges that might come during testing

• Ethics: what ethical considerations need to be addresses? In method testing, the main ethical concerns are likely to relate to privacy of participants information.

• Techniques for gathering data must be identified, such as controlled experimentation, questionnaire(design, length, structure, wording, types and scales), interviews (structure, style, settings, and recording) and focus groups (break into groups and facilitate discussion) and eye tracking

• Analysis techniques must be selected such as GOMS [2], KLM [3],

• Reporting of findings.

Many of the tests of methods that have been conducted are comparative

--

[2] John, B. E. and Kieras, D.E 1996a, ’The GOMS family of user interface analysis technique: comparison and contrast.’ ACM Transaction on Computer-Human Interaction 3(4): 321-55

[3] Kieras, D.E 2001, ‘Using the keystroke-level model of estimate execution times.’ Unpublished manuscript, University of Michigan

Design Considerations: Dependent and Independent Variables

Designing an experiment is another very important aspect to think about before jumping right away into the possible available ones. As most of the aspects of HCI design is not based on quantitative evaluation but rather concise qualitative variables that represent corresponding measurement parameters.

One possible technique can be the use of dependent and independent variables. During the experiment one can change and see the effect of these variables on the result found out. The variable is intentionally varied is referred to as independent variable and that which is measured is dependent variable. In HCI, these changes might be to interaction design, interface features, participant knowledge and so on. [1] The value of dependent variable depends on value of independent variable. There might be multiple dependent variable (e.g. time to complete the task, error rate etc) within one experiment, but there should normally be only one independent variable (this is applicable to only for simple experiments).

The failure to prove the null hypothesis provides evidence that there is a casual relationship between independent and dependent variable.

--

[1] Card, S., Moran, T. and Newell, A. 1983: The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates