BluePages - User Tests 1

Anusha, Monica and I have finished a first (short) round of user tests. We got very interesting feedback from them which we will use to design a first online version of the BluePages. We will have a second round in maybe a month time.

In the mean time I am going to give you a brief overview of the process we designed for the user tests.

• We recruited a small number of testers from within the Library Services unit plus 2 external voluntiers. Most of these testers do not necessarily belong to any of the groups identified in the Stakeholder Analysis ("researchers", "administrators", and "strategists and disseminators".) As this was the first round and as we had a basic mockup we first needed general input.
• We designed a script which had 2 main objectives: (1) to refine our initial design and (2) to identify and embed clear purposes and uses of Research Activity Data in the BluePages. Note: I am aware that this last objective involves more than one or more sets of user tests. However we are using these interactions with testers to start making sense of the features that need to be enhanced to make the BluePages transmit ideas such as collaborations, connections and membership. With this I think we can get acceptance which is an important part of the embedding process.
  

Introductory Questions 1.Ask tester to browse the site for 1 minute (or just to see the frontpage?). From looking at this site, what kinds of information do you think you could get from this site? 2.Who do you think this site is designed for? Why? Tasks 1.Name the sponsor for one project run by Prof. AAAAA. 2.Obtain a list of collaborators of Prof. BBBBB and email that list to a friend 3.You are an academic in the area of statistics who wants to explore new areas of research. For that you are looking for some reading material. You are looking for information about genome-related diseases particularly from studies that use big samples (over 10000). 4.You are the Head of the Management department and are going to build a new Decision Making Support laboratory as part of a program to boost research among your staff. As you or your staff do not have any experience with building laboratories that need special computers and software you need to find someone in Oxford who could offer you some guidance. That person could be someone who is an expert in computers and/or works in laboratories. 5.You just set up your new project’s website and would like it to be listed in the BluePages. What do you need to do? Briefing 1.Ask again: what kinds of information do you think you could get from this site? 2.What is your overall opinion of the BluePages? 3.Would you use a site like this? What kinds of information would you be interested in?

• We always had two observers during the tests (either Anusha and I or Monica and I) We took turns to ask the questions. We had copies of the script and wrote notes separately. We also gave testers the list of tasks on a sheet of paper.
• When we finished I transcribed all the notes and classified them by task or question. I then wrote a summary which highlighted main issues under each task or question.

After this we met to discuss our next strategy. This involves a clearer idea of the data "objects" that are searched for and browsed in the BluePages and the way these objects can be presented and connected. But I won't say more because this is a topic for another post.

BluePages - User Tests

Anusha and I are running a first round of user tests. For this we are using a mockup of the BluePages that Monica made with Balsamiq. Nice tool by the way, you can see some screenshots of a previous version of our mockup here.

I am using the term user tests as we are aiming to do a bit more than just asses the usability of our initial design. With these tests we are also trying to assess the perception of usefulness of the BluePages. That is, how useful a tool like the BluePages and/or the information that it offers is to its users' work. I think perception of usefulness plays an equal or perhaps more important role than usability in the acceptance of a technological tool. If a tool is user friendly but users do not think is useful for their work, they will not use it.

The stakeholder analysis revealed that people around the University already deal with research activity data by using their own systems (manual, computer or mixed). These systems are sometimes not as efficient as they could be but people trust them becasue they work for them. Therefore new tools like the RII and the BluePages should allow users to:

• do what they are already doing in a more efficient and faster way (less rekeying, less duplication of data,)
• do things they could not do or struggled to do before (collect a list of experts in one field from across the University,)
• discover new uses to this information (find connections between people and between projects through their keywords.)

Anusha and I designed a user test script, comprised of some questions and tasks. In one of the tasks, for example, we ask testers to find someone with a particular expertise who can guide them in the design of a laboratory.

At the end of the tests we have informal conversations (with tea/coffee and buscuits!) We ask testers their overall opinion of the site and if they would like/need to use a tool like this for their work.

Tests have been very useful so far to refine our initial design but most importantly to define clear purposes and uses for the BluePages. With this we want to position the BluePages within our stakeholders work environments and with that to improve user acceptance.

Main points we want to emphasise in the BluePages are collaboration, connections (networks) and membership. These are ideas that we want to emerge from the BluePages through its use. In other words we want people to think that the BluePages is a tool for collaboration, for making connections and for improving membership.

• Collaboration means that users can find who is collaborating with whom in Oxford. It also means that people listed in the BluePages are potential collaborators for whoever is browsing them. Therefore if anyone would like to collaborate in or outside Oxford they would probably like to be in the BluePages.
• Connections means that the BluePages can allow people to discover connections which were not clear before. For example connecting people who did not know each other by their research interests, or connecting research activities by their keywords or themes.
• By emphasising the concept of Membership we would like encourage Oxford research staff to contribute their data to make their research community more visible and more accessible within and outside the University.

Developing the Bluepages

The BRII project is very much on track.

I finished the Stakeholder and User needs Analysis. Will be ready for distribution soon.

Ben has been working on the development of the foundations of the Research Information Infrastructure (RII). He has also created an online store for the vocabularies resulting from the BRII project. The vocab site has been given a single, central location in Oxford and now consitutes an institutional service.

Anusha is working on the harvesting of data for the RII. At the moment she is dealing with data about the structure of the University (a very complicated task!) which should form the basis of the RII. I will meet with her soon to talk about this as we have to document data and process audit. She has also promised to give me a technical overview of her work so far so I can post it in this blog.

Monica is working on a mock-up of the Bluepages, one of the proposed outputs of the BRII project. As part of the development process we are planning a series of user-test sessions with a selection of users from the University. Monica is following an iterative approach to development. She will produce a first, basic version of the Bluepages which will be tested by users. She will then work on their feedback to produce an improved version. We will do a few rounds of testing until users and Monica feel satisfied with the product.

This is what Monica says:

After a few meetings and brainstorming with the team, many layout considerations were made and an initial timeline for the Bluepages was created. We were able to gather enough information about what data to include and the website's behaviour after data harvesting. Now we have a good idea about the direction we want to go with the Bluepages.

The outcomes of the meeting include a definition of the look and feel of the website and a list of many possible features that could be included depending on time of development and resources available. I will create a mockup with a few screens that will represent the initial Browse by People and Profiles section of the website. (See pictures below.)

Click on pictures to view larger versions.

The following is a list of layout considerations and ideas:

General design ideas

• Have a 'web 2.0' style of page, where simplicity and boldness will capture user's attention
• Introduction paragraphs that tells what the website does (purpose) before we show how and why is good for the user.
• attention map: what is more important bolder/bigger, less important/less used items smaller
• fewer columns, 2 or 3 max

Gadgets/widgets/functionalities

• "share this" button including email friend
• download as PDF
• "shopping basket" to add profiles/links to be printed later (as a list of 'things' the user wants to keep)
• enable links in profiles to: people with same interests (clicking in keywords), people in same projects, funder's page, project's page, etc (we have to decide which one links to what)
• collaborators sections in profiles (articles written together, include projects too?)
• RSS?

Homepage

• logos of University, BRII, JISC (bottom right), any other?
• quick search on top
• identification that is a BETA with a badge/star
• different tabs for different areas: Browse (active from the homepage), Advanced search, How to contribute, Add your data, About, Help (others???)
• Welcome blurb and max of 3 short paragraphs that can say WHAT IS THIS ABOUT, HOW TO USE and BENEFITS/Why
• "Browse by" area with buttons (icons too maybe?)
• Research Activity button will have a 'tooltip' type of baloon that will show what includes in Research activity, we can also add this info on Help page.
• "slider" showing groundbreaking research
• slider with random profiles ???

People's profiles

• Name
• Affiliations
• Roles
• Research topics/interests/keywords
• Themes - controlled vocab/top level
• Projects or other Research Activities -> link to the projects page inside blue pages
• Funder -> link to funder page inside blue pages
• Websites: department/college/personal page in each of the previous/"personal" personal page
• Publications -> link to publications if available
• Source of information
• Report problem/correct your information button

Funder

• Project
• People
• Department
• Subject

We estimate to have this finished in the next few weeks. This initial design will serve for the first tests and useful to evaluate the speed at which I can develop the next items on the list.

About the user tests

User tests will start as soon as Monica gives us the green light. We would like to gather small groups in sessions that should last between 30 to 45 minutes. When possible we would have the sessions in our base (Osney One). However we are prepared to visit our testers in their own office. Testers will be asked to perform a particular set of tasks and give us their opinion. We will gather feedback from the users by observing their behaviour, from user notes and comments in the debriefing. Group testing and refinements will repeat until a good representation of the future users could test the website and the corrections stop/are reduced to a minimum

Now I am looking for testers. So if you work in the University of Oxford AND you are curious and want to see what we are doing AND/OR you are interested in participating in a user test please email me!

Note: you may have noticed that I am using the word Bluepages in this post instead of Blue Pages, we are thinking on what would be the best way to call them. What do you think?