Blue Pages video clip

As the project will end soon (31st of March) I wanted to show you what we have done so far with the Blue Pages. We will not stop working though. We will continue working on harvesting more data and user engagement. Anusha and Mat have been working tirelessly on the Blue Pages and although there is still a lot of work to be done on it I think it is at a stage where I can show you a short demo. This is a work in progress video and we are using a few sets of harvested data. The video below lasts 3:14 minutes and has no audio.



Open wmv version on a separate window.

Project Evaluation

We just finished our Project Evaluation report. The project's summative evaluation took place on the 19th of February and was directed by Neil Beagrie from Charles Beagrie Ltd. As part of this evaluation Neil ran an online survey with a sample of our interviewees and testers. The results were very positive. We chose people who knew about BRII and who had at least seen a demo or tested the Blue Pages. The reason for this was that we needed respondents who understood the basic concepts of BRII’s entity registry and RAD (Research Activity Data) and and were able to assess the potential benefits that sharing and re-using RAD could have for the University.

For the purpose of the survey we asked respondents to have a look at the working version of the Blue Pages. As this tool is still under development we have limited access only from Oxford networks. However I also compiled a series of screenshots for respondents who were not on campus. You can see the status of the Blue Pages six weeks ago.



and these are the results of the survey (click on + to zoom in):

Uncovering User Perceptions of Research Activity Data

I recently wrote a paper about the user testing of the Oxford Blue Pages. It has been published in Ariadne in their January 2010 issue. Here you have a link to the article in Ariadne www.ariadne.ac.uk/issue62/loureiroKoechlin The title is Uncovering User Perceptions of Research Activity Data. I chose that title to emphasise the aspect of the user testing which we think helped us the most to engage and understand our users. Beside improving software usability the user tests provided us with data which confirmed and complemented the findings of the Stakeholder Analysis. In this article I discuss the concepts of Usability, Perceived Ease of Use and Perceived Usefulness and provide some examples from feedback.

I have also deposited a copy of the article in the Oxford University Research Archive ORA here http://ora.ouls.ox.ac.uk/objects/uuid%3A1d61ea76-1a6a-4595-a011-52a8303c549b
You can also find the ORA record (with the full text) for the BRII Stakeholder Analysis here http://ora.ouls.ox.ac.uk/objects/uuid%3A1df69991-cd37-445b-a4c7-3573ce80c36e


Note: I have added a section titled "BRII Papers, Reports and Presentations" in the right column of this blog (scroll down a bit.) It contains links to relevant outcomes produced by BRIIl.

Blue Pages - User Tests 3

We have just started a third round of user tests of the Oxford Blue Pages. This time tests focus on research collaborations. Collaborations can happen in projects, when writing (books, articles, etc,) or they can be informal exchanges of knowledge. The Blue Pages will display the collaborators of researchers as far as data is available. Names of collaborators will be extracted from publications, project websites and personal websites. See screenshot below.

Users will be able to change the grouping of data between two views. In the example below collaborations are organised by people. When the user expands one person the Blue Pages will show the nature of that collaboration, which in the case of the example is one academic article and that person being listed in the researcher's website. When the user clicks on the group collaborations button the Blue Pages will organise data by sets of collaborations, for example by research projects or academic articles which when expanded will show their paticipants. Whenever data is available within the Blue Pages names of collaborators and research outcomes will have links to their corresponding profiles.

Note: data is real but connections between people were made up for this screenshot. Click to see full picture.

We are also testing how useful connections between data objects are to users. Data objects for us are People (researchers,) Research Activities (e.g. projects), Academic Units (e.g. departments) and Funders. The Blue Pages can connect all of these between them. For example: departments with people, people with people, people with research projects.

The Blue Pages also use research keywords to find and connect objects. For example search for research projects under a subject field, or find projects in similar areas to the one displayed on screen. Although some the of mentioned examples have not been implemented yet we ask testers how they would like to access and see these data.

Blue Pages - User Tests 2

After 3 intense weeks Anusha, Monica and I have completed a second round of user tests. This time no mock ups but with the real thing, which of course is the Blue Pages v0.00 :) We did 15 tests with a selection of academics, students, administrators and one research facilitator. Again feedback has proved to be an eye opener, and is helping us on our design.

Tests lasted an average of 45 minutes each and took place mostly in our testers offices. Monica and I (and sometimes Anusha and I) visited them, so lots of buses and taxis. We used our laptop which was setup to access our server. Testers were told that we were using a work in progress version of the software and that they should expect some errors. However we had very little. We used samples of data from 4 areas of the University: Faculty of Philosophy, Department of Cardiovascular Science, Department of Phisiology Anatomy and Genetics and Gray Institute for Radiation Oncology and Biology. That is aproximately 250 people. Anusha had previously added data representing the organisational structure of the University, which includes divisions, departments, faculties, institutes, centres of study and research groups.

Our questionnaire had a similar structure to the one we used in the first round of tests. See below. We altered some of the questions we had in the first 2 or 3 tests as Anusha was adding more data. We also felt the need for repeating the same first 2 questions we had in the first round. These questions tried to capture the testers perceptions of the site struture and look and feel. As we were using the real web-based version of the Blue Pages the look and feel was completely different from the one we had in the mock up version.

v2.0

Introductory Questions 1.Ask tester to look at site’s Homepage for 10 seconds. From looking at this site, what kinds of information do you think you could get from this site? 2.Who do you think this site is designed for? Why? Tasks 3.Find Prof AAA BBB’s profile and display it in the screen. How would you find another person in the Blue Pages who does work in the same department? 4.Obtain a list of/display CCC DDD’s publications 5.Email a list of researchers in the Philosophy department to a colleague. 6.What you would do if you find a misspelling or missing information? 7.You just setup your new project’s website and would like it to be listed in the Blue pages. What do you need to do? 8.Find a list of research projects (activities) under the keyword "cardiology" Briefing 9. Do you think the information displayed in the Blue Pages is useful for Research Work? Would you trust the information provided in this site? 10.What kinds of uses of the Blue Pages can you think of which are related to your work and or studies? 11.Do you think it would be good for you to be in the Blue Pages? What kinds of benefits can you think of? (only for academics) 12.What changes and or additions would you suggest in the Blue Pages?

Yesterday, Anusha, Monica and I met to discuss the feedback. They ended up with a long to-do list. I was able to able to dig out a few outcomes, suggestions for features to add or complement the Blue Pages which are not necessarily within BRII's scope:

• Visual (or textual?) representation of networks of people connected by their collaborations and research interests. Anusha and Monica got excited by this as this could be an interesting challenge from the technical point of view.
  
• A time line of people (and maybe projects) - this involves the changes of people's profiles across time. In this way we can know about their previous roles as they rotate across areas and previous research interests as they progress in their careers. This will also include the display of information of people who have left Oxford. One aspect of this is already covered by the display of research outcomes (e.g. publications) as each item has attached to it a date of production.
  
• An extensive work on research subject ontologies - a sort of backbone of subject matters which Anusha could use to find and highlight keywords across descriptions (as in free text.) These keywords can then be used to interconnect people and activities, enabling the discovery of hidden connections.
• Inclusion of information about funders' calls for funding applications - one of the outcomes of these tests is that academics would mostly use the Blue Pages when they are thinking on starting something: projects and/or collaborations. This is when they would like to find relevant people. However as they are in that stage they would welcome information about possible sources of funding.

BRII Inside O.R.

This week I got a copy of the Inside O.R. magazine - November issue with a short article about BRII. I wrote this article after the OR51 conference (Operational Research) where I presented a paper called Making Sense of Research Activity Information in the Information Systems stream.




Thanks to Rajan Anketell, Inside O.R. editor.

Click here to download whole issue.

Developing the Bluepages

The BRII project is very much on track.

I finished the Stakeholder and User needs Analysis. Will be ready for distribution soon.

Ben has been working on the development of the foundations of the Research Information Infrastructure (RII). He has also created an online store for the vocabularies resulting from the BRII project. The vocab site has been given a single, central location in Oxford and now consitutes an institutional service.

Anusha is working on the harvesting of data for the RII. At the moment she is dealing with data about the structure of the University (a very complicated task!) which should form the basis of the RII. I will meet with her soon to talk about this as we have to document data and process audit. She has also promised to give me a technical overview of her work so far so I can post it in this blog.

Monica is working on a mock-up of the Bluepages, one of the proposed outputs of the BRII project. As part of the development process we are planning a series of user-test sessions with a selection of users from the University. Monica is following an iterative approach to development. She will produce a first, basic version of the Bluepages which will be tested by users. She will then work on their feedback to produce an improved version. We will do a few rounds of testing until users and Monica feel satisfied with the product.

This is what Monica says:

After a few meetings and brainstorming with the team, many layout considerations were made and an initial timeline for the Bluepages was created. We were able to gather enough information about what data to include and the website's behaviour after data harvesting. Now we have a good idea about the direction we want to go with the Bluepages.

The outcomes of the meeting include a definition of the look and feel of the website and a list of many possible features that could be included depending on time of development and resources available. I will create a mockup with a few screens that will represent the initial Browse by People and Profiles section of the website. (See pictures below.)

Click on pictures to view larger versions.

The following is a list of layout considerations and ideas:

General design ideas

• Have a 'web 2.0' style of page, where simplicity and boldness will capture user's attention
• Introduction paragraphs that tells what the website does (purpose) before we show how and why is good for the user.
• attention map: what is more important bolder/bigger, less important/less used items smaller
• fewer columns, 2 or 3 max

Gadgets/widgets/functionalities

• "share this" button including email friend
• download as PDF
• "shopping basket" to add profiles/links to be printed later (as a list of 'things' the user wants to keep)
• enable links in profiles to: people with same interests (clicking in keywords), people in same projects, funder's page, project's page, etc (we have to decide which one links to what)
• collaborators sections in profiles (articles written together, include projects too?)
• RSS?

Homepage

• logos of University, BRII, JISC (bottom right), any other?
• quick search on top
• identification that is a BETA with a badge/star
• different tabs for different areas: Browse (active from the homepage), Advanced search, How to contribute, Add your data, About, Help (others???)
• Welcome blurb and max of 3 short paragraphs that can say WHAT IS THIS ABOUT, HOW TO USE and BENEFITS/Why
• "Browse by" area with buttons (icons too maybe?)
• Research Activity button will have a 'tooltip' type of baloon that will show what includes in Research activity, we can also add this info on Help page.
• "slider" showing groundbreaking research
• slider with random profiles ???

People's profiles

• Name
• Affiliations
• Roles
• Research topics/interests/keywords
• Themes - controlled vocab/top level
• Projects or other Research Activities -> link to the projects page inside blue pages
• Funder -> link to funder page inside blue pages
• Websites: department/college/personal page in each of the previous/"personal" personal page
• Publications -> link to publications if available
• Source of information
• Report problem/correct your information button

Funder

• Project
• People
• Department
• Subject

We estimate to have this finished in the next few weeks. This initial design will serve for the first tests and useful to evaluate the speed at which I can develop the next items on the list.

About the user tests

User tests will start as soon as Monica gives us the green light. We would like to gather small groups in sessions that should last between 30 to 45 minutes. When possible we would have the sessions in our base (Osney One). However we are prepared to visit our testers in their own office. Testers will be asked to perform a particular set of tasks and give us their opinion. We will gather feedback from the users by observing their behaviour, from user notes and comments in the debriefing. Group testing and refinements will repeat until a good representation of the future users could test the website and the corrections stop/are reduced to a minimum

Now I am looking for testers. So if you work in the University of Oxford AND you are curious and want to see what we are doing AND/OR you are interested in participating in a user test please email me!

Note: you may have noticed that I am using the word Bluepages in this post instead of Blue Pages, we are thinking on what would be the best way to call them. What do you think?

BRII Blue Pages

Creating Oxford University Blue Pages is one of the objectives of the BRII Project (see http://brii.bodleian.ox.ac.uk.) The Blue Pages will be a directory of expertise. Through them you will be able to search for research activities and experts in Oxford University. As I see it, it will be a sort of mega tool allowing you to see what is in Oxford's Research Information Infrastructure from different angles, at different depths, and through a variety of search options.

This morning I had a meeting with Sally and Monica to start sketching a mock up. We made a few (technical) design decisions (which I could probably be able to comment on in a few weeks time, if Monica helps me!) After the meeting I ended up with the feeling that these Blue Pages are not going to be just some “Blue Pages”. Blue Pages is a nice name. It relates to the white and yellow pages where you were able to find telephone numbers or addresses of people and businesses. White and yellow pages were very useful in their time. However, those are old concepts.

BRII’s Blue Pages will be more than a list of something. Try to picture the Research Information Infrastructure as a sphere containing all information we harvest about research activities in Oxford. (The multidimensional cube analogy I used a few posts ago still applies. I used cubes to describe research activity objects.) Within that sphere you have information collected from different sources in Oxford and now connected by using semantic web technologies. There will be so much information that you will need something powerful to start exploring it and finding what you want. You will need to hold that sphere and turn it around as you wish until you find what you want (the way you need it) or at least until you find a starting point from where you can start digging (angle and depth).

The blue pages will do all that for you.

As I have mentioned in previous posts, the outcomes of the interviews I have done emphasise the need to provide access to information from different angles (are you interested in publications, in current activities, in collaborations?) and depths: broad perspectives – narrow perspectives, zoom in – zoom out. So for example if you type Pathology, the Blue Pages could give you a list of research sub fields, or research projects under pathology. If you are not interested in details, that list would give you an overview of the sort of research done there. If you are interested in detail, you can search for specific names, or click on the links resulting from your Pathology search.

At departmental and University levels there are needs for discovery of hidden connections which may lead to future collaborations. These needs may emerge from sponsors wanting to fund original, interdisciplinary research, or from the departments’ research strategies which see gaps or weaknesses in their current research. The Blue pages will be able to connect information offered at any level to whatever other information is available in the infrastructure. A straight forward example would be the Research Topics of Interest in a Researcher's Profile. With one click on a profile you will be able to find other people or projects related to the same topics.

Note: the above are just ideas. Not sure if all them will be implemented as I presented them or if we will need some changes.

--------------------
On Tuesday we had our Assembly. The event turned out to be very interesting and everyone left the place happy. I will post on this in 1 or 2 weeks after I finish a webpage with all the presentations, comments and outcomes from the meeting.