What is going on in BRII

Since September we have been carrying out user tests of the Oxford Blue pages. These tests have been very useful to improve the Blue Pages usability and to get the perspectives of potential users. These perspectives were not limited to the Blue Pages but to the BRII registry. After seeing the Blue Pages testers asked about the sources of data and the processes of updating these in the registry. Some testers raised issues about accuracy and validity of data, others asked about data coverage (within and outside Oxford) and the processes to update or synchronize these data with changes in the original sources and scope. Regarding this last one, scope, some academics suggested harvesting their personal websites and blogs (i.e., to not only harvest data from official departmental sources.) They said personal sites "reflect more their career and research experience as they are not limited to their activities in Oxford but relate previous experiences.” Other testers suggested ways to organising and presenting information. For example how to present a list of publications. Some people were interested in the abstracts while some others just wanted basic information such as the title, year and author.

All these feedback and questions are food for thought for us. User tests are an ongoing activity. Although were are not doing as many as we did last year we are still running them whenever we get an opportunity.

In addition to the user tests, we have also run some dissemination activities, such as meetings and demos with interested parties. We recently had a meeting with some people in the Social Sciences division. They are working on a divisional system which will cover areas such as teaching, HR and research within their division. Most of our conversation focused on the nature of the data harvested. We discussed if the aggregation of data in the registry and the Blue Pages could change their original nature and intentions. We also talked about how the Blue Pages will make these data, which is already publicly available, more visible. The Blue Pages will enhance all the positive aspects of research within the University. However it will also make some data problems more obvious. Social Sciences participants realised that some inconsistencies could be made more obvious in the Blue Pages. As for example, wrong names or affiliations. Correcting these inconsistencies is of course not a task for BRII but for the people responsible of the original sources. However BRII can potentially help them by letting them know where these problems are, e.g. Blue Pages users can report errors in data.

We have also been working on a new Graduate Opportunities website for the Medical Sciences division. This is a second planned output of BRII. Anne Bowtell is in charge of this. She is using the BRII API and data from the registry to populate this website. These are the data Anusha is harvesting from Medical Sciences sources which of course will also be available from the Blue Pages.

On a different matter, in a couple of weeks I will attend the next JISC Innovation Exchange event. This will take place in Birmingham. One of the main events will be a kind of trade fair where we will have to sell to and buy products from other JISC Institutional Innovation projects. This sounds exciting! Now I need to think on how to sell our BRII products and need to get some updates of the status of other projects so I know in advance what will be on offer then.

JISC MEAoT Assembly

Yesterday I attended a JISC assembly organised by the Modular e-Administration of Teaching MEAoT Project run by CARET - Cambridge University. The aim of the event was to explore ways in which stakeholders and other parties can be encouraged to adopt tools developed in JISC projects. Below you can see my presentation:



The second slide briefly explains the nature of BRII and the Entity Registry we are creating. The registry is a container of Research Activity Data and it is these data which has been semantizised that we are trying to sell around the University. BRII is doing this at two levels. First, being the registry an abstract concept, users find it difficult to understand. Therefore BRII needs to provide tangible examples that use its data. Slide 3 shows this as level 1: providing practical examples of data use, one of which is the Blue Pages. However, as the Blue Pages is a new piece of software BRII needs to find ways to promote it across its user base. Through user tests (level2 ) on the Blue Pages we make sure it fits the needs of our potential users, but at the same time we sell the idea of Research Activity Data being available in our registry for another users.

There were other presentations of course. The MEAoT people have written a summary about the event in their blog:

The Centre for Applied Research in Educational Technologies (CARET) in Cambridge University hosted a JISC assembly for the Modular e-Administration of Teaching (MEAoT) project on Thursday, 10 December 2009.

The list of participants included:

Richard Prager, MEAoT project PI, Cambridge University Engineering Department (CUED)
Laura James, CARET
Anne Clarke, MEAoT project, CARET and CUED
Guy Chisholm, MEAoT project, CARET
Avi Naim, MEAoT project, CARET
Cecilia Loureiro-Koechlin, JISC BRII project, Oxford University
Bridget Taylor, DAISY project, Oxford University
Jenny Mackness, JISC liaison for MEAoT
Rachel Tuley, Teaching Administrator, CUED
Jen Pollard, Computer Officer, Cambridge University English Department
Carmen Neagoe, Teaching Administrator, Cambridge University Judge Business School
Helen Marshall, Teaching Administrator, Cambridge University Physics Department
David Goode, Computer Officer, Cambridge University, Department of Divinity

The programme was designed to make the meeting a pleasant event. It started with a historical overview of the project (by Prof. Prager). This was immediately followed by lunch at a nearby restaurant, which helped break the ice and make everybody more comfortable.

Read more here: http://modular-e-admin.blogspot.com/2009/12/modular-e-administration-of-teaching.html

BRII's Entity Store

In our internal meetings we use the term Registry or Entity Registry to refer to the Research Information Infrastructure. Wanting to know a bit more about the meaning and technological features of such kind of store I asked Anusha. She gave me the following lecture:

Cecilia: What is an Entity Registry and what is the difference with conventional stores?

Anusha: Lets first list our entities, so we are clear about what we mean when talking about an entity - person, organisational unit, publications (journal articles, books, chapters...), funder information and research activity info.
Now, the main difference between an entity store and conventional store (typically databases) is that in a conventional store, the columns relate to attributes of each entity and it needs to be created at the time of creation of the database. So how is this a problem? Well,

1. We need to think up of all the attributes relating to the entity (example: all the attributes that make up a person) at the time of creation and we cannot change our structure very easily later on. (cost of change rises exponentially with time)
2. All the people have to follow the same structure. So you cannot account for variations very easily.
3. You cannot accomodate all of the multiple relationships easily. Eg - person belongs to multiple departments/colleges, person has multiple roles, person has multiple titles, person has multiple names

When aggregating data from multiple sources, points 1, 2 and 3 are crucial. To design a common structure having all the attributes we are going to come across in the future is going to be near impossible.

Cecilia: This sounds obviously relevant to BRII because we are collecting information from all kinds of sources around the University, and most importantly because we do not have control over the content or format of that information.

But beyond these technological advantages, what are the benefits that this way of organising data brings to scholars?

Anusha: The extra benefit to visitors is that we can show them multiple and different relationships between entities very easily (like collaborators, linking funders, research activity, people, departments in whatever way we want).
Or the entity registry will be transparent to them, as they will only see things like the Blue Pages.
The entity registry will be transparent or open to all to access our data. If they are interested in the data, they can build tools to analyse the data in whatever they want (we haven't yet done this, but will be doing so).
With a service like the Blue Pages, for a keen observer, the entity registry will be noticable. For other users no and rightfully so, as they need not know what's happening at the back (for example: we see nothing about how google does its work). They can however observe that some of them have a lot of information and are linked in multiple ways to other entities, while some others hardly have any information. The key thing is that the data can be linked very easily to other entities in multiple ways.

The power of the Blue Pages is mainly derived from three things

1. Quantity of data - The more we have the better
2. Variety - A one stop shop, visit one website rather than 10 different websites
3. The way we present our data and the deductions / analysis we perform on our data (like finding collaborators). If we can think of more deductions like this, it would be useful and make our web service more powerful.

The Blue Pages is just one way of displaying the data. We can do much more with the information, like create a graph of collaborations and areas of research across Oxford University for example.

Cecilia: and how do you see all this helping scholarly communication?

Anusha: scholarly communication is more than just Journals. Journals were and to some extent still are primary sources of communication but they aren't the only sources. We now have institutional repositories which are helping with this. Also, "scholarly" in scholarly communication does not refer to the people, but to the type of communication. So its anyone (not just scholars) communicating on a scholarly topic.

Cecilia: yes and I guess that having all these connections facilitates these communications.

Blue Pages - User Tests 3

We have just started a third round of user tests of the Oxford Blue Pages. This time tests focus on research collaborations. Collaborations can happen in projects, when writing (books, articles, etc,) or they can be informal exchanges of knowledge. The Blue Pages will display the collaborators of researchers as far as data is available. Names of collaborators will be extracted from publications, project websites and personal websites. See screenshot below.

Users will be able to change the grouping of data between two views. In the example below collaborations are organised by people. When the user expands one person the Blue Pages will show the nature of that collaboration, which in the case of the example is one academic article and that person being listed in the researcher's website. When the user clicks on the group collaborations button the Blue Pages will organise data by sets of collaborations, for example by research projects or academic articles which when expanded will show their paticipants. Whenever data is available within the Blue Pages names of collaborators and research outcomes will have links to their corresponding profiles.

Note: data is real but connections between people were made up for this screenshot. Click to see full picture.

We are also testing how useful connections between data objects are to users. Data objects for us are People (researchers,) Research Activities (e.g. projects), Academic Units (e.g. departments) and Funders. The Blue Pages can connect all of these between them. For example: departments with people, people with people, people with research projects.

The Blue Pages also use research keywords to find and connect objects. For example search for research projects under a subject field, or find projects in similar areas to the one displayed on screen. Although some the of mentioned examples have not been implemented yet we ask testers how they would like to access and see these data.

Blue Pages - User Tests 2

After 3 intense weeks Anusha, Monica and I have completed a second round of user tests. This time no mock ups but with the real thing, which of course is the Blue Pages v0.00 :) We did 15 tests with a selection of academics, students, administrators and one research facilitator. Again feedback has proved to be an eye opener, and is helping us on our design.

Tests lasted an average of 45 minutes each and took place mostly in our testers offices. Monica and I (and sometimes Anusha and I) visited them, so lots of buses and taxis. We used our laptop which was setup to access our server. Testers were told that we were using a work in progress version of the software and that they should expect some errors. However we had very little. We used samples of data from 4 areas of the University: Faculty of Philosophy, Department of Cardiovascular Science, Department of Phisiology Anatomy and Genetics and Gray Institute for Radiation Oncology and Biology. That is aproximately 250 people. Anusha had previously added data representing the organisational structure of the University, which includes divisions, departments, faculties, institutes, centres of study and research groups.

Our questionnaire had a similar structure to the one we used in the first round of tests. See below. We altered some of the questions we had in the first 2 or 3 tests as Anusha was adding more data. We also felt the need for repeating the same first 2 questions we had in the first round. These questions tried to capture the testers perceptions of the site struture and look and feel. As we were using the real web-based version of the Blue Pages the look and feel was completely different from the one we had in the mock up version.

v2.0

Introductory Questions 1.Ask tester to look at site’s Homepage for 10 seconds. From looking at this site, what kinds of information do you think you could get from this site? 2.Who do you think this site is designed for? Why? Tasks 3.Find Prof AAA BBB’s profile and display it in the screen. How would you find another person in the Blue Pages who does work in the same department? 4.Obtain a list of/display CCC DDD’s publications 5.Email a list of researchers in the Philosophy department to a colleague. 6.What you would do if you find a misspelling or missing information? 7.You just setup your new project’s website and would like it to be listed in the Blue pages. What do you need to do? 8.Find a list of research projects (activities) under the keyword "cardiology" Briefing 9. Do you think the information displayed in the Blue Pages is useful for Research Work? Would you trust the information provided in this site? 10.What kinds of uses of the Blue Pages can you think of which are related to your work and or studies? 11.Do you think it would be good for you to be in the Blue Pages? What kinds of benefits can you think of? (only for academics) 12.What changes and or additions would you suggest in the Blue Pages?

Yesterday, Anusha, Monica and I met to discuss the feedback. They ended up with a long to-do list. I was able to able to dig out a few outcomes, suggestions for features to add or complement the Blue Pages which are not necessarily within BRII's scope:

• Visual (or textual?) representation of networks of people connected by their collaborations and research interests. Anusha and Monica got excited by this as this could be an interesting challenge from the technical point of view.
  
• A time line of people (and maybe projects) - this involves the changes of people's profiles across time. In this way we can know about their previous roles as they rotate across areas and previous research interests as they progress in their careers. This will also include the display of information of people who have left Oxford. One aspect of this is already covered by the display of research outcomes (e.g. publications) as each item has attached to it a date of production.
  
• An extensive work on research subject ontologies - a sort of backbone of subject matters which Anusha could use to find and highlight keywords across descriptions (as in free text.) These keywords can then be used to interconnect people and activities, enabling the discovery of hidden connections.
• Inclusion of information about funders' calls for funding applications - one of the outcomes of these tests is that academics would mostly use the Blue Pages when they are thinking on starting something: projects and/or collaborations. This is when they would like to find relevant people. However as they are in that stage they would welcome information about possible sources of funding.

BRII Inside O.R.

This week I got a copy of the Inside O.R. magazine - November issue with a short article about BRII. I wrote this article after the OR51 conference (Operational Research) where I presented a paper called Making Sense of Research Activity Information in the Information Systems stream.




Thanks to Rajan Anketell, Inside O.R. editor.

Click here to download whole issue.

Institutional Repositories: acceptance and adoption

I have been reading some literature about institutional (digital) repositories. I am interested in the human and social issues surrounding their development and implementation as well as their embedding. I think this literature is very relevant to BRII as it reports experiences in similar implementations (not necessarily technically but conceptually) in similar kinds of institutions.

Technically speaking BRII is not building a digital repository but a Research Information Infrastructure (so a bit much broader in scope.) However the issues, technical and non-technical, surrounding implementation of institutional (digital) repositories apply. The reasons are:

• BRII has an institutional scope and it is situated within an academic context. It aims to collect information about research in all academic areas and its target audience is everyone within the University who carries out research-related activities.
• BRII deals with Research Activity Data: information about research. So, not information used and generated by research (as in datasets and publications) but descriptions of research (people and activities.) This kind of information is essential to facilitate scholarly communication.
• BRII core users are Researchers and Faculty: as content contributors of research descriptions and as users of the information deposited in the infrastructure.

Being now in the "development stage" of the project we are starting to think on how to make our products more relevant to our core users, so they understand them and use them. Of course to make users understand the Research Information Infrastructure we first need to understand our users. Our work in BRII should not be techocentric only but should expand to reaching out to our users, speaking their own language and observing them in their own working spaces. This can be a very difficult task as we are dealing with heterogeneous groups of academics and administrators, each one with different needs and perspectives (as per their research culture.)

Understanding our users is a crucial first step to achieving acceptance and adoption of the RII.

Research in the area of Institutional Repositories field has reported some issues concerning this.

In a study of implementation of an Institutional Repository (IR) at the University of Rochester, Foster and Gibbons (2005) report a "misalignment between the benefits and services of an IR with the actual needs and desires of faculty." They state that the benefits of Institutional Repositories are attractive only to the institutions which host the repositories. This is because Institutions (including Librarians and IT developers) see IRs as a way to facilitate access, reduce costs and improve efficiencies at collecting and managing information that is produced in the Institution. On the other hand academics see themselves as belonging to research communities (not institutions) and are not concerned about the proccesses of managing all that data they produce. This is, perhaps, because they see the problem as affecting others and not themselves. (Colbertt, 2009)

At the centre of this all is the Scholarly Communication crisis. In 2002 the Scholarly Publishing and Academic Resources Coalition (SPARC) (part of the association of Research Libraries (ARL)) stated that Institutional Repositories could help combat dissatisfactoin with the "monopolistic effects of the traditional and still pervasive journal publishing system" (Crow (2002) quoted in Maness et al (2008)). Rieger (2008) states that to combat this crisis, academic institutions aim at "reducing costs of producing and acquiring publications and gaining control of processes from commercial publishers." Seen from this angle, institutional repositories are good and sensible solutions. Attention therefore is devoted to technical efficiencies and control of the information produced within the institution. A consequence of this is a bias towards computer and librarian approaches to developing IRs, which neglect perspectives from relevant groups’ interests.

From BRIIs Stakeholder analysis we have learned that technical efficiencies, metadata, repositories are terms which are meaningless to academics. This concurs with Foster and Gibbons (2005) view. Seen from this technocentric perspective IRs do not provide benefits to academics. In addition having institutional labels suggests researchers that the IRs will support the needs of the institution and not their own individual needs. Academics are indeed a different crowd:

• Academics think in terms of reading, researching, writing and disseminating.
• Academics have strong ties with the people interested in their own field of research, or with whom they are interested to collaborate. Their geographical location is not important to them.
• Academics are interested in a relatively small subset of research information, that one of their own research field. They have acquired skills to search for and use that information for their own work.

BRII Stakeholder Analysis (Loureiro-Koechlin, 2009)

On the other hand implementers of repositories (IT and library practitioners) possess an institutional perspective, they are interested in developing new forms of scholarly communication, control costs and improve data manipulation.

An example of institutional perspective is the mapping of people and information under the umbrella of their institutions and departments. This is an obvious mismatch as researchers see themselves as belonging to their research communities. Other issues connected to the particular ways in which research is carried out (technical and ethical) also accentuate these differences. One example of this is the differences of methodologies and nature of data collected by natural scientists and social scientists. Carusi and Jirotka (2009) report ethical issues of archiving qualitative data in a digital format, such us consent, anonymity and privacy…They state that reusing research data for other purposes will go against research participants wishes and therefore researchers ideals.

Misalignments like the above are reasons why academics do not feel institutional repositories are relevant to their work. Therefore as Foster and Gibbons (2005) report there is a need to get to know researchers in their own space so we can develop tools which are useful to them as well as to the institution. This improves the chances of IRs to being accepted and adopted by their users.

One example is Maness et al's (2008) study on needs and goals of institutional repository. It reports results which contradicted initial assumptions by IR designers and decision makers. While designers and decision makers assumed users wanted an open-access archive for research outputs, the study found out that in reality users wanted a network to share learning and teaching materials, where collaborators could be identified and where their research could be promoted to institutional colleagues.

Carusi and Jirotka (2009) From data archive to ethical labyrinth. Qualitative Research.
Corbett (2009) The Crisis in Scholarly Communication, Part I: Understanding the Issues and Engaging Your Faculty. Technical Services Quarterly, vol 26 p.125-134.
Foster and Gibbons (2005) Understanding faculty to improve content recruitment for institutional repositories. D-Lib Magazine, Jan 2005.
Maness, Miaskiewickz and Sumner (2008) Using Personas to Understand the Needs and Goals of Institutional Repository Users. D-Lib Magazine, Sept-Oct 2008.
Rieger (2008) Opening up institutional repositories: Social Constitution of Innovation in Scholarly communication. Journal of Electronic Publishing.