evidence » virtually a librarian

towards a research-led profession (or, how to become a practitioner-researcher)

ebl, evaluation, evidence, research No Responses »

Feb 212012

At LibraryCamp Australia 2012, there was a discussion about building a culture of research in library and information practice. Lots of really practical ideas were shared during the session, and I wanted to follow up here with a few of my ideas on how professionals can get involved in doing research. So here goes!

Every project is an opportunity for research

Starting out a new project with clear, measurable objectives is not only good project management, but it also provides you with an opportunity to publish. Setting objectives and putting strategies in place to collect the data you need to measure achievement of those objectives sets you up to evaluate the success of your project once you hit the finish line. Publishing on the results of your project and your evaluation of its success is a great way to help build the evidence base.

Collaborate

If you’re new to research (or even if you’re not!), collaborating with your colleagues can be a great way to break down some of the fear you might have about doing research and getting published. Collaborate within your institution and beyond it. Connect with people that have similar professional interests and just start talking about what interests you. You’ll be surprised by how many ideas you come up with. Also consider collaborating with academics – we’re always looking for opportunities to work with practitioners on real world projects. Your professional expertise and their methodological expertise will complement each other.

Get a mentor

There are many, many active practitioner-researchers in the library and information professions. Find someone whose work you admire and ask them if they’ll mentor you through the process of undertaking your first research project and looking for publication opportunities. You might already have a professional mentor, but your research mentor is probably a different person: someone who has experience in designing and seeing through research projects and getting published.

Can’t find a research mentor? Drop a comment here and I’ll see if I can help you.

Look at alternative channels

Journals and conferences are probably the two most obvious places to publish the results of your research, but have you thought beyond that? Blogging is a great way to document your research journey and a really valuable way to disseminate your research. Academic publishing timelines are laggy and can mean that your cutting edge project isn’t so cutting edge any more by the time your journal article gets published. A series of blog posts on your findings can help you disseminate any time-critical aspects of your research as you progress.

Get lots of traction out of a single project

Remember, one project does not necessarily equal one publication. You can get much more traction than that out of your research! By publishing some of your findings on a blog, for example, you don’t preclude yourself from publishing about that research in a journal or presenting at a conference. There are always a variety of angles you can use for publications about a single research project. A project I led last year has so far resulted in three publications, and I’ll be publishing again out of this project this year (more than once). Each of these publications focuses on a different aspect of the research and was designed for a different audience. Even after five or so publications, there will still be data (and different views of the data) that I haven’t published on.

But that’s okay, because the other thing you need to remember about research is your data has a longer shelf life than you might imagine. When I first made the switch to academia, I was always in a rush to publish while my data was fresh, but I learned very quickly that I actually have quite a big window of time to disseminate the results of my research. Don’t publish one surface-level view of your research in your haste to get your findings out there. Instead, take a narrower but deeper view of your research in a series of publications over a longer period of time. You’ll do your research more justice this way and your publications will be more useful to others.

Start with a literature review

We’re librarians. We know how important it is to review the literature before we start a project. But instead of just writing up a short literature review in a publication on your findings, consider writing a more holistic literature review before you get started on your research, and publish it independently. Our professional literature could definitely use more of this type of publication. The process will help you focus your research on the gaps in the literature and your review will benefit other researchers. You might consider, for example, publishing a literature review in the new review section of the EBLIP journal.

Do a research degree

If you’re keen to get involved in research, you might like to consider getting a research degree. As librarians and information professionals move into research support, research training is going to be increasingly relevant for practitioners. At VALA a couple of weeks back, research support was probably the hottest topic on the program. Increasingly, information professionals are taking a really practical approach to research support and contributing to research in really tangible ways, through activities like literature searching, grant writing, and advising on publication opportunities.

If your professional qualification is a Graduate Diploma, you might be thinking about upgrading to a Masters. Instead of upgrading by completing more coursework, consider a research degree. The benefits of this in terms of your skills and knowledge are obvious, but the benefits for your wallet shouldn’t be discounted. Masters by research programs are fee-free.

We have an awesome research community at QUT… Come and play with us!

Just do it!

Stop thinking about it, and just do it! You’ve probably got at least one project on the go right now that you could be publishing about. So do it! The Research Applications in Information and Library Studies seminar is a perfect venue to publish research in practice, because the focus of this event is on connecting researchers and practitioners and fostering a culture of research in the profession.

Abstract submission for RAILS8 closes this Thursday, 23 February. You’ve got plenty of time between now and then to prepare and submit a 300 word abstract! The Organising Committee eagerly await your contribution!

redesigning evidence based practice for wicked problem solving

conference presentations, ebl, evidence No Responses »

Jul 052011

I spent the better part of last week at the Sixth International Evidence Based Library and Information Practice Conference (EBLIP6) in Manchester, where I presented a paper co-authored with the wonderful Zaana Howard. Our paper was a conceptual one – it presented an untested model for evidence based practice (EBP), which is essentially a hybrid of design thinking and EBP models. This paper was born out of a discussion (or several discussions really) that Zaana and I had about the instances where EBP doesn’t cut it as a model for dealing with a problem – for example, situations where there is no existing literature, or situations that require innovation. We also had some doubts about particular aspects of the EBP model, including its reliance on published literature.

This presentation was a lot of fun to work on, not the least because it gave me an opportunity to get more familiar with Zaana’s field of design thinking, which I think has a lot to offer information practitioners. As always, it was a pleasure to work with Zaana, too!

Our presentation was well received – we got interesting questions during the session, and were pleasantly surprised to win both the delegates’ choice and committee’s choice best paper awards for the conference.

You can see our slides and references over at my other site, katedavis.info. We’d love to hear your thoughts on the hybrid model, too – we plan to write this up as an article, so all feedback is very welcome. You can either leave comments here or email us (davis.kate@gmail.com – I’ll forward feedback to Zaana).

libraries, saas, the cloud and more: participate in our survey

evidence, library2.0 6 Responses »

Oct 202009

A colleague and I have been spending a lot of time lately thinking and chatting about libraries’ use of SaaS solutions and free, third party proprietry cloud-based services to host their systems and software. We’ve thought about it so much that we decided to take that one step further and do a spot of research.

At the VALA 2010 conference (one of Australia’s leading ITish conferences for librarians), we’re presenting a paper called ICT as core business: will we prosper or drown? Here’s the abstract:

Information Communications Technology (ICT) is core business for libraries. Every day, libraries deliver ICT services to their customers, in the form of public access computers, wireless Internet connectivity and technology training programs. We increasingly rely on the Internet and the World Wide Web as a core service and collection delivery channel. And ICT is the single most important set of tools in allowing us to carry out ‘traditional’ library functions such as collection management and circulation.

At the same time as we are seeing our dependence on ICT reach new heights, we are also seeing IT departments locking down and standardising organisational ICT environments in response to an increasing need for control in order to meet efficiency and governance requirements. Often, these efforts occur in response to the organisational needs of the parent organisations to which libraries belong, not in response to the library’s needs. The move towards shared service models for ICT services means that libraries have to compete for ICT services and support, and as a result, do not always obtain the needed support.

Libraries are surrounded by tools and systems that provide new and exciting options for service delivery, but that require a move away from the traditional ICT model. The authors’ conversations with colleagues throughout the industry, commentary in the biblioblogosphere, and even the library literature, suggest that with the proliferation of these new tools and their uptake by libraries, there has been a disconnect between some libraries and their IT support groups. Many libraries are adopting, or at least investigating new models, including Software as a Service (SaaS) options for major systems, cloud computing for hosting of services and resources, and open source systems and software solutions. How does this fit within the broader ICT framework of parent organisations? Often, it simply doesn’t fit at all.

Why is it that libraries have chosen to position services in the cloud, to move core systems to SaaS environments, and to seek out open source alternatives to proprietary software and systems? This paper considers whether the adoption of these tools and environments by libraries has occurred as a result of a lack of appropriate and necessary ICT solutions and support within our corporate ICT environments. Did the disconnect cause libraries to seek out new models and tools, or did our adoption of the new models and tools cause the disconnect?

Given that libraries are adopting strategies and implementing services in this new ICT sphere, what do libraries need to consider in order to ensure sustainability, supportability, and ultimately, success? As the end users become controllers of ICT – as libraries implement ICT solutions quite independently of the ICT groups within their broader organisations – is it enough for library staff to be able to depend on the intuitiveness of the tools, or are there skill sets – ICT, writing, publishing, design and other skill sets – required by library staff, skills that have perhaps been taken for granted as being provided by the organisational ICT group?

If you work with IT in libraries, if you adminstrate a library management system, if you’ve ever implemented a blog or wiki or Facebook page for your library, then we need your input to get a clear understanding of why libraries are looking for alternative models for system and software hosting, and what that means for skill requirements.

If you’ve got 15 minutes to spare, we’d really appreciate you taking the time to complete our survey.

Walt Crawford's Public Library Blogs: essential reading if your library is (even thinking about) blogging

evaluation, evidence 2 Responses »

Dec 132008

This weekend, I’m working on the bordering-on-mythological paper on mpow’s blog pilot. To that end, I’ve been trawling the interwebs looking for blog posts, articles… anything documenting libraries’ and the corporate world’s strategies for evaluating the success or otherwise of blogging projects. I’ve been trawling for a while, but I live in hope that it’s just my search skills letting me down, and I’m going to miraculously find the very article I need at the eleventh hour.

As I’ve lamented earlier, libraries are not publicly documenting their evaluations of blogging projects, which is a problem because it makes benchmarking near impossible. Sure, you can still come up with a bunch of metrics and work out a number for each, but how do you know if the number you’re getting it good or bad?

Luckily, through a serendipitous Twitter experience, I’ve managed to track down a couple of people who were willing to share their data. But I need more. More, I say!

Enter Walt Crawford’s Public Library Blogs: 252 Examples. I have to confess, I’ve been meaning to buy and read this book for ages. I finally did buy and read it today, and I wish I’d read it a while back. If you’re responsible for a blogging project, you need to read this study. Especially useful is the grouping of the examples by population served – this allows you to compare your library’s blog’s performance for key metrics against like-sized libraries. I think it’s also a useful tool in setting realistic expectations, especially when it comes to converstatinal intensity (or the number of comments libraries tend to get per post). I wish I’d bought this and shared it with our staff ahead of the pilot.

While Crawford didn’t have access to usage stats, this is still an incredibly useful book.

I do wish, though, that someone would go begging to the library community at large to supply site visits, page views, links clicked and a few other metrics besides, so that we could have a ginormous study alla Walt Crawford’s that includes the kind of statistics he unfortunately didn’t have access to. PhD thesis, anyone? Even to do it for a grouping of libraries (say, public libraries serving populations between 400,000 and 500,000 – obviously, my reasons are entirely altruistic) would be incredibly useful, and you could extrapolate for libraries of different sizes.

As an aside, I love that I can buy a book online and be reading it within two minutes. I wonder if publishers will ever sort out the DRM debate and get pricing right on eBooks so we can do this with anything we might want to read.

Right, enough procrastinating.

a blink, a metric and some thinking out loud

evaluation, evidence 4 Responses »

Nov 092008

So, I blinked… and suddenly six months went by without a peep on this blog. Life has been doing what it does best – barreling along while I try to keep up. Where have I been? Right here, but buried under a pile of projects, a gaggle of committees, and a series of attempts at getting non-library, non-technology related hobbies.

Life’s not easing up but blogging has been on my mind lately, in more ways than one.

In my last post, I raised some questions measuring the success or otherwise of 2.0ish projects. I’m still thinking about this stuff – constantly. The time for me to evaluate mpow’s new blogging project is rapidly approaching, and I’m starting to look in earnest for literature on other libraries’ evaluations of similar projects. I’m still not turning up a lot.

I’m spending a fair bit of time thinking about what success for a project like this looks like. It’s a difficult thing to conceptualise. And obviously it’s something that needs to be conceptualised before you can figure out what sort of data you need to measure that success.

Of course, success for this project will be measured against our aims and objectives for the project – some of which relate to topics quite apart from the level of usage the blog has garnered, including aims like trialling blogging as a service delivery platform, and providing staff with an opportunity to get familiar and comfortable with blogging in the public domain (and with the technology itself).

But what other things should we be looking at? What, in general terms, makes a blog successful?

Conversational intensity

This is something I think a lot of bloggers get hung up on, so it gets its own sub heading. To what extent is success in blogging about “conversational intensity”?

We’re not getting a great deal of comments (I’ve got a theory about why that is, which I’ll probably blog about later), and I’m not particularly phased by that at this point. I had a chat with a colleague about the appropriateness of using blogs without being too concerned about generating conversation. She indicated she thought that a blog without multi-way conversation (ie with little commenting) misses the point of blogging. Her feeling is that conversation is a fundamental element of blogging.

I think I agree, to a certain extent, but I’m not convinced that blogs that exist without active commenting don’t have their own role to play. After all, we know that there are lots of different types of participants in this participatory web: consumers of information; occasional content producers (commenters); active content producers; and so on. (And this doesn’t even take into account the idea of using a blog as a CMS of sorts – people do great stuff with WordPress-driven websites. But that’s a little different.) Does it really matter if you don’t get a whole host of comments? Is there a ratio of comments to page views* that indicates a blog is successful in facilitating conversation?

In my opinion, level of conversation is a measure you should get hung up on only if it’s a primary aim for your blogging project.

Attributes of a successful blog

So, if the success of a blog does not hinge on conversational intensity, on what does it hinge (other than the blog’s aims and objectives)?

According to Asterisk, a successful blog is:

Well written
Frequently updated
Consistent
Open
Responsive
Well designed
Aware of its audience
Varied in topic
Personal
Thick skinned
Honest
Accountable
Funny

Are these the kind of success measures against which libraries should be assessing their blogs? Are subjective measures like these valid? And how do we measure against them? These measures do appeal to me. Or some of them do – some are obviously personal preference things, and dependent on the type of blog (like funniness, for eg) but others could be useful.

An obvious thing to measure is readership – subscribers, site visits, post views and so forth. But how do you decide whether the level of readership is enough to mean success? And, in the case of our project, whether the level of readership is enough to warrant a transition from pilot to permanent service? Do we work with a ratio of site views for the blog versus site views for the library’s website? And if so, do we compare to page views for the library’s home page, or do we look at page views for the young people’s page on our website and compare to that? Do we look at site views for the blog versus population in our region for the target audience?

Is their any validity in looking at participation in polls? What if we created polls for the express purpose of getting a feel for the number of people who might participate? What would that tell us?

Clearly, my thoughts on evaluating blogging projects are still fairly nebulous, and I know I need to do some research outside the library field to see how (or even if) other industries are evaluating the success of customer facing, service oriented blogs. But I do want to here from other library-types on this, and it appears I aint gonna find what I need in the literature.

So now that that I’ve just suggested that conversation may not mean much in an assessment of a blog’s success, I’m going to try and start one. Tell me, readers (if there are any of you left, after my six month hiatus): what do you think the markers of a successful library-land blog might be? What are the attributes of a successful blog? If you were evaluating a blogging project, what data would you be collecting and what would you be comparing to? How would you decide if your blog is a success?

* Conversational intensity is often measured by dividing total comments by total posts, but I think another useful metric would be to get a feel for the number of visitors who feel compelled to get involved in a conversation.

do we want to know whether library blogs are succeeding in the big, bad web world?

ebl, evidence, library2.0, soapbox 3 Responses »

Mar 142008

There’s been plenty of talk around the Library 2.0 theme on the idea of evaluation or assessment. At Information Wants to be Free, Meredith Farkas says what she wanted to see come out of Library 2.0 was a greater focus on assessment. I certainly want to see libraries have a greater focus on assessment, too, and I want to see them publishing about it. (Particularly public libraries. We just don’t publish enough.)

Why aren’t we (libraries in general) publishing about the success (or failure) of our 2.0 projects? Why is there virtually no data to be found that quantifies some of the outcomes of 2.0 projects? We’ve been on this 2.0 bandwagon long enough for studies and assessments and evaluations to have been undertaken. For a movement that’s intrinsically tied up with quick publishing channels like blogs and wikis, it seems strange that there is a real dearth of published studies on 2.0 projects. Why is that?

Walt Crawford had this to say in a recent post on his two blog survey books:

Maybe there’s a clear desire not to know how library blogs are doing in the real world, other than a few cherry-picked examples. I’d like to think that’s not the case. It would be unprofessional to tell people about how wonderful library blogs are, and encourage them to create such blogs, without giving them honest and broad-ranging information on what’s actually happening with such blogs.

I’d like to think that’s not the case, too. But I wonder. I wonder a few things:

Is the lack of publishing indicative of a lack of success? (And a fear of talking about it?)
Is the lack of publishing indicative of a perceived lack of success, a perception that might be formed because we’re not collecting the right data? (eg. How are we measuring ROI? Do we just count comments on blog posts? Or do we look at exit links, time spent on the page, holds on titles blogged about, impact on online resource usage stats…? I certainly hope all of these metrics and more are informing libraries’ evaluations of their blogs, because if we’re just relying on comments to measure user engagement, then we’re not seeing the full picture.)
Is the lack of publishing indicative of a lack of evaluation? (And if so, why aren’t we evaluating? Because we don’t know how? Because we don’t have time? Because we don’t want to know?)
Or, is it just that we’re not publishing about our evaluations?

I’ve got a blogging project in the pipeline at mpow. It’s germinating quite slowly, because I want to see it well planned. We want a well planned implementation, but also a well planned, multi-faceted evaluation. If it works, I want to know about it, and I want us to be able to reflect on what we did and make links to what worked. If it doesn’t work, I want to know about it just as much (if not more), because I want to be able to reflect on what we did, look for ways we could improve, and ultimately, pull the pin if that’s what we need to do.

Blogs (and all things shiny and 2.0) are just great. They’re fun for staff to work on, and have huge potential to engage our users. But none of us have time to run services that don’t work. If we don’t evaluate, we have no ability to know whether

We know that “because we always did it that way” is not a good reason to keep doing the things we’ve always done, whether they work or not. But neither should a failure to evaluate be the reason we keep on keeping on with our 2.0 services.

If you have evaluated your 2.0 service, publish about it! And if you have published, I’d love to receive some links.

evidence…

ebl, evidence, soapbox No Responses »

Sep 042007

A brief but pertinent post from Lorcan Dempsey today on the use of evidence – or rather, the non-use of evidence – to back up discussions on bibliographic data and catalogues.

I’ve got evidence on the brain at the moment. This area of librarianship is not alone in the fact that its discourse sometimes lacks grounding in evidence. But the future of the catalogue is certainly a dominant topic of conversation at the moment – a conversation that could only benefit from the existence of and reference to a solid evidence base.

[Aside: While I'm talking about the future of the catalogue... I'll be very interested to see the papers from the Australian Committee on Cataloguing seminar that's on this Friday in Brisbane. Aptly titled Promise for the future, or legacy of the past? : cataloguing in a changing world the program includes papers on a range of topics from educating future cataloguers to tagging to RDA. Should be some good food for thought there.]

virtually a librarian

towards a research-led profession (or, how to become a practitioner-researcher)