do we want to know whether library blogs are succeeding in the big, bad web world?
There’s been plenty of talk around the Library 2.0 theme on the idea of evaluation or assessment. At Information Wants to be Free, Meredith Farkas says what she wanted to see come out of Library 2.0 was a greater focus on assessment. I certainly want to see libraries have a greater focus on assessment, too, and I want to see them publishing about it. (Particularly public libraries. We just don’t publish enough.)
Why aren’t we (libraries in general) publishing about the success (or failure) of our 2.0 projects? Why is there virtually no data to be found that quantifies some of the outcomes of 2.0 projects? We’ve been on this 2.0 bandwagon long enough for studies and assessments and evaluations to have been undertaken. For a movement that’s intrinsically tied up with quick publishing channels like blogs and wikis, it seems strange that there is a real dearth of published studies on 2.0 projects. Why is that?
Walt Crawford had this to say in a recent post on his two blog survey books:
Maybe there’s a clear desire not to know how library blogs are doing in the real world, other than a few cherry-picked examples. I’d like to think that’s not the case. It would be unprofessional to tell people about how wonderful library blogs are, and encourage them to create such blogs, without giving them honest and broad-ranging information on what’s actually happening with such blogs.
I’d like to think that’s not the case, too. But I wonder. I wonder a few things:
- Is the lack of publishing indicative of a lack of success? (And a fear of talking about it?)
- Is the lack of publishing indicative of a perceived lack of success, a perception that might be formed because we’re not collecting the right data? (eg. How are we measuring ROI? Do we just count comments on blog posts? Or do we look at exit links, time spent on the page, holds on titles blogged about, impact on online resource usage stats…? I certainly hope all of these metrics and more are informing libraries’ evaluations of their blogs, because if we’re just relying on comments to measure user engagement, then we’re not seeing the full picture.)
- Is the lack of publishing indicative of a lack of evaluation? (And if so, why aren’t we evaluating? Because we don’t know how? Because we don’t have time? Because we don’t want to know?)
- Or, is it just that we’re not publishing about our evaluations?
I’ve got a blogging project in the pipeline at mpow. It’s germinating quite slowly, because I want to see it well planned. We want a well planned implementation, but also a well planned, multi-faceted evaluation. If it works, I want to know about it, and I want us to be able to reflect on what we did and make links to what worked. If it doesn’t work, I want to know about it just as much (if not more), because I want to be able to reflect on what we did, look for ways we could improve, and ultimately, pull the pin if that’s what we need to do.
Blogs (and all things shiny and 2.0) are just great. They’re fun for staff to work on, and have huge potential to engage our users. But none of us have time to run services that don’t work. If we don’t evaluate, we have no ability to know whether
We know that “because we always did it that way” is not a good reason to keep doing the things we’ve always done, whether they work or not. But neither should a failure to evaluate be the reason we keep on keeping on with our 2.0 services.
If you have evaluated your 2.0 service, publish about it! And if you have published, I’d love to receive some links.
March 15th, 2008 at 3:34 am
Excellent comment. As should be obvious from my own commentary, I’m troubled by the repeated attention given to one or two Highly Successful Library Blogs, ones that strike me as being exceptional cases–and what I see as a failure to admit that, at least by some measures, most library blogs haven’t done all that well at engaging users.
There are, as you note, other measures (most of them only available for the blog owner), which is one reason I’ve avoided ever suggesting that any library blog is a “failure.” When a library blog repeatedly coaxes the readers to comment (as a few have), and there are no comments…well, that says something. But most library blogs don’t do that.
I think some of the metrics you suggest, most of them unavailable to an outside observer, are distinctly worth considering–and that it’s worth discussing the results.
But then, I’ve said for a while that Library Success would be even more valuable if it included “Learning Successes”–that is, failed programs that help us move on. I don’t see that happening, and (with one or two major exceptions), it’s never been much of the literature.
March 17th, 2008 at 2:46 pm
Thanks for your post - very timely for me. Today I’m writing a survey for 15 bloggers who will be part of a 6 month pilot project where the uni hosts blogs using WordpressMU. This is designed to create a best practice model if we want to extend it to the rest of campus.
The survey is designed to work out their existing blogging skills and the support they need, and to work out the template features, widgets and plugins I’ll need to set up. I am also asking them at the end “Which things would you consider when evaluating whether the blogging trial was a success for you?”.
I want to embed evaluation as part of the pilot project, so will be surveying participants at the point of getting their blog (June) and three months later when the trial ends. I will be looking beyond the participants’ opinions to things like time cost, reliability of the software, load on IT etc.
As for publishing - I’m in two minds. I’m guessing that the telephone and the bookmobile (which is often held up as the ultimate Library 2.0 service) were introduced in libraries without published studies or evaluation. I’m also guessing that many of the small, busy libraries who are introducing blogs are using their intuitive professional judgement and knowledge of their communities, rather than taking time out to use more formal methods or even think about the luxury of publishing.
I do think that we need published, valid studies of what works and what doesn’t. Like you, however, I am unsure what a lack of published studies means.
Good luck with your blogging project.
March 17th, 2008 at 7:41 pm
@walt: wrt the metrics i refer to not being available to anyone other than administrators: it certainly is a shame, but fodder for the argument for publishing. i think we need to explore best practice for blogging on a number of levels - including statistical standards. would be great if we (as a profession) could collectively think about what’s needed to measure the success of a blog project, and come up with a set of metrics that we can use for benchmarking.
@kathryn: your project sounds v. interesting. i’d love to have a look at your survey instrument…
you’re right about the bookmobile and telephone. i just think that when we have so much click-to-publish technology at our fingertips, we should take advantage of it (i’m thinking getting ‘published’ in some way was possibly not as easy/accessible around the time the phone had its greatest impact on changing the face of reference). i’m not talking peer reviewed, printed publications - any kind of writing, really: blogging, news-y articles, contributions in wikis like library success…
to look at a more ‘formal’ or ‘traditional’ forum: take the recent vala conference, for example. there wasn’t a huge deal there for public libraries, and that’s nobody’s fault but our own. we’re doing great stuff, but we aren’t particularly good at self promotion. how much more would we all get out of these big conferences if we contributed to the programs?
as for publishing as a luxury: this is interesting. to me, a stretch of uninterrupted time to write at work would indeed be a luxury. it just doesn’t happen, and not for lack of support for these sorts of endeavours from management, nor for lack of enthusiasm on my part. there is just so much other stuff to do, and in a lot of ways, i’d rather write at home so i can potter around and have clear headspace in which to do it.
for me, more than being a luxury (and i know it is a luxury, of sorts, that my personal and professional life affords me the space to write), writing is a professional responsibility, on three levels: to my self (because it makes sure i stay engaged and keep on learning), to my organisation, and to my profession. it’s also something i do because i love to do it.
what i find luxurious (and i think this is a shame) is when i get to work on a project where there is a considerable body of professional literature of some description from which to draw inspiration, background, data…
the reasons for a lack of published studies are certainly many. i think, as you suggest, that one of the reasons is very likely that we’re too busy doing our jobs to get writing (apologies - i didn’t make that point in my original post). and there’s certainly a lot to be said for intuition and professional judgment… i don’t mean to devalue that at all.
i just hope that libraries are doing some thoughtful, meaningful evaluation of their 2.0 projects. and for my own selfish purposes, i’d love them to publish their findings!
thanks to both of you for a couple of thoughtful and thought provoking comments.