I posted recently about the importance of publishing, and in that post I talked about reflection and how important I think that process is. And in my last post, I focused on evaluation – and reflection is a key tool in the personal evaluation toolkit.

So I was interested today to see a tweet about a new article on reflection and Evidence Based Library and Information Practice (EBLIP). EBLIP rock star Andrew Booth has just published an article called Upon reflection: five mirrors of evidence based practice.

Whether you’re for or against (or perhaps ambivalent about) EBLIP, this article is well worth a read.

via @dkouf

I have lots of soap boxes, a number of which I’ve posted about here before. Evaluation (otherwise known as assessment) is probably my favourite soap box of all.

It’s also something that I don’t believe we take as seriously as we should. Librarians are really good at collecting statistics – perhaps *too* good at it. The issue, I think, is that we focus on the collecting with such intent that we sometimes miss out on the analysing. We measure everything that’s measurable, and some other stuff too, but I’m not entirely sure that we work with the data we collect in a way that tells us something meaningful about what we’re doing.

Do we actually change how we do things or how promote our services because of what the data tells us? And on an even more fundamental level, do we even take a considered look at the data we collect, or do we just send it straight up the line to meet our reporting requirements?

Evaluation of what we do on a day to day basis is critical. It’s critical because we owe it to our customers to be offering services that meet their needs and that are used. It’s critical because we are constantly asked to do more with less resources, and we need to be rational and thoughtful in deciding where to invest our energies. It’s critical because without meaningful evaluation, how can we ever improve?

Evaluation is also a critical phase in managing any project, and in my experience, it is a phase that is perhaps undervalued (or at least, under-emphasised) in what some might call 2.0 projects. I have observed that, in implementing online services that make use of freely available web based tools, practitioners tend to focus on agility. I’ve seen this result in a lack of planning, in terms of service design, project management and evaluation, and it’s my contention (and a profound one at that!) that in order to be meaningful, evaluation must be planned and aligned with the aims and objectives of the project. You need to know in advance what you’re going to measure in order to collect the right data along the way, and to ensure that you have adequate time allocated to evaluation.

I’m thinking about evaluation at the moment because I’m working on materials for a new unit that I’m running next semester, and working on a plan to evaluate how the unit performs in terms of allowing students to realise the unit’s learning objectives. I’m also thinking about the two units I taught in first semester, and reflecting on both my performance and how the unit structure and content worked. There are many inputs in terms of data: student feedback on my performance, and my personal scores on the Learning Experience Survey (LEX); student feedback on the units, and the unit scores on the LEX; student performance; informal feedback received from students; and students’ reflections (students write one reflection for each assessment item). Of course, working with all of this data will take time, but I think the evaluative process is critical, so I’m setting aside the time to do it.

Do you give evaluation the time, energy and thought power it deserves?