When a Marker is More than a Marker 

Picture Credit: Zombeiete from Flickr Creative Commons

User experience is all around us. In libraries, we often think the assessment of user experience relates to web interfaces, or building way finding and navigation. We might, ask, “Is the language that we use on the website clear to non-librarians?”  “When visitors come into the library, are they provided sufficient affordances  for orientation to the services and spaces available? “

Of course these are questions we already have on our plate for exploration, particularly now as we deal with issues of user experience in a very new library building, the Charles. 

But dry erase board markers? That seems like a pretty small operational decision. We either make them available for check out, or we don’t. But when the option of providing markers to students arose, it got a bit more complicated, and everyone had an opinion.  

Charles Library has 36 study rooms each equipped with whiteboards. These are quite popular, as evidenced by the sprawling, specialized, and creative work we see in the rooms. It is gratifying to see how this simple tool sparks collaboration among students.  Exactly the behaviors we hoped to see in these new library spaces. 

In providing study rooms, there are operational decisions to be made, from how we manage room reservations to policies on use of the rooms.   When the rooms opened, the issue of markers was raised. Should we provide them? And how? Multiple options were discussed, and each might be evaluated on a kind of user experience. 


Make markers always available in study rooms

Make markers freely available at the service desk, but don’t check them out

Check out markers at service desk

Make markers available for purchase in vending machine

Make students responsible for bringing markers for use in study rooms


There may be other solutions, of course. It’s clear that there is a range of options, and each has implications for the user experience. Each option needs to be balanced against library operational concerns, including staff time and effort (creating records in catalog for checkout, preparing the material for checkout, time for transaction at checkout, collecting fines for lost markers) and of course, the outright cost of the markers.  

We may decide that while students might love to have each each study room supplied with an array of colored markers, all full of ink, each time they visit – that may not be the experience we can afford to provide, given other organizational priorities and expectations.

Fortunately, students seem happy to bring their own markers,  as we see many wonderful expressions of collaborative work in the study rooms. While there is no right or wrong answer as to providing markers,  it’s always useful to remind ourselves that 1) there is a range of solutions available to us and 2) the solution we choose may impact user experience.  

Posted in library spaces, usability, user experience | Tagged , | Comments Off on When a Marker is More than a Marker 

Are There Any Meetings on Library Assessment?

Assessment is a growing topic of interest at American Library Association meetings and last weekend I had the privilege of participating in several meetings to discuss trends and challenges. 

Look at How Far We’ve Come: Successes

Assessment practice is evolving from the solo librarian to the assessment conducted in multiple domains – user experience, collections analysis, space design. We started the ACRL Assessment Discussion with a sharing of successes. Grace YoungJoo Jeon at Tulane demonstrates that one librarian can accomplish alot. In her first year as Assessment and User Experience Librarian, she talked with everyone about assessment, learned about their needs, created a list of potential activities, and began to prioritize the work ahead. Grace described reaching out to other units on campus, including the Office of International Students and Strategic Summer Programs. She worked with them to design and moderate focus groups with international students.   All in one year!  

Penn State Libraries’  success this year is a growing department for assessment and metrics headed up by Steve Borelli.  Prioritizing assessment needs through the lens of budgetary operations, they are currently advocating for a position in collections assessment for a department of four. 

Joe Zucca at the University of Pennsylvania  is using the Resources Sharing Assessment Tool (Metridoc) as a space for collecting interlibrary loan statistics, enhanced with MARC data from the consortium’s individual library holdings. With connections to Tableau, data visualization enhances the ability to evaluate inventory and use, and provides potential for collection development at a collaborative level. 

We Still Have Some Challenges

In the example of RSAT, merging data from 13 institutions creates some challenges. There is a “near total absence” of data governance, including some 600 designations for academic departments. This lack of standardization makes cross-institutional analysis very difficult to do. 

Of course this isn’t just a problem for large-scale analysis across libraries. One assessment librarian discovered her public services departments have a “home grown” system for tracking reference and directional questions. While standard definitions provided by ACRL and ARL can provide some guidance, libraries may not want to be limited to these, more traditional, metrics alone. There is a spectrum of opinions as to how to count, what to count. How best to define a transaction? 

This lack of agreement related to counting has ramifications down the line, particularly if these metrics are used in performance review. What is to prevent someone from “bumping up” her numbers?  We talked quite a bit about how the library “reduced to bean counting” is no way to tell our story. Librarians may very well feel that a focus on counting diminishes the work that they do. 

The Rearview Mirror

We shared concern that assessment practice is “always looking through the rear view mirror”. When we look at trends only at annual review time, we fail to understand those trends to plan for the future.  We may prefer to ignore the trends.  We tend to keep our data silo’ed, making it difficult to see the full picture, or inter-relationships.  A great example is this one: Less questions about finding Huckleberry Finn (a decrease in numbers at the reference desk) could mean that our discovery systems are working even better.  Fewer page views on our website may result from a more efficient, user-friendly interface. We need to look at our numbers in a more integrated way. 

It was good to talk about our challenges, our successes, and best practices with a group of understanding peers. Then on to the next meeting, LLAMA Assessment Community of Practice, Hot Topics!

Posted in uncategorized | Tagged , | Leave a comment

Furniture feedback in Charles Library

When we opened Charles Library in August of 2019, we knew right away that we needed to increase the seating capacity in the building. During the day, a walk through the upper floors of the building gives the impression that we are at, or quickly approaching, full capacity.

The first physical space UX project I did in Charles was to gather data about the current furniture. In the fall, I worked with Rachel Cox, Nancy Turner, and Evan Weinstein to collect data on furniture use and preferences. We’re using that data now to recommend additional furniture and space improvements.


To kick things off, Rachel and I conducted floor-by-floor walkthroughs, counting the number of seats occupied and recording observations about furniture use. We carried out the daily walkthroughs for one full week in October. We entered the headcounts and observation notes on a worksheet. The student building supervisors handled data collection at night and on the weekend.

Worksheet for recording observational data

The sweeps gave us a sense of how full the building was at different hours and how students were using the space. But, we also wanted to know if students liked the new furniture.

To find out more about this, Rachel and I built a survey that asked what students came to the library for, if the furniture met their needs, and what they liked and disliked about it.

survey p. 1

We got 213 responses. Rachel and I analyzed over 600 survey comments over a couple of days, sorting the comments into categories.

Survey analysis


The majority of students (78.4%) came to the library to study alone but studying as a group was also very common (48%). Other reasons to visit included resting between classes, finding a book, going to the cafe, visiting the Scholars Studio, Graduate Study Area, Special Collections, or Student Success Center, and hanging out with friends. 54% responded that the furniture didn’t meet their needs.

The over 600 survey comments provided us with more detailed information about what students wanted:

  • More seating generally
  • Comfortable seating  and “cozy” spaces
  • More spaces to support individual study
  • An environment mostly free of distraction that also allowed some level of chatting/interacting with others

More seating

The headcounts revealed, somewhat surprisingly, that we rarely reached over 50% occupancy on any floor of the building, even during the day when the building looks full. Survey comments told us that students sometimes cannot find a seat in the library, and a few asked specifically for more seats. Students also described the existing seating as being “too close” to other students and too small for spreading out their work. Lack of personal space at the large open tables was a common thread throughout the survey comments.


Right now, the open study areas in Charles Library are furnished mostly with open seating tables. This is a dramatic shift from Paley Library where students were accustomed to semi-private, partitioned desks and armchairs with individual side desks. Students miss the private seating they found in Paley; one commented,

“Please, provide some cubicles or private study spaces. It takes so much time to book a study room because they’re rarely available. The cubicles that used to be in Paley fit my needs well and I have not been nearly as productive since moving to Charles.”

Others spoke directly about distractions, both noise and visual, at the open table seating,

“…perhaps these spaces could be furnished with … divided desks to remove distraction from the (sometimes embarrassingly noisy) library environment and to center focus on the work I came here to do…”


Another common theme was the desire for comfortable seating. Students frequently asked that we add “comfortable” furniture and make the space cozier. “Comfortable” and “cozy” were used to describe soft seating like beanbags and couches, as well as upright tables and chairs. The comments overall conveyed a strong preference for furniture that supports work, and in many cases, “comfort” meant seating that is ergonomically designed for sitting and doing work over extended periods,

“[I want] couches and seating like you would find at a cozy coffee shop. The furniture I have used has been very uncomfortable and I cannot do work for a long period of time without something in body aching. The library is absolutely gorgeous, but the furniture is probably the biggest complaint…”

Even our lounge furniture doesn’t meet students’ expectations for comfort. They described the lounge chairs and small polygon tables as too “low to the ground” and lacking “function.” One student demonstrated their distaste for the lounge furniture with a drawing of a stick figure hunched uncomfortably in one of our lounge seats to reach a laptop on the small polygon table. Others asked for armchairs and other soft seating,

“I don’t really feel that there are any comfortable arm-chair like seats at the library like there were at Paley. I can’t get comfortable anywhere, and as a result, I don’t feel welcome to stay long…”

During the building walkthroughs, we mostly saw students using the lounge seating to do work, rather than for socializing or other short term activities more suitable to lounge seating. But, the lack of a work surface at those seats meant that students either hunched over the small tables or created work surfaces using benches, laps, window ledges, or a second lounge chair. The survey made it clear that students do not like the lounge seating because it’s not ergonomically suited for doing work.


Despite increases in group and collaborative work, student space needs are still strongly tied to individual study. The survey showed that students primarily come to the library to study alone and they want to do that in a comfortable environment that is relatively free of distraction.

The survey provides us the evidence for prioritizing future purchases of furniture, including pods or carrels for individual study. But we’re also exploring ways of re-configuring our current furniture to provide semi-private spaces with minimal distractions. Table partitions and different furniture arrangements can create barriers, providing students with a psychological sense of privacy.

tables around ledge with students studying

Extra seating for exam period

When we added the individual table seating around the ledge during exams, it was immediately popular. Though these seats are in high-traffic areas, they allow students to face away from others, providing a buffer from visual distractions.

We also want to get a better understanding of study room use and continue to gather student input as we select potential new furniture.

Posted in library spaces, service assessment, surveys | 1 Comment

We Don’t Want to Work with Mummies

At Charles Library we are experiencing a more open office environment. I saw an  extreme version at the Penn Museum this weekend ; the conservator’s workspace is actually in the gallery, on view several hours a day. But the office mate, a mummy, is very, very quiet.

These issues of privacy, of noise, and of how we move around our space without disturbing co-workers, surfaced in several workplace norms conversations that I conducted in September. The meetings, organized by work area, were designed to surface issues of concern about the kinds of environment (and behaviors) that would enhance, and detract, from the “ideal” working conditions. Would it be a problem to eat tuna salad at one’s desk? How can we signal to colleagues that we are not “interruptable”?

Some anxiety about establishing workplace norms emerged when we talked one-on-one with staff as part of the Envisioning our Future project. Even before the move, staff were concerned about the more open workplace environment and the need to set guidelines for behaviors. Recognizing as well that multiple types of work would be taking place in our new work spaces. Would this lead to more collaboration in person because of casual interactions,  or just more use of electronic communication?

The discussions were summarized and shared with all of the staff in that workspace, including those who were not at the in-person conversations. A summary email was sent to all staff.

Did the workplace norms conversations make a difference?  Last week I launched a short survey to staff to gauge the effectiveness of the process. It’s never easy to ask for feedback, risking the possibility that it did not make a difference at all, or worse, it was considered a waste of time. While that didn’t happen, I did learn some lessons, about the process and about assessment.

About 40 staff members participated in the five sessions. 15 of those people responded to the survey. They were from all work areas, 

  • 66.7%  participated in the session
  • 86.7% read the document resulting from the session
  • 66.7% read the email to staff

I asked three additional questions:

From your perspective, what degree of change has occurred in attending to workplace norms in your area? (0 is “none at all” and 5 “a good bit”)

From your perspective, what degree of change has occurred in attending to workplace norms in your area based on issues discussed in the facilitated conversations/shared documents? (0 is “none at all” and 5 “a good bit”)


Although this is a very small sample, it seems that time in our new spaces has had more of an impact than the actual conversations. (I base this on the fact that the average goes down for the second question).  The comments provide a bit more insight into this interpretation:

The process elicited quite a bit of variation between work areas in some aspects of the work environment, like tolerance for noise. Some were disappointed that the outcome did not result in concrete decisions about policy. Others felt that the conversations led to more comfort in discussing the workplace norms.  But they were not the real impetus for changes in developing norms that staff are perceiving.  Those arise from just just being together. And there are outstanding areas of negotiation, like the use of the breakout rooms. 

What did I learn as a facilitator? That I can be challenged by remaining neutral in these situations.  I need to follow my own ground rules in providing equal voice to all participants.

But overall, I feel positive about:

  • providing an opportunity for staff to get together and
  • opening up a dialogue that may  not have come about as readily without these meetings

Mainly, I am appreciative of my colleagues who participated in this effort. We don’t necessarily want mummies as our work mates, but it’d be nice to play our music as loud as we like.  

Posted in organization culture and assessment, process improvement | Tagged , , | Comments Off on We Don’t Want to Work with Mummies

Celebrating 10 Years at Ginsburg: A Time for Review

The Ginsburg Health Sciences Library is celebrating its 10th birthday and used the occasion to conduct an intensive review of current operations and planning for the future. Barbara Kuchan, Director of the Health Sciences Libraries, worked with a team (Patrick Lyons, Jenny Pierce, Natalie Tagge and Will Dean) to plan the process. Since the review included a hefty mix of assessment activities, I was invited to participate as well.  

In addition to looking at 10 years of activity statistics, the group decided on three assessment methods: focus groups, a whiteboard for user feedback, and space usage analysis with heat mapping.

Five focus groups were conducted with students, with faculty, and with staff. I asked them about how the library’s spaces, technologies and services support their work. What works well? What are the challenges? If you could wave a magic wand, what would your ideal library be like?

“Magic wand” questions can be a good way of getting at library qualities that are important to users. For instance, students described their dream library as high tech, but with “green” attributes – trees, a courtyard, and natural light. It has lots of bathrooms. It has secluded areas (nooks and crannies) but also an open feel. The temperature is not too cold or too hot. Having a Starbucks in the library would also be nice. 

We gathered additional feedback in a very simple way: a large whiteboard was placed in the library’s lobby.  Library staff queried in-person visitors about their use of the library, impressions of short-comings and strengths, and topics of general interest. This approach is lightweight (i.e. easily and cheaply done) and provides an opportunity for anonymous feedback.

Will Dean led an analysis of space usage, conducting regular audits of seating to determine the most popular areas for library work – a heat mapping exercise. Using the library’s floor plans, staff divided the library space into a series of zones. The staff then counted the number of individuals in those zones at various times of the day, during busy and quiet times of the year. A heat map of the data with the “hot spots” in red and not-so-hot spots in “blue” gives us a visual display of how popular certain areas are, and identifies not-so-popular areas.

In this example from the lower level, the tables by the windows (Zone 2) are popular, as expected for the natural light provided there. Unexpectedly, the carrels near  the stacks (Zone 7) are about as popular, appealing to those requiring more secluded, private spaces for study.


What is most exciting about the report is not just the richness of the data collected, and the wide-sharing of the findings (at an all staff meeting and via email), but the many recommendations for changes to spaces and services that came about based on the assessments.

For instance, student needs for quiet space and interest in stress-reduction may result in a redesigned room as a “green study lounge.”

Staff interviews led to a recommendation for improved on-site IT support. This would allow Ginsburg to meet user requests in new ways, like the live streaming of  workshops.

Assessing for the future is hard. How do you ask users what their future needs will be?  How do you determine the best direction to take when users may not know what services the library can provide? The team addresses this dilemma in its conclusion: 

 Taking the time to assess current needs and plan for evolving ones is a worthwhile investment. Now that a health sciences library assessment team has been established, evaluation of programs and services will become part of continuing operations. For this benchmark assessment, work now focuses on prioritizing and implementing key recommendations.

For more information about the project, contact Barbara Kuchan at b.kuchan@temple.edu



Posted in assessment methods, data-driven decision making, library spaces, qualitative research | Tagged , , | Comments Off on Celebrating 10 Years at Ginsburg: A Time for Review

Stretching Boundaries, Crossing Thresholds


Today I needed a metaphor to suit my photograph.  It’s the immersive artwork Flight Paths by Steve Waldeck installed at Atlanta’s airport, constructed of thousands of laser cut “leaves” and incorporating sounds and images of birds. The artwork makes walking the terminals a memorable experience.  

I was returning from the Designing Libraries conference and considering how the recent move to Charles, this very different physical space, impels us to think differently about our work and the organization. How do buildings shape our organization? How will Charles stretch us? 

The session on Using Innovative Ideas and Team Building to Drive Organizational Change addressed these questions. Mary Ann Mavrinac (University of Rochester) talked about how collaboration drives organizational change.  She compared organizational structures to rubber bands – we can stretch them, but they will tend to go back to hierarchy as a default. So we must continue to stretch and build teams for collaboration.  

Charles is stretching our organizational muscles in more than the exercise we get on the stairs. The more open spaces, for both public and for staff, test the “territorial” boundaries that our administrative structures can create.  Here are some examples, 

Just before DL, I facilitated a series of  workplace “norms” conversations with staff from work areas at Charles. This process was not about setting new policies. It was about talking together, in groups that mixed departments to surface what an ideal work environment was like and what challenges that ideal. We generated ideas for enhancing the positives in our work environments, from setting aside space for sharing food, to a desire for space that was not so silent as to feel monastic. 

So the move to Charles provided an opportunity to engage us in a process that is:  

  • Non-hierarchical
  • Participatory
  • Where all have equal voice and
  • Department agnostic

But the norms process also brings to light the different ways in which we conduct our work with colleagues and patrons. Limited resources for space requires us to work together in resolving conflicts and coming up with creative solutions. 

The openness of the building is forcing issues as well. Many of our services and special spaces, like the Loretta Duckworth Scholars Studio,  are more visible.   As students are actively looking for seats and computer workstations, this accessibility needs to be balanced with use policies that meet their needs. These needs may require more “permeable” boundaries for usage. 

The One Stop Assistance desk and consultation spaces nearby necessitate a rethinking of boundaries as well.  In a small space, we are providing IT services, reference services, access consultations and circulation, as well as managing a busy self-service holds shelf. We process guest computing applications, handle media equipment and train faculty in course reserves. To provide these services seamlessly we must cultivate more permeable boundaries between departments, allowing us to share information, decision-making and resources in new ways. 

At last year’s Designing Libraries conference, Greg Rasche (North Carolina State) claimed that new buildings provide us with once in a lifetime opportunities. He said,  

Hunt was a meteorite opportunity. If you miss it, it will set you back. If you overdo it, you can always scale back, but if you miss that opportunity for change, you’ll never get there. Use failures and successes as a way of evolving your organization.

Our local experience brings this advice home,  as we learn how changes of services and space can propel organizational growth.  Let’s not be afraid to fail, and learn, as we cross those thresholds into new territory. 

Posted in conference reports, organization culture and assessment | Tagged , | Leave a comment

Envisioning our future: Will the view forward ever be clear?

This was a thrilling week at Temple University Libraries, as many of the staff moved into their new office spaces at Charles Library. The windows provide an abundance of natural light throughout the building – at last we have escaped our cinderblock bunker-like offices.  

This week also saw the distribution to staff of the self-study Envisioning the Future (see August 12 email to libstaff). It’s another way of shedding light, as the research team asked staff to anticipate how the new spaces will impact their work, particularly with users and with colleagues.  The research project was part of an ARL initiative for assessment, and their prompt question for our research was this one: (How) do library spaces facilitate innovative research, creative thinking and problem solving? 

We wanted to look at the question from the perspective of the staff.  If new library spaces facilitate innovation, staff must be supported in helping users to be innovative. What are the best practices for supporting staff as they help users in new ways? Here at the Charles, we will be introducing to campus a whole new way of accessing and discovering our collections, new ways of using technologies for research and instruction,  and new ways of providing services. The space is part of the infrastructure, of course, but the people will be what makes it all work. 

In the course of 29 interviews with staff at all levels of the organization, we asked,

  • “How do you anticipate your work with users changing?”
  • “What do you think the opportunities and challenges will be?”
  • “How might your work with colleagues change?”

We also asked about the ways in which we were preparing for the changes, and how we felt best supported in making those preparations.

What did we learn? At the time of the interviews, staff concerns centered most about the loss of private offices and the convenience for small meetings and consultations those provided. But they acknowledged that private offices can create isolation from colleagues, and a lack of visibility to students. The new spaces will provide far more opportunities for interacting with colleagues in informal settings. Another potential cost will be the shared spaces that are noisy and full of distractions. Yet for many staff members, these have always been part of their work environment. And they appreciate the collegiality and fun that can develop in these environments. 

Communication is key as staff prepare for changes. While most recognize that information may not be available, even at the highest levels, and is always subject to change, many cited the “Ask Me Anything” (AMA) sessions as one of the most useful approaches for sharing information with staff. The regular reminders (and incentives) for purging offices of paper were important as well.

In the months prior to the move, the anxiety level of many staff was pretty high. That needs to be acknowledged and allowed to be discussed in safe ways. But only a few staff took advantage of the change management workshops sponsored by peer groups, which was interesting.  Many staff also look forward to more conversations about norms in shared office spaces. Now that we are actually in the space, we’ll have more opportunity to know what those issues will be. 

A big thank you to the research team.  (Olivia Given Castello, Rachel Cox, Jessica Martin, Urooj Nizami, Jenny Pierce, Jackie Sipes, Caitlin Shanley, Stephanie Roth)Envisioning our Future is just the first step in a longer-term project to look at how staff work in the building.  Will the Charles Library shape our organizational culture? Will it inspire staff to be more creative and innovative, as it is designed to support those behaviors for our patrons? And how can we best engage staff in helping to shape that future?  This kind of qualitative research is time consuming and messy. But we come away with a much richer picture of the organization at this very exciting and unique point in the libraries’ development.  And perhaps, with a clearer view of the best way forward.  

Posted in organization culture and assessment | Tagged | Leave a comment

A Yogi’s Reflections on Change, and Charles

Charles Library

For a couple of years now I have been practicing yoga. I live near the studio and my routine is to have an early coffee then head over to the 6:30 session. It’s a good way to start the day and puts me in a positive frame of mind.

But this month two instructors have left the studio and we students now practice with “Jane”, who is just out of teacher training. It was awful. Nothing she did was right, from confusing left and right to selecting unpleasant music.  I would emerge from practice annoyed, and at times bitter and more stressed than when I arrived.

I was frustrated with my own impatience. I told myself to respect the process, to allow Jane the space to learn. That’s a value I tout all the time. But I was having some trouble practicing what I preach.

Yesterday’s class was quite different. It was still Jane. We chatted before class and got to know each other better. In practice, the shapes seemed not so strange and the music not so grating. I came away from practice refreshed and energized for the day ahead. Something had changed, but it wasn’t just Jane.

Change is annoying, and stressful, and makes us uncomfortable. Here at Temple University Libraries/Press, we are going through a lot of it.  The Envisioning our Future interview project was designed to explore with staff aspects of that change as we transition to Charles.

We are now wrapping up its first phase, providing for some self-reflection on our organization and our individual work. How do we anticipate our work with colleagues and with users will be impacted? What opportunities and challenges will we face? We hope to identify best practices when supporting this kind of dramatic change. The many participant voices have also brought to light the anxieties we are facing now, including my own.

But like a yogi  trying always to improve my practice, I feel I can adapt to that change. And will be both stronger and more flexible because of it. Next post, from Charles Library!

Posted in organization culture and assessment | Tagged | Comments Off on A Yogi’s Reflections on Change, and Charles

“why can’t life just be easy !?”

One of our strongest educational partnerships here at Temple Libraries is our collaboration with the First Year Writing Program. Nearly all entering first-year students at Temple take at least one of the courses in this program, so it’s a great opportunity for library staff to reach a large population of undergraduates. Each section of the three largest courses (ENG 802, 812, and 902) is scheduled for two workshops with a library staff instructor. These workshops are designed to build on each other to support students as they work on increasingly complex research writing assignments throughout the semester.

One of the ways we assess our success in this program is by having students complete a brief, five-question survey after each workshop. In summary, the questions we ask on the survey are:

  • Who was your librarian?
  • What’s one new thing you learned today?
  • Rate your confidence BEFORE and AFTER this workshop.
  • What is one question you still have about using sources (after Workshop 1) / the research process (after Workshop 2)?

Though brief, these surveys are a rich source of data about our instruction. We tend to get a high response rate, because we have students complete these surveys while they are still in the classroom. While there are a number of potential findings from these results, in this post I will focus on major trends across all instructors. The results I discuss are from the Fall 2018 and Spring 2019 semesters.

Student Confidence

When designing these surveys, our curriculum working group struggled with the best types of information to gather in this particular context. If we quiz students on concepts we discussed in the workshop just minutes before, are we sure that their responses mean they can apply and retain these concepts? One measure we believed students could effectively self report on was their confidence about doing research. We hoped to learn if students felt increasingly confident after our workshops.

Our data suggests that students’ self-reported confidence did increase due to our workshops. The blue lines on the chart below indicated confidence before, while the green indicates confidence after. The results are on a scale from one to four, with the exception of Workshop 2 in Spring 2019, where we updated the scale to run from one to five, based on survey best practices. That’s why the very last measure is higher overall.

What Students Learned

One of our survey questions asked students to report one new thing they learned in each session. Overall, student responses were relevant to the workshop content, suggesting that students are paying attention and leaving with some of the concepts we discuss in mind. Students recalled specific databases by name, especially JSTOR and Academic Search Complete. Students also recalled the BEAM method that we discuss, indicating that perhaps acronyms and other mnemonics can help with learner recall.

Sometimes, though, student responses suggested a broader understanding of research than what we cover in the workshops. For instance, students often claimed that they left knowing “everything” about how to use the library, databases, and/or the library website, when we cover just a fraction of the available resources in each workshop. Students also sometimes misinterpreted concepts like BEAM, suggesting that while they remembered the name they may not have learned how it applies to their research process.

Student Questions

In my opinion, the survey question that asks students to share any remaining questions they have is one of the richest sources of data we have. I picked up on three common themes when reading through student responses.

Straightforward answers

Students want specific numbers when it comes to finding sources for their research papers. Some examples of the kinds of questions include:

  • How many years is too old for an article?
  • How many sources is too many (or too few)?
  • What time length is necessary to commit to doing the research portion of the paper?

Value judgments

Students want librarians to tell them the difference between “good” and “bad” when it comes to their research. Examples of these questions include:

  • How do I tell if a source is bad?
  • How do I tell if I have a good topic / thesis / article / database?
  • Why are databases better than the internet?
  • Is it okay to [fill in the blank with various research practices]?

Genuine curiosity

For the most part, students took these evaluations seriously, and answered sincerely (though there were occasional humorous answers – this blog post’s title being one of them). In addition to reflecting on what they had learned, students demonstrated interest in the library beyond what we covered in the workshops. Examples of these questions:

  • How many types of librarians are there?
  • Do we use the dewey decimal system?
  • How many books can we take out?
  • Are librarians available on the weekends?


It’s difficult to immediately apply these results to our instruction practice (e.g., should we be giving students value judgments on source types? I think most of us would say no!). There are also definitely limitations to using a survey to assess student learning – we don’t get to see if students actually apply the skills they learn to their writing, and we can’t ask follow-up questions when we get a confusing response. But these evaluations represent an important part of the prism of student learning assessment that we do here at the Libraries. I hope that by sharing these results with teaching librarians, we’ve continued to shift our practice to meet changing student needs and to improve our instruction overall.

I look forward to continuing conversations with my colleagues about the following questions:

  • How might these survey responses help us improve our teaching?
  • How could we be using the data we have to demonstrate our impact?
  • What other data would be helpful? How might we gather it?
Posted in instruction and student learning, surveys | Tagged , | Comments Off on “why can’t life just be easy !?”

Cookies, User Research, and an Iterative Design Process

Cookie Monster invites you to help with the library website…

In late February, the Library Website Redesign project turned its focus to incorporating user research into the design process. With the help and support of Cynthia Schwarz, Nancy Turner, David Lacy, the UX group, and others, Rachel Cox and I began a month long “UX intensive.” The goal was to refine how content is organized on the site and to identify the primary site navigation.

During the UX intensive, we tested design prototypes with 48 users, recruiting passersby in the Paley lobby with cookies, granola bars, and coffee. The UX intensive also afforded us the opportunity to consider how to add user research and iterative design to the project on an on-going basis.

Organizing Content

When Rachel and I started our work, the site content was loosely organized based on the site’s backend infrastructure. The entity model allows us to group content into different entity types, such as services, policies, spaces, etc., and ultimately this model will make it easier for users to locate website content through search.

Cardsorting with library staff in 137

Our first step was to review the site content and organize it in a way that makes sense to users — we agreed that the organization of the front-end user interface shouldn’t necessarily mirror the backend entity structure. While the website’s infrastructure promotes discoverability of content, we didn’t think this was the most intuitive structure for navigating the site. Plus, we wanted to offer users a way to discover our content through browsing, in addition to searching.

Building on user research from earlier in the winter on the categorization of library services content, we did a holistic review of all of our site content. With the help of library staff we spent a day and a half in room 137, sorting our content into categories. We also spent time brainstorming category labels that accurately described the content within and were free of library jargon. Based on the card sort, we created two sets of content groupings, one a more “traditional” (and minimal) navigation similar to many other academic library websites, and an “action-oriented” navigation based on what users can do on our site.

Testing the Navigation with Users & Library Staff

The UX group then conducted tree testing with users on the two navigation menus.

screenshot of navigation menu

“Traditional” navigation

Ultimately, the “traditional” navigation tested better. Users were able to locate information with less effort and some commented that they liked the simplicity of fewer navigation options.

screenshot of navigation menu

“Action-oriented” navigation

However, one category from the action-oriented menu, “Visit & Study” was very successful. We went back to the drawing board, creating a single navigation menu comprised of the best elements of each.

We also asked interview questions to get a sense of what users expected to find in each of the category and whether the terminology we chose made sense.

word cloud of what users expect to find in the About category

What users expect to find under About

What users expect to find under Research

What users expect to find under Research

Refining Content Categories

From there, we continued refining the content and categories of content within the top-level navigation categories, i.e., we decided what site content should go under “About” “Visit & Study” “Services” and “Research.”

At this point, we realized we needed to meet with internal stakeholders. There was a lot of content about library services that we simply weren’t familiar with. We met with library staff from the Research Data Services Strategic Steering Team, the Scholarly Communication Strategic Steering Team, and the Digital Scholarship Center to learn more about their emerging services.

We shared the revised wireframes with library staff at an open forum in February where attendees gave feedback about labels, navigation, and organization of content. This feedback helped us to finalize the architecture of the site.

An early attempt at finalizing the navigation and content categories 

Building a Site Prototype

Once the site structure was finalized, Rachel created landing pages to correspond with the primary navigation, i.e. the pages where a user ends up after clicking a top-level navigation category. We used Balsamiq to design low-fidelity prototypes (or wireframes). Balsamiq allowed us to create interactive, clickable, prototypes that could be shared and tested with users.

The first landing page prototype was fairly bare bones; we listed content as text within each category. 

In the first round of user testing, we observed that participants spent a lot of time scanning the landing pages for the information they needed. Participants found the pages overwhelming and overlooked content in the categories. Some of the subcategory labels were also confusing. For instance, participants didn’t realize that “Libraries and Collections” meant Temple’s libraries and collections.

Based on results, we revised some of the terminology (e.g. “Libraries and Collections” was renamed “Temple Libraries & Collections”) and reorganized some of the content, but the biggest change was the elimination of the subcategory links from the landing pages. We designed icon-based prototypes of the landing pages hoping the icons would make the pages easier to scan. We tested the icon-based design with users just two days later with great success.

screenshot of landing page

Icon-based landing page

Since February, we’ve continued to make user research a routine part of the design process. Earlier in the website redesign project, design concepts often went from a basic prototype to final product with minimal staff or user input in between. Testing initial prototypes with users and library staff helped us to identify usability issues and iterate on the design before it moved to the final coding and review stage.

Today the UX group ran our last user research session in the lobby of Paley before the Library closes on May 9th. I look forward to continuing user research in Charles Library (as well as other campus locations) and integrating user experience into a variety of projects.

Posted in usability | Tagged , , | Comments Off on Cookies, User Research, and an Iterative Design Process