Assessment on the Ground

Many Temple Library Staff, particularly those in RIS, are already familiar with the qualitative research project that Jenifer Baldwin, Anne Harlow and Rick Lezenby have been working on for the last two years. (Yes, qualitative research, at this depth, takes time!) They have already presented on their work several times at Temple Libraries and conducted a workshop on their approach to interviewing at the Maryland Library Association conference.  Currently, they are wrapping up the data collection phase, which included interviews with faculty and co-viewing sessions with peer librarians. Using a grounded theory approach, they’ve been analyzing their data all along. Now they’re ready to talk about it.

NT: What was your question?

JB, AH, RL: Our first question related to how faculty used Blackboard to provide course content to students. As we interviewed participants our question evolved to a more general one of exploring how faculty choose and share course content.

This content is predominantly course readings, from published articles or book chapters to lecture and lab notes, or reflections. We became interested in how faculty understand their own expertise and how they model that to their students.

NT: What method did you use and why?

JB, AH, RL: We were inspired by the work of Nancy Foster at the University of Rochester – I (Jenifer) took a CLIR workshop on the ethnographic method at MIT along with Peter Hanley from Instructional Support. Then we hosted a CLIR workshop here at Temple, so Anne and Rick were able to take advantage of this training as well. We already had lots of quantitative information here at Temple, we wanted to see what the qualitative research process would look like. We wanted to use in-depth, semi-structured interviews with faculty about their work practice. We had a method and we needed to find a question to apply it to!

Once we developed our questions we went through the Institutional Review Board for approval to conduct the research. In the interview we asked the faculty member to show some recent examples of materials they used for a class. We would ask them to talk through the process of deciding on what content to use and demonstrate how they located the materials and organized them. We asked them what they expected the outcome would be for their students. Each interview was video taped and we then conducted co-viewing sessions with librarians.

We had the recordings transcribed and we coded these (using Atlas.ti) for themes that emerged from the interview texts. We employed a grounded theory approach – where the theory emerges from the research rather than applying the research to a pre-formed theory. The co-viewing process became part of the data and analysis as well. For instance, one of the products was a spreadsheet of suggestions of initiatives and projects that might be developed out of needs expressed in the interviews.

NT: Tell me about your results? What did you learn?

JB, AH, RL: Several themes emerged. One is the tacit knowledge that scholars bring to their work. There is a lot that they know about their discipline that they might have trouble articulating to a student new to that discipline. When asked, well how did you know to include this article they might say, “”Well I just know”. Often they find out about something in what seems to them to be serendipitous ways, but in fact they are predisposed to the literature, invisible college, attuned to the environment that relates to their work

So they might say to a student, “just go and browse the shelves” – this is a behavior that they might usefully undertake because they know what to look out for. But a student might have a very different, less successful, experience with this kind of “browsing.”  Faculty try many ways of modeling these “expert” behaviors for their students.

We heard lots and lots of stories about faculty experiencing serendipitous discovery. The use of video and popular culture is pretty ubiquitous, so faculty might get ideas for class by watching cartoons or movies, at a social event, wandering a bookstore, or reading the newspaper or magazines.

We learned some interesting things about expectations for reading.  There are differences between the kind of reading that is expected of undergraduates compared to graduates,  that was not what you’d intuit. Undergraduates are expected to read in a more transformative and analytical way – graduate students need to read more broadly. If something is really important it would be distributed in class and paper, perhaps even read together.

NT: Is there anything you’ll change based on your findings?

JB, AH, RL: We hope to have a full discussion of implications and ideas for service initiatives and outreach this spring as one of several products of the project.  The co-viewing process provided us with practical ideas for outreach and push notification, for example. Most faculty talked about new book announcements that had an impact on them. This led us to ask what other kinds of things could we push.

Faculty consult with their peers for ideas about content. If librarians have a well-grounded relationships, they will take our suggestions seriously as well.

Another example – we saw a need to create easily and rapidly accessible resources for students who are also practitioners in the field– resources that they can readily access when they themselves are in a classroom or other practitioner setting. We think we might be able to help out in this area.

NT: If you did this again, what would you do differently?

JB, AH, RL: Atlas.ti, for textual analysis and coding, was cumbersome to use, particularly in a networked environment. We’d like to interview a broader group of faculty, a less “self-selected” group of participants. And currently, our library staff is small and stretched so we don’t have all the time to focus on projects such as these. But the “rewards are worth it.”