Assessment & Analytics Community of Practice “Kick Off” : A Recap

Posted on August 17, 2017 by Nancy Turner

Yesterday here at Temple University Libraries we kicked off a new approach to developing our culture of assessment, an Assessment & Analytics Community of Practice discussion to which all staff were invited.

The Kick Off  focused on two recent research projects conducted here: a qualitative interview project with religious studies faculty and the Ithaka S+R /Temple faculty survey. What I want to facilitate at these sessions is the opportunity for staff to think together about how we connect user research to strategic library practice. To that end, I asked the Ithaka Research Team members to consider these questions, provided here with some key points:

In terms of the work you do with faculty, what was the most significant finding?

  • Faculty (religious studies) are using social media to create informal communities to a greater extent than we thought. Many use Twitter to keep up with other researchers in their field.
  • Faculty are incredibly busy.
  • While faculty do want a wide audience for their scholarship, they perceive “open access” as less targeted than publication in more traditional vehicles.

In understanding faculty work practice, and how the library can support that, what were the particular benefits of the research method? What was problematic?

Interviews

  • Interviews provide robust data and good “story-telling” potential.
  • Processing that data take time; we were fortunate in having our audio interviews professionally transcribed.
  • Yet even with a small sample (12) we were able to discern themes.
  • In the end, a narrative actually was easier to write with the qualitative data compared to the quantitative.

Survey

  • Surveys are hard to design. While they provide a way of gathering from a broad group, the question format can be limiting, and we never know if respondents understand the question in the same way.
  • We worried about skewed results, i.e. respondents were mostly faculty who like and want to support the library.
  • Survey data takes time to analyze. We spent lots of time massaging the numbers, and it was not easy to turn those numbers into a good (and even-handed) narrative.
  • That said, the “hard numbers” may be more usable for “talking points” and advocating for the library’s value to the University, particularly as we compare our results with other research institutions.

In using the findings to consider next steps (actions, not additional research), provide an example. Connect something you learned to something you will do.

  • Services like document delivery to faculty offices are important as they save the time of scholars.
  • There is plenty of room for educational outreach. Areas like open access, open educational resources, institutional repositories  are not commonly understood. There is also room for educating faculty about how we can support their needs in scholarly communication, e.g. negotiating a license with a publisher
  • Providing liaison librarians with “talking points” related to survey findings would be helpful, particularly those that point to the librarians’ value towards improving student critical thinking and research skills.
  • There are implications here related to collection development as well, particularly as we see the extent to which faculty use media in their classrooms and assign media-related assignments to students.

Some thoughts in general:

No method provides ready “next steps” for action. And much of what we learned was not surprising, nor “ground-breaking”. Our faculty are, for the most part, like faculty at other research institutions. So it behooves us to pay attention to the literature – to other research. An example was the recent Ithaka S+R report on Rethinking Library Liaison Programs for the Humanities, and its insights into changing paths for library work with faculty.

Next steps:

We had a great turnout yesterday, with staff from Health Sciences, Research & Instruction, Outreach & Communications, Access & Media Services, Special Collections Research Center, Administration, Library Technology, Scholarly Communication, and Digital Library Initiatives.

We’re already planning the next few months’ session – there is lots to talk about and lots of interest in assessment activity – usability and undergraduate education are upcoming topics, as well as a working group for Alma Analytics skills-sharing.

Thanks especially to members of the Ithaka Research teams : Rebecca Lloyd, Justin Hill, Kristina DeVoe, Annie Johnson, and Lauri Fennell for sharing their thoughts to make this first program a success. (And Fred Rowland, present in spirit)

Posted in organization culture and assessment, qualitative research | Tagged , , , | Leave a comment

Improving our Understanding of Faculty Work: Two Research Projects

Posted on August 15, 2017 by Nancy Turner

Over the last couple of years, Temple librarians have engaged in two significant research projects with potential for informing our work with faculty. This week’s kickoff for the Libraries/Press Assessment Community of Practice provides an opportunity to discuss these projects with members of the research team, focusing on how assessment research can be turned into strategic “next steps”.

This post excerpts highlights from the summary research reports and I hope, provides a “teaser” for Wednesday’s session, when we’ll focus our attention on methods and implications for the research. That session is scheduled for Wednesday, August 16 at 3:00 in the Paley Lecture Hall. All are welcome to attend!

Religious Studies Scholarship at Temple
In October 2015 Temple University Libraries joined 40 other libraries to explore the research practices of faculty in religious studies. Coordinated by Ithaka S+R, the project sought to understand the resources and services these scholars need to be productive and successful in their research. Here at Temple, we conducted 12 in-depth interviews. Members of the team included Fred Rowland, Justin Hill and Rebecca Lloyd. This “ethnographic” method of listening to faculty, offered us new and different insights into the perceptions of our users. It deepened our understanding of the many challenges scholars face in conducting research and establishing a secure place in the academic world. Some takeaways:

• The Temple University Libraries and academic libraries in general are strong advocates and supporters of open access publishing. However, librarians need to be sensitive to the institutional pressures that faculty face in the areas of tenure, promotion, and merit-based pay increases. The current academic reward system prioritizes selecting a publisher based on prestige and frequently on the speed of publication. In contrast to the sciences and social sciences, concerns about metrics related to citation count or social media impact were less evident in faculty decision-making. Selecting the correct target audience is very important.

• One inference we draw from our interviews is that faculty perceive open access publications as offering no meaningfully defined audience. It is easy to imagine the open web as a vast ocean of undifferentiated content. Traditional publishing serves as a locus of attention, a node on a network, in which a definable audience can be imagined. This is an area where education may come into play. As our outreach efforts foster a greater awareness of the open access institutional structures that promote discoverability and coherent research communities these options may become more attractive.

Faculty Survey
In the Fall of 2016 we also conducted a localized version of the national Ithaka S+R Faculty Survey , deployed to 3,678 faculty members with a response rate of 15% (548 responses). In many ways our survey overlapped with the content areas of the interviews, with questions related to scholarly communication and publication practices and attitudes towards open access in addition to perceptions of student research skills and undergraduate instruction practice. Members of the analysis team included: Fred Rowland, Kristina DeVoe, Rebecca Lloyd, Annie Johnson and Lauri Fennell.

Key Survey Findings
• 80% of faculty strongly agree in the importance of the library as it helps undergraduates develop research, critical analysis and information literacy skills. This figure is 10% higher than other research institutions responding to the national survey. Over half of faculty strongly believe that undergraduate students have poor skills related to locating and evaluating scholarly information, although improving those research skills is very important to the majority (70%).

• The roles librarians play are also highly valued. Less than 12% of faculty believe that “with easy access to academic content online”, the role of librarians at Temple University is less important.

• 90% of faculty respondents strongly agree that it is important the library pays for materials they need for their research and teaching. And while the library’s collections are still of prime importance for faculty research and teaching (89%), 71% are supplementing this with freely available online resources.

• Over 80% prefer the print format of a scholarly monograph for reading in depth. A smaller majority (65%) prefer the digital format for searching. This preference of format is significantly different for journals. Most faculty (70%) are “completely comfortable” with the cancellation of a print journal if the electronic format is available.

• And of interest as we consider our own institutional repository, more Temple faculty say that in the past five years they have shared their research via a preprint (65%) than via a blog post or social media post (50%). In addition, a majority (72%) of faculty support mandatory publicly funded research be made freely available online.

• 46% of Temple faculty respondents have received or are receiving government or other external funding for their research. This is of interest for our library in planning research data management services, when federal funding typically requires a plan for making research data more openly accessible. 83% of faculty report storing their research data on their local computer.

Final Thoughts on Survey
The survey data provides us with a head start in understanding faculty needs, as well as potential “gaps” in their awareness of library services related to open access, data management, as well as our rich multimedia collections. But a survey like this does not provide us with specifics or clear cut future directions. For those insights and ideas, we will continue to rely on our close relationships with faculty and understanding of changing work practice. While the survey provides us with a broad view across all disciplines, the survey method must be mixed with a more in-depth, nuanced, qualitative approach to understanding the research and teaching practices of our users.

Posted in qualitative research, surveys | Tagged , , , | Leave a comment

Library Assessment Repository Project

Posted on July 26, 2017 by Nancy Turner

Over the last several years, the American Library Association’s LLAMA (Library Leadership & Management Association) Assessment Community of Practice has been exploring community interest in a repository for library assessment resources. The repository would provide open access to survey instruments, case studies, raw data – providing the community with a single space for locating and sharing materials.

This spring I conducted a survey on behalf of LLAMA to gauge interest –  379 responses.The majority, 86.8% (n=329) of respondents were from academic libraries, and 7.7% from public.

I spent much of my ALA conference in June reporting on the survey findings, but I think they are of general interest and worth relating here as well. Particularly because some of the issues and concerns raised might apply to establishing other types of open access repositories – whether institutional or disciplinary.

The survey was short – just 11 questions. This may be a reason we received lots of comments from librarians who wanted to tell us even more!

Question: Please indicate how likely you would be to use these types of resources if they were part of a freely available online Library Assessment Repository:

By a large margin, the types of resource most likely to be used are assessment instruments (surveys, questionnaires, rubrics). Least likely to be used are raw data.  Raw data was also least likely to be deposited.

Scale Green=High; Red =Low

Question: What factors would contribute to your willingness to use these resources?

Prospective users are looking for quality resources that are relevant to their immediate research need. Out of 272 free-text comments, themes included:  quality, relevance to the work, peer-review, ease of access and free availability, robust search functionality, metadata and overall organization of the resources.

Question: How likely would you be to deposit the following types of materials?

Survey responses indicate somewhat less likelihood of deposit of assessment resources than the likelihood of use.  264 respondents would be likely or most likely to deposit assessment instruments.  This is highest ranking type of material for deposit, with case studies also ranking high for potential deposit.

Scale Green=High; Red =Low

Question: What factors would contribute to your willingness to share your resources?

Prospective contributors to the repository are also looking for ease of use and accessibility. Other themes from the 244 free-text comments: concerns that one’s own work would not be “worthy” of sharing, lack of permission from library administration or IRB (Institutional Review Board), and privacy concerns. There is fear that submitting raw data would result in misunderstanding or misusing it. How would that data be “framed” or annotated so that its analysis made sense?  Another concern:  “Why would I submit my work to a repository when that doesn’t count towards my tenure/promotion process?”

Next Steps:

What started as a great idea has taken on some “legs”, but has also turned into a complex project with high expectations. Our assessment community members would like lots of rich, quality, peer-reviewed content that meets their particular research needs for their particular library type. The repository must be easy to access and deposit must also be easy, yet resources must be described thoroughly with rich metadata to insure discoverability.

The need seems greatest for small libraries who may not have many opportunities for professional development. Librarians who are seeking support and information as they take on new responsibilities in assessment expressed a particular need for the resources and networking opportunities made available through a repository.

LLAMA Assessment is now looking at how best to more forward, particularly as it seeks partners for building technical infrastructure and soliciting deposits.  The survey confirmed the value and need for a repository of library assessment resources, but is just an impetus. Now the real work begins.

Note: To read the report: AssessmentRespositoryNeeds_SummaryReport

Posted in surveys | Tagged , , | Leave a comment

Notes from the Field (and Lake) : ALA 2017 Chicago

Posted on July 5, 2017 by Nancy Turner

Photo: D. Lacy

This June many of us headed off to Chicago for the annual American Library Association conference. The weather was perfect – warm temperatures with great views of the sky and lake, enjoyed off hours, of course.   I asked for contributions to this post –  for colleagues to share a program that was particularly useful or provocative. I’ve excerpted from those accounts, but please do feel free to contact folks individually with questions, follow-ups.

Brian Boling reports on the session, How Are Our Instructors Truly Using Media? A Multifaceted Approach to Developing Departmental Course Media Use Profiles by Scott Spicer from University of Minnesota.  “Scott has developed a workflow for ingesting course syllabi into NVivo [a textual analysis tool] and coding the syllabi for mentions of required viewing and media production assignments.  He shares his analysis with subject liaisons with an aim towards informing their outreach to faculty – particularly important as faculty may not be aware of the rich offerings at the library (both collections and services).”

Steven Bell also reported on different ways librarians can reach out to faculty. He describes an ARL-sponsored program on changing liaison roles, Talk So Faculty Will Listen; Listen So Faculty Will Talk. “It addressed the common frustration liaisons face when it seems that faculty ignore efforts to build bridges that could lead to enhanced use of library services and resources for students and faculty. The gist of the program, and this might not come as a surprise, is that it takes more than informational email blasts and participating in orientations (good but low impact activities) to build productive relationships with faculty.” Four strategies for talking with faculty, suggested by panelists:

  • Mistakes to avoid: 1) You do most of the talking 2)Focusing on “We have a resource to solve that…” 3) Ignoring emotions/feelings
  • If these mistakes sound like you…you need to talk less and listen more. Start by developing open ended questions to engage faculty in a conversation where they do most of the talking.
  • Four enabling questions were suggested: 1) What are your challenges as a researcher and teacher? 2)How is your field changing? 3)What are the emerging areas of emphasis for the department? 4)What are your hopes for library services?
  • Be concise in messages to faculty that are informational. Emphasize time-saving and productivity aspects of resources/services.

Adam Shambaugh also cites a program on emerging roles for librarians, ALCTS’ Creating the Future of Digital Scholarship Together. “Laurie Allen spoke about the digital initiative that began at the Penn Libraries to preserve federal climate data in the wake of the 2016 presidential election. This initiative—known as Data Refuge—would eventually include librarians, preservationists, and scholars from across the country. Laurie’s presentation, showcasing a variety of emerging roles for librarians and information professionals, was one of the most interesting and insightful talks I attended at ALA this year. “

Justin Hill attended a meeting of the SHARES group to discuss interlibrary loan practices and concerns. “Among the topics discussed was the investigation of the viability of allowing on-site access to visiting scholars from member institutions.  This conversation is indicative of the types of things being explored by consortia to reinvent what’s possible within groups of like-minded institutions. Where activities within consortia has traditionally been limited to just supplying ILL to members, the discussion has now shifted to begin discussing other possibilities like on-site access, shared programming, consortial collection building activities, and group purchasing powers.”

David Lacy attended pre-conference focusing on Search Engine Optimization with the “ambitious” theme: “If someone is searching Google for something that our Library holds, it should be at the top of the list”. Dave says, “the session provided me with an abundance of ideas that we can apply to our new online environment.”

Joe Lucia  attended a session presented by Chicago YWCA Chief Executive Officer Dorrie McWhorter, [formerly a CPA and management consultant for a Big Ten accounting firm. “Her theme was ‘the business of social impact’ and her focus was on how she’s worked over the past four years to build a workplace environment for the YWCA staff that is emotionally healthy, supportive & positive. Her intention in fostering that environment is to enable people to deal with the stresses and demands involved in bringing services to women and girls who are often in situations of critical need. Much of her presentation focused on interpersonal dynamics, relationship-building, and the core perception that people who feel valued, recognized, and respected are likely to carry those traits into the field in their daily work. She talked about a commitment to self-care for her staff and the benefit of regular (weekly) check-in meetings between managers and their team members that addressed current challenges, recent successes, continuing problems, and emotional well-being. Sounds touchy-feely, of course, but she reported a fundamental change in workplace culture in the direction of continuous connection and continuous professional growth that has allowed the organization to completely eliminate performance reviews.”

That would be nice.

I asked for some “fun facts” as well. Justin gets the prize for most weird hotel, rumored  to be haunted by the ghost of Al Capone, and Dave’s meal sounds most interesting:  at Maude’s Liquor Bar,  he enjoyed “ liver pate, steak tartar, roast duck salad, and Sazarac.” Me,  I had the mixed popcorn special at the airport – half caramel and half cheese – sounds gross but it was quite good!

Thanks to everyone for contributing.

Photo: N. Turner

Posted in conference reports | Tagged | Leave a comment

Assessment for Planning: Special Collections Research Center Uses Data to Drive Location Decisions, and More

Posted on June 20, 2017 by Nancy Turner

Planning to move the Special Collections Research Center  from its current space to the new library building takes careful planning, with different challenges than the general collections. For one, there are manuscripts AND books to be stored, with collections that vary in size from a few pages to many, many boxes. Not every item is described at the same level. Much is very rare, and much is fragile. And yet, through our robust discovery systems and extensive finding aids, an item could be requested at any time.

Fortunately, the Special Collections Research Center (SCRC) staff including Jessica Lydon (a member of the Libraries’ Assessment Group), and the rest of the “Mapping the Move” team, have already started to collect and analyze data that will inform the decision making as to what materials go where.

For instance, SCRC collections will be housed the new library in two separate compact shelving installations, open shelving in the reading room, and in a dedicated aisle  within the Automated Storage Retrieval System (ASRS), as well as off-site at the library depository.  Each of these locations has strengths and challenges in terms of proximity, accessibility, access, and time to retrieval.

Jessica and the team are using reports from AEON (the SCRC registration, circulation, and tracking  system) and DB/TextWorks (the collection management database), as well as other reference and cataloging statistics to help map material to appropriate storage. A collection that is frequently used, but not too large, might  go to onsite storage. A collection that is extensive and housed in many, many boxes, may go to the depository  where contents can be searched by staff prior to delivery, or to the ASRS. Any material heading to the ASRS will need to be listed at the box level, with each box added as an item to the collection’s catalog record.

While data is useful, it also must be reviewed with a human eye to make meaningful decisions. For instance, the Albert Einstein Medical Center records received lots of use this last year, but that demand was influenced by their 150th  anniversary and planning that went into an online exhibit and a history of the center.  That collection may now be stored in a less accessible space.  The Philadelphia Special Investigation Commission (MOVE) Records receive constant use from many types of patrons (students, community, media) so this will be housed onsite.  The Science Fiction book collection, which is used less frequently and cataloged at the item level  will go to the off-site vault.

Data also will inform decisions about public services. The move to the new library may  necessitate closing the reading room for some periods. Data shows us that the type of patrons using special collections in the summer (visiting scholars) have a different use pattern than the heavy afternoon use when classes are in session (Temple undergraduates).  This kind of information will be useful as well when the communication plan is put into place.

Staff in SCRC are asking questions of other services as well.   In October 2016, they  conducted an analysis of their reference transactions, applying a READ (Reference Effort Assessment Data) scale to understand the complexity of the reference work they were doing. This month they are doing that exercise again, in order to compare the two time frames, when different types of patrons use SCRC. While the READ scale was not difficult for staff to include in their record-keeping, they admitted that a generic tool like READ does not adequately reflect transactions in special collections — reference requiring a combination of expert knowledge of the collections as well as complex strategies for addressing questions, and specialized interview skills.

Using special collections data to guide decision-making is never a straight path, and seemingly straight-forward questions may require multiple data sources.  “What is the time frame between the creation of a finding aid, or the cataloging of a rare book, to its subsequent use?” We have anecdotal evidence, but not concrete numbers. To get at this answer, SCRC staff members  consulted reference statistics and in-house use documented in the  AEON system, technical services processing data, and looked at Google Analytics for web use information. And there may be no discrete dates to compare:  the processing of a collection can take months.

This kind of fuzziness in addressing questions would make most of us throw up our hands in frustration and despair! Fortunately, in SCRC, we have dedicated folks who are willing to handle the complexities as we consider the special nature of their collections move.

 

Posted in data-driven decision making | Tagged , , , | Leave a comment

LACUNY 2017: What’s Next for the Academic Librarian?

Posted on May 25, 2017 by Nancy Turner

Last week I had the opportunity to attend the LACUNY 2017 Institute, a small conference sponsored by the Library Association of the City of New York at  LaGuardia Community College. The theme was The Future of Librarianship: Exploring What’s Next for the Academic Librarian. I was drawn to the conference anticipating several lively panel discussions of interest to me: the application of organizational development towards more inclusive environments at work, egalitarian teams for grassroots organizing, and building the post-collection library (i.e. no physical books at all).

What a wonderful professional development opportunity for librarians.

In this post I want to share the themes from Barbara Rockenbach’s rich keynote address, Thinking about IMPACT on Research, Teaching and Learning. Rockenbach currently serves as Interim Associate Director for Collections and Services at Columbia; her talk drew upon broad experience,  from computational methods applied to humanities scholarship to undergraduate research and instruction. I especially liked how she  connected the themes of space, collections, partnership and context to both current research and  real life examples from her home institution.

The library can be a space that puts the user front and center, preparing spaces that are flexible, that facilitate conversations, that are moving from passive to active space in support of student success. The space may be quiet, social or collaborative. For instance,  Studio @ Butler is “co-owned and co-programmed by faculty” and provides open labs, instruction in tools like R and Python, and “talk shops” that incorporate both formal presentation and experiential workshop.  A great example of David Lankes’ “library as conversation” with the community.

Rockenbach used Lorcan Dempsey ‘s ideas to talk of the future of library collections – valuing less what a library “owns” to the service a library facilitates.  We will be thinking less about “our collections, our systems, our buildings” to considering the communities we serve. Thinking less about the life of the library towards thinking about the library in the life of the user.

The library is a partner in the work of scholars. Columbia participated in a collaborative Librarian Liaison Institute pilot with Cornell, ARL, and the University of Toronto. Its much discussed report (here at Temple too!) on the evolving role of liaison librarians had this  as one conclusion:  librarians need to focus away from the work of librarians to that of scholars, and to develop engagement strategies based on their needs and success factors.

The April 2017  US Library Survey suggests a disconnect between university administration and library leadership, in terms of how the library contributes to the teaching and research mission of the institution.

Library directors are pursuing strategic directions with a decreasing sense of support from their institutions. There is evidence across the survey that library directors feel increasingly less valued by, involved with, and aligned strategically with their supervisors and other senior academic leadership.

Pointing to Columbia’s efforts, Rockenbach says that libraries engaged with strategic planning efforts are less likely to suffer this fate. Libraries need to commit its values as well as its objectives to the larger institutional context.  For instance, what does it mean to be a diverse and inclusive library?

The library provides space for learning, but also serves as  an alternative to the formal classroom. The idea was captured eloquently by Ta-Nehisi Coates’ in his Between the World and Me :

I wanted to pursue things, to know things, but I could not match the means of knowing that came naturally to me with the expectations of professors. The pursuit of knowing was freedom to me, the right to declare your own curiosities and follow them through all manner of books. I was made for the library, not the classroom.

And, a big thank you to Barbara Rockenbach for allowing me to re-cycle her perspective on the  future of academic libraries.

Posted in conference reports | Tagged , , | Leave a comment

Improving How We Support New Professionals: The Resident Librarian Program

Posted on April 24, 2017 by Nancy Turner

Temple’s Resident Librarian program is now in its third year, and we have started the recruiting process for our 2nd cohort of resident librarians. I sat down with the program administrators Richie Holland, Director of Administration,  and Sandi Thompson, Head of the Ambler Branch library, to learn more about how they incorporate an assessment process to insure that this important initiative is successful.

NT: Tell me a little about the resident librarian program here at Temple. What was the impetus for initiating the program?

RH: Well, this has been on my back burner for a long time. The University of Delaware had a long standing program named in honor of civil rights leader and historian,  Pauline A. Young. Here at Temple Libraries, our overall goal is to provide mentoring and support the diversification of librarians in the profession.

As the library’s Human Resources person, I’ve attended several meetings at ALA and I participate in the ACRL Residency Interest Group. I reached out to colleagues at other institutions with programs like this, and also talked with participants in resident programs. There is a bit in the formal literature, but most of my “research” was talking with others with experience.

ST: And then we also had the Temple/Drexel/Penn staff development day, with Jon Cawthorne. He’s also part of the ACRL Diversity Alliance.

NT: What does a successful program look like? What are you aiming to accomplish?

ST: We want to instill in our resident librarians confidence as they move into a career position. They will have experience in several areas of librarianship, and have solid accomplishments, or projects that they’ve completed.  They know what it takes to be a professional. And of course we want them to thrive in their next professional job.

We had to think about how best to accomplish this. For instance, we deliberately organized the program to fit the interests and needs of the resident. They take responsibility for developing a program tailored to their interests and the experience they want.  Then we provide them with lots of support – regular meetings with us, with their mentors, with site supervisors, and healthy travel support. We picked the brains of residents at other libraries on how to structure our program to make it really attractive to prospective applicants.  One thing that came out loud and clear was to hire more two, providing the librarian with a ready cohort.

Once our resident librarians were in the program we set up different ways of “checking in” with them to insure all was going smoothly. For each rotation, the librarians wrote up their experiences, and we talked with site supervisors about what was working; the strengths and weaknesses for each experience.

NT: I know you have been carefully tracking on the program over the last two years. Tell me about some changes you are making this next cycle, based on that experience.

RH: Yes,  we asked the current residents to provide us with feedback, as well as others involved with Temple’s program. We made a couple changes to the job description.  For instance, candidates need to know they will be working with diverse management styles, and be amenable to that. Of course, that flexibility is a life-skill for all of us. Since we’re building a culture of assessment here at Temple, we added language about decision-making informed by data.

We want to recruit candidates who are a good fit for these positions, and having the right job description gives a more accurate sense of what the resident experience is like.

We’re also evaluating the timing of the different rotations – we did four, some libraries do as many as five. We want the librarians to have adequate time to complete a substantive project in their rotations. Then of course in their second year they stay in one department.

With each step of the process, from recruitment to the program structure itself, we’re continually tweaking, to make sure it works for everyone.

NT:  We don’t always think of “assessment” taking place in the HR area, but you both have modeled a process that fits that definition – you identified a goal, or something you wanted to accomplish, you conducted research into other programs, you set up the program and sought feedback continuously and from multiple sources on how well the program was working and how it could be improved, and then finally, you are making adjustments to improve the program going forward.

So thanks for sharing the process.

Posted in organization culture and assessment | Leave a comment

ACRL Conference Points the Way: Reports from the Field

Posted on March 30, 2017 by Nancy Turner

Last week many of us headed down to Baltimore (along with 3400 other librarians) to attend the ACRL (Association of College and Research Libraries) biennial conference. By all reports, it was great learning experience, plus it’s always fun to catch up with colleagues from here and other institutions. Since there was so much going on, I asked my Temple colleagues to share their highlights.

We were all inspired by the keynotes – an exceptionally diverse and stimulating set of speakers included data visualization “rock star” David McCandless (Information is Beautiful), writer and feminist Roxane Gay on the meaning of diversity, and our amazing Librarian of Congress, Carla Hayden.

Steven Bell reports that he was asked about Temple’s new library about 100 times. He attributes this to his participation in the panel discussion Ready or Not? Pressing Trends, Challenges, and Tech for Libraries – about the NMC Horizon Report 2017 Library Edition.

Fred Rowland found Andrea Brooks’  Shifting the Discourse: Information Literacy as an Opportunity to Address Intellectual Virtues  of interest, the concept that intellectual virtues like open-mindedness, self-awareness, respectfulness, curiosity, and flexibility are embedded in the ACRL Framework. Citing Jason Baehr, Brooks argued that instruction librarians should invest more energy in instilling these intellectual virtues in the students they are addressing. Also provocative for Fred was Addicted to the Brand?: Brand Loyalty Theory as a Means of Understanding Academics’ Scholarly Communication Practices Cara Bradley applies the perspective of branding theory from marketing to understand the role “brand” plays when scholars and scientists choose to submit their work. Open access journals might compete for effectively if attention was paid to this kind of branding.

The panel Rebecca Lloyd found particularly interesting was Reference: The New Dirty Word. She reports, “The topics addressed were very similar to the discussions we’ve had here about shifting librarian roles, student worker training, librarian visibility, etc.  Even though it wasn’t new territory, it was reassuring to hear that other libraries are also struggling with these questions and finding that no matter which approach they choose, there are trade-offs and aspects of the service model that need further improvements.  The main take-away was that there is no “right or wrong” on the future of the reference desk.  It’s individual to each institution and there are variety of viable paths forward.”

A presentation that stood out for Natalie Tagge was Nicole Cooke’s invited talk, “How would you like to be remembered? Expanding your pedagogy and professional practice”. The presentation described Univ of Illinois’ School of Information Sciences classes dedicated to issues of diversity, social justice, and race, gender and sexually, attempting to infuse these concepts throughout the curriculum.

A useful session for Erin Finnerty was,  ‘Everything you wanted to know about predatory publishing but were afraid to ask.’  providing a comprehensive snapshot of the current state of predatory publishing, as it relates to the role of librarians and researchers.  Erin reports that “the Q&A session was particularly fruitful, and she gave useful recommendations for educating the different members of our user community – undergrads, pre-PhD students, faculty, etc.”

I asked my colleagues what they thought they might try here at home. Natalie says this: “I may consider experimenting with making course guides organized by research process instead of organized by type of source. I saw a poster (Pathfinder or Pedagogical: Transforming Course Guides for Student Success) about a research study indicating that students retained more from guides organized around the research process. I think this could actually work well for any instruction sessions focused on evidence based practice.”

Noting how many academic libraries are involving students for peer support, Steven would like to continue our own conversations about how work and train students in making such a program a success here.

Assessment continues to be of interest.  Several of us attended Metrics Selection across the Research Life Cycle, featuring Chris Belter, the Bibliometrics Informationist at the National Institutes of  Health Library. She pointed out (perhaps this is obvious) that metrics need to be question centered, and that having “lots and lots” of metrics, though each imperfect, allows us to triangulate to establish a better view of reality.  Erin notes that NIH has an “incredibly informative LibGuide that outlines their major data aggregation and visualization tools, and provides article recommendations and examples of their work.“ Take a look at: http://nihlibrary.campusguides.com/bibliometrics

Annie Johnson’s take on the conference: “The best part of ACRL was meeting people from other libraries who are working on similar initiatives and projects.”  Rebecca also enjoyed networking: ” I am still quite new to being a ‘history librarian,’ and it is very helpful to chat with librarians at other institutions who are dealing with similar responsibilities and challenges.”

Congratulations to Annie on her poster presentation, as well to Steven Bell (a panelist and a poster-presenter).  And thanks to everyone who shared their ACRL stories.

Finally, the food highlights: Jack & Zach’s lunch counter veggie burger,  falafel, beets and an ancient grains salad from Cava Mezze, broiled rockfish, and for me, a sausage, peppers & provolone sandwich to take out and eat on a park bench, once it hit the 70s.

Note: The ACRL conference proceedings are freely available online at: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2017/ACRL2017_A.pdf  All 778 pages.

Posted in conference reports | Tagged , | Leave a comment

Grounded or Toppling Over? The Three-Legged Stool of Assessment Culture within the Organization

Posted on March 1, 2017 by Nancy Turner

Today marks my third year at Temple University Libraries – a good time for reflection on how the organization has developed a culture of assessment. One thing I have learned. An assessment librarian does not a culture of assessment make. The practice of continuously asking questions of ourselves, our value to the institution, considering the metrics and research methods appropriate to those questions and values – it takes time to embed that practice into our work.

When I interviewed for this position, I began my presentation to the staff with this slide, composed of a word cloud based on definitions of the word culture:

I asked, “What would a culture of assessment look like?”, suggesting that it meant having a shared value system among staff about the benefits of assessment practice in how we do our work – that practice then becomes a pattern of behavior and is incorporated into the thinking about our work and  improving our service to users. I think this concept still holds true.

Organizational Development

A culture of assessment must be more than the “doing” of assessment. It requires us to analyze, reflect on, and act upon our findings. For instance,

  • We must consider changes based on our focus group data, our surveys, our usability testing, our interviews with faculty.
  • We must then share our findings with staff and external audiences, and make explicit the changes we are making (or not) based on research findings.

This feedback loop is essential, and is continuous.

How do we empower our staff with the skills to do that research? It’s easy to gather data, but we need methods that insure the data and their interpretation are sound. This constitutes the organizational development aspect of building culture – in which assessment is not limited to a few but all staff are empowered with the skills to participate.

Strategic Planning

In 2014 here at Temple Libraries/Press we established an annual strategic action planning process that included an expectation for assessment with each objective. As new initiatives and ongoing ones are considered, we ask ourselves:

What does success look like? Ideally, that answer will be more than a count, but a measure of impact or growing reach into our communities.

  • Is our instruction program effective? How do we know that our students are incorporating critical thinking skills into their research and coursework?
  • Are we reaching new audiences? How are those communities engaged in our programs and how might they support the library in other ways?

These measures of impact (and return on investment) are much harder to determine. Gathering that evidence requires a commitment to planning and sustained support for a robust technical infrastructure for data collection, analysis and access.

Finally, a culture of assessment requires that these questions of effectiveness, self-examination and advocacy are systematically integrated into the fiber of the organization.

  • Are we reviewing our operations in light of our values for transparency, developing staff and collaborating across departments?
  • At budget time, do we review our spending and our work flow efficiencies as we justify new funding requests?
  • When a position goes vacant, do we reconsider that position within the organizational direction and new priorities?

In my three years I’ve learned (and as Lakos and Phipps discussed 13 years ago*) that organizational development, strategic planning and assessment serve together to balance that three-legged stool, and only with alignment in these areas will the stool stand firmly on the ground.  Equally essential are the staff, the participants in this culture, that bring assessment alive as a true shared value for the library.

Three years in and we are definitely on the ground.

*Lakos and Phipps, Creating a Culture of Assessment: A Catalyst for Organizational Change (portal: Libraries and the Academic, Volume 4, Number 3).

Posted in organization culture and assessment | Tagged , | Leave a comment

From Assessment to Leading Change in Windy, Wet Atlanta

Posted on January 30, 2017 by Nancy Turner

Coke is king here in Atlanta, so I was surprised to see comedian Paula Poundstone with a Pepsi for refreshment during her hysterical stand-up routine in front of 500+ librarians at the American Library Association’s midwinter meeting. She made much fun of our love for meetings, meetings to discuss meetings, and meetings to discuss those discussions.  I’m more fortunate than some, with only four official meetings to attend, from the ARL Survey Coordinators meeting to the editorial board meeting of ACRL Library Trends & Statistics, LLAMA Assessment Organizational Practice and ACRL’s Assessment Discussion Group. Thank goodness for the comedic relief of Poundstone!

But it was a productive conference, and I had the opportunity to learn quite a bit as well as meet up with former colleagues.  That’s one of the highlights of conferences for me. We learn from these personal networks as well, of course. At the ACRL Assessment Discussion Group, one of our topics was what libraries are doing to support student retention. Anne Moore (former colleague at New Mexico State, now Dean of Libraries, UNC-Charlotte), updated us on what’s going on in her library.

At UNC Charlotte card readers are installed throughout the library – outside study and instruction rooms, at the circulation desk, outside the building, and at the Speaker Center. (This space  is run by Communication Studies and is more than a presentation room. It is staffed with graduate students who can coach and critique practice presentations). Space user data is connected to not only Banner data (for demographics) but also responses to the NSSE (National Survey of Student Engagement). This is all pulled into a SQL database to which Tableau points, allowing UNC Charlotte  to create compelling data visualizations that demonstrate relationships between use of the library spaces (accessible to all, 24 hours a day) and students’ grades.

An ongoing programmatic theme for ALA is the future of research libraries, changing roles of library staff and strategies for supporting that change.

One of the best was Joseph Zolner’s Managing Change & Fostering Innovation. Hofner is Senior Director, Harvard Institutes for Higher Education. He led the ACRL Leadership group through an activity where our table came up with an initiative (ours was to increase diversity in ACRL membership, and then identify various types of change resistance: from loss of control to fear of uncertainty, to recognition of real threats. To learn more, take a look at Rosabeth Kanter’s article, Managing the Human Side of Change in Management Review.

Kathleen deLong (University of Alberta Libraries) spoke as part of an ACRL program on Leadership for New Roles. Referencing the Transition Leadership Wheel, she spoke of the need for balancing opposing “forces.” Capitalizing Strengths but Going again the Grain; Instilling a Sense of Urgency yet Being Realistic and Patient. Key to getting the right balance is trust.

Good message, and one that resonated with me.  Change is hard for us all, but trust and balance are key.  And I suppose, being true to our core values. Like Paula and her Pepsi.

 

Posted in conference reports | Leave a comment