An Assessment Librarian Converses with ChatGPT

Posted on February 29, 2024 by Nancy Turner

Inspired by Hyangeun (Jenny) Ji’s workshop out of the Scholars Studio yesterday on How to Talk with AI: Prompt Engineering, I continued my exploration of how generative AI tools might support my work in data analysis and assessment. Jenny’s lesson #1: The best way to learn about AI is to experiment, and her tips and guidance provided just the motivation to do so.

We need to understand trends in the use of the Charles Library. We see thousands of students come through the door each day. But our knowledge of day, time, and demographics can be anecdotal. We do get weekly gate reports, .csv files that include date, time, college, and student level. In this experiment, I asked ChatGPT to analyze two gate reports from a busy week in February (February 19-25, 2024 and February 20-26, 2023). My starting instructions: “Analyze these two csv files and then I will ask questions of you to compare the data in each.” 

ChatGPT proceeds to output a data overview of each file, detailing each column name, such as date, time, academic level and college, and entrance used. It’s rather dry and technical, just raw numbers. But kindly, ChatGPT suggests,

“If you have specific questions or analyses you’d like to perform on these datasets, please let me know! “

What a nice invitation! I did have some questions. I asked ChatGPT to compare traffic by days of the week, by time of day, and also by college. Since I like visualizations, I asked this data to be shown as a bar graph. ChatGPT complied: 

I’m impressed. But the analysis is not perfect. For instance, there is no school of Theology at Temple, nor is there a Creative Arts school. Chat GPT is using a two-letter code and predicting what that might stand for. But still… 

Chat GPT as Consultant

What would it look like to have an external data “consultant” review this data and provide their perspective? I asked ChatGPT to take on the consultant “persona” and provide me, an academic library administrator, with some insights.  Here’s the result:

  • Resource Allocation:

The clear peak usage times around midday (10 AM to 2 PM) suggest a need for optimal staffing and resource allocation during these hours to ensure efficient operations and user satisfaction. Consider scheduling more staff or offering additional support services during these peak times.   

  • Space Utilization:

With weekdays experiencing significantly higher traffic than weekends, consider evaluating the effectiveness of the space and resources available during these peak days. This could also be an opportunity to promote the library as a study space during less busy times, like weekends, to even out usage.   

  • Targeted Programs and Services: 

The data shows varying access patterns among different colleges. Tailoring programs or workshops to the needs and schedules of students from colleges with the highest foot traffic (e.g., Science and Technology, Liberal Arts) could improve engagement and utilization of library resources.  

For colleges with increasing access year over year, such as Engineering and Medicine, consider collaborating with faculty to offer specialized resources or services that support these disciplines.

  • Security and Access Control:

The analysis did not directly address security, but consistent monitoring of access patterns can help in identifying any irregularities or security concerns. Ensuring that access control systems are up to date and conducting regular reviews of access logs can enhance security measures.

  • Observations:

The significant access by the Science and Technology college, particularly its increase from 2023 to 2024, suggests a high engagement level that might correlate with specific resource needs or study patterns. This could warrant a closer look to ensure resources meet demand.

The decrease in accesses for the Health Professions and Liberal Arts colleges might reflect changes in enrollment, curriculum delivery methods (e.g., more online resources), or shifts in study habits. It would be useful to investigate these trends further to adapt library services accordingly.

The notable increase in Medicine and slight increase in University College suggest growing areas of academic focus or expansion. These trends could guide future resource acquisitions or service expansions.

Human Reflections

The analysis provided by ChatAI is quite thorough, even insightful, given the limited quantity of data provided. Imagine what we might learn with more numbers, or the efficiencies we might gain by refining these queries. Of course we need to be concerned about replicating our analysis, and hallucinations can be problematic. In this case, ChatGPT (4) allows me to peek under the hood by viewing the Python codes used for analysis. I can also share the query and response with others through a QR code or link. But I like how ChatGPT presented its analysis in an organized and usable style that is easily shared with others, like colleagues in access services and facilities. What a wonderful way to initiate a conversation with human stakeholders.

I incorporated another hint from Jenny, one based on research. She suggests we use polite, respectful and encouraging language when conversing with ChatGPT. While this can actually provide better responses, it also created a relationship that felt like a partnership as we worked through the analysis together. The conversation became a collaboration, and that tight division between human and AI seemed less obvious to me.

Thank you to Hyangeun and to all the library staff who provide guidance (through workshops and consultations) on best practices as we explore and understand AI tools in our work.

Posted in assessment methods | Tagged , | Comments Off on An Assessment Librarian Converses with ChatGPT

A Better Understanding of Library Work Effort

Posted on January 11, 2024 by Nancy Turner

Happy New Year! What a year we had in 2023. Tumultuous. I predict that 2024 will be equally filled with change and uncertainty. Before we move on to those new challenges, a brief update on an important assessment effort conducted last year, the Strategic Resources Review.

The Review was undertaken to get a clearer sense of how staff, in the aggregate, are spending time towards providing library services.

  • How many staff are engaged in work related to collection management?
  • Preparing for instruction?  
  • What percentage of our time is spent with administrative work?
  • Are there ways in which we should be shifting our efforts towards a different future of work? For instance, do we need more time to spend on licensing electronic resources? More time developing workshops for students and faculty on the use of artificial intelligence tools?  

Our aim was to systematically document how time was being spent on what types of activities This descriptive picture has potential to benchmark current work effort as we identify emerging areas of service to guide future investment.

The approach  

Understanding how staff spend time is complicated. We might have asked everyone to document each hour of each day’s work for a year! Instead, we asked each department and unit head to provide a list of all activities and services provided by their unit and the percentage of time (over a year) that staff, in the aggregate, spend on this work. For each department, the total work effort should add up to 100%.

All this was compiled into a large, shared spreadsheet (293 rows), with percentages multiplied by the number of staff in each – amounting to a rough calculation of how much effort we expend, as an organization on services. To strengthen the analysis, we asked about service audience (internal, external) and whether work in that area was increasing or decreasing. We asked how trends were documented (for instance, tracking transactions).

We designated this descriptive phase of the review our Services and Work Effort Catalog.  Admittedly subjective and broad, this kind of analysis allowa us to see at a high level, how we use our time.  

Early findings 

Findings from the effort inventory were not too surprising but describe in a new way how we are currently investing in categories of service at the libraries.

For instance,  

  • 1/3 of our activity is devoted to “Resource Access” – this includes circulation, cataloging, acquisition of new materials and stacks management. 
  • Almost 1/5 relates to Administration – this might be HR functions, committee and professional service activity, or time spent with internal communications. 
  • We learned that for many services, we’re just not tracking our time investment in a consistent, systematic way. This makes sense. It’s easy to count our instruction sessions, less straightforward to calculate time spent preparing that instruction.

While this descriptive exercise was interesting, there are bigger, unanswered questions. Does this allocation of effort reflect our values as an organization? Is the effort aligned with the impact we desire to have on our communities in support of the work of students, faculty and the community? And how do we measure that value and impact? Perhaps our biggest takeaway is the difficulty quantifying our work in this way, that “return on investment” and true productivity and value are more than numbers that can be calculated with a formula.

–  

Posted in data-driven decision making | Tagged , | Leave a comment

Asking for help from ChatGPT

Posted on November 2, 2023 by Nancy Turner

I’ve been exploring prompt engineering this week in my efforts to learn more about the uses of AI – for personal interests and professional work. It connects to our thinking about the use of AI tools for instruction and by students – a recent study reported in Inside Higher Ed indicated that college students are far more likely to use AI than their instructors. Temple is ahead of the game in developing policies for student use of AI. But what does “use” mean exactly?

ChatGPT can do far more than write homework assignments. My early learning in prompt “engineering” through Coursera gets into these complexities and possibilities. I highly recommend it.

In learning about the Chain of Thought prompt, I asked ChatGPT to Provide steps for a novice viola player to take in practicing a part for in a chamber ensemble. Break out the practice routine into simple steps for 30 minutes of daily practice. 

The prompt yielded this output:

Practicing a part for a chamber ensemble on the viola requires a structured and focused routine. Here’s a step-by-step practice routine broken down into simple 30-minute sessions for a novice viola player: 

Day 1: Establishing the Foundation (30 minutes) 

  • Warm-up (5 minutes): Start with some basic warm-up exercises to get your fingers and muscles ready. Practice scales or simple finger exercises to improve dexterity. 
  • Review the Sheet Music (5 minutes): Spend a few minutes looking over the sheet music for your part. Familiarize yourself with the key signature, time signature, and any challenging passages. 
  • Basic Technique (10 minutes): Focus on the fundamental techniques such as bowing, finger placement, and intonation. Play through the section slowly, paying attention to correct fingerings and bowing direction. 
  • Rhythm and Timing (5 minutes): Work on the rhythm of your part. Use a metronome to ensure you’re playing in time. Start at a slow tempo and gradually increase it as you become more comfortable. 
  • Play Through (5 minutes): Play through the entire part at a slow tempo, concentrating on maintaining proper technique and rhythm. 

Day 2-4: Refining Technique (30 minutes each day) 

  • Warm-up (5 minutes): Begin with warm-up exercises to prepare your fingers and muscles. 
  • Isolated Practice (15 minutes): Break your part into smaller sections or phrases. Work on one section at a time, paying close attention to intonation, bowing, and dynamics. Repeat each section until you’re comfortable. 
  • Integrate with Metronome (5 minutes): Combine the sections you’ve practiced with a metronome to improve your overall timing and coordination. 
  • Play Through (5 minutes): Play through the entire part at a slow to moderate tempo, focusing on smooth transitions between sections and maintaining correct technique. 

Day 5-6: Musical Expression (30 minutes each day) 

  • Warm-up (5 minutes): Start with your usual warm-up routine. 
  • Expression and Dynamics (15 minutes): Pay attention to the musicality of your part. Experiment with dynamics, phrasing, and articulation to make your playing more expressive. Try different bowing techniques to achieve the desired musical effect. 
  • Slow Practice (5 minutes): Play your part at a slower tempo, concentrating on expressive playing. Use vibrato and bow control to add depth to your performance. 
  • Play Through (5 minutes): Play through the entire part, focusing on musical expression and dynamics. Try to convey the emotions and nuances of the music. 

Day 7: Putting It All Together (30 minutes) 

  • Warm-up (5 minutes): Begin with your warm-up exercises. 
  • Full Tempo (15 minutes): Play your part at or close to the intended performance tempo. Focus on maintaining all the techniques and expressions you’ve practiced. 
  • Play Through (10 minutes): Play through the entire part multiple times, gradually increasing your confidence and fluency. 

Remember to take short breaks between practice sessions to prevent fatigue. Consistent daily practice and attention to detail will help you prepare your viola part for a chamber ensemble effectively. Adjust the practice routine as needed to accommodate your progress and specific challenges in the music. 

Citation: Chat GPT 3.5 November 1, 2023

This program made a lot of sense to me – I’d love to hear from other musicians about it.

In addition to being a novice viola player, I enjoy photography. I took lots of pictures when traveling in Ireland, and I particularly liked this one:

I asked ChatGPT, How could I make this photograph more impactful? and again, the bot’s suggestions are worth considering:

The photograph you’ve shared captures a serene urban scene with beautiful natural reflections on the water. Here are some suggestions to make the photograph more impactful: 

  • Cropping: Consider a tighter crop to eliminate unnecessary distractions. For example, cropping out some of the sky can place more emphasis on the reflections in the water. 
  • Enhancing Colors: Increase the saturation slightly to make the colors pop, especially the golden hues from the sunlight and the greens from the trees. 
  • Contrast and Brightness: Adjusting the contrast can help to make the image look more dynamic. Brightening specific areas can also help draw the viewer’s attention to key elements. 
  • Highlighting Reflections: Emphasize the reflections in the water by increasing clarity or sharpness. This will add depth to the photo. 
  • Leading Lines: The railing in the foreground is a strong leading line. You might want to ensure it leads the viewer’s eyes towards the main subjects, such as the bridge or the line of buildings. 
  • Sky Enhancements: If the sky appears too bland, consider using gradient filters to darken the top part, bringing out more details in the clouds. 
  • Vignetting: Add a slight vignette to the edges of the photo. This darkens the corners slightly and can help draw the viewer’s eye to the center of the image. 
  • Remove Distractions: Consider removing any small distractions using a tool like the clone stamp. For example, any litter on the pavement or small objects floating on the water. 
  • Experiment with Black and White: Sometimes, turning a photo black and white can give it a timeless, classic feel, emphasizing shapes, lines, and contrasts. 
  • Add a Focal Point: While the scene is beautiful, adding a focal point such as a solitary person on the bridge or a boat on the water might create a story or theme for the viewer to latch onto. 

Remember, the best edits will often depend on the mood or emotion you want to convey with the photograph. Always keep your intended audience and the purpose of the photograph in mind when making edits. 

Citation: ChatGPT 3.5 November 1, 2023

Finally, I asked ChatGPT, What famous paintings are similar to this picture in terms of color and content? While I’m not familiar with all these paintings, the suggestions seemed to me quite useful, providing a helpful starting place for further exploration.

The photograph you’ve shared showcases an urban scene with buildings, a bridge, calm water with reflections, and a rich golden hue, reminiscent of the warm lighting found during the golden hour. Here are some famous paintings that share similarities in terms of color palette and content: 

  • “The Bridge at Argenteuil” by Claude Monet: Monet painted many scenes featuring reflections on water, bridges, and buildings. The colors and tranquility of this specific painting resemble the mood captured in your photograph. 
  • “Boulevard Montmartre at Sunset” by Camille Pissarro: The warm colors and urban setting of this painting align with the mood and content of your photograph. 
  • “Houses of Parliament, Sunset” by Claude Monet: The golden and warm hues of the setting sun, along with the reflection on the water and the silhouette of a city, have similarities to the photo you’ve shared. 
  • “View of Delft” by Johannes Vermeer: Although the color palette is cooler, the focus on buildings, water, and reflection has parallels to your image. 
  • “The Starry Night Over the Rhone” by Vincent van Gogh: This painting has a night scene with reflections on water and buildings on the side. The colors are richer and darker but evoke a similar tranquil mood. 
  • “Evening on Karl Johan” by Edvard Munch: The urban setting, with a play of light and shadow, creates an atmosphere that can be related to your photograph. 

While these paintings might not be exact matches, they each share certain elements of color, content, or mood that resonate with your photograph. If you’re interested, you might want to explore these works further to see how artists have historically captured similar themes and scenes. 

Citation: Chat GPT 3.5 November 1, 2023

These examples are just for fun, but as we consider our policies related to the use of generative AI in higher education, it’s easy to see how challenging and complex the determination of “use” might be. For what purpose might we use AI and at what stage of the research or creative process? We would never discourage a student from asking for assistance from their human instructor about best approaches to practice, improvements on a draft or creative work, or recommendations for related content to explore. And what an opportunity for learning – to learn how best to mine large learning models, ask better questions, and to critique the results.

Posted in instruction and student learning, research work practice | Tagged , | Leave a comment

Designing Libraries: Finding New Adjacencies to Improve Experience

Posted on October 19, 2023 by Nancy Turner

The ways in which we use libraries is changing. How we conduct research. How we use (and don’t use) print collections to seek information. How we design our physical spaces to welcome students and create a sense of belonging for diverse communities. A theme throughout the Designing Libraries conference last week was how situating services and spaces together and creating unexpected aesthetic experiences provide for both convenience and a welcoming environment for students today. Browsing the book stacks is no longer what drives traffic to the library.

The conference was held at the University of Arizona’s sprawling Tucson campus. Imagine a Student Success District comprised of three adjacent buildings over nine acres! (Well, space is different in the West).

Designing the exterior landscape that connects those buildings presented many engineering and aesthetic challenges, according to keynote speaker Christine Ten Eyck. In Tucson, providing for shade is necessary and the plants must be able to thrive in the hot, arid climate. With little maintenance. The landscape design is inspired by the city’s surrounding arroyo’s – meandering paths for waterflow, here providing for an urban oasis that attracts both students and wildlife. Ten Eyck’s themes of aesthetics and nature ran throughout the conference. Others spoke to how outdoor landscaping and indoor plants contribute to students’ sense of well-being.

Another example of situating aesthetics in new ways to support student wellness and belonging is the Artist in Residence Program at Geogia Tech. This is a school that is predominantly graduate level STEM (Science, Technology, Engineering & Mat) – disciplines not frequently associated with the arts. Artist residents are asked to engage the community in an artistic and aesthetic exploration of the role of the libraries in society.  The project, now in its third year, supports artists, and as importantly, engages students in an experience that encourages them to think about art in new ways.  

A centerpiece for the conference was the University’s Student Success District. The main library is part of the district alongside technology help (and lending), career services, academic support and tutoring, study rooms, a data visualization lab and a space for makers.

CATalyst Maker Space at the Student Success District
Technology tools to borrow

While the luxury of space provided by three buildings is not something we all can afford, the Success District demonstrates also creative thinking and attention to student needs made possible by collaboration between multiple administrative organizations. These services are not all under the control of the library! But the shared goal of providing a welcoming space for students that supports their learning in a variety of ways – that can be replicated by all of us.

The convenience and ready access to a variety of services is not limited to books, technology or academic support. Food is at the ready too, with these darling robots that travel autonomously throughout campus to deliver food and drink. A service that works better when there is sunshine 350 days out of the year.

Student pets robot. Credit: Chris Richards/University of Arizona

Posted in conference reports, library spaces, user experience | Tagged , , | Leave a comment

Assessing the Impact of Purchasing (and Promoting) Etextbooks 

Posted on September 26, 2023 by Nancy Turner

We start the semester off with a post from Karen Kohn and members of the Open Education Group (Steven Bell, Andrew Diamond, Courtney Eger, Kristina De Voe, Janeen Lamontagne, Alicia Pucci) that describes in detail how Karen and the team are evaluating the effectiveness of promoting this important but time-intensive effort. Karen’s account is a great example of assessment, but also how hard we work towards ensuring our collections support the curricular needs of our students.

Since Spring 2018, Temple University Libraries have been systematically purchasing electronic copies of books assigned in courses. We began this effort in response to research showing the difficulties students have in paying for textbooks. (See previous posts on the Library News blog.) Building a partnership with the campus bookstore, we request a list each semester of all the books faculty have asked the bookstore to order for their courses. As Collections Analysis Librarian and part of the Acquisitions department, I (Karen Kohn) am responsible for reviewing the list and ordering textbooks. I check the list against our library holdings to see what we have as an ebook, and then I check which additional texts we’d be able to buy as ebooks. For many years, Media Services Librarian Brian Boling has been maintaining an Etextbook Database to facilitate students’ finding the library copies of their texts. 

After the Libraries’ Open Education Group formed in fall 2019, we added an assessment component to the etextbook acquisition and promotion process. The original goal of our assessment was to estimate how much money the Libraries had saved students by purchasing electronic copies of their textbooks. After seeing a presentation by Villanova librarians John Banionis and Marianne Watson in fall 2020, I mapped out a process that the Open Education Group has since used for estimating savings. The calculations have turned out to be useful in ways beyond our initial intention. 

The Details of Calculating Cost Savings

In theory, the estimated amount of money the Libraries save students is the cost of the book multiplied by the number of students who didn’t have to pay that cost. In practice, the calculations are a little more complicated. Student workers record the final enrollment for each course and the price of the textbook. We instruct them to record the “Buy Used” price, as this falls between the cost of a new book and the cost of a rental. Once the semester is over, I collect information on how many times each book was used. If a book is available on several sites, I add together the usage from all the sites. Usage is measured with COUNTER reports, which follow an international standard. COUNTER reports provide a metric called Unique Title Requests, which tallies how many sessions were spent in a given book. 

Including usage data allows us to adjust the savings estimate downward if not every student used the book. I calculate the estimated savings using whichever is lower–the number of enrolled students or the Unique Title Requests. An example can illustrate the reason for doing this. If a course had 25 students and its textbook was used 100 times, we have only saved at most 25 students the cost of buying the book. If a course had 25 students and its textbook was used 5 times, then at most 5 students used it, and we only saved those five the cost of purchasing the book. Multiplying the lower number by the Buy Used price produces an estimate of the amount the Libraries have saved students by providing an electronic copy of their textbook. 

Promoting the Etextbook Purchase Program

A surprise for the Open Education Group initially was the number of etextbooks that were not used at all! While the number of unused books varies by semester, it is still notable that any book assigned in a course and available through the Libraries would have no use! We decided to try promoting the books more to see if it would make a difference. 

Looking at the textbook usage over time has allowed us to see the impact of various changes in our processes and promotion efforts. In fall 2022 for the first time the Open Education Group used a mail merge to notify every faculty member that the Libraries had their textbook. The mail merge feature lets us customize emails by pulling information from a spreadsheet, so each email contains the professor’s name, the course number, title of the book, and a link to our catalog. We got a generally positive response to this, and a slight decrease in the number of books that were unused.

Promotion Is Working

So far, the lowest-ever number of unused etextbooks was the past spring. Two parts of the process might have affected this number. One was the mail merge, which we used that semester for the second time (thanks to Andrew Diamond). Another factor was that for the first time, the bookstore sent us the textbook list ahead of the start of the semester, and we were able to notify faculty before the semester started. In fall 2023 we were not able to notify faculty until the end of the first week of classes, so we will watch to see if that leads to fewer books being used. Over time, assessment will help us see the impact of both our purchases and our promotion efforts, so we can make the etextbook collection as useful as possible to Temple students. 

Posted in collections | Tagged , , | 2 Comments

Letters from the Field: Evidence-Based Research for Librarians

Posted on August 2, 2023 by Nancy Turner

This month’s post is from Sarah Bauman, Head of the Charles A. Kraus Library at Temple’s Schools of Podiatric Medicine. Sarah was so excited by the recent conference she attended, I asked her to post about it.

Here’s Sarah’s account:

Last month I had the pleasure of attending the Evidence-Based Practice for Health Sciences Librarians: An Introductory Workshop at MUSC (Medical University of South Carolina) in Charleston, SC. The workshop was an introduction to essential topics for conducting evidence-based practice. It covered the following questions: 

  • How do we select an appropriate study design for a research question? 
  • How do we find evidence? 
  • How do we create a good clinical foreground question using the PICOTT approach (Patient, Intervention, Comparison, Outcome, Type of Question, and Type of Study)?  

The workshop consisted of lectures and small group work, where we learned to assess standard research methods used in the health sciences, like systematic reviews. We used a traditional a CASP checklist for the assessment in our critical appraisal by looking at whether the review, in general, addressed a sensible clinical question, the search was relevant and exhaustive, the studies found were of high methodological quality, and the selection and assessment of studies were reproducible.       

Librarians in the health sciences are not usually required to evaluate the calculations used in medical research. Still, we learned some “basics,” like how to use nomograms to specify likelihood ratios and create tables to calculate the sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios. Understanding what’s behind these calculations helps one appreciate how the efficacy of trials and treatments for patient care is rigorously evaluated before publication—or at least the best practices.         

I liked learning about this so much; I plan to take the Critical Appraisal Institute for Librarians course this spring semester to delve deeper into the topic. And the workshop was not all work – I also enjoyed restaurant “dine-arounds” and even a ghost tour of Charleston. 

Ghostly Graveyard (Photo by N Turner, Charleston, SC, 2016)
Posted in conference reports, research work practice | Tagged , , | Leave a comment

There’s Always Room for Improvement

Posted on July 20, 2023 by Nancy Turner

In my capacity as coordinator of library assessment, I’m often consulted about survey design. I even represent the Libraries on the University’s Survey Coordinating Committee. So I should know and use best practices.  

But frequently we want to toss out a quick survey to gauge reactions to a situation –  surveys are generally considered a straightforward and suitable method for this. I wanted to gather feedback from library and press staff on the perceived effectiveness of activities put into place to build staff engagement.  The survey followed was our staff engagement event (retreat) conducted in December. Six months post event seemed like a good time to conduct a brief assessment using examples of staff activities taken directly from the report’s recommended next steps.

It was a short survey. The first question,  

On a scale of 1-5, how effective is this activity in helping you connect to others in the organization? 

A minor mis-step.  

The question references a scale of 1 to 5, but the options were not numbered! I doubt that this confused too many, but it was, admittedly, a bit sloppy on my part.

But the second question did cause some confusion. Using the same set of activities, the survey asked,  

On a scale of 1-5, how effective is this activity for communication and building trust?   

This is an example of a double-barreled question. A common survey question mistake, asking about two things at once. Communication is not always the same as building trust. When a survey question is not clear and precise, responders are justly confused, and the data collected is difficult to analyze. Because the question is not understood in the way it was intended.  

Fortunately, the stakes here were not too high. The survey yielded important feedback and suggestions for improvement. But for those large-scale efforts, the University Survey Coordinating Committee, and the Office of Institutional Research & Assessment, serve an essential purpose. This office protects Temple students and faculty from being over-surveyed. Additionally, its review process puts lots of expert eyes on surveys that go out to our community. Ultimately, our committee work supports better response rates as it prevents survey-fatigue, and the careful review helps ensure the survey data we do collect is sound.  

And of course, it’s a lesson for me and other assessment professionals — to always know there is room for improvement!

Posted in surveys | Tagged | Leave a comment

Creating Spaces for Community and Connection 

Posted on June 29, 2023 by Nancy Turner

The crowds are back! For ALA’s annual conference, almost 16,000 registrants gathered at Chicago’s McCormick Place Convention Center. The return to fully in-person meetings, coupled with sessions on the new hybrid work environment, has me revisiting the question of how space, both physical and digital, contributes to fostering connection and community within an organization. The topic has interested me since the move to Charles Library’s open office spaces and the subsequent pandemic, as it impacts our work, our communication and our sense of cohesion as an organization.

We had perfect weather for a tour of library renovations sponsored by ALA CORE’s Buildings for College and University Libraries Committee, on Innovative Learning Spaces: A Tour of Chicago-Area Academic Institutions.

I was most interested in the staff space renovation taking place at Regenstein Library, a 1970’s brutalist structure, almost 600,000 square feet and housing 3+ million volumes (including an onsite ASRS).  John Carey, Director of Building Services and Operations, described a post-pandemic shift in building use. While the library sees over 5,000 visitors a day, there is a shift towards pandemic-weary undergraduates seeking in-person experiences as well as wifi and power outlets.  

I was particularly interested in the staff areas. The renovation plan deliberately included an “investment” in staff spaces, towards all librarians having an “enclosed” space.  We toured these new office suites, including both large (for supervisors) and smaller for librarians. There are consultation spaces for work with patrons, a staff kitchenette, conference room, and a common table for informal unit gatherings.  While enclosed, the suite is located near open stacks and study areas and available for drop-in service.  Only one staff member occupied the space on our Friday visit, but according to a UC colleague, staff are using up vacation time. Plus the upcoming NASCAR race wreaking havoc with commutes!  So it will be interesting to see how these spaces work when classes are in session and library staff are onsite once again.

Renovated space for librarian offices at the University of Chicago.

Hybrid Work Culture Post-Pandemic: Challenges and Opportunities from Geographically Diverse Libraries was a panel discussion on how pandemic lessons are informing our increasingly hybrid work environment. Panelists from North Caroline, Virginia and Miami shared experiences and best practices.

From the University of Miami:

  • Think creatively about the library organization structure 
  • Expect to train managers of hybrid teams  
  • Understand that achieving fairness and inclusion may be more difficult than expected
  • Maintaining a healthy work/life balance and engaging in networking can also be challenging

Glen Wiley, Associate Dean for Access and Discovery Services

North Carolina State University also has:

  • a renewed emphasis on staff well-being, touching bases regularly with “How are you doing?” surveys
  • is rethinking office and meeting space to ensure that meeting rooms are hybrid-enabled
  • reconsidering the current practice of assigning office space in a “hierachical.” 
  • has established in-person days for many departments, when everyone is onsite
  • makes an effort to find remote work for all staff, with managers typically working on site at a higher level, “as a way of supporting staff.”  

David Goldsmith, Associate Director for Collections & Research Services

Space for Community and Connection  

Whether our workspaces are physical or digital, in the end it’s not the space, per se, but how our space use supports communication, connection and community. Whether we are in-person, working in a more hybrid environment, or fully remotely, we need to design our spaces to support new communication modes and recognize that optimal use of these modes requires new skills. And for some kinds of community-building at work, there continues to be value in the face to face, in the flesh connection.

Chairs can be checked out at the Illinois Institute of Technology

Posted in conference reports, library spaces | Tagged , , , , | Comments Off on Creating Spaces for Community and Connection 

Improving on the Charles Library Welcome 

Posted on May 8, 2023 by Nancy Turner

Walking into the vast atrium space of Charles Library is awe-inspiring. It can also be overwhelming, particularly to new students when the semester first starts.  While we have many ways of counter-acting that feeling – helpful security guards, Emily Schiller’s friendly and creative whiteboards, Access Services also instituted a greeter program in October 2022. 

Student greeter Kyler Epstein

Access Services’ Alex Rosen supervises the students who work at the One Stop Assistance Desk (OSAD) and now, the kiosk, providing two complimentary service points on the first floor.

Working with Jackie Sipes, they developed an assessment plan using both metrics and qualitative data to understand and improve the greeter service. They carefully analyzed transaction counts, question types, and traffic patterns to optimize the use of students at the two service locations.  In addition, students regularly provide reflections on their experience through an online form designed by Jackie.

I spoke with Alex last week about the project, what they’re learning and the new kinds of questions the assessment process generates.  

Nancy: Tell me more about how you decide to deploy students where? 

Alex: As part of their regular shifts at the desk, student workers are deployed at the kiosk. Originally, we hired someone expressly for the kiosk, but out of necessity, each student is now assigned one hour a week. There is the option to pick up more. Mostly we’re basing this off the data tracker (transactions) number, averages, considering the traffic and balancing with interactions at the OSAD, which can be very busy.  So we try to make sure the kiosk is staffed when we expect it to be busy elsewhere.  

We are also learning about how important it is to place the kiosk where it is visible, especially because students don’t always see the OSAD. The kiosk gets the most visibility when it’s outside the Events Space, but that can interfere with traffic to the space itself, or attendees think the kiosk is for information about the event. It’s uncomfortable when the kiosk is stuck inside a tour group of high school students. What works best is pairing the white board with the kiosk.  

One advantage of the portability of the kiosk is we can try different locations, and also be adaptable to other things happening in the atrium. Combined, time and placement influence the traffic the most.

Nancy: Do the students get special training? Is this something they opt into or is it part of the regular duties of the OSAD student?

Alex: Each student is scheduled to work at the kiosk one hour each week, but some prefer working at the main desk, where they feel less “exposed”.   The kiosk has been useful in helping students to navigate the atrium area’s services. Interaction statistics indicate that the majority (61%) of questions are directional, but students also handle a good number of basic reference questions.

Question Types

The students’ reflections provide additional insight into what is going on at the desk:

“I noticed a lot of people who seem to be hesitant to ask for help at the OSAD desk are a lot more willing to ask for help at the greeter kiosk. I assume because the experience is more intimate for them without a desk and plexiglass in between us. That is probably the most rewarding part of the greeter kiosk, as it is easier to help and connect with people who are more shy/timid”

“what is hard is hearing how busy OSAD is and not being able to help”  

Alex: The irony too is that currently, students working at the kiosk are limited in the kind of information they are supposed to provide – so that can be frustrating to them. But the potential for side-by-side review of the computer screen makes some kinds of information sharing easier. Students assume that everyone does the same thing and can be frustrated with the response to their question of, “I’m not a librarian, let me give you three handouts.”

Nancy: The challenge for you is to figure out how best to balance the types of services available at each of the service points – in a way that is natural to the space and the traffic flow as well as the kind of information needs that you’re seeing. So we avoid bouncing students from one desk to another! 

It sounds as though you are learning a lot. The assessment process is providing some useful insights into scheduling needs, but your analysis informs our understanding of the different types of interactions supported by different physical setups. Thanks for sharing your experience with us.

Posted in library spaces, statistics, user experience | Tagged , | Comments Off on Improving on the Charles Library Welcome 

Assessment in Research Libraries: Unlocking the Secrets to Success*

Posted on April 14, 2023 by Nancy Turner

If you’ve ever set foot in a research library, you know that it’s a magical place filled with books, articles, and information galore. But what makes a research library successful? How do they know what their patrons need and how to provide it? The answer lies in assessment. 

Assessment is the process of gathering and analyzing data to inform decision-making. In research libraries, assessment helps us understand how our resources and services are being used, and how we can improve them to better meet the needs of our patrons. 

Take Temple University Libraries, for example. They use assessment to better understand their patrons and to make data-driven decisions that improve their services. Let’s explore some examples of their assessment efforts. 

One of Temple’s most impressive assessment initiatives is their annual user survey. The survey asks patrons to rate their satisfaction with various aspects of the library, such as the collections, the physical space, and the staff. The survey also includes open-ended questions that allow patrons to provide feedback on specific areas of interest. This data is then used to inform decision-making around resource allocation, service improvements, and strategic planning. 

Another example of Temple’s assessment efforts is their analysis of circulation data. By tracking the number of times a book or article is checked out, the library can better understand which resources are most popular and which ones may need to be removed or replaced. This analysis also helps the library identify trends in usage, such as an increase in demand for resources related to a specific subject area. 

In addition to these assessment initiatives, Temple also uses focus groups and usability testing to gather feedback on specific projects or services. For example, they conducted a focus group to gather feedback on their website redesign, and used the feedback to make improvements to the site’s navigation and search features. 

All of these assessment efforts help Temple University Libraries better understand their patrons and improve their services. But assessment isn’t just important for research libraries. It’s also important for anyone who wants to make data-driven decisions. 

Whether you’re a business owner, a student, or just someone who wants to improve their own productivity, assessment can help you understand what’s working and what’s not. By gathering and analyzing data, you can identify areas for improvement and make changes that will help you reach your goals. 

So, next time you’re in a research library, take a moment to appreciate the assessment efforts that go into making it a success. And if you’re looking to improve your own performance, consider using assessment to unlock the secrets to your own success. 

* To the reader: This post was written entirely by ChatGPT 3.5, using the prompt, “write a blog post on the topic of assessment in research libraries for a novice audience in a intriguing style with examples from Temple University Libraries at https://sites.temple.edu/assessment/ “

Consider this: How long did it take you to discern the difference between this AI-produced post and a “human” written one? What were the clues? While this post exercise was “just for fun”, in the future I’d like to explore ways in which AI might support our work in library assessment – identifying themes in survey comments, polishing a post, designing images, or conducting quick literature reviews. It’s both exciting and scary!

DALL-E Prompt: Research library in the style of Edward Munch
Posted in assessment methods | Tagged | 1 Comment