Assessment Community of Practice: User Experience

Posted on November 5, 2018 by Jacqueline Sipes

Last week’s Community of Practice was my first opportunity to talk with colleagues from across the organization about user experience (or UX) in libraries. I presented an overview of some principles of UX and a brief update on two current UX projects (Blacklight/Library Search and service design for the new library).

Rather than rehashing highlights of my presentation, I want to return to two questions that I posed to the audience. I intended for these questions to be a “thinking together” exercise where attendees could reflect on how we, as an organization, might embrace UX and more fully integrate it into our every day work. We touched on these briefly toward the end of the COP meeting, but I want to delve into them more here and provide my own perspective.

How is UX different from what we’ve always done in libraries? (i.e. we’ve always been committed to user-centered design and great customer service). 

In the book Useful, usable, desirable : applying user experience design to your library, librarians Aaron Schmidt and Amanda Etches tell the story of a colleague who tentatively asked them, “How is [UX] different from what we’ve always done in libraries? We’ve always cared about our patrons. We’ve always been user-centered. We give great customer service!” As an erstwhile public services librarian who values good customer service, this question resonated with me. At the desk, we strive to provide the best service possible; we listen closely, we ask questions, and we do our best to make sure the person in front of us is satisfied. When planning an instruction session, we talk to instructors; we learn about the students we’ll be working with and their assignments, and we design instruction based on that information. Library staff have a long history of considering user needs when providing services.

For me, user experience is an evolution of that user-focused tradition. It gives us a framework to put the user’s needs at the center of all of our design and decision making.

My interest in assessment and user experience grew over the last few years in my previous position as the Emerging Technologies Librarian and Education Liaison. In that role, I was uniquely positioned to both contribute to the design of library technologies while also observing firsthand how students and faculty used, and sometimes struggled with, those technologies. Those interactions highlighted for me the need to know more about our users’ needs and expectations when making decisions about the design of our website and search tools. I began to approach all of my work, including technology projects and reference and instruction, from the question of “how can we make this better for our users?” I’m now lucky enough to have a job where it’s my responsibility to ask that question every day, not just of our online spaces, but also of our services and physical spaces.

To create services, websites, and spaces that work well for our users, we need to understand and empathize with our users. User experience asks us to build that empathy and understanding with intention. We might immerse ourselves in a user’s journey (i.e. put ourselves ‘in our users’ shoes’) using a particular service or tool to identify pain points or frustrations along the way. We might observe how users naturally utilize a space or service. Or, we might work directly with them through user research using methods like usability testing, interviews, or focus groups. When we see how users actually engage with the library — what they like or find challenging or what we, as well-meaning information experts designing services and websites for non-expert users, might have unwittingly missed — we start to empathize with our users and better understand them. And, hopefully we come up with solutions that enhance their experience with the library.

UX also requires that we approach design holistically, acknowledging that users do not experience our individual services in isolation. My interactions with students and faculty at the desk and in the classroom also afforded me the opportunity to see the connections between our services and physical and online spaces. A faculty member who wants a book uses our website or catalog to find or request the book, asks for help using our chat service, reads stacks or other wayfinding signs in the building, and finally uses self-checkout kiosks or interacts with a staff member at the service desk. A student who wants to reserve a study room uses our website, submits a request through our room scheduling system, approaches the service desk staff, and finally uses their study room. Good customer service might ensure the user has a pleasant, if not successful, interaction with staff at a service desk and in many cases even throughout the remainder of the experience. But, even the strongest customer service ethos does not acknowledge the robust interconnectedness of library services and spaces; nor does it ask us to work collaboratively, and sometimes uncomfortably, across library departments to holistically design services and tools for our users.

How can we prioritize user experience work in the organization? How do we integrate UX into our existing processes?

If great customer service doesn’t necessarily make for a great overall user experience, neither does the work of one, solo User Experience Librarian, even with her small, but enthusiastic group of colleagues who have signed on for UX work on the Library Search and Website Redesign projects. We need to work together as an organization to integrate user experience into our work. I look forward to expanding my own work beyond the user experience of our online tools to our services and physical spaces. I also hope to help my colleagues develop capacity in user research and design and foster an organizational culture that puts the user at the center of every decision we make.

Posted in usability | Tagged , , | 1 Comment

Setting the Path towards a New Library

Posted on September 24, 2018 by Nancy Turner

The “Link” at the Calgary Airport

Last week I had an unexpected 12 hours to spend at the Calgary International Airport, providing me plenty of time to consider all that I learned at the Designing Libraries conference.  While the airport provided fun rides between terminals and the wi-fi was robust, the distractions (like food, bookstores, cozy chairs), unfortunately,  were minimal.

Designing Libraries brings librarians, researchers, designers, planners and architects together to discuss the future of libraries, particularly how new technologies support new ways of doing research and supporting students in their learning. The conference is currently sponsored by the Coalition for Networked Information (CNI), North Carolina State University Libraries, and the University of Calgary Libraries. 

Joan Lippincott (CNI) summed up the key discussion threads in this way:

We are:

  • Integrating our thinking about what we want users to be able to do in relation to space
  • Articulating more clearly the relationship of programs to institutional priorities
  • And we have a growing “appetite” for digging deeper into post-occupancy assessment.

Joan was persistent in asking those hard questions of assessment:

  • How do you demonstrate how the new space relates to the creation on new knowledge?”
  • How are spaces contributing to learning?
  • How do you determine impact, either quantitatively or qualitatively?

Assessment is a topic I’d like to see addressed again at future conferences, and something we need to plan for at Temple as we near the opening of the Charles Library.  Assessment is important because it requires us to consider, before any project is initiated, what it is we want to accomplish with the new space, the new service, the new resources? 

As we plan for the opening of the Charles Library, let’s ask:

  • What are our new expectations for the building, and the kind of research, learning, and partnerships it will support?
  • What does that kind of engagement on the part of faculty, students and staff look like?
  • While we’re focusing on the digital scholarship and instruction aspects of the space,  what are the other opportunities for assessment? What are the different ways the collection is being used?  More use of electronic?  Less use of the physical collection, or different parts of the physical collection? Is this due to the different access, or is it due to different discovery opportunities?
  • Certainly, we anticipate opportunities for collaboration and communication within the space. How else might the building facilitate a different kind of organizational culture?

Greg Raschke said of the Hunt Library at NCSU (a Snohetta project, like the Charles Library)  “Hunt was a meteorite opportunity. If you miss it, it will set you back. If you overdo it, you can always scale back, but if you miss that opportunity for change, you’ll never get there. Use failures and successes as a way of evolving your organization.” 

Architect Craig Dykers, of Snohetta, also spoke of the transformative value of the library – it can be a place for both safe dialog AND disagreement; both a welcoming space AND a challenging one. The library is something always transforming, but should serve as a transformational space as well.  Sort of like the path along Calgary’s Bow River.

 

Path along the Bow River, Calgary.

 

Posted in conference reports, organization culture and assessment | Tagged , , , | Leave a comment

Moving Forward: Lessons Learned from the Last 10 Years of Risk Modeling

Posted on September 5, 2018 by Nancy Turner

Last week the Libraries’ Assessment Community of Practice kicked off the year by hosting Alexandra Yanovski-Bowers (Assistant Director for Undergraduate Strategic Initiatives) and Michele Lynn O’Conner (Associate Vice Provost for Undergraduate Studies). The discussion was Undergraduate Studies’ work to build a predictive statistical model for identifying “at risk” students at Temple and provide positive interventions to support these students.

While the idea of collecting, mining and analyzing personal data on students is concerning to some, Temple’s research program utilizes data already collected by the University as part of doing business, and uses very selected data points to target students who are at greater statistical risk of not returning to school at the beginning of the subsequent year. This is how “at risk” students are defined.

Alexandra described the various ways in which data can be used – we can collect it, we can watch the trends and report out on what’s happening. But we can also take a more “pro-active” approach to using the data that we are already collecting and use that data for supporting student success. This is predictive analytics. 

 

In the course of its business, the University collects data related to admissions, enrollment, orientation registration, gpa, housing, financial aid, as well as its New Student Questionnaire, completed by all incoming students. The modeling process uses historical data to make a determination of the key variables within these data sets that correlate with a students not returning. That “model” is applied to the current student population to identify students potentially at risk. This means that the model is constantly changing as the student population and policies for administration and registration change.

Undergraduate Studies wanted to be more pro-active in helping students – not stepping back and waiting for students to seek assistance on their own. So prior to any data work, the University beefed up its advising program.  Each school as a risk adviser, and those advisers all undergo Risk Liaison Intervention Training. There are a variety of interventions to meet a student’s particular need.   The intervention may be extra academic advising, but it is as likely to result in a job closer to campus, or a specialized financial aid workshop.

From the floor there were questions about how confidentiality of students was protected. For the Library, a core value is protecting confidentiality of the individual in terms of what they read or the information they access. We talked of the responsible ways of using predictive analytics. The fact that Temple has created its own model means less risk of external parties getting access to student data.

What might the Library’s role be?  Students don’t always know how the librarian fits into their educational experience.  To that,  Alexandra says, “We want the library to be an intervention, not a factor. “ In other words, we serve a greater purpose by connecting with students, providing space for their work, advising on their research, and being a friendly face as they face the stresses of being a college student on this big campus.

Posted in statistics | Tagged , , | 1 Comment

Attending the American Library Association’s Annual Conference can be overwhelming – the crowds, the number of programs, the sheer size of the convention center and all the walking that comes along with that. This year I asked three colleagues to reflect on their experience, as first timers.  I heard from everyone that it was hot and steamy. And the food… 

Thanks, Kaitlyn Semborski (Library External Affairs & Advancement), Beckie Dashiell (Editor)  and Urooj Nizami (Resident Librarian) for their  thoughtful responses, and tempting, interesting pictures.   

 

What was the most useful session you went to, in terms of how you might apply what you learned to your work here?

Kaitlyn:     My favorite session that I went to was “Bringing Life to Your Library Services with 360˚ Virtual Tours.”  I felt the session gave me a learning takeaway that I might not have learned about without attending a conference where people share their ideas. It inspired me to want to work on a similar 360 project when we move into the new library to help connect our users with resources. It’s also an idea for how we can get students and community members familiar with the new layout in the new building. Hopefully we can make it happen.

Beckie: I went to a session titled “Marketing Strategy, Marketing Plan, and Marketing Tactics: Why You Need All 3!” which I found to be very useful. It served as an important reminder that there are multiple steps in strategic marketing, and all of them should be in service of your organization’s mission/vision. Starting at the high level (who, what, why) to the how to the specific actions you take to support a marketing plan will help you develop the most successful campaign. I think this kind of planning will inform the work we do as we prepare for Charles Library.

Urooj: I attended a really cool, very well attended, session entitled Open Education Resources (OER): Where Libraries Are and Where We are Going that aligned with my own interests. As an early career librarian with an interest and passion for the openness movement, I found this panel particularly helpful as it eased my imposter syndrome. Sometimes the very act of being in a room with folks who are interested in the same domain allows you to feel like your ideas and plans are aligned with other professionals’ and institutions’ strategies and are not completely out of left field. This session sparked ideas that I hope to integrate into my own capstone project plans here at the Temple University Libraries.  

What was the most surprising thing that you discovered/learned at ALA?

Kaitlyn: Since this was my first time at ALA, a lot surprised me. I was just amazed at the amount of other library workers in the city and to learn how big of a conference this really is. It was fun to meet other people doing similar work across the country. Something that I learned (but wasn’t too surprised about) was that it seems like we are certainly doing a lot right! Especially with our programming, we seem to be doing a great job.

Beckie: How huge the exhibition hall was! I hadn’t really thought about all the different kinds of vendors I would encounter. The library world is vast! And people told me about the free books, but I had no idea just how many there would be for the taking.

Urooj: I never quite realized the size and breadth of our profession until I was in NOLA at ALA among tens of thousands of fellow librarians trying to nab a seat at the same lunch spots! I was exposed to librarianship of public and special libraries in a way I haven’t been unable to engage with before. I found it really interesting to think about the overlap within the particularities of our fields while also considering the realms each distinct library type excels and how we can share and learn from our colleagues’ strengths. At the same time, I also realized that while ALA is a really helpful foundation conference, it would best serve a professional if it were complimented by a discipline specific, more focused conference.

What was the best food you ate, or the most interesting experience that was non-ALA related?

Kaitlyn: I went to Café Du Monde and tried my first Beignet. I love donuts or anything fried, so it was really amazing! I also did a bicycle tour of the city that took us through the French Quarter, Garden District, and Business District where I got to learn a lot about the city’s history.

Beckie:  Turkey and the Wolf–this tiny, kitschy sandwich shop that I dragged Kaitlyn and Urooj to in the 100+ weather! But I had a good reason–in 2017, Bon Appetit named it the best new restaurant in America, so I’ve been wanting to try it ever since.

Urooj concurs with Beckie’s assessment of the food. She says, The food was great, but it was even cooler to spend time with and to get to know colleagues outside the office!

Thanks everyone.

Nancy: Just curious, is that chocolate pudding and marshmallows with french fries?

Beckie: Actually, soft serve with magic shell and potato sticks!

Posted on July 17, 2018 by Nancy Turner | Leave a comment

Strategic or Operational? That is the Question

Posted on June 14, 2018 by Nancy Turner

Photo courtesy Geof Wilson

 

Or is it? This last year we’ve had many lively conversations at the Libraries/Press about distinguishing the strategic work from the operational work we do. Those conversations, coupled with this morning’s early yoga class, have me reflecting on how strategy and operations need to work side by side.  

How do we balance these two ways of describing our work? The conversation often goes, “Well, there is just so much time for strategic (perhaps code for “new”) initiatives when we have our operational work to do.” That makes sense.

But what is that operational work? And if we describe it as operational, does that provide it some immunity from scrutiny or assessment? Will we continue to purchase books using the same procedures, just because it’s “operational” or “what we do”? The reality is that we are continually changing up our methods, procedures and operations in acquisitions and collection development – to save money, to meet new needs, to save staff time, to explore new access models. It’s continuous improvement, and it can be labeled strategic as readily as the purchase of a 3 -D printer for the Digital Scholarship Center.

Another great example is the mapping collections project in the Special Collections Research Center. With use data, the staff is making strategic decisions about where collections should reside, at what level they need description, and what collections might be digitized for wider accessibility. How we optimize our space, staff skills, and staff time while providing for improved access / or preservation  – those are strategic moves.

Perhaps it’s my own rosy lens on the Libraries/Press but I’d like to call all of our work strategic, in that it has intention and direction – the work is continually changing  (some aspects more quickly than others) to meet new needs of the organization, institution and community.

Rather than separate,  strategic and operational are balancing forces that are dependent on one another. To grow and meet changing needs, we need to strategize our operational work. Likewise, we must consider how our strategic work can be operationalized, with goals, objectives and measures of success.  The yin and yang that keeps us both grounded and moving forward.

Photo courtesy Geof Wilson

Posted in organization culture and assessment | Tagged , | Leave a comment

Springtime Refresh at Ambler

Posted on May 9, 2018 by Nancy Turner

Continuous improvement is a kind of assessment that we don’t usually think of as assessment per se – there are no statistics, there isn’t a formal plan for data collection, and our efforts may not result in a report to stakeholders. But the work that Jasmine Clark, our resident librarian, has done with staff at Ambler definitely falls into the category of assessment towards workflow improvement. In this blog post I’ll use the assessment components of identifying needs analysis and measuring success – to frame Jasmine’s work with staff that has created efficiencies, standardized workflows, and fostered change at the Library. I sat with Jasmine and Sandi Thompson, Head of the Ambler Campus Library, this week to talk about the project.

Photo credit: Darryl Sanford

 

 

 

 

 

 

 

NT: Tell me about the details of to your stint at Ambler?

JC: I needed to pick a rotation and I was interested in higher-level decision making, how organizations are run. Sandi & Andrea [Goldstein] expressed a desire to take a more comprehensive approach to the existing documentation at Ambler.

ST: For many years much of the information on policies and procedures was kept in a physical notebook at the public service desk. We were having trouble keeping it up to date, particularly with the migration to Alma. 

JC: I was able to bring my past experience with creating documentation to bear on this project, which involved collecting policies and procedures, moving documents to an online environment, standardizing workflow, and training users in using the system.

I looked at various technologies – Google drive, Slack, JIRA, and Confluence. We didn’t want to get too fancy, and needed to take into account the current skills of staff, their interest in technology, as well as the amount and nature of the data we’d be working with. So I decided to use Confluence, linking out to Google documents when necessary. As it turned out, Confluence is a perfect solution for our current needs. 

At Ambler, the “print” was the primary location for documentation. This made it hard to access, and hard to maintain. Now, Confluence: Ambler Campus Library is primary and if we need a printed copy, it’s exported as PDF, printed, and placed in the reference binder.

NT: How do you know if you’ve been successful in accomplishing this change – which is about both technology, but also the organization and how it shares information?

ST: Moving to the Confluence environment has had a multitude of benefits. When a student has a request, we’re not dependent on a particular individual to provide that service. Having everything in a centralized location online allows for other staff to comfortably fill in when someone is absent.

We’ve drastically improved the accessibility and the sharing of our knowledge and awareness – from notifying everyone that a student will be late for work  to how to process an interlibrary loan.  We are less silo’ed in our work, and this has led to a lot of “cross-training”.

JC: Yes, and I’ve seen staff who have taken real ownership of the site. They go beyond using it in a passive way, but also contribute to its accuracy – making corrections, interacting with the documents, updating on their own.

Training is an important part of the process, of course. Our workflow is realistic and based on everyone’s level of comfort and pace. I provide support as they are learning. I let staff know, you will not be “looked over” even when you are not familiar with technology.

NT: Are there other success indicators?

ST: There is a social media function, so we see an uptick in commenting on the blog post. Just the fact that people are using it for everything – having a place to go where everything is current and everything has made it the“go to” place for information. 

Reviewing the documentation has forced us to look at procedures in a different way, with “fresh eyes” looking at the work we do and how it might be changed. This was an unexpected result.

JC:  Something like this changes workplace culture. It’s become the norm to share information. It’s started discussions about new problems to solve.

NT: If you could describe the benefit of the project in one word, what would it be?

JC: Efficiency!

ST: Collaboration!

NT: Taken together, those two things really do speak to using process improvement work as an approach to building a team; working together to create a shared knowledge base. And really improve our service to users. Thanks for sharing this with us.  

Posted in process improvement | Tagged , , | 2 Comments

An Agile Approach to Assessment

Posted on April 2, 2018 by Nancy Turner

The Epitome of Agile. Photo by Philippe Rouzet

Last week I had the pleasure of talking with Emily Toner, our Technology Projects Librarian, about how she works with developers in Library Technology Development to conduct ongoing assessment of their work on the Blacklight project. (Blacklight is the software that will be used for the upcoming enhanced version of the Library Search tool, due for general release in June 2018.)  In her role as project manager, Emily coordinates between what the developers are working on and what users (librarians, patrons) need and expect in a discovery tool.

The programmers/developers work in an “agile” framework – an approach that incorporates the principles of iterative, flexible development and continuous improvement. The project work is divided into short “sprints” – concentrated effort on a specific feature or collaboration opportunity. The team’s typical sprint lasts 2 weeks. When the sprint is wrapped up,  Emily facilitates a “retrospective” session with the team – they reflect together on what is working and what isn’t working so well.

In the world of agile development, there’s a whole tool box of techniques for doing retrospectives. One of the group’s favorites is the “Four L’s”. The exercise was developed by Mary Gorman and Ellen Gottesdiener and works like this: The group asks together of the sprint:

  • What did we Like?
  • Learn?
  • Lack?
  • Long For?

The exercise can be done with big sheets of paper mounted on the wall, but since TUL developer David Kinzer works remotely, they do the brainstorming entirely online now. Emily leads the team in a brainstorming session to generate feedback on the positive and negative things that happened during the sprint.

  • What was productive during the sprint?  This is a LIKE.
  • Did we figure out how to resolve a problem, or learn about a new technology? This goes into the LEARNED category.

Likes and Learns serve to highlight the positive – to bolster the energy of the team and to appreciate the good work getting done. It also serves to build the team as a cohesive group.

  • But there are also breakdowns in communication, flaws in process, occasional lack of support for developing a certain feature – these are LACKS.
  • And finally, LONG FORS – identifying items that are absent from the current project.

In conducting the exercise, team members take time to write down their personal thoughts, then these are exchanged and talked through. As a group, common themes are identified.

The retrospectives serve a couple of purposes: One is to identify which project outcomes were successful and which not. Also, a retrospective provides an opportunity for the team to think about how the team is performing as a team.  How is the team communicating? Too much, Too little? Is our work as a team effective? How can we make it better? 

And how is this assessment? The process is all about continuous improvement; the principle that we can always reflect on our work and what’s working about it, and what can be improved. And the retrospective serves a practical purpose –  putting that reflection into next steps for making the team’s work better and more effective.  It’s a kind of process improvement but not just about efficiency, about effectiveness.

Getting in the habit of regular self-reflection on our work – both celebrating the positive and recognizing what is challenging, leads to team building, trust and creative innovation.

Thanks, Emily

For more about the Agile Retrospective process, check out:

Posted in assessment methods, process improvement | Tagged , , | Leave a comment

Improving Temple Libraries’ System for Systematic Reviews

Posted on March 5, 2018 by Nancy Turner

Stephanie Roth, Biomedical and Research Services Librarian at the Health Sciences Libraries,

Stephanie

doesn’t always think of herself as doing “assessment.” But my conversation with her about the evolving service and education she’s doing with Systematic Reviews is a good example of how librarians at Temple are being strategic and continuously improving how we support faculty and students, even when we don’t think about it in those terms.

 

What is a Systematic Review, exactly?

It’s like a literature review on steroids. It’s the most comprehensive type of literature review using a very specific protocol. The work must be transparent and reproducible. Anyone who claims to do a systematic review needs background and knowledge of the seven stages to be followed. Take a single question and try to figure out what the evidence is; then critically analyse the evidence and form a consensus at the end on the current state of evidence. The author creates own analysis  based on what is published; It’s important that the process does not introduce any personal bias.

It sounds like a really good opportunity for librarians to support faculty research? So how did you get involved in this work?

I’ve been here for almost three years. I had some gaps when I came. I’d been out of health care for a couple of years, and although I had conducted systematic reviews (from nuts to bolts), at the co-author level, the technology had changed by the time I got to Temple. For instance, there are many more options for citation managers, and Temple also has lots of journals and databases. I had had no formal training in Systematic Reviews, and there is so much to know. So I had a huge learning curve.

In your time here, what have you learned about the specific needs of Temple faculty and students?

Graduate students are often assigned systematic reviews but not given information about them. They may understand this to be an ordinary literature review. We spend a lot of time with researchers (faculty, students, hospital staff) so we understand the kind of review that they actually need for their work.

The word about how we can help is getting out, informally through word of mouth but also guest lectures and workshops.  I see these consultations as educational  – I used to be a school teacher. We are trying to move from  “this is a service we can do for you” to “this is what you need to ask yourself to know what approach to take.” 

I’ve done a lot of work with the other HSL librarians as well. For instance, on how to know when a question is more than it first appears. A patron might arrive at the desk asking for help finding articles, but upon some probing, we learn that it’s not for a single course assignment, but is more like a systematic review. The librarian can help them to select the appropriate review type.

As these services become better known, how do you manage the additional work?

A couple of things. We have a new protocol form that faculty need to fill out before we begin working with them. In the past, faculty were requesting a full systematic search but they hadn’t done much in the way of preparation. Our request form requires them to think carefully about their question, their inclusion and exclusion criteria. Completing this work streamlines the process, and also demonstrates a level of investment on their part. Having the search “protocol” developed before the process begins is important to eliminating bias. I also ask up front for co-authorship.

We’ve also implemented team training. This was something that Barbara Kuchan (Director, Health Science Libraries) suggested, and it’s making a lot of sense. I’ve created a formal model for this process that can be replicated by other librarians. I also use the Open Science Framework to share the model more widely.

In addition to creating a more sustainable process for supporting Systematic Reviews at Temple, I wanted to model what “open access” might look like. I used to be worried that I would attract critique, or my work would be stolen, but now I think about it as a benefit to other librarians. And I think that overall, it will improve the quality of systematic reviews published by Temple authors. 

So you are taking more of a collaborative role with faculty and students for this type of research, you are developing and training a team to support the work, AND you are using an open access platform to share your work. 

It’s a great example of how we are expanding our roles as librarians. Thanks for sharing your experience.

 

Posted in process improvement, research work practice | Tagged , , , , | Leave a comment

An Interview with Gina Calzaferri: Temple’s Director of Assessment & Evaluation

Posted on January 25, 2018 by Nancy Turner

Assessment at Temple University Libraries is part of a robust institutional culture of assessment, fostered by offices like Institutional Research and Assessment. I visited the IRA office last week to talk with Gina Calzaferri, just promoted as Director of Assessment & Evaluation. This office is essential to many University efforts, from providing for centralized and authoritative data for external needs, to helping drive decision-making by University administration.

Institutional Research & Assessment is comprised of two organizational areas, and in my job I get to work with both. Data Reporting & Analysis serves to collect centrally the statistics required by agencies like IPEDS (Integrated Post-Secondary Education Data System) as well as the many ranking publications like U.S. News & World Report. They produce the annual At a Glance brochure each year, a useful tool to find accurate figures on enrollment (34,349) or high school GPA of incoming freshman (3.54).

Gina directs Assessment & Evaluation, providing the infrastructure for program reviews, the support for the assessment of student learning and continuous improvement towards institutional effectiveness.  Her commitment to this mission relates closely to her personal story. She herself was a first generation college student from rural Pennsylvania and her dissertation research (for an EDD in Higher Education from Penn) concerned the kinds of access issues that rural students face in earning a college degree.  When the position in Evaluation & Assessment opened at Temple, it was a perfect fit for Gina. She’s been here for over 4 years.

Assessment & Evaluation coordinates multiple surveys each year, including the Temple New Student Questionnaire and the National Survey of Student Engagement.  The results of these surveys are freely available on the web site, and may be of interest to library staff in understanding our student body. For instance, the New Student Questionnaire asks why students choose to go to college, and why they choose Temple.

Gina chairs Temple’s Assessment Planning Committee, with representatives from all schools and colleges (as well as Student Affairs and the Libraries). This is the committee that helps the IRA office to roll out initiatives, to communicate about upcoming events and training. Members coordinate the tracking of student learning outcomes assessment within each academic program.

Of course Gina’s office is also engaged with the accreditation process. While some might assume accreditation is the driver of assessment, she disagrees.

When it comes to assessment, “The point we are trying to drive home is that [accreditation] should really be the last thing on our minds. Assessment that is ongoing and systematic is for program improvement. It is to understand what you are doing well and not doing so well. We need to be accountable to our  stakeholders and assessment helps us to do that; students, parents, government officials;  Whether it is assessment of student learning or institutional effectiveness as a whole”

Gina continues. “Assessment helps us to tell our story; this is what we do very well here. It lets people know why they should come to Temple.”

Institutional Research & Assessment provides us with resources and support for all of us to show how planning, assessment and resource allocation are connected.  We want to be able to say, “Here are the numbers that demonstrate that we need another faculty or staff member. Assessment supports that demonstration of need.”

I couldn’t have said it better myself. Thanks Gina!

 

Posted in organization culture and assessment | Tagged , , | Leave a comment

A New Year’s Reflections and Resolutions

Posted on January 2, 2018 by Nancy Turner

The beginning of a new year is a somewhat artificial, but useful time to reflect on the past and look toward the future. Reviewing the 2017 archives of this blog reminds me of all that has happened this year.  One big change is how assessment activity is organized –  evolving from a  rather traditional Assessment Committee to a growing Community of Practice. And it is growing; the last assessment meeting attracted 22  staff members to engage with Cynthia Schwarz and others about the recent library website user survey.

Cultivating this assessment community includes drawing upon staff here at Temple and beyond to contribute to this blog,  from our own colleagues’ experiences at conferences to Swarthmore’s Mary Marissen – relating her experience as an assessment librarian. 

Some of my favorite posts are those that speak to broad issues: How do we develop a culture of assessment in an organization?  If for nothing but the title, I liked this one, “Grounded or Toppling Over: The 3-Legged Stool of Assessment Culture”.

But enough reflection, and on to the future.  The recent ARL Assessment Program Visioning Task Force recommendations (published December 4, 2017) was just released, and has particular relevance for what we are also accomplishing locally. 

“Research libraries need to define the values by which they want to be measured, rather than trying to manifest values out of the data that they have.”

Rather than holding fast to those important, though conventional library metrics (e,g, number of volumes, number of participants in programs) we need to take the lead on developing metrics (and telling stories) that serve to better describe our value. Not just in relation to other libraries (i.e. rankings) but value to the institution and its strategic priorities.  

So as I consider my own resolutions for the upcoming year: My mantra will be impact. I plan to think more, and talk more with staff, and external colleagues,  about how best we use metrics to understand impact. 

With the Administrative Council I’d like to explore together how we can better demonstrate impact as we report out and share our activities, and how we connect those activities to the strategic contexts.  Can we ask ourselves:

  • What library work will have the greatest impact? On student success? On faculty research productivity ? On support for faculty instruction? 
  • How do we prioritize the work we do to make our collections most accessible to the widest range of users (geographically, for instance) and with the most effective metadata?
  • How do we organize our own time in ways that have the most impact? That may involve pausing in our routine work to learn something new, to allow ourselves the time for brainstorming and creative thinking.

Perhaps considering impact can serve to insure that we’re collecting the data (both quantitative and qualitative) we need to demonstrate value.

Here’s to keeping those resolutions in 2018, and continuing to appreciate all that we’ve accomplished already. Thanks for a great year. 

Posted in organization culture and assessment, statistics | Tagged , , , | Leave a comment