Like many, I have learned a lot about myself as a teacher and my students as people through the pandemic. We all had to make adaptations, some of which will disappear when (if?) the virus eventually does. One that I have decided to keep is my new “pause button” approach to late work.
In March 2020, everything was panicky and confused and, well, not anything any of us had experienced. While I primarily teach online and have no kids that I suddenly found at home as many of my colleagues and students did, I was not immune to their distress their work-life balance shifted so dramatically after spring break. And with that, I took a deep breath and let go of some control, and boy did it feel good! This is the message I sent my students about our new “late work” policy:
If you can’t get work completed on time, don’t panic. I want you to be happy with what you submit, and sometimes life takes over and school needs to be put on pause while we cope. For my class, hit the pause button if you need to. Just email me and let me know. You don’t have to share details, just “I need to hit the pause button this week” works. No penalties for late work. This lets me know you’re still out there and trying your best vs just giving up on the class completely. I hope no one gives up.
I gave this grace to both my undergraduate students and my graduate students. My initial concern about implementing such a policy was that they’d all “hit pause,” and then I’d end up with loads of incomplete students. But that did not happen. I had a few students per class who had to “hit pause”—some worked in health care, some suddenly had several children at home 24/7, and a few actually got sick, with 2 needing prolonged hospital care. But you know what? Every single one of them completed their courses. One of them needed about 10 days beyond the final exam period to complete the final paper, but everyone else finished on time.
I received many emails thanking me for taking the pressure off and knowing that they had this option, even if they never used it. The students who did use it said it was the difference in their being able to rest easier knowing that I meant what I said—to focus on themselves and let me know when they were “pushing play” again and what questions they had before they got started.
In Spring 2020, even I had to “hit pause” when things got to be a little too much as I helped colleagues who had never taught online before shift to that modality way too quickly—and students let me know it was okay if I needed a break, that they completely understood. When you give grace, you get it in return.
I have continued this policy every term since then. And I will keep doing it. I still have had not even one student take advantage of it. (I know it will happen eventually, but I’ll deal with it when it does.)
I think the one silver lining we have of this whole pandemic experience is we all had a chance to learn something—not in spare time (!!!), but from the experience of it all. I learned that a little kindness and transparency go a very long way to creating a welcoming learning environment and stronger relationships with my students, and I look forward to continuing to allow this human touch, this little bit of grace, in all of my classes.
Wren Mills is Pedagogical Assistant Professor at Western Kentucky University’s School of Leadership and Professional Studies.
We are all teaching in a new reality created by powerful text-generation tools like ChatGPT that allow us and our students to compose text on demand. Lori Salem, Assistant Vice Provost and Director of the Student Success Center, and I wrote an initial post about this last semester. As instructors, we will all need to think hard about how to manage and harness the power of this tool. I use the words “manage” and “harness” intentionally here, as we cannot pretend that we can entirely ban these tools nor can we rely on anti-AI detectors (that I can assure you will not be foolproof). In addition, we have a responsibility as educators to guide our students in the ethical and effective use of AI tools that will be available to them beyond the university in their workplaces and in their daily lives. There is no body of research (yet) that can guide us in using AI for teaching, so we are all feeling our way along by reading, debating, and experimenting with some best ways forward.
While the teams at the CAT and the Student Success Center are working towards developing a set of guiding principles for managing AI in our classrooms, the way to start right now is by considering how PI can help us manage AI. What is this magical PI, you say? Does it have something to do with Tom Selleck (if you’re my age, you get that joke)? Is it a fancy new counter-AI robot that will solve all of our problems? No, my dear colleagues, it is simply an invitation to examine the fundamentals, the Pedagogical Intelligence that should be the first stop on the road to a set of principles for thinking about teaching in the presence of Artificial Intelligence. In the CAT’s new spring series, Using PI to Manage AI, we will be exploring, both on our blog and on our CAT Tips series on social media, these pedagogical fundamentals as a way to start this conversation. The topics we will explore on our EdVice Exchange blog are all evidence-based ways of designing student assessments of learning in ways that will encourage academic honesty, motivation, and a desire to learn. We will follow each blog post with a CAT Tips video on social media outlining a few concrete ways to implement these assessment strategies in your classes. If you have not recently done a deep dive into evaluating how useful your assessments are for evaluating learning–and also for furthering learning by engaging students in meaningful learning tasks–now is the time!
We will start the series by exploring how to design assessments that are meaningful for students, allowing them to connect to what we are teaching in ways that help them see the value of engaging in the work. The following blog post in the series will discuss how to use learning assessments to build student self-efficacy in ways that help them to be able to do the work well and to feel confident in what they are learning. Then we will unpack iterative work that provides feedback and allows for revision along the way. We will subsequently examine summative assessments and strategies for supporting students to think reflectively about how they prepare for these usually higher-stakes assessments. Finally, we will complete the series by introducing some educational technology tools that can assist us in implementing better assessment protocols.
It will be important to approach this new challenge as an opportunity. It will necessarily push all of us to think deeply about how we are teaching and how we are assessing learning, and in so doing, lead us to more effective practices. We may surprise ourselves by discovering that AI itself can be useful in exciting new ways for learning. In the meantime, know that we at the CAT are on this journey with you, and will be working to support you as you support our students’ learning.
Note: If you are intentionally using ChatGPT to teach in your classrooms this semester, please email us at cat@temple.edu and tell us about it. Consider also that you can engage in the Scholarship of Teaching and Learning (SoTL) by designing a classroom study to evaluate the impact of ChatGPT on student learning. If you want to learn how to design a study related to your use of ChatGPT in the classroom, contact Benjamin Brock at bbrock@temple.edu for assistance.
Follow our Using PI to Manage AI blog series at EdVice Exchange
Throughout the recent semesters, as Temple and the world continued in pursuit of a “return to normal” after the pandemic, many of us found our classrooms to be anything but normal.
Sentiments that years of disrupted learning, increased mental health concerns, social unrest, and various other factors have led to a new and more seriously underprepared population of students radiated through faculty conversations. “I’m used to students not knowing…, but now they don’t even know…” became a common phrase amongst instructors. Attempts at improving student ability increasingly yielded lower rates of assignment completion, higher rates of student burnout, and feedback with phrases such as “unreasonable amount of work” and “pointless assignments,” thus creating a seemingly unbreakable cycle of underpreparedness and higher DFW rates. I know that I, at least, started to wonder: is it the students who are underprepared for my class, or am I underprepared to teach this population of students?
To address this issue and hopefully find reassurance that both the current population of students and I could find success, I joined the Center for the Advancement of Teaching’s Faculty Learning Community (FLC) on Underprepared Students in the Fall 2022 semester. Joined by twelve other faculty members representing the College of Liberal Arts, College of Public Health, College of Science and Technology, Kornberg School of Dentistry, Fox School of Business, Katz School of Medicine, and the School of Tourism and Hospitality Management, our group met biweekly throughout the semester to discuss research about, experiences with, and potential solutions to underprepared students.
Perhaps the biggest takeaway from our group came in one of our final sessions. A simple change in terminology, suggested by a participant from CST–based in turn on a conversation with a colleague at another institution–managed to encompass nearly the entirety of our conversations. When asked if there is a better phrase by which to refer to “underprepared” students, the colleague replied that the current scholarly phrase being used is “at-promise.” This singular phrase got directly at the heart of what we had spent months discussing. As described by a participant from CLA, this shift away from the deficit-minded “underprepared” label demonstrated that these students are working toward something rather than working without something.
Acknowledgement of this fact changes not only the way we view these students, but also one’s view of self and sense of one’s role as an instructor. Rather than working to “fix” what is missing with these students, our focus can be redirected toward finding ways to support these students in their pursuit of fulfilling their promise. But to provide this kind of detailed support for each student in our classes seems like a massive undertaking! How can we possibly take this on, while still teaching the necessary content and maintaining our professional responsibilities?! It turns out, we had really been answering that exact question all along throughout our FLC sessions – we just didn’t know it at the time.
Our early conversations focused on identifying what was meant by the term “underprepared” and exploring what this looks like in our classrooms. Our discussions found that while we were from a variety of disciplines, we all saw similar presentations of underpreparedness in our students. Many of us initially thought being underprepared was largely related to content knowledge, but through deeper exploration of this, we identified that academic underpreparedness is only one piece of a much more complex puzzle. We then explored the factors that contribute to underpreparedness, and this was perhaps the most eye-opening conversation in our early sessions. Through research, conversation, and reflection, we identified long lists of personal, academic, economic, social, and institutional barriers to student success, causing them to appear underprepared on the surface. Simply recognizing the existence of these challenges as an obstacle course of barriers for our students to overcome is a critical component to reframing our thoughts towards our “at-promise” students and their pursuit of success.
The other essential component of our role in helping these students is admittedly more involved than acknowledgement of barriers and was discussed throughout the second half of our FLC sessions – using strategies to create an atmosphere in our classrooms designed for all students to truly learn. Again, this seems like an enormous task made up of sweeping pedagogical changes, but, actually, success in this venture can be found through a series of small modifications, most of which will improve learning not only for the at-promise students in your class, but for all! Seeking out these tasks may seem daunting at first, but once you know what can be helpful to these students, there is plenty of information and support! As stated by an FLC participant from CPH, “If I had known what to look for [before the FLC], this would have been really helpful!”
And our FLC group is here to help YOU with exactly this! Upon realizing how valuable all of this information was, we wanted to create a resource and means to share with the University at large. We want all faculty to benefit from the rich discussions, deeper understanding, and lingering questions we have found so important over the past several months. In service of this goal, we have created this document containing some of the information and strategies we found to be critical in understanding and supporting our at-promise students. In this document you will find information regarding the barriers students face, tips for classroom modifications, and related articles, blog posts, and other resources we believe to be most helpful.
Additionally, we wanted to not only continue our conversations, but open them up to the Temple community through round-table discussions. In these sessions, we hope to share even more insights that we have gained, as well as the questions and concerns we still have, and invite you to join the conversation with your questions and insights as well!
With the ever-changing population of students needing our support, our Faculty Learning Community hopes that these resources will serve to provide you, our colleagues, with the same sense of deeper understanding of and appreciation for our at-promise students that we have developed through these sessions and inspire changes to support these students at all levels and in all disciplines university-wide.
Jessica Babcock is Assistant Professor of Instruction in Temple’s Department of Mathematics and serves as Director of Developmental Mathematics.
The premortem is an activity used by companies or their coaches as a strategy to reduce the chances that a project will fail. I first heard about it while listening to psychologist Gary Klein on The Knowledge Project in August 2022 (Ep. #144). The premortem is a twist on preventing ‘bad’ outcomes in that it assumes the project has already failed. That’s right, as Dr. Klein presents it, he has a crystal ball and can see into the future. In that future, the project has failed – it’s true he says – he has seen it. Dr. Klein then takes his participants/trainees through a process where first individually and then collectively they brainstorm all the reasons the project failed. That is part one, and its done like a nominal group process so that all the problems are listed, and the main ones are highlighted. In part two, the participants again generate a list individually and collectively. This time, they find the most likely steps to prevent the adverse outcome. According to Dr. Klein, this activity is effective in reducing failure AND building a shared sense of responsibility for the success of the project.
The premortem can be used in the classroom even when a group project isn’t involved and with similar success. It’s true, I have done this! In this post, I provide two examples of using the premortem with students. And if you’re like me, you will start to see the many ways that you can apply this in your own classes.
At the beginning of each semester, instructors consider ways to set the tone for an engaged classroom (e.g., Ep. #41 of Faculty Focus Live). Many of us ask our students to consider the best and worst classroom experiences they’ve had, or what they think an instructor could do or bring to the classroom and what they, as students, could do or bring to the classroom to make it one of those ‘best’ experiences. The first time I used a premortem in class it was meant to do this. Specifically, I wanted to set the stage for a successful outcome with students in a two-semester course. This particular course is a requirement for the Public Health major. Students must receive a minimum passing grade of C in the first semester in order to progress to the second. If not, they must wait until the following fall to repeat the course. Thus, the stakes are pretty high.
On the first day–after I spoke to the students about the course and we did a few ice breakers to get to know each other–I explained that I put a lot of thought into how I teach the course and I care deeply about each student’s progress. I told them I knew the class was demanding and I wanted to do everything I could to help students succeed. But it doesn’t always work out. And just like Dr. Klein, I told my students I had a crystal ball that could see into the future. It was December 2022, and a student was not going to pass the course. I emphasized that the student was NOT passing and then asked them to first spend a few minutes writing down all the things that went wrong for this student. I told them to think broadly on causes, for example, the students actions or inactions, the instructors actions or inactions, and other life or contextual events. Once the students had time to write their thoughts, we went around the room (about 20 students), with each person stating the first item on their list, or the first item that had not already been stated. The white board was filled with responses such as did not come to class, did not turn work in on time, took too many courses, poor time management, instructor didn’t give clear instructions, had to work full time. We repeated the activity on what the student and instructor could do to keep the student from failing and closed the activity by highlighting the top 3 things students could do to ensure they would earn at least a 73 this semester (i.e., coming to class, asking for help, turning things in on time). I shared what I would do as well (i.e., preparing for class, providing rubrics and feedback and grading their work within days of submission).
My second use of a premortem was embedded in a case study and supported by a Poll Everywhere up vote activity. I created this activity after for a different course on the topic of people who use drugs. Before starting the activity, I got to know the students a bit and highlighted a few things about substance use, dependency, and addiction. The premortem began with a story about a person named Jackson. Jackson is a 30-year-old Black cisgender male. One week ago, Jackson was arrested for possessing 2 grams of heroin. He was released from jail this morning to await trial. A few hours ago, he died from an opioid overdose.
After pausing for the story to sink in, I asked students, “how did we get here?” and told them to make a list of all the things that might have happened in this man’s life to get us to this point. Some additional prompting was given, for example: think immediately and distally, think about this persons actions and interactions with people, institutions, and society. Students then wrote their reasons on a poll everywhere thread and upvoted the reasons that were most important. Students then made a list of ways we could prevent deaths from drug overdoses and upvoted the most important ideas. The details about the person in the case study can also be altered to see what different problems students might identify. Because this was the first time I tried this activity, I included a brief exit survey with the following questions.
What do you think was accomplished by the activity we did today?
What suggestions do you for how today’s activity could have been improved?
Is there anything else about today’s activity or topic that you’d like to share?
I was pleased with both premortem activities and believe the activity itself can be adapted for use in any course and any topic. I am happy to brainstorm ideas for you classroom and can be reached at deirdre.dingman@temple.edu.
[And, as always, the CAT is available for one-on-one consultations on this or any other learning activity you are planning for your Temple students. -Ed.]
Deirdre Dingman, DrPH, MPH, CHES is Associate Professor of Instruction in the Department of Social and Behavioral Sciences and Chair of the Collegial Assembly of the Temple University College of Public Health.
More than 220 faculty from universities across the region came together with the CAT for the 2023 Annual Faculty Conference on Teaching Excellence co-sponsored by Temple Libraries, the Office of Digital Education, Information Technology Services, and the General Education Program. We were delighted to be back in-person on January 11th and 12th to celebrate the 20-year anniversary of the CAT. The CAT was founded in 2002 as the Teaching and Learning Center (TLC). In 2016, the TLC merged with the Instructional Support Center and was renamed the Center for the Advancement of Teaching. In celebration of the 20-year anniversary, the CAT invited and honored past directors of our center.
This year’s theme was Achieving Rigor Without the Mortis: Keeping High Standards While Rejuvenating Our Students, Ourselves, and Our Communities. Participants grappled with the notion of rigor–its different meanings, how the pandemic may have reshaped our notion of rigor, and how that is being played out in the policies and practices in our courses.
Keynote and Plenary Address
Marcus Johnson and Derek Bruff
The CAT was honored to have Dr. Marcus Johnson, Professor in the Educational Psychology and Educational Research and Evaluation programs at Virginia Tech, as our keynote speaker and Dr. Derick Bruff, author, consultant and Visiting Associate Director of the Center for Excellence in Teaching and Learning at the University of Mississippi, as our plenary speaker.
Dr. Johnson’s day one keynote was titled, Motivation and Rejuvenation with Rigor + Care. After defining motivation and its characteristics, he shared common misconceptions about what motivates students. He then discussed three constructs from different theories of motivation that could guide us as we consider ways to spark students’ motivation through our assessments, learning activities, and classroom environment: purpose, mastery, and autonomy. By considering our students’ diverse motivations, experiences, and identities, we can target a variety of motivational strategies that can lead to better outcomes for more of our students. Dr. Johnson concluded by encouraging us to help rejuvenate our students by supporting their well-being, helping them to become better self-directed learners, and by giving them opportunities to be inspired and engaged cognitively, behaviorally, affectively, and socially.
Dr. Bruff’s Plenary on day two was titled, Intentional Tech: Reconnecting Our Students to Learning. He discussed three principles and then asked us to generate and consider a variety of technologies to address these principles, which are to
structure ways for students to learn from and with each other to enhance learning for all and for creating a sense of community;
incorporate multiple modalities (audio, visual, embodied) for teaching content that can help more learners succeed;
connect our students to authentic learning experiences to raise the value of what they’re learning and to help students see themselves in their desired majors and as future professionals in their field of interest.
He reminded us to base our choice of technologies on whether they help students meet the learning goals for the course.
Building on the Keynote and Plenary Sessions
In addition to keynote and plenary speakers, the conference featured interactive workshops, breakout sessions, lightning talks, and poster sessions–all designed to generate discussion and share ideas for creating challenging, inspiring, inclusive, and equitable learning experiences.
New this year was our resource fair, where representatives from a variety of Temple support units explained their services and provided informational materials. Representatives from the Student Success Center, Temple Libraries, The Wellness Resource Center, CARE Team, Disability Resources and Services, and Instructional Technology all joined us for this fair. Additionally, two tables were set up for CAT staff to provide information about the Scholarship of Teaching and Learning (SoTL) scholarship and another table for information regarding CAT’s Teaching in Higher Education Certificate.
This year we invited representatives from a variety of Temple support units to provide resources and explain their services during the luncheon on day 2. Representatives from the Student Success Center, Temple Libraries, The Wellness Resource Center, CARE Team, Disability Resources and Services, and Instructional Technology joined us to share resources. In addition, CAT staff who are leading efforts to expand its outreach for Scholarship of Teaching and Learning (SoTL) scholarship and another table for information regarding CAT’s Teaching in Higher Education Certificate.
Let’s Continue the Conversation
Our annual faculty conference generated many thought-provoking questions and teaching strategies, so let’s continue talking! Here are some ways you can keep the conversation going:
Share what you’ve learned with your program/department faculty.
Visit our faculty commons to ask questions, pose ideas, and get feedback from the CAT and faculty across disciplines.
Schedule a consultation with the CAT to work on incorporating what you have learned into your courses.
In honor of Dr. Martin Luther King, Jr. Day, the CAT invites you to reflect on his legacy and impact on education, which was a core principle of the civil rights movement. As we begin a new year and semester of teaching, what lessons can we take from his leadership and principles as we think about our role in education and the purpose of education–not only on this important day of commemoration, but throughout the year?
In an acceptance speech he gave when he was honored by the United Federation of Teachers in 1964, Dr. King shared that “For most of the past decade the field of education has been a battleground in the freedom struggle.” … “It is precisely because education is the road to equality and citizenship that it has been made more elusive for Negroes than many other rights” (Strauss, 2012). Education continues to be a “battleground” for contesting rights over free speech and critical perspectives of US history, book bans, vaccination and masking, gun violence, and gender issues–and, most recently, over trans student rights (Obeng, 2022). In 2023, Dr. King’s message about education, critical thinking, and the ability to evaluate evidence continues to have a strong sense of urgency and relevance.
“Education must enable one to sift and weigh evidence, to discern the true from the false, the real from the unreal, and the facts from the fiction.”
“The function of education, therefore, is to teach one to think intensively and to think critically.” (from “The Purpose of Education,”)
Dr. King dedicated his life to addressing and fighting against structural and systemic inequalities, and we as educators can continue the work he began by reflecting on our own teaching practices. Do we consider how historical and structural factors impact our students’ lives, opportunities, and outcomes? How can we, as teachers, create more equitable and inclusive classrooms to honor his memory and legacy?
At Temple, we teach and learn in a vibrant intellectual community. There are many events on campus that give us and our students opportunities to engage more deeply with his ideas and legacy through discussions and acts of service on this day, and throughout the year. The new Center for Anti-Racism Research and IDEAL are hosting a critical conversation on intergenerational thoughts and actions to promote a more inclusive and socially just society for this year’s MLK 365: Keeping the Dream Alive; the Lewis Katz School of Medicine is featuring a talk by Linda Villarosa to honor his legacy of activism and service; the Maurice H. Kornberg School of Dentistry will host talks related to Dr. King from January 16-20; and there is a group reading of Dr. King’s “I Have a Dream” speech at the Temple University Bell Tower led by Klein professors David Brown, David T.Z. Mindich, and Karen Turner.
One of our most cherished resources on campus is the Blockson Collection, which houses an extensive collection of materials by and about Dr. King. The available resources include photographs and printed materials such as his speeches, letters, sermons, and articles. Next month the CAT will co-host a special workshop with the Blockson Collection for faculty interested in learning more about the Collection and resources they can use for more inclusive classes across the disciplines. Our Teaching for Equity Institute, the Can We Really Talk? series with IDEAL, and other workshops, book groups, and learning communities offer additional spaces for reflection and action as educators throughout the year.
To honor Dr. Martin Luther King, Jr’s memory, we hope that you will choose one of these opportunities to engage with others in reflecting on ways to create a more equitable educational environment for our students, one that helps fulfill Temple’s historic mission to educate “acres of diamonds.”
The earlier posts in this series discussed how to apply the lenses of self, colleagues, students and scholarly literature to the evaluation of teaching. In the final post in this series, we will reflect on how assessment of student learning at the course and program level allows us to take a step back and ask whether what we’re doing is working. Assessment of student learning in our classes helps us evaluate whether students have met the learning goals for the course: it tells us what our students know and can do, what they have yet to learn and are still working on, and whether our instructional decisions have been effective. Assessment of student learning at the program level helps us evaluate whether students have met the learning outcomes of the program (the curricular requirements, degree or Certificate): it tells us whether the program is designed to deliver the promised outcomes and is structured coherently and in such a way that information needed in later courses is adequately scaffolded in earlier courses. When thoughtfully designed, course and program assessment can foster reflection and dialogue that ultimately benefits the students in our classes and programs.
In this post we discuss student learning goals, learning outcomes, and the relationship between the two. At Temple, learning goals refer to what a student should know or be able to do at the end of a single course, whereas student learning outcomes generally refer to outcomes at the major, minor, certificate or curricular (e.g., GenEd or Writing Intensive) level. Assessment of student learning is most useful when it takes into account both course learning goals and student learning outcomes. For this reason, we encourage you to take into account the following factors when you design assessments.
Curriculum Alignment
Ideally, course assessment and program assessment begin with student learning outcomes, or the overall goals of the degree program or curricular sequence your course is embedded within. While you may not have control over course sequence or student learning outcomes, simply knowing where your course fits into the bigger picture can help you thoughtfully design assessments aligned not just with your own course goals, but with the trajectory of student learning both before and after your course. The learning goals specific to your course should be designed to deliver the larger student learning outcomes of the program your course is nested within.
Identify or Construct Learning Goals and Outcomes
Course learning goals and program-level student learning outcomes inform your overall course content, class activities and assessments. For this reason, it’s important that they are specific and measurable. By “measurable” we don’t mean that you need to be able to quantify student learning in relation to all of your goals. Rather, a goal should be written in such a way that you can devise a method to determine whether a student is making progress toward it and whether it has been met. For more information on writing course goals, visit our EDvice Exchange post, Learning Goals: Dream Big!
When Designing Classroom Assessments, Begin with the Goals
Create assessments and activities that allow students to demonstrate whether they have met the learning goals you have established. Your assessments are an opportunity for your students to highlight their learning and development across the semester, and an indicator of your effectiveness teaching the content you set out to deliver. To learn more, see our post Looking for Evidence in all the Right Places: Aligning Assessments with Goals.
Consider Program Assessment
Course-based assessments may then be used in program assessments. You may want to work with colleagues in your program to review these artifacts using a rubric that aligns with one or more program-level student learning outcomes. Colleagues may also be called upon to help you interpret your students’ gains across the semester. While reviewing course-based artifacts such as exams, essays and written reflections, aim to identify areas of the course or curriculum in need of revision for future semesters. In other words, be sure to use the results of your assessments.
The Scholarship of Teaching and Learning (SoTL)
If this is beginning to sound a bit like research, that is the idea . . . once this all becomes more systematic we can move into what is called the Scholarship of Teaching and Learning (SoTL). Systematically inquiring into our pedagogical practices allows us, as instructors, to make evidence-based decisions about our teaching, our classroom activities, and our assessments. Engaging in SoTL ensures our students are learning and developing as best they can in our classroom (Brock & Rouder, in press).
Where Student Feedback Forms (SFF) are up to our students, peer review requires the input of our colleagues, and assessment of courses and students learning generally falls to the department or program. When we focus on the instructor in the classroom we realize that this systematic, continuous evaluatory process aimed at pedagogical improvement is solely in our own hands as faculty. This process allows us the opportunity to provide evidence that we are performing highly as teachers and that our students are, in fact, learning. It is also an opportunity to consider why we might want to routinely assess our teaching and our students’ learning: are our actions aimed at developing (as opposed to demonstrating) our pedagogical knowledge, competencies and skills, and how might this further our motivation to do so over time? We can think of assessment of student learning as a means to communicate empirical evidence regarding our instructional practices and our students’ experiences. It can be used to demonstrate how we are consistently evolving our pedagogical practices so that our teaching can be as impactful as possible.
For support with designing assessments, schedule a consultation with a CAT specialist. For help with developing SoTL projects, look for SoTL consultations on our CAT consultations page.
Brock, B. & Rouder, C. (in press). Celebrating the Scholarship of Teaching and Learning (SoTL). Faculty Herald, Temple University.
Linnenbrink-Garcia, L. & Patall, E. A. (2015). Motivation. In L. Corno & E. M. Anderman (Eds.), Handbook of Educational Psychology (3rd ed., pp. 91-103). Routledge. https://doi.org/10.4324/9781315688244
Dana Dawson and Benjamin Brock work at Temple’s Center for the Advancement of Teaching.
While reflection on one’s teaching, as well as student and colleagues’ feedback, are better-known methods for evaluating teaching, perhaps the most overlooked method is to consider how we use the scholarly literature on teaching and learning to improve our teaching. Instructors who engage with the literature of the scholarship of teaching and learning develop a vocabulary and way of thinking that moves them beyond replication of teaching methods they experienced as a student or ones that were taught to them when they were teaching assistants or junior faculty. Familiarity with this literature allows us to engage in reflection and experimentation that continually evolves our teaching practices. The insights gained from engaging with the extensive body of work on teaching and learning includes both validating effective practices you may already have been using, and of course, opening up new ways of teaching, designing curriculum, assessing learning, and supporting students that we may have never considered. It also clarifies for us why certain methods may work better than others.
It is clear to me why this criterion is often overlooked. When I started working at our teaching center after having taught for over 25 years, I was introduced for the first time to the scholarly literature on teaching and learning. I had dabbled a bit with very specific literature on teaching English as a second language, and I had read a little bit about oral proficiency methods for teaching world languages, but I never moved beyond these limited forays into this kind of scholarship. I don’t think my lack of awareness was unusual. Immersed in my disciplinary research, as most faculty are, I had never had occasion to explore the wealth of scholarship that provides guidance and evidence on how students learn. In my new role at the center, a whole world opened up to me that I never knew existed.
I remember in particular a brand new book that had come out just as I started my role at the center—How Learning Works: 7 Research-Based Principles for Smart Teaching. It was an incredibly good entry point as each chapter pulled together the research on teaching and learning on a variety of topics in coherent form and then suggested strategies we can employ in the classroom. The chapter on student motivation was transformational for me. It validated much of what I had been doing, especially around creating a positive environment for learning, but also provided so many ideas for how to support student learning in more effective ways. When I went back into the classroom, my newfound knowledge really helped me rethink my teaching and implement concrete changes that saw exciting results. If I had been asked to demonstrate how I utilized the literature on teaching and learning to improve student learning as part of a process for evaluating teaching, I could have pointed clearly to the changes I made as a result of this book and the impact those changes had on student engagement and motivation.
So how can you use this lens to evaluate teaching? In particular, you can demonstrate how you have engaged in a process of continual scholarly teaching by taking advantage of professional development opportunities that allow you to delve into the literature on teaching and learning. For instance, have you attended workshops at the CAT, met with an educational development or educational technology consultant at the CAT, or attended other similar programming offered by professional organizations in your discipline? Have you taken a deeper dive by enrolling in longer-term, intensive opportunities focused on particular aspects of teaching and learning? For instance, perhaps you have attended our 12-hour Teaching for Equity series, or you have met monthly with a cross-disciplinary group to explore a teaching topic in a faculty learning community. Maybe you have simply gotten your hands on some excellent literature (the CAT has a lending library available on all kinds of topics!) and have made changes to your teaching based on what you have read. And, of course, taking this a step further, you might contribute to the scholarship on teaching and learning by investigating how teaching or curricular changes you have implemented have impacted student learning, and then presenting or publishing on those findings.
If you have never before considered this particular lens, I urge you to give it a try! Faculty who begin that journey into the scholarship on teaching and learning find it a fascinating and energizing way to evolve their teaching and curricular practices.
Stephanie Fiore is Assistant Vice Provost and Senior Director of Temple’s Center for the Advancement of Teaching.
Peer review of teaching gets a bad rap. It conjures up images of being judged, of one’s teaching put under a microscope. Faculty express discomfort and nervousness at being observed in class, and, interestingly, they also resist the idea that they are “qualified” to provide feedback on a colleague’s teaching. That is, of course, if they even give feedback. I have a distinct memory of my chair coming into my class (unannounced), sitting in the back and writing furiously the whole time. Afterwards, I never received any feedback, but I knew that his mysterious impressions of my teaching were written in a report and filed somewhere with my name on it. And, of course, while faculty need a letter written by a peer reviewer for certain summative purposes, such as promotion, merit, or awards, these letters are often little more than a checkbox exercise written by a well-meaning colleague, and certainly aren’t intended to improve teaching.
But it doesn’t have to be this way!
Formative peer review of teaching (and by formative I mean, peer review intended to support continued growth in teaching excellence) should contribute to what Shulman calls making teaching community property. Just as we would never evaluate scholarly research on the basis of offhand comments made around the water cooler, nor should we evaluate teaching in this way. A community of colleagues can provide feedback in both our research and teaching worlds to help us improve the quality of our work. This word—community—is so important here. Done well, peer review should build community in your departments and colleges as you talk to each other about teaching and learning, promote shared educational goals, and of course, create natural support structures when our teaching goes sideways. Within this community of colleagues, a well-designed peer review process helps to encourage reflection and more intentionality in teaching, and energizes us as instructors as we gain more insight into our practices. Note that peer review can take the form of classroom observations, as well as review of a Canvas course, a syllabus, or other teaching artifacts (such as assignments, assessments, and materials). If your department is considering peer review as a professional development practice, the CAT can help you create a protocol that works for your specific department’s needs.
Well-designed peer classroom observations should be a rewarding collaboration that contributes to the professional development of both the reviewer and the reviewed, as both gain insight into effective teaching practices through this process. There are three stages to an effective peer classroom observation: the pre-observation discussion, the observation, and the post-observation debrief.
The Pre-Observation Discussion
Before the observation, the colleague conducting the review should try to learn as much as possible about the class goals and other helpful details, and any specific areas of concern the instructor may have about their teaching so that the reviewer can pay special attention to those areas during the observation and provide targeted feedback.
The Observation
For the observation itself, it is very helpful to use an instrument to guide the reviewer. The CAT has recently created a new comprehensive instrument that may be useful for your peer observations, and there are other models we can share as well. Here are some helpful recommendations for conducting the observation adapted from “Twelve Tips for Peer Observation of Teaching” from Siddiqui et al, 2007):
Be objective. Focus on specific teaching techniques and methods that were outlined in the instrument. You should communicate your observations, not your judgments.
Resist the urge to compare with your own teaching style.Being peers does not necessarily mean that the two of you will have the same teaching style. Concentrate on the teaching style of the person and the interactions that you observe.
Respect confidentiality.Your professionalism and trustworthiness is essential in building a peer review relationship with your partner, so confidentiality is important.
Make it a learning experience.For the reviewer too, the process of conducting a peer observation is a learning experience, which both builds the reviewer’s skill at providing constructive feedback, and may spark new ideas useful for the reviewer’s teaching.
The Post-Observation Debrief
Providing supportive and constructive feedback in a timely manner is key to making this experience meaningful to your colleague’s professional development. But this is, of course, the part that worries faculty most. We often advise reviewers to think of the debrief as a discussion between colleagues, focused more on asking questions than telling a colleague what went right or wrong. The guidelines below will help you give helpful feedback in peer observations:
Give your colleague an opportunity first to self-assess what they did well, what they have questions about, and what they might do differently.
Limit the amount of feedback to what the receiver can use rather than the amount you would like to give (we recommend no more than 3 strengths and 3 areas of discussion and improvement)
Your feedback should be based on observations rather than inference
Provide your feedback in descriptive rather than evaluative language, using “I” statements rather than “you” statements. “I saw that some students in the back were disengaged”, rather than “you should have really done something about the disengaged students in the back”.
Begin with some (genuine) positive comments.
Offer constructive ideas, framed as possibilities for consideration. It can help to frame these ideas as questions. “Have you considered trying…?”
Invite dialogue about your comments and questions.
Adapted from:
Ende, J., M.DEnde, J. (1983). Feedback in Clinical Medical Education. JAMA; 250: 777-781; and Oxford Learning Institute. Giving and Receiving Feedback. http://www.learning.ox.ac.uk/rsv.php?page=319
Peer review can be a rewarding and meaningful part of our professional development if designed with care and transparency and in the spirit of doing our best to support student learning. It can help us build community with our colleagues through a shared sense of responsibility and mentorship about our development as teachers, and encourage personal reflection about our teaching practice. Ultimately, of course, its purpose is to deepen student learning, a goal we share as educators.
In the next part of this series, we’ll discuss evaluating teaching using outcomes and assessments.
Stephanie Fiore is Assistant Vice Provost of Temple’s Center for the Advancement of Teaching and Linda Hasunuma serves as an Assistant Director at the CAT.
So says the title of a recent article in Vice that has been making the rounds at Temple. The article describes a new tool called Open Ai Playground, that generates text on demand. Playground uses GPT-3, a newly developed machine-learning algorithm, to compose the text. GPT-3 is also the power behind Shortly-Ai, another text-generation tool offering a somewhat different set of features. The sentences generated by both programs are surprisingly good – they flow, and they have clear and simple prose style. A student could theoretically type their essay prompt into Playground or Shortly, and the program would generate the essay for them. And because the sentences produced by GPT-3 are entirely original, the resulting text would not be flagged by a plagiarism detector like Turnitin.
So, is this the end of writing instruction as we know it? We think not. But these new programs do have implications for teaching, and that’s our focus in this post.
We tested both tools to get a sense of what they can do and what it is like to use them. Both tools make it easy to produce short (paragraph-long) texts that clearly and coherently state a few relevant facts. It’s possible to imagine a student using them to produce short “blog-post”-type essays, which is exactly what the students in the Vice article say they do. At least for now, neither program would make it easy to produce a longer text, nor to produce a text that was argument-driven, rather than factual.
But more importantly, these programs don’t—and can’t—help with the real work of writing. They can create sentences out of sentences that have already been written, but they can’t help writers find the words to express the ideas that they themselves want to express. If the purpose of writing was simply to fill a page with words, then the AI tools would suffice. But if the writer wants to communicate something, and therefore cares what ideas and arguments are being expressed, then AI writing tools are not helpful.
Don’t take our word for this. In the sidebar, we provide information about how to access and use Playground and Shortly. Try them and see if you can get them to write something that you can genuinely use.
If you find, as we did, that AI writing tools are not useful when the writer cares about the content of the writing, then we’re halfway to solving the problem of students using AI tools to plagiarize.
The Plagiarism Arms Race
Just because AI generated texts are undetectable right now, doesn’t mean that will always be the case. Someone somewhere is probably already working on a tool that will detect texts written by GPT-3, because of course they are. Students figure out ways to cheat, and companies invent tools to catch them, and then they sell their inventions to us. This is just the latest iteration of that cycle.
To that point, have you seen the YouTube videos instructing students on how to beat Proctorio at its own game? The same Proctorio for which we pay a hefty annual subscription fee?
There has to be a better way, right?
A better way, part I: Encourage Academic Honesty by Creating Better Assignments
This new AI tool is a “threat” to academia only insofar as we ask students to complete purposeless writing assignments, and ones that rely on lower-level thinking skills that ask students to reiterate factual information. The real answer to cheating systems that become more sophisticated is to create better assessments and to create conditions in our classrooms that encourage academic honesty.
There is some very good research on what works to encourage academic honesty. This is a longer discussion than we will take here, but in essence, we should think about what the factors are that lead to cheating behaviors and work to reduce those factors. These include 1) an emphasis on performance (rather than learning); 2) high stakes riding on the outcome; 3) an extrinsic motivation for success; and 4) a low expectation of success. There are very intentional steps that we as instructors can take to reduce these factors, including adjusting our assessment protocols to rely less heavily on high-stakes one-and-done writing assignments, centering writing assignments on issues students care about, and scaffolding writing assignments to allow for feedback and revision.
We also need to look at the kinds of assessments we are using in our courses. The more we move towards authentic assessments and grounded assessments (designed to be unique to the course you are teaching in the moment. They often include time, place, personal, or interdisciplinary elements to make them something not easily replicable), the better off we are. There is a lot of work to be done here, as we often rely on the kinds of assessments we had as students, very few of which were either authentic or grounded. It is much harder to cheat on these kinds of assessments.
Finally, findings from some interesting research on academic honesty suggest that communicating with students about academic honesty works better than you would think, reminding them of their ethical core and focusing on what academic honesty looks like and why it is expected. This is especially effective when timed close to an assessment.
Try it for yourself! Open Ai Playground How to try it: Use the link above to open the website and make a free account. From the home screen, click on the “Playground” tab (top right.) Then enter an “instruction” in the main text box. The instruction might be something like “Describe [topic you are writing about.]” Or “Explain [something you are trying to explain.]” Click “submit,” and your results will appear. If you don’t get what you were looking for, you can keep refining and resubmitting your instructions. ShortlyAIHow to use it: Use the link above to open the website and make a free account. Enter a title and a sentence or two and set the output length to “a lot.” Then click the “write for me” button. If you like the way the text is going, you can type another sentence or two and click “write for me.” Or you can refine your original title and first sentence and start over. Please share your results! Copy the text(s) that you “write” and email them to Lori.salem@temple.edu along with any comments you care to offer about the texts or your experience producing them.
A better way, part II: Adapt instruction to reflect new writing practices
Once upon a time, writing instruction centered around penmanship and spelling. Those days are gone because developments in the technology of writing (from pens, to typewriters, to word-processors) drove changes in writerly practice, which eventually led to changes in writing instruction.
Automated text generators are just the latest technological innovation, and they have already changed the practice of writing in journalism, online marketing, and email. And why not? There is great value in making certain kinds of writing more efficient.
Our approach to writing instruction will need to adapt to this new reality. It’s not hard to imagine a future in which universities teach students how to use AI tools to generate text for some situations, even as they disallow the use of AI tool for others.
Lori Salem serves as Assistant Vice Provost and Director for the Temple University Student Success Center. Stephanie Fiore is Assistant Vice Provost and Senior Director of Temple’s Center for the Advancement of Teaching
