A Guide to Creating an Analytic Rubric

Dana Dawson, Ph.D.

Rubrics are tools used by faculty to guide our assessment of student performance and to make our expectations transparent for students. Using a rubric can help make grading more efficient for faculty and fair for students, but when constructed well and shared along with assignment or activity descriptions, they also benefit student learning. Rubrics explicitly represent our performance expectations and allow students to direct their effort toward the intended goal of an activity or assignment. By asserting ahead of time our highest expectations, we encourage students to reach toward those high standards. The use of rubrics promotes more specific feedback and guidance on future performance which allows students to target specific areas for improvement. When we encourage students to review the rubric ahead of time and reflect on feedback after the fact, we can help our students develop the habit of reflecting on their learning.

Rubrics take a variety of forms from checklists of attributes that would demonstrate competence to analytic rubrics featuring descriptions of levels of competence in relation to different criteria. In this post, I will guide you through the steps of creating an analytic rubric (a rubric that features discrete dimensions and criteria descriptions of performance standards for each of those dimensions) featured in our rubric creation worksheet.

Parts of an Analytic Rubric

Analytic rubrics consist of dimensions, scale labels and descriptions of performance standards. A rubric may feature any number of dimensions, but including too many dimensions may make the rubric difficult for a student to interpret. Dimensions of a rubric may be weighted differently to indicate to students those that are most crucial to success on the assignment. Scales generally range from 3-5 criteria levels.

Scale label 1Scale label 2Scale label 3
Dimension 1(Number of points)Description of performance standardsDescription of performance standardsDescription of performance standards
Dimension 2(Number of points)Description of performance standardsDescription of performance standardsDescription of performance standards
Rubric Scale Example

Steps to Create a Rubric

1) Reflect

Begin with a freeform reflection on your goals for the assignment or activity. What are the main things you want this activity/assignment to accomplish; in other words, what are your goals? What content knowledge and skills is/are needed to productively complete this assignment/activity? What behaviors demonstrate achievement of the assignment’s goals? What are the highest expectations you have for students on this assignment? What evidence can students provide that would show they have accomplished what you hoped they would accomplish when you created the assignment/activity? What would the worst demonstration of this assignment look like? 

2) List

Use your reflection to formulate a list of the most important attributes of success on the activity/assignment. What would an excellent submission or performance look like? What specific characteristics would it have? What are the most important attributes of success for this assignment/activity? Include a description of the highest level of performance you expect for the item.

3) Group

Group items with similar performance criteria and give your groups titles. These groups will become your rubric dimensions. For example, your list may look something like this:

  • Presentation is cogent
  • Presentation is organized
  • Thesis demonstrates thoughtful analysis of the text
  • Thesis and evidence demonstrates familiarity with the text
  • There is evidence for the thesis
  • Presentation anticipates counter-points
  • Specific position (perspective, thesis/hypothesis) is imaginative, taking into account the complexities of an issue*
  • Limits of position (perspective, thesis/ hypothesis) are acknowledged
  • Organizes and synthesizes evidence to reveal insightful patterns, differences, or similarities related to focus
  • Central message is compelling (precisely stated, appropriately repeated, memorable, and strongly supported)

The above list may be grouped as follows:

Thesis

  • Thesis demonstrates thoughtful analysis of the text
  • There is evidence for the thesis
  • Central message is compelling (precisely stated, appropriately repeated, memorable, and strongly supported)

Textual analysis

  • Thesis and evidence demonstrates familiarity with the text
  • Addresses complexity of text

Supporting points

  • Presentation anticipates counter-points
  • Limits of position (perspective, thesis/ hypothesis) are acknowledged

Creativity

  • Specific position (perspective, thesis/hypothesis) is imaginative, taking into account the complexities of an issue
  • Organizes and synthesizes evidence to reveal insightful patterns, differences, or similarities related to focus

4) Apply

You may now use your groups to fill in the left hand side of the rubric and your notes from the reflecting and listing phases of this exercise to establish your criteria. This worksheet includes a blank table that you may use to begin drafting your rubric. Remember to consider whether you want to assign point values to each of the dimensions in your rubric.

A Note on Scale Labels

Too often, the language we use for our scale labels can read as harsh and judgmental for students reading the scale. For example, scale labels such as “Weak,” “Poor” or “Unacceptable” do not imply for our students a belief that they can improve. Here are some suggestions for scale label language that is less likely to be discouraging.

Advanced, intermediate high, intermediate, novice

Exceeds expectations, meets expectations, developing towards expectations

Exceeds expectations, meets expectations, progressing, not there yet

Distinguished, proficient, intermediate, novice

Mastery, partial mastery, progressing, emerging

Sophisticated, highly competent, fairly competent, not yet competent

Concluding Thoughts

Taking the time to reflect on your goals for an activity or assignment and to concretely articulate your expectations will not only improve the quality of the rubric you create, but will help guide your instruction. Clearly identifying what you expect your students to know or be able to do will allow you to work backwards from those expectations to the exercises and materials needed in order for students to build the necessary skills and content knowledge.

For help designing and implementing rubrics, feel free to book an appointment with a CAT educational developer or educational technology specialist. Go to catbooking.temple.edu or email cat@temple.edu.

*Some of the performance criteria description language used here is borrowed from the AACU Value Rubrics.

Dana Dawson serves as Associate Director of Teaching and Learning at Temple University’s Center for the Advancement of Teaching

Using Reading Prompts to Promote Students’ Academic Reading

Zeenar Salim

Do you have concern around students attending classes without pre-reading? Ever wondered how you can make them read? Students in higher education are expected to comprehend the text, connect their prior experiences with the text, evaluate the text, and consider alternative view-points to the text. Reading prompts are considered to be a way to motivate students to read. It improves students’ comprehension and critical thinking skills by engaging them actively with the reading material. 

Provision of reading cues/prompts helps the learners to actively read as well as analyze their own thoughts during and after reading to expand, clarify or modify their existing thinking about the concepts or idea at hand. The reading prompts can be categorized into six categories a) identification of problem or issue b) making connections c) interpretation of evidence d) challenging assumptions e) making applications, and f) taking a different point of view. Sample questions for each category are as follows:

  1. What is the key issue/concept explained in the article? What are the complexities of the issue? (Identification of problem or issue)
  2. How is what you are reading different from your prior knowledge around the issue/topic? (Making connections)
  3. What inferences can you draw from the evidence presented in the reading? (Interpretation of evidence)
  4. If you got a chance to meet the author, what are the key questions that you would ask the author? (Challenging assumptions)
  5. What are the lessons for your practice that you have drawn from this reading? (Application)
  6. If you wrote a letter to your friend who has no expertise in this subject area, how would you explain to him the theoretical concept presented in the article? (Taking a different point of view)

Generally, students are asked to complete the reading prompts before the next class by writing a paragraph-long response to each question. Teachers may ask some or all questions depending upon the learning objectives of the session and may adapt the question(s) to gauge specific information around the text. For more sample questions and detailed literature around reading prompts, please read Tomasek (2009). 

Reference

Tomasek, T. (January 01, 2009). Critical Reading: Using Reading Prompts to Promote Active Engagement with Text. International Journal of Teaching and Learning in Higher Education. Retrieved from https://files.eric.ed.gov/fulltext/EJ896252.pdf

Zeenar Salim is a Fulbright PhD Candidate at Syracuse University, where she works at RIDLR (Research in Design Learning Resources).

creative commons license This post is released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license.

Full Disclosure of the Terms of Success: Nine Things to Tell Your Students

Dana Dawson, Ph.D.

In a 1997 essay entitled “For Openers… an Inclusive Course Syllabus,” Terence Collins argues for the importance of what he calls “full disclosure of the terms of success” – making explicit the “befuddling mores, assumptions, work habits, background knowledge, key terms, or other markers of the academic subculture too often left implicit, inaccessible to outsiders.” By the time most college instructors or TA’s teach a course, lab, studio or recitation for the first time, we have been embedded in the context of higher education for long enough to have forgotten what we found mystifying and incomprehensible in those early days on campus. It’s important to periodically remind ourselves that what is obvious to us needs to be made explicit to our students.

So, in service of encouraging full disclosure of the terms of success and in keeping with a genre of pseudo-journalism I often find irresistible, I present 9 things all professors should tell their students (including their graduate and professional students).

1. It’s normal to feel like an imposter. 

What we have come to call “imposter syndrome” is the feeling that we do not have the requisite skills or knowledge to be where we are and that we have somehow tricked others into believing we are something we’re not (Clance and Imes, 1978). Unfortunately, such negative self-beliefs, however unfounded, can have very real effects on learning and persistence (Holden et al., 2021). Reassure your students that it was not an accident that they found their way into your classroom or program. Share experiences you or your colleagues have had with feelings that you don’t belong and how you overcame them. 

2. You can ask for things. 

You may have noticed that while some students don’t hesitate to ask for extensions, help, accommodations or clarifications, others suffer in silence even where there are supports they could be taking advantage of. Your students may worry that asking for help is a sign that they don’t belong (see #1 above) or feel unsure of what they can ask you about and when it’s appropriate to ask. Make it clear that they can ask, even if the answer may not always be yes. 

3. Treat your learning as a never-ending research project.

There is no one-size-fits-all approach to succeeding in one’s studies, so it’s important that our students regularly ask themselves whether what they’re doing is working. Encourage your students to use metacognitive strategies to interrogate their study practices and find opportunities for improvement (McGuire and McGuire, 2015).

4. All students can benefit from academic support.

The best way to ensure students who need academic support will seek it out is to reinforce the idea that all students benefit from academic support (Thomas and Tagler, 2019). Remind your students that even star athletes receive coaching. Academic support will benefit any student and will most benefit those who seek support early and often. Remember that students coming to your campus from high school may be completely unfamiliar with student support centers, mental health counseling centers, student health clinics and other student supports. Transfer and graduate students who are new to your campus might be familiar with such supports but not where to find them. Be sure to include this information in your syllabus and course site, and to bring it up in class.

5. We are all still learning.

Another way of saying this is that there are no bad questions. Be transparent about your on-going learning, for example, research findings that surprised you and changed how you thought about your field or an article, book or conference presentation that taught you something new. 

6. What your discipline does and how your course fits into that framework.

When I started my undergraduate degree, I had never heard of Sociology, the discipline I ultimately chose as a major. As soon as I started taking Sociology courses, I knew I was in the right place but struggled to explain to my family what I was going to do with the degree because I wasn’t entirely sure how the content taught in my courses was applied outside of an academic context (or in an academic context, for that matter). Pull back the curtain on your discipline. What are the big questions? Why do they matter? Where does what your course covers fit into the fabric of your discipline? How do people use the skills and knowledge specific to your field in non-academic contexts?

7. What you assume your students already know and can do at the start of your course and what to do if they’re missing any pieces.

Are there concepts, authors, formulas, procedures, methods, etc. that your students should be familiar with? Are there courses you’re assuming they’ve taken? Being explicit about anticipated prior knowledge in a pre-semester questionnaire or early in the semester will give your students an opportunity to fill in gaps sooner rather than later.

8. Preferred communication guidelines.

Do you expect to be called Dr. ___? Would you rather not be called Dr. ___? Do you refuse to read emails that don’t begin with “Dear ___,” and end with a period? Should students nudge you if they haven’t received a response to an email within a couple of days? A couple of weeks? In addition to ensuring you are communicated with in a manner with which you are comfortable, this is an important part of our students’ professional development.

9. You’re glad they’re in your class.

I’m glad you read this far! There. Now didn’t it feel good to read that?

References

Clance, Pauline Rose, and Suzanne Ament Imes. “The Imposter Phenomenon in High Achieving Women: Dynamics and Therapeutic Intervention.” Psychotherapy: Theory, Research and Practice, vol. 15, no. 3, 1978, pp. 241-247.

Collins, Terence. “For Openers, An Inclusive Course Syllabus.” New Paradigms for College Teaching, edited by W. E. Campbell & K. A. Smith, Interaction Book Company, 1997, pp. 79-102.

Holden, Chelsey L., et al. “Imposter Syndrome Among First- and Continuing-Generation College Students: The Roles of Perfectionism and Stress.” Journal of College Student Retention: Research, Theory & Practice, 2021. DOI 15210251211019379.

McGuire, Stephanie, Saundra Yancy McGuire, and Thomas Angelo. Teach Students How to Learn: Strategies You Can Incorporate Into Any Course to Improve Student Metacognition, Study Skills, and Motivation. Routledge, 2015.

Thomas, Christopher L., and Michael J. Tagler. “Predicting Academic Help-Seeking Intentions Using the Reasoned Action Model.” Frontiers in Education. Vol. 4. Frontiers Media SA, 2019.

Dana Dawson serves as Associate Director of Teaching and Learning at Temple University’s Center for the Advancement of Teaching

A Brief Introduction to Ungrading

Jeff Rients

In recent years the term ungrading has been circulating in various educational circles. But what does it mean? Is there an opposite to grading and, if so, how can ungrading be used in the classroom? Does ungrading mean students don’t receive a final grade? In this blog post we’ll break down what ungrading is and offer some suggestions as to how you might practice it in your classroom.

The Problem with Grades

Grades are a quantitative, objective proxy for the qualitative, subjective student experience of learning. In theory, grades tell us whether or not students are learning. In theory, the prospect of getting better grades helps to motivate students to do the coursework prerequisite to learning. In theory, grades provide the administration a snapshot of how things are going in our classroom. But how successful are grades at doing that? 

The short answer to that question seems to be that the efficacy of grading can vary wildly. Ken Bain, in his seminal work What the Best College Teachers Do, reports how students in a physics class could achieve A’s in the course without fundamentally shifting how they thought about mass, energy, and motion. Any instructor who has taught the second or third course in a sequence knows the frustration of needing to reteach concepts from previous semesters despite students getting A’s or B’s on their finals just weeks before. 

Furthermore, Alfie Kohn, in his work Punished by Rewards, documents the extensive research demonstrating that attaching an extrinsic reward to a classroom assignment actually undermines intrinsic student motivation. It seems that if we tell students that a task is worth ten points in the imaginary economy of our course, then they will never value the task itself, only the ten points. That puts us in the precarious position of sending mixed messages about the intrinsic value of learning itself. If, as seems to be the case, our grades can never be more than proximate (even inaccurate) indicators of some student learning AND grading every task can have the effect of demotivating students then how valuable can collecting numerous grades really be?

Decoupling Grades from Assessment

Although much work has been in terms of developing better assessments, communicating priorities to students via rubrics, and other attempts to refine our grading practices, those involved in the ungrading movement attack the problem by assuming that grading itself is the problem. From this perspective grading is a largely unnecessary appendage to the practices we need to focus on with our students: feedback and assessment. We, like most educators, are professionally obliged to report final grades and to communicate to our students at the start of the semester how we will arrive at those grades. Although there are many ways to implement ungrading, the basic approach seeks to shift student attention from grades to learning by deferring (for as long as possible) that final moment of collapsing the student learning experience–in all its complicated, messy, nonlinear multidimensionality–into a quantitative approximation such as a single letter or number.

Instead, that messiness, the student learning itself, becomes a core component of the course content. Through ungrading students are challenged to directly interrogate their learning as well as respond to feedback from the instructor. Here are a few examples of ungrading practices:

  • Grade-Free Zones – Designate an early portion or experimental unit of your course as entirely grade free.
  • Self-Assessment – A few times during the semester students assign their own grades, providing evidence in a separate document they write and/or filling out a rubric. You accept or veto their assessment in dialogue with the student.
  • Process Letters – Students write directly about their learning process. The instructor provides feedback focused on meeting the course learning goals.
  • Minimal Grading – Instead of the usual percentage and letter schemes, assign one or more assessments complete/incomplete, pass/fail, or another simplified schema.
  • Authentic Assessment – Students are assigned a relevant real-world task that serves as their primary or sole graded activity, such as a cinema class organizing a film festival. 
  • Contract Grading – Students are given “to-do” lists for achieving an A, a B, and a C in the course. At the start of the semester they choose which list they complete for the assigned final grade.
  • Portfolios – Students build portfolios over the course of the semester, which serves as the sole object of assessment for the course. The portfolio must supply evidence of having achieved the learning goals of the course.
  • Peer Assessment – Similar to self-assessment (which it can be paired with) students work in small groups to discuss and evaluate each others’ contribution to the learning in the course.
  • Student-Made Rubrics –  A few times during the semester you devote a day of class time to collaborate on building a rubric which the students will then use to self- or peer-assess their progress in the course.

(Adapted from Stommel, “How to Ungrade” in Blum 36-8.)

Moving away from a constant stream of grading and points to a minimal number of graded objects does not mean you stop giving feedback! In fact, you will probably find yourself giving more feedback to students and having more conversations with them regarding their progress in the course. It is critically important to approach these interactions with respect for the student and a firm belief that they will want to learn if provided the right support.

Another important thing to keep in mind is the fact that ungrading is slower than many traditional assessment methods. Meeting with students to discuss their progress towards the course goals, reading reflective essays, writing specific feedback, etc. all takes time. You may find it necessary to pare back the total number of assessments you offer, especially if you normally use tools like auto-graded quizzes on Canvas. 

However, this slowing down is a feature, not a bug. Learning requires time to process and reflect. Guiding that learning also necessitates extra time. Resist the ever-present urge to rush through the semester. Accept the invitation to slow down and linger over what really matters in the classroom: our students and their learning.

Getting Started

If ungrading interests you but you find the prospect of revising your course overwhelming, please know that you don’t have to ‘ungrade’ your whole course! You can try it for just one week or just one unit without upsetting the apple cart for the rest of the semester. If you try ungrading at a small scale, it will be important to clearly communicate that you are trying something new for a portion of the course and to unambiguously identify what portion of the course will be ungraded.

Messaging is always important when introducing an innovation of any sort to your learning environment. Remember, your students have years of experience informing their idea of what teaching and learning is “supposed” to look like! As Stephen Brookfield notes in The Skillful Teacher, student resistance in the face of change is “normal, natural, and inevitable” (238). Expect it and plan a response to it.

Temple faculty wanting support in implementing ungrading in their own courses can book an appointment for a one-on-one consultation with a pedagogy specialist. For any other teaching-related needs you can email the Center for the Advancement of Teaching at cat@temple.edu

Resources

  • Bain, Ken. What the Best College Teachers Do. Cambridge, Mass: Harvard University Press, 2004.
  • Blum, Susan D., ed. Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia UP, 2020.
  • Brookfield, Stephen. The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. Jossey-Bass, 2006.
  • Kohn, Alfie. Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes. Boston: Houghton Mifflin Co, 1993. 
  • Sackstein, Starr. Hacking Assessment: 10 Ways to Go Gradeless in a Traditional Grades School. Times 10 Publications, 2015.

Jeff Rients serves as Associate Director of Teaching and Learning Innovation at Temple University’s Center for the Advancement of Teaching.

Hit the Pause Button: A New Kind of Late Work Policy

Wren Mills, Ph.D.

Like many, I have learned a lot about myself as a teacher and my students as people through the pandemic. We all had to make adaptations, some of which will disappear when (if?) the virus eventually does.  One that I have decided to keep is my new “pause button” approach to late work.  

In March 2020, everything was panicky and confused and, well, not anything any of us had experienced. While I primarily teach online and have no kids that I suddenly found at home as many of my colleagues and students did, I was not immune to their distress their work-life balance shifted so dramatically after spring break.  And with that, I took a deep breath and let go of some control, and boy did it feel good!  This is the message I sent my students about our new “late work” policy: 

If you can’t get work completed on time, don’t panic. I want you to be happy with what you submit, and sometimes life takes over and school needs to be put on pause while we cope.  For my class, hit the pause button if you need to.  Just email me and let me know. You don’t have to share details, just “I need to hit the pause button this week” works. No penalties for late work. This lets me know you’re still out there and trying your best vs just giving up on the class completely.  I hope no one gives up.

I gave this grace to both my undergraduate students and my graduate students. My initial concern about implementing such a policy was that they’d all “hit pause,” and then I’d end up with loads of incomplete students. But that did not happen.  I had a few students per class who had to “hit pause”—some worked in health care, some suddenly had several children at home 24/7, and a few actually got sick, with 2 needing prolonged hospital care.  But you know what? Every single one of them completed their courses.  One of them needed about 10 days beyond the final exam period to complete the final paper, but everyone else finished on time.  

I received many emails thanking me for taking the pressure off and knowing that they had this option, even if they never used it.  The students who did use it said it was the difference in their being able to rest easier knowing that I meant what I said—to focus on themselves and let me know when they were “pushing play” again and what questions they had before they got started. 

In Spring 2020, even I had to “hit pause” when things got to be a little too much as I helped colleagues who had never taught online before shift to that modality way too quickly—and students let me know it was okay if I needed a break, that they completely understood.  When you give grace, you get it in return.  

I have continued this policy every term since then.  And I will keep doing it.  I still have had not even one student take advantage of it.  (I know it will happen eventually, but I’ll deal with it when it does.)

I think the one silver lining we have of this whole pandemic experience is we all had a chance to learn something—not in spare time (!!!), but from the experience of it all. I learned that a little kindness and transparency go a very long way to creating a welcoming learning environment and stronger relationships with my students, and I look forward to continuing to allow this human touch, this little bit of grace, in all of my classes.

Wren Mills is Pedagogical Assistant Professor at Western Kentucky University’s School of Leadership and Professional Studies.

This article is released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).

From “Under-Prepared” to “At-Promise”: Reframing Student Preparedness

Jessica Babcock

Throughout the recent semesters, as Temple and the world continued in pursuit of a “return to normal” after the pandemic, many of us found our classrooms to be anything but normal.

Sentiments that years of disrupted learning, increased mental health concerns, social unrest, and various other factors have led to a new and more seriously underprepared population of students radiated through faculty conversations.  “I’m used to students not knowing…, but now they don’t even know…” became a common phrase amongst instructors.  Attempts at improving student ability increasingly yielded lower rates of assignment completion, higher rates of student burnout, and feedback with phrases such as “unreasonable amount of work” and “pointless assignments,” thus creating a seemingly unbreakable cycle of underpreparedness and higher DFW rates.  I know that I, at least, started to wonder: is it the students who are underprepared for my class, or am I underprepared to teach this population of students?

To address this issue and hopefully find reassurance that both the current population of students and I could find success, I joined the Center for the Advancement of Teaching’s Faculty Learning Community (FLC) on Underprepared Students in the Fall 2022 semester.  Joined by twelve other faculty members representing the College of Liberal Arts, College of Public Health, College of Science and Technology, Kornberg School of Dentistry, Fox School of Business, Katz School of Medicine, and the School of Tourism and Hospitality Management, our group met biweekly throughout the semester to discuss research about, experiences with, and potential solutions to underprepared students.

Perhaps the biggest takeaway from our group came in one of our final sessions.  A simple change in terminology, suggested by a participant from CST–based in turn on a conversation with a colleague at another institution–managed to encompass nearly the entirety of our conversations.  When asked if there is a better phrase by which to refer to “underprepared” students, the colleague replied that the current scholarly phrase being used is “at-promise.”  This singular phrase got directly at the heart of what we had spent months discussing.  As described by a participant from CLA, this shift away from the deficit-minded “underprepared” label demonstrated that these students are working toward something rather than working without something. 

Acknowledgement of this fact changes not only the way we view these students, but also one’s view of self and sense of one’s role as an instructor.  Rather than working to “fix” what is missing with these students, our focus can be redirected toward finding ways to support these students in their pursuit of fulfilling their promise.  But to provide this kind of detailed support for each student in our classes seems like a massive undertaking!  How can we possibly take this on, while still teaching the necessary content and maintaining our professional responsibilities?!  It turns out, we had really been answering that exact question all along throughout our FLC sessions – we just didn’t know it at the time.

Our early conversations focused on identifying what was meant by the term “underprepared” and exploring what this looks like in our classrooms.  Our discussions found that while we were from a variety of disciplines, we all saw similar presentations of underpreparedness in our students.  Many of us initially thought being underprepared was largely related to content knowledge, but through deeper exploration of this, we identified that academic underpreparedness is only one piece of a much more complex puzzle.  We then explored the factors that contribute to underpreparedness, and this was perhaps the most eye-opening conversation in our early sessions.  Through research, conversation, and reflection, we identified long lists of personal, academic, economic, social, and institutional barriers to student success, causing them to appear underprepared on the surface.  Simply recognizing the existence of these challenges as an obstacle course of barriers for our students to overcome is a critical component to reframing our thoughts towards our “at-promise” students and their pursuit of success.

The other essential component of our role in helping these students is admittedly more involved than acknowledgement of barriers and was discussed throughout the second half of our FLC sessions – using strategies to create an atmosphere in our classrooms designed for all students to truly learn.  Again, this seems like an enormous task made up of sweeping pedagogical changes, but, actually, success in this venture can be found through a series of small modifications, most of which will improve learning not only for the at-promise students in your class, but for all!  Seeking out these tasks may seem daunting at first, but once you know what can be helpful to these students, there is plenty of information and support!  As stated by an FLC participant from CPH, “If I had known what to look for [before the FLC], this would have been really helpful!”

And our FLC group is here to help YOU with exactly this!  Upon realizing how valuable all of this information was, we wanted to create a resource and means to share with the University at large.  We want all faculty to benefit from the rich discussions, deeper understanding, and lingering questions we have found so important over the past several months.  In service of this goal, we have created this document containing some of the information and strategies we found to be critical in understanding and supporting our at-promise students.  In this document you will find information regarding the barriers students face, tips for classroom modifications, and related articles, blog posts, and other resources we believe to be most helpful.   

Additionally, we wanted to not only continue our conversations, but open them up to the Temple community through round-table discussions.  In these sessions, we hope to share even more insights that we have gained, as well as the questions and concerns we still have, and invite you to join the conversation with your questions and insights as well!

With the ever-changing population of students needing our support, our Faculty Learning Community hopes that these resources will serve to provide you, our colleagues, with the same sense of deeper understanding of and appreciation for our at-promise students that we have developed through these sessions and inspire changes to support these students at all levels and in all disciplines university-wide.

Jessica Babcock is Assistant Professor of Instruction in Temple’s Department of Mathematics and serves as Director of Developmental Mathematics.

Pre-Mortem: Preventing failure and looking forward to ensure success

Deirdre Dingman, DrPH

The premortem is an activity used by companies or their coaches as a strategy to reduce the chances that a project will fail. I first heard about it while listening to psychologist Gary Klein on The Knowledge Project in August 2022 (Ep. #144). The premortem is a twist on preventing ‘bad’ outcomes in that it assumes the project has already failed. That’s right, as Dr. Klein presents it, he has a crystal ball and can see into the future. In that future, the project has failed – it’s true he says – he has seen it. Dr. Klein then takes his participants/trainees through a process where first individually and then collectively they brainstorm all the reasons the project failed. That is part one, and its done like a nominal group process so that all the problems are listed, and the main ones are highlighted. In part two, the participants again generate a list individually and collectively. This time, they find the most likely steps to prevent the adverse outcome. According to Dr. Klein, this activity is effective in reducing failure AND building a shared sense of responsibility for the success of the project.

The premortem can be used in the classroom even when a group project isn’t involved and with similar success. It’s true, I have done this! In this post, I provide two examples of using the premortem with students. And if you’re like me, you will start to see the many ways that you can apply this in your own classes.

At the beginning of each semester, instructors consider ways to set the tone for an engaged classroom (e.g., Ep. #41 of Faculty Focus Live). Many of us ask our students to consider the best and worst classroom experiences they’ve had, or what they think an instructor could do or bring to the classroom and what they, as students, could do or bring to the classroom to make it one of those ‘best’ experiences. The first time I used a premortem in class it was meant to do this. Specifically, I wanted to set the stage for a successful outcome with students in a two-semester course. This particular course is a requirement for the Public Health major. Students must receive a minimum passing grade of C in the first semester in order to progress to the second. If not, they must wait until the following fall to repeat the course. Thus, the stakes are pretty high.

On the first day–after I spoke to the students about the course and we did a few ice breakers to get to know each other–I explained that I put a lot of thought into how I teach the course and I care deeply about each student’s progress. I told them I knew the class was demanding and I wanted to do everything I could to help students succeed. But it doesn’t always work out. And just like Dr. Klein, I told my students I had a crystal ball that could see into the future. It was December 2022, and a student was not going to pass the course. I emphasized that the student was NOT passing and then asked them to first spend a few minutes writing down all the things that went wrong for this student. I told them to think broadly on causes, for example, the students actions or inactions, the instructors actions or inactions, and other life or contextual events. Once the students had time to write their thoughts, we went around the room (about 20 students), with each person stating the first item on their list, or the first item that had not already been stated. The white board was filled with responses such as did not come to class, did not turn work in on time, took too many courses, poor time management, instructor didn’t give clear instructions, had to work full time. We repeated the activity on what the student and instructor could do to keep the student from failing and closed the activity by highlighting the top 3 things students could do to ensure they would earn at least a 73 this semester (i.e., coming to class, asking for help, turning things in on time). I shared what I would do as well (i.e., preparing for class, providing rubrics and feedback and grading their work within days of submission).

My second use of a premortem was embedded in a case study and supported by a Poll Everywhere up vote activity. I created this activity after for a different course on the topic of people who use drugs. Before starting the activity, I got to know the students a bit and highlighted a few things about substance use, dependency, and addiction. The premortem began with a story about a person named Jackson. Jackson is a 30-year-old Black cisgender male. One week ago, Jackson was arrested for possessing 2 grams of heroin. He was released from jail this morning to await trial. A few hours ago, he died from an opioid overdose

After pausing for the story to sink in, I asked students, “how did we get here?” and told them to make a list of all the things that might have happened in this man’s life to get us to this point. Some additional prompting was given, for example: think immediately and distally, think about this persons actions and interactions with people, institutions, and society.  Students then wrote their reasons on a poll everywhere thread and upvoted the reasons that were most important. Students then made a list of ways we could prevent deaths from drug overdoses and upvoted the most important ideas. The details about the person in the case study can also be altered to see what different problems students might identify. Because this was the first time I tried this activity, I included a brief exit survey with the following questions.

  1. What do you think was accomplished by the activity we did today?
  2. What suggestions do you for how today’s activity could have been improved?
  3. Is there anything else about today’s activity or topic that you’d like to share?

I was pleased with both premortem activities and believe the activity itself can be adapted for use in any course and any topic. I am happy to brainstorm ideas for you classroom and can be reached at deirdre.dingman@temple.edu.

[And, as always, the CAT is available for one-on-one consultations on this or any other learning activity you are planning for your Temple students. -Ed.]

Deirdre Dingman, DrPH, MPH, CHES is Associate Professor of Instruction in the Department of Social and Behavioral Sciences and Chair of the Collegial Assembly of the Temple University College of Public Health.

Houston, We Have Liftoff! Successfully Implementing Your Course Design

Cliff Rouder

Congratulations! You’ve put in the time and hard work to think meaningfully about course design for significant learning. You’ve considered your course’s unique situational factors and then developed meaningful learning goals, authentic assessments that measure students’ achievement of those goals, and learning activities that enable them to get practice and feedback to prepare them for your assessments. Everything is in perfect alignment. You’re totally psyched to begin the semester now that you’re armed with, if you do say so yourself, a course designed to help learning take off. Nothing much left to do except hit autopilot and watch the magic unfold. Well…not so fast.

In this last blog post of the Course Design Summer Series, we now look at how to implement the design for maximum impact. To keep your course humming smoothly, here are four key ways to successfully implement your course design.

1. COMMUNICATE, COMMUNICATE, COMMUNICATE. 

We want students to be excited about the course, realize its value, and feel like they can meet your high expectations. One way we can do this is by making transparent why you’ve selected the learning activities and assessments, and how both align to the learning goals.

Communication is key. When and how should we do this? Early and often is the mantra, and here are some ways to do it:

  • Send a pre-semester welcome email or video. Start piquing students’ interest by telling them why you’re passionate about the course, your hopes and goals for them, and how you’ll support their learning.
  • Include messaging throughout the syllabus. Take that (usually) boring course description that is required on the syllabus and give it a face-lift. Whet your students’ appetite by telling them what big and meaningful questions this course will answer, what important ideas or issues they’ll grapple with, and what valuable skills they’ll be equipped with for future courses (and for life)! Explain their role as active learners in the course and why that’s of value. Be transparent with your high expectations as well as the things you will do to support their learning. 
  • Include messaging throughout the semester. At every class period, you can articulate the value of what they’re learning and the purpose of the activities they’re doing in and out of class to aid their learning. Better yet, ask them to articulate the value! Same goes for why you’ve chosen the types of assessments you’re giving them. Keep connecting the activities and assessments to the course goals so students can see the big picture.

2. KEEP A LOG.

Take a moment right after class (or as soon as possible) to reflect on how the day’s activity or assessment went. “Was it a hero or a zero?” as Laurie Grenier from the TV show Shark Tank asks. If it was a zero, don’t let that dissuade you from trying again next semester. See if you can determine what went awry and find ways to tweak it. Were students adequately prepared for the activity? Were the directions and prompts clear and did they hit that sweet spot of being challenging without being too far above your students ability? Was there a tech fail? If you do notice something not working in the moment and aren’t sure why, remember that you could always ask your students.  

3. SOLICIT COURSE FEEDBACK. 

After students get their grades on the first major assessment, think about getting formative course feedback from them. What elements of the course are working or not working for them? What can you be doing differently to help support their learning? What could they be doing differently to support their learning? Be sure to address the feedback in your next class session. Tell them what you’re going to keep doing and what you’re going to change (and why!) and be sure to follow through. If you’d prefer to call in an educational developer from the CAT, we can do a mid-semester instructional diagnosis. We would meet with your class without you present to get consensus feedback and then prepare a report to review with you and discuss strategies for implementing any changes you’d like to make to the course.

4. TAKE A TRIP DOWN MEMORY LANE. 

At semester’s end, incorporate an activity that asks students to review their body of work and let them self-assess whether they believe they’ve met the course goals (and why or why not). You could also ask them to write a letter to future students about the design of the course and what advice they might have for maximizing their success in the course.

* * *
So, take pride in the work you’ve done to design (or redesign) your course, make the design elements transparent for your students from the start, be open to self-reflection and student feedback on the design, and have the best fall semester ever! And remember that you are not alone. As always, the CAT is here to help you design and implement your course via 1-1 consultations, teaching observations, and mid-semester instructional diagnoses.

Cliff Rouder, Ed.D., is Pedagogy and Design Specialist at Temple’s Center for the Advancement of Teaching.

The Heart of the Course: Learning Activities!

Jeff Rients

Along with a set of significant learning goals for your students to achieve and an array of assessments that allow you (and they!) to know they’ve met them, you also need to develop what the students actually are going to do in your course. These learning activities include anything that prepares the students to succeed at the assessments and gain the practice (and feedback) needed to achieve the learning goals.

We often think about learning activities in terms of  ‘delivering content’, e.g. lectures, readings, videos, etc. But instead of thinking about these items as the things we the teachers deliver to students, it helps to flip the script and start thinking about what students will do in the course to learn more deeply.  This shift in thinking about learning matters–because once you take the emphasis off of you delivering the content and put it on them interacting with it, you are then in a position to ask yourself this key question: what else can the students do (besides consume more content) in order to achieve the goals of the course?

That’s where active learning techniques often come into play. These techniques give students an opportunity to engage with course material directly and authentically while affording them a chance to practice the skills they will need to succeed at your assessments. Some courses come with readymade activities that you may already be using, such as students working on problem sets in a math course, group critiques in an art course, or role-played client interactions in a therapy course. However, a wide variety of active learning techniques exist that will work in nearly any classroom. These techniques are designed to go beyond the flawed idea that students will simply absorb the content like sponges (which we humans aren’t really that good at, cognitively). Rather they put students into the position of interacting with the content, and each other, in order to gain a deeper understanding of course concepts and content.

Here’s a simple active learning technique that we love at the Center for the Advancement of Teaching:

Think-Pair-Share

  • Students are given a prompt (a question or problem) relevant to the learning goal and course for the day. Students silently think (and perhaps write) individual responses to the prompt for a minute or two. Then the students pair up for five or ten minutes to compare their responses. Finally, the instructor facilitates a whole class discussion as pairs share their results, such as any insights they discovered or areas where they got stuck. This last phase typically takes between ten and twenty minutes.

Among other things, the Think-Pair-Share helps make space for students who would otherwise be reluctant to speak up in class. The period of individual reflections followed by the comparative discussion with a peer gives them the opportunity to test their own understanding and hone their ideas with peers, helping the students to feel more confident in sharing their own thoughts to the whole class. It also gives students a chance to help them solidify their own understanding by requiring them to articulate it to a classmate.

Many, many other techniques exist for students to wrangle with content in your class. You can get started with this CAT handout, but you might also want to check out Elizabeth F. Barkley’s Student Engagement Techniques (Jossey-Bass 2010) and Discussion as a Way of Teaching by Brookfield and Preskill (Jossey-Bass 2005). Both books cover a variety of classroom learning activities (and we have them available for Temple faculty to borrow at the CAT).

As you plan your learning activities–whether they be classroom activities, homework, readings or whatever else the students need to succeed–it can be useful to plot out the activities using a three-column table, and how they align with the learning goals and assessments. Below is a sample excerpted from a literature class course design, with the readings omitted from the Activities column for brevity.

Learning GoalAssessment(s)Activities
Demonstrate the ability to build an evidence-based argument about a text (Application Goal)Textual annotation, using comments feature on Google DocsTraditional literary explication paperSocial annotation using PerusallReverse engineering a reading: find the evidence cited in the original context, look for contradictory evidence.
Relate a work of literature to the context of its composition (Integration Goal)Knowledge Grid (Barkley & Major, Learning Assessment Techniques, 208-13)Chalk Talk activity (Brookfield, The Skillful Teacher, 94-5)
Value themselves as participants in the joint adventure of world literature. (Caring Goal)Student-created zineWrite Literary Autobiographies (as per “Autobiographical Reflections” in Barkley, Student Engagement Techniques, 301-4)

With all of the elements of a sound course redesign – goals, assessments, and learning activities – thus aligned, your course will make sense to your students in a new way. As a result, they’ll be far less likely to assume that any task you ask them to perform is mere busy work. Instead, your students will be able to see that you want them to achieve great things and you are providing the practice and support they need to achieve them.

Don’t miss our last installment in the Summer Course Design Series: Pulling It All Together!

Jeff Rients is an Assistant Director at Temple’s Center for the Advancement of Teaching.

Looking for Evidence in all the Right Places: Aligning Assessments with Goals

Dana Dawson

You’ve written the learning goals for your course and are now ready to design learning assessments that align with your course goals, offer opportunities for formative feedback and are educative. Well-designed learning assessments will:

  • Provide evidence that students have met your learning goals;
  • Support students in progressing toward accomplishing your learning goals;
  • Allow students to assess their learning process and progress; and
  • Help you discern whether your learning materials and activities are effective.

Learning assessments are often described as formative or summative. Formative assessments are designed to give students feedback they can use for future work and are most commonly low stakes and assigned early and often in a unit or course. Summative assessments provide a snapshot of a student’s learning at a point in time (at the end of a unit or course, for example). Another way to frame this is using Dee Fink’s description of Auditive versus Educative Assessments. Auditive assessments are backward looking and are used to determine whether students “got it.” Educative assessments have clear criteria and standards (through the use of rubrics, for example), help us ascertain whether students are ready for a future activity, and provide opportunities for high quality feedback from the instructor and self-assessment on behalf of the student.

Here are some things to keep in mind as you design your assessments.

Start with your goals

You have determined what you want students to be able to do or to know by the end of your course and articulated those ambitions as learning goals. Now you must determine the activity or product that would provide the best evidence as to whether your students have reached a particular goal? What can your students do or create to demonstrate they have gained facility with the content or skills the course promises to deliver?

The previous post in this series outlined the six categories of goals that constitute Fink’s Taxonomy of Significant Learning (see the table below). The type of assessment you select will depend on the nature of the learning goal it is designed to address. For example, while a multiple choice quiz may be a good option for assessing foundational knowledge, it may not be a good fit for integration or caring goals. Here are some suggestions for types of assessments or assessment strategies that align with the dimensions of Fink’s Taxonomy of Learning. Note that many of the suggestions listed below will address more than one dimension. For example, a carefully constructed research poster assignment might assess how students define key concepts or methods (foundational knowledge), use communication skills (application), articulate the significance of the project (caring), consider their audience in designing the poster (human dimension) and pull together research skills taught and practiced throughout the semester into a coherent whole (integration).

Elements of Fink’s Taxonomy of Significant LearningExamples of Assessments
Foundational Knowledge
What key information is important for students to understand in this course or in the future?
Multiple choice quiz, guided notes, classroom polling, quotation summaries
ApplicationWhat kinds of thinking are important for students to learn? What important skills do they need to gain?Briefing paperdyadic essaylab reportannotated bibliographyproblem-based learning
IntegrationWhat connections (similarities and interactions) should students recognize and make in this course and with other courses or areas of learning? Or within their own personal lives?Reading prompts, learning portfoliocase studyresearch poster
Human DimensionWhat could or should students learn about themselves and others?Asset-mapping, role playtest-taking teamsstudent peer reviewdyadic interviews
CaringWhat changes/values/passions do you hope your students will adopt?Positive projects, contemporary issues journal“what, so what, now what” journal, class participation, critiquesWikipedia assignment
Learning How to LearnWhat would you like for your students to learn about how to be a good student, learn in this subject, and become self-directed learners, and develop skills for lifelong learning?Ask students to prioritize areas of feedback, advance organizers, self-reflection assignmentstwo-stage exams

Use this worksheet to reflect on assessments that align with your goals and whether your goals and assessments address all six elements of the Taxonomy of Significant Learning.

Don’t forget those situational factors

Assessments designed for first-semester undergraduates ought to differ from those assigned to graduate students. When designing your assessments, you will need to put on your own Human Dimension hat and transport yourself back into the shoes of a learner taking their first lab, completing their BFA exit portfolio, doing rotations, and so forth. You may need to design assessments that also align with department, program or accreditor goals and assessment efforts. Factors such as the number of students in your section and instructional modality will influence assessment decisions.

Use assessment to support student learning

If assessments are infrequent or completed only at the end of a unit or course, they will not give students an opportunity to practice prior to summative assessments or to use your feedback. Remember that learning assessments do not have to be graded. There may be times that the primary purpose of an assessment activity is to help students gauge their own understanding or for you to get a big-picture sense of whether students are following you. In-class or low stakes Learning Assessment Techniques can be used throughout the semester to give students immediate feedback. Consider whether there are opportunities to build revision into your assignment design.

Assessments give you information – use it!

A classroom polling activity may tell you that your lecture on a topic didn’t land with a significant number of your students and that you need to spend a bit more time on it in the next session. A series of ineffectual peer reviews or critiques may tell you that you need to provide more guidance on how to conduct peer reviews or critiques. Learning assessments provide feedback on our students’ progress and on our own work as educators. Take time to reflect on what assessment results tell you not only about your students’ learning but also about your instructional strategies.

When aligned to your learning goals and designed to accommodate situational factors, address the six elements of Fink’s Taxonomy and guide future effort, your assessments will be an essential component of successful course delivery.

For support in designing learning assessments, don’t hesitate to book a consultation with a CAT specialist.

Dana Dawson is Associate Director of Temple’s Center for the Advancement of Teaching.