Course Design – Page 3 – EDVICE EXCHANGE

The Heart of the Course: Learning Activities!

Jeff Rients

Along with a set of significant learning goals for your students to achieve and an array of assessments that allow you (and they!) to know they’ve met them, you also need to develop what the students actually are going to do in your course. These learning activities include anything that prepares the students to succeed at the assessments and gain the practice (and feedback) needed to achieve the learning goals.

We often think about learning activities in terms of ‘delivering content’, e.g. lectures, readings, videos, etc. But instead of thinking about these items as the things we the teachers deliver to students, it helps to flip the script and start thinking about what students will do in the course to learn more deeply. This shift in thinking about learning matters–because once you take the emphasis off of you delivering the content and put it on them interacting with it, you are then in a position to ask yourself this key question: what else can the students do (besides consume more content) in order to achieve the goals of the course?

That’s where active learning techniques often come into play. These techniques give students an opportunity to engage with course material directly and authentically while affording them a chance to practice the skills they will need to succeed at your assessments. Some courses come with readymade activities that you may already be using, such as students working on problem sets in a math course, group critiques in an art course, or role-played client interactions in a therapy course. However, a wide variety of active learning techniques exist that will work in nearly any classroom. These techniques are designed to go beyond the flawed idea that students will simply absorb the content like sponges (which we humans aren’t really that good at, cognitively). Rather they put students into the position of interacting with the content, and each other, in order to gain a deeper understanding of course concepts and content.

Here’s a simple active learning technique that we love at the Center for the Advancement of Teaching:

Think-Pair-Share

Students are given a prompt (a question or problem) relevant to the learning goal and course for the day. Students silently think (and perhaps write) individual responses to the prompt for a minute or two. Then the students pair up for five or ten minutes to compare their responses. Finally, the instructor facilitates a whole class discussion as pairs share their results, such as any insights they discovered or areas where they got stuck. This last phase typically takes between ten and twenty minutes.

Among other things, the Think-Pair-Share helps make space for students who would otherwise be reluctant to speak up in class. The period of individual reflections followed by the comparative discussion with a peer gives them the opportunity to test their own understanding and hone their ideas with peers, helping the students to feel more confident in sharing their own thoughts to the whole class. It also gives students a chance to help them solidify their own understanding by requiring them to articulate it to a classmate.

Many, many other techniques exist for students to wrangle with content in your class. You can get started with this CAT handout, but you might also want to check out Elizabeth F. Barkley’s Student Engagement Techniques (Jossey-Bass 2010) and Discussion as a Way of Teaching by Brookfield and Preskill (Jossey-Bass 2005). Both books cover a variety of classroom learning activities (and we have them available for Temple faculty to borrow at the CAT).

As you plan your learning activities–whether they be classroom activities, homework, readings or whatever else the students need to succeed–it can be useful to plot out the activities using a three-column table, and how they align with the learning goals and assessments. Below is a sample excerpted from a literature class course design, with the readings omitted from the Activities column for brevity.

Learning Goal	Assessment(s)	Activities
Demonstrate the ability to build an evidence-based argument about a text (Application Goal)	Textual annotation, using comments feature on Google DocsTraditional literary explication paper	Social annotation using PerusallReverse engineering a reading: find the evidence cited in the original context, look for contradictory evidence.
Relate a work of literature to the context of its composition (Integration Goal)	Knowledge Grid (Barkley & Major, Learning Assessment Techniques, 208-13)	Chalk Talk activity (Brookfield, The Skillful Teacher, 94-5)
Value themselves as participants in the joint adventure of world literature. (Caring Goal)	Student-created zine	Write Literary Autobiographies (as per “Autobiographical Reflections” in Barkley, Student Engagement Techniques, 301-4)

With all of the elements of a sound course redesign – goals, assessments, and learning activities – thus aligned, your course will make sense to your students in a new way. As a result, they’ll be far less likely to assume that any task you ask them to perform is mere busy work. Instead, your students will be able to see that you want them to achieve great things and you are providing the practice and support they need to achieve them.

Don’t miss our last installment in the Summer Course Design Series: Pulling It All Together!

Jeff Rients is an Assistant Director at Temple’s Center for the Advancement of Teaching.

Looking for Evidence in all the Right Places: Aligning Assessments with Goals

Dana Dawson

You’ve written the learning goals for your course and are now ready to design learning assessments that align with your course goals, offer opportunities for formative feedback and are educative. Well-designed learning assessments will:

Provide evidence that students have met your learning goals;
Support students in progressing toward accomplishing your learning goals;
Allow students to assess their learning process and progress; and
Help you discern whether your learning materials and activities are effective.

Learning assessments are often described as formative or summative. Formative assessments are designed to give students feedback they can use for future work and are most commonly low stakes and assigned early and often in a unit or course. Summative assessments provide a snapshot of a student’s learning at a point in time (at the end of a unit or course, for example). Another way to frame this is using Dee Fink’s description of Auditive versus Educative Assessments. Auditive assessments are backward looking and are used to determine whether students “got it.” Educative assessments have clear criteria and standards (through the use of rubrics, for example), help us ascertain whether students are ready for a future activity, and provide opportunities for high quality feedback from the instructor and self-assessment on behalf of the student.

Here are some things to keep in mind as you design your assessments.

Start with your goals

You have determined what you want students to be able to do or to know by the end of your course and articulated those ambitions as learning goals. Now you must determine the activity or product that would provide the best evidence as to whether your students have reached a particular goal? What can your students do or create to demonstrate they have gained facility with the content or skills the course promises to deliver?

The previous post in this series outlined the six categories of goals that constitute Fink’s Taxonomy of Significant Learning (see the table below). The type of assessment you select will depend on the nature of the learning goal it is designed to address. For example, while a multiple choice quiz may be a good option for assessing foundational knowledge, it may not be a good fit for integration or caring goals. Here are some suggestions for types of assessments or assessment strategies that align with the dimensions of Fink’s Taxonomy of Learning. Note that many of the suggestions listed below will address more than one dimension. For example, a carefully constructed research poster assignment might assess how students define key concepts or methods (foundational knowledge), use communication skills (application), articulate the significance of the project (caring), consider their audience in designing the poster (human dimension) and pull together research skills taught and practiced throughout the semester into a coherent whole (integration).

Elements of Fink’s Taxonomy of Significant Learning	Examples of Assessments
Foundational Knowledge What key information is important for students to understand in this course or in the future?	Multiple choice quiz, guided notes, classroom polling, quotation summaries
ApplicationWhat kinds of thinking are important for students to learn? What important skills do they need to gain?	Briefing paper, dyadic essay, lab report, annotated bibliography, problem-based learning
IntegrationWhat connections (similarities and interactions) should students recognize and make in this course and with other courses or areas of learning? Or within their own personal lives?	Reading prompts, learning portfolio, case study, research poster
Human DimensionWhat could or should students learn about themselves and others?	Asset-mapping, role play, test-taking teams, student peer review, dyadic interviews
CaringWhat changes/values/passions do you hope your students will adopt?	Positive projects, contemporary issues journal, “what, so what, now what” journal, class participation, critiques, Wikipedia assignment
Learning How to LearnWhat would you like for your students to learn about how to be a good student, learn in this subject, and become self-directed learners, and develop skills for lifelong learning?	Ask students to prioritize areas of feedback, advance organizers, self-reflection assignments, two-stage exams

Use this worksheet to reflect on assessments that align with your goals and whether your goals and assessments address all six elements of the Taxonomy of Significant Learning.

Don’t forget those situational factors

Assessments designed for first-semester undergraduates ought to differ from those assigned to graduate students. When designing your assessments, you will need to put on your own Human Dimension hat and transport yourself back into the shoes of a learner taking their first lab, completing their BFA exit portfolio, doing rotations, and so forth. You may need to design assessments that also align with department, program or accreditor goals and assessment efforts. Factors such as the number of students in your section and instructional modality will influence assessment decisions.

Use assessment to support student learning

If assessments are infrequent or completed only at the end of a unit or course, they will not give students an opportunity to practice prior to summative assessments or to use your feedback. Remember that learning assessments do not have to be graded. There may be times that the primary purpose of an assessment activity is to help students gauge their own understanding or for you to get a big-picture sense of whether students are following you. In-class or low stakes Learning Assessment Techniques can be used throughout the semester to give students immediate feedback. Consider whether there are opportunities to build revision into your assignment design.

Assessments give you information – use it!

A classroom polling activity may tell you that your lecture on a topic didn’t land with a significant number of your students and that you need to spend a bit more time on it in the next session. A series of ineffectual peer reviews or critiques may tell you that you need to provide more guidance on how to conduct peer reviews or critiques. Learning assessments provide feedback on our students’ progress and on our own work as educators. Take time to reflect on what assessment results tell you not only about your students’ learning but also about your instructional strategies.

When aligned to your learning goals and designed to accommodate situational factors, address the six elements of Fink’s Taxonomy and guide future effort, your assessments will be an essential component of successful course delivery.

For support in designing learning assessments, don’t hesitate to book a consultation with a CAT specialist.

Dana Dawson is Associate Director of Temple’s Center for the Advancement of Teaching.

Learning Goals: Dream Big!

Linda Hasunuma, Ph.D.

Take a close look at your syllabus. What do your learning goals (if you have them) say about what students are going to learn and achieve in your course? Often, our goals or course descriptions focus entirely on foundational knowledge and some application of that knowledge, but what about learning goals that go beyond facts, concepts, formulas, and theories? In this blog post, the third in our summer series on course design, we focus on how we can articulate learning goals that integrate our highest aspirations for learning and what Dee Fink calls our Big Dream for our students. What do we want students to take away, do, and remember years later from their time with us? Fink reminds us in his guide to creating courses for significant learning that we should lead our course design not with the content we will cover but instead with the goals we are hoping our students will reach.

So, what is your Big Dream and how can you craft that into a learning goal? Fink created a taxonomy to help you do just that. Fink’s Taxonomy of Significant Learning encourages instructors to think broadly about their goals for their students. A course goal might be focused on basic information you need students to know or on applying that foundational knowledge (the right side of the taxonomy), but goals focused on learning about oneself or others, or learning how to learn are equally important (the left side of the taxonomy). See below and think about where your current course learning goals are versus where you could go if you dared to dream big and include more of what is on the left side of his taxonomy. Most of us build goals in the foundational knowledge and application areas, but what can we do to include integration, the human dimensions, and caring into the learning experiences we create for our students?

By articulating goals that include more pieces of this pie, we can challenge ourselves to develop new and creative activities, assignments, and assessments that help our students make connections to one another and to the world. We can make our course content more meaningful to our students and their lives and can intentionally and thoughtfully build transformative and significant learning experiences.

The following questions can also help you brainstorm and draft learning goals so that we can aim to have students try to reach more of the goals on the left side of the pie:

Big Dream: A year or more after this course is over, what do you want and hope your students will do?
Foundational Knowledge: What key information (facts, formula, terms, concepts, relationships, etc.) is/are important for students to understand in this course or in the future?
Application Goals: What kinds of thinking are important for students to learn (critical thinking, in which students analyze and evaluate; creative thinking, in which students imagine and create; and practical thinking, in which students solve problems and make decisions)? What important skills do they need to gain?
Integration Goals: What connections (similarities and interactions) should students recognize and make in this course and with other courses or areas of learning? Or within their own personal lives?
Human Dimension Goals: What could or should students learn about themselves and others?
Caring Goals: What changes/values/passions do you hope your students will adopt?
Learn how to learn goals: What would you like for your students to learn about how to be a good student, learn in this subject, and become self-directed learners, and develop skills for lifelong learning?

As we expand our understanding of learning goals to make them more ambitious and think about what we want students to actually DO, the verbs we choose to write the goals make all the difference in helping to create an authentic, transformative and significant learning experience. At the CAT, we suggest using Noyd’s 2008 table of verbs based on Fink’s Taxonomy as you think about developing, revising, or refining your own learning goals for your classes and students.

After brainstorming some draft goals, you may want to review them with a colleague to make sure they are effective and clear. Are your draft goals too narrow? Are they written in language your students will understand? Do they motivate and challenge your students? Which areas of the pie are represented in that learning goal? We don’t just teach content; we teach human beings. Though we may not have been encouraged to include the human and caring dimensions in our syllabi and courses during our own education and training, this framework and taxonomy remind us to keep the bigger picture in mind and to be bold in articulating our dreams for our students. Those dreams and hopes can be part of your learning goals!

Now that we have provided a framework for thinking about and designing your course and learning goals, we turn to assessments for the next post in this series. Working backwards from that Big Dream and our more ambitious learning goals, how can you evaluate learning and progress toward those goals?

References:

Fink, Dee L. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. San Francisco, Jossey-Bass, 2013 (pp.83-84).
Noyd, Robert K. and the Staff of the Center for Educational Excellence, (white paper 08-01), A Primer on Writing Effective Learning-Centered Course Goals, 2008. Colorado Springs, CO. US Air Force Academy.

Linda Hasunuma serves as an Assistant Director at Temple’s Center for the Advancement of Teaching.

Context Matters: Considering Situational Factors in Course Design

H. Naomie Nyanungo

Imagine trying to plan a trip with limited knowledge of your destination. Maybe you know dates of your departure and return and that you will have some travel companions but not much else. You don’t know the weather at your destination, or even how you will get there? You don’t know how many travel companions you will have or anything about them. If you are like me, who likes to feel prepared before embarking on any adventure, this sounds like a nightmare situation. I hope you can see where I am going with this – it is hard to plan for something without considering the context. This is true for planning a trip as it is for designing a course.

The courses we design and teach take place in specific contexts, they do not happen in a vacuum. The situational factors in our context should inform the decisions we make about our learning goals, activities, assessments and feedback strategies. For example, the types of teaching and learning activities that I use in an asynchronous online course will be different from those in an in-person course. A well-designed course is one that takes into consideration relevant contextual factors. When we fail to consider the situational factors in the process of designing courses, we run the risk of setting unrealistic expectations of student performance and alienating our students. It could also result in poor alignment with standards set by departments, programs or accrediting agencies. Ultimately, it leads to frustration for both instructors and students.

Consideration of situational factors is the first step of Dee Fink’s Integrated Course Design Model. The model identifies five categories of contextual factors listed below (with examples of questions for each category):

Specific context factors: E.g. what classroom will be used for the course, how many students, how often will the class meet, how instruction will be delivered?
Expectations of others: E.g. what are the expectations placed on this course by the university, department, accreditation agencies, and the students?
Nature of the subject: E.g. is the subject primarily theoretical, practical, applied, or some combination?
Characteristics of the students: E.g. what are the characteristics of students who take this class? Are they working professionals? Are they majors in this field?
Characteristics of the teacher: E.g. what are the factors about your approach to teaching that are relevant to this course? What is your level of knowledge or familiarity with the subject? What is your level of comfort teaching in the specific modality?

It is important to note that not all of these factors are relevant to all teaching situations. You will need to determine which of these are relevant for you. As teachers we usually don’t determine which students will enroll in our class, the classrooms we will teach in, or the expectations of accrediting agencies. The challenge for us is to design good courses knowing the parameters beyond our control in the teaching context.

We encourage you to think about issues of equity and inclusion when assessing the situational factors of your course. In Inclusion by Design: Tool Helps Faculty Examine Their Teaching Practices, Moore and his colleagues share some helpful questions to guide our thinking about equity and inclusion in situational factors.

With some knowledge of the contextual factors in our teaching situation, we can be more confident about the decisions we will make when designing our courses, starting with the next step in this process – Setting Learning Goals.

H. Naomie Nyanungo is Director of Educational Technology at Temple’s Center for the Advancement of Teaching.

Design Your Course for Significant Learning! A Step-By-Step Guide

Stephanie Fiore

The first time I taught, I was a first-year graduate student assigned to teach a section of an upper-level multi-section Italian Cinema course. I had never taught before, and certainly didn’t feel that I possessed the expertise to teach upperclassmen about the subject. The faculty fed me notes from their lectures and handed me the syllabus I would need to follow. It was the usual syllabus with a list of policies, a schedule of topics—i.e. the list of films that we would discuss—and the dates of two midterms and the final exam. There was no explanation about why the course was structured in this particular way (except that it was chronologically organized by date of film) nor was there any instruction on how to teach it effectively. I muddled through as so many do when we begin teaching in higher ed, but I must admit that I had little idea what the overarching goals of the course were and what I was hoping my students would achieve by the end of it. I knew only that I needed to do a lot of research on each film and then ask interesting questions to get a discussion going, focusing on some of the salient points from my research.

My experience is not unusual. The truth is that very few of us are provided guidance in graduate school about designing courses that will lead our students to what L. Dee Fink calls “significant learning.” As Fink explains it, “for learning to occur, there has to be some kind of change in the learner. No change, no learning. And significant learning requires that there be some kind of lasting change that is important in terms of the learner’s life” (Fink, 2013). But how to achieve that significant learning? This summer’s EdVice Exchange Course Design Blog Series offers a step by step guide to Fink’s Integrated Course Design model in order to lead you and your students towards that significant learning experience we all wish for.

In Fink’s model, we begin by considering the situational factors that define the learning environment in which we are teaching. Then we start designing our course, starting with learning goals. This is a meaningful change from the content-first way we often plan our courses (What content do we have to cover? What order is the textbook in?), shifting our focus instead towards the learning goals we hope our students will achieve. We then identify assessments that provide evidence students have reached the learning goals, and activities that provide the practice students need to achieve mastery. This is not a linear process but instead an integrated one in which goals, assessments and activities work together to provide the significant learning we seek. We will discuss each of these elements of course design in greater depth as we move through the series, providing time between each piece for you to apply what you are learning to your course(s).

EdVice Exchange Course Design Series:

Part 2: Considering Situational Factors

Part 3: Articulating Meaningful Goals

Part 4: Aligning Assessments with Goals

Part 5: Developing Teaching Activities

Part 6: Implementing Your Course Design

This year, I taught a similar course to the one taught years ago. Using Fink’s model, I designed the course by focusing on the goals I wanted my students to achieve. Of course, I wanted them to develop an appreciation for Italian culture and film. But I also wanted them to develop analytical skills for critically discussing and writing about film. I wanted them to explore the complex interactions between film and its historical, cultural, and political context. I also wanted them to learn how to build new knowledge through productive and collaborative discussions. By clarifying for myself exactly what the learning goals were for the course, I was able to design learning activities that would get them there and assessments that would provide evidence that they had indeed achieved those goals. Student self-assessments at the end of the semester reflected their own awareness of having experienced significant learning. They spoke about formulating new ways of thinking that challenged their previous perspectives, looking at stories as an expression of culture, and learning to critically analyze film through an understanding of historical, cultural, and political factors. One commented, “discussions allowed us to practice analyzing these films, and hearing what other people noticed broadened our perspectives of not only the film itself, but the director, their choices, and the process of filmmaking.” The important thing to notice here is that designing our courses for significant learning means both we and our students understand where the learning journey is taking us and how it will fundamentally change us along the way. That is a powerful outcome for any one of us!

Interested in reading more about designing courses for significant learning? See L. Dee Fink’s Self-Directed Guide to Significant Learning.

Stephanie Fiore, Ph.D., is Assistant Vice Provost at Temple’s Center for the Advancement of Teaching.

Jumpstart Student Engagement and Curiosity with ABC

Meg Van Baalen-Wood

Early in my first semester of grad school, a mentor introduced what was then a counterintuitive teaching strategy: Activity Before Content, or ABC. The strategy is simple: have students explore concepts and ideas before presenting them with new content. For example, before introducing a new concept, ask students to define it, drawing from their experiences and expectations: What do they think it means? What does it remind them of? How have they encountered it in previous situations?

ABC can be used with equal impact in both face-to-face and virtual learning environments.

Why use ABC?

ABC has many advantages. Among them, it

Engages students in active learning
Enables students to retrieve existing information and make predictions about new information
Provides opportunities for students to review what they know, or think they know, before piling on new information
Creates a preliminary foundation for new content
Encourages students to share both knowledge and questions with their peers
Fosters a classroom community that values socially constructed knowledge
Positions you as a collaborator and member of the classroom community, rather than a sage on the stage

How does it work?

Many familiar teaching strategies lend themselves to ABC, for example,

Write or project a question on the board for students to answer before class begins. Using a polling application (e.g., PollEverywhere), share responses as they accrue in real-time or immediately after everyone has responded, so students can see what their classmates are thinking. As a class, discuss the responses; be sure to explore any misconceptions/outliers as well as correct responses.
Start class with a one- or two-minute freewrite. Discuss students’ responses as a class or in small groups before sharing them out with the entire class. In online classes, leverage asynchronous discussion threads for these small group explorations.
Create a low-stakes quiz that students complete in small groups at the beginning of class. Provide a few minutes at the end of class for groups to revisit/revise their answers after you’ve presented and discussed the content.

Once you get started, you’ll think of many more ways to pique your students’ curiosity, engage them in active learning, and create a vibrant classroom community with ABC.

Meg Van Baalen-Wood is from the Elbogen Center for Teaching and Learning at the University of Wyoming.

Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).

Navigating the New Information Ecosystem: Helping Students Thrive in the Digital Age

Dana Dawson, Caitlin Shanley and Stephanie Fiore

The most common forms of information literacy education in recent decades have focused on developing our students’ ability to find sources and use them effectively and ethically in support of a thesis. The development of skills related to source identification and evaluation for scholarly purposes is undoubtedly an important outcome for college-level studies. However, the revised Framework for Information Literacy in Higher Education, developed by the Association of College and Research Libraries (ACRL), Trudi Jacobson and Thomas Mackey’s work on metaliteracy and numerous studies conducted by Project Information Literacy have shown the need for a revised approach and provided new tools for addressing information literacy. Given the volume of information our students are exposed to on a daily basis, it is critical that they graduate with an enhanced awareness of how the information ecosystem works, how emotion influences the information we believe and act upon, and how investigative skills used for scholarly study can and should be applied in our everyday lives. Here are some approaches to consider for your own classroom.

Metacognition

Metacognition involves reflecting on our cognitive processes and developing an awareness of how we think and learn so that we can adjust our behaviors on the basis of what was effective. Applying metacognitive strategies allows our students to take control of their educational process and make connections between prior knowledge and course content. The metacognitive process for learning involves an iterative cycle of planning, monitoring and evaluating one’s learning with focused self–questioning at each stage.

This same metacognitive process can be used to develop a reflective stance on the information we’re exposed to in our studies and everyday lives. To help our students become more critical and reflective consumers of information, encourage them to apply the tools of the metacognitive process to their information consumption. In order to improve information literacy, students can ask themselves questions such as::

Planning – What sources have I selected to read/watch/listen to and why? Will I look at multiple sources on an event or topic? Why am I reading about/watching/listening to this?
Monitoring – What kinds of information are more likely to appear at the top of my feed? How am I feeling as I read/watch/listen to this? Do I agree with the information being presented because it has been verified or because it aligns with what I already believe? Am I investigating claims before forwarding them to others?
Evaluating – Did I spend more time on certain types of sources than others? Whose information was I more likely to believe and share with others? Were my efforts to investigate sources and claims using other sources successful?

Ask good questions

Generalized skepticism is among the challenges instructors face in relation to information literacy and “I’m just asking” is one of the most potent tools for spreading distrust of substantiated information. However, learning to ask good questions for the purposes of investigation and verification is a foundational skill for scholarly study and conscientious consumption of information. Dan Rothstein and Luz Santana’s Question Formulation Technique is a strategy instructors can use to prompt students to not only generate questions but reflect on and prioritize the questions that will generate the most productive investigation.

Lateral reading

In a small but provocative study comparing the website evaluation strategies of Stanford students and history professors with those of professional fact checkers, Wineburg and McGrew found that fact checkers applied two strategies neither students nor faculty immediately went to – “taking bearings” and lateral reading. Rather than evaluate a source by digging deep into the source itself (for example, applying the CRAAP test), fact checkers started by learning more about the website or source and then opened new tabs and found additional sources to investigate claims and sources on the website as they read content on the initial site.

SIFT method

The SIFT method combines elements of metacognition and lateral reading into a simple mnemonic for evaluating claims.

Stop
Investigate the source
Find better coverage
Trace claims to their source

Connect classroom activities with everyday life

Our students come to our classrooms with some awareness of how information is manipulated and with strategies for finding and evaluating sources. However, the very human tendency to compartmentalize information means that we need to be proactive in connecting what happens in the classroom with the strategies applied outside the classroom. If you assign a research paper or project, start with a reflection on how students find information in their everyday life. Use analogies such as crime scene investigations or learning more about a possible romantic partner to encourage deeper thinking about how we ask questions and seek additional information. LaGarde and Hudgins encourage instructors to have students use personal devices such as cell phones to do some searching to help make the connection between search strategies learned in the classroom and those employed in everyday life.

Inoculate against mis/disinformation

Finally, recent research on protecting against mis- and disinformation has suggested the efficacy of “inoculating” students against future exposures. Strategies such as explaining misleading argumentation techniques, demonstrating how content is designed to manipulate emotion and showing how certain types of content are more likely to be prioritized on platforms students access can reduce the influence of misleading information going forward. Sites such as Bad News Game can be used to help students understand how platforms and content creators play on our emotions and pre-existing beliefs to increase viewership and readership. Lorenz-Spreen et.al. have also shown that such interventions in combination with self-reflection on aspects of our personality or behavior that might be targeted is particularly effective in boosting one’s ability to detect manipulative strategies.

Additional Resources

Algorithmic Justice League www.ajl.org
Digital Citizenship Resource Platform https://dcrp.berkman.harvard.edu/
News Literacy Project https://newslit.org/
Project Information Literacy: https://projectinfolit.org/

Sources

Downey, A. (2016). Critical Information Literacy : Foundations, Inspiration, and Ideas. Library Juice Press.

Dana Dawson is Associate Director of Teaching and Learning at Temple’s Center for the Advancement of Teaching.

Caitlin Shanley is the Coordinator of Learning & Student Success at Temple University Libraries.

Stephanie Fiore is Assistant Vice Provost at Temple’s Center for the Advancement of Teaching.

Advance Organizers: Setting the Stage for Learning and Retention

Dana Dawson

The Advance Organizer is a tool you can use to focus student attention during lectures, improve retention of course content and connect new information with prior knowledge. Advanced Organizers can take a variety of forms from brief expository overviews to help students make meaning of course content to graphic representations that provide a framework for information.

Overview

As an expert in your field, it can be hard to put yourself in the shoes of someone new to the information you’re sharing. The connections between content delivered in separate lessons, how a theory applies to a specific case or even the meaning of frequently used terms in your discipline may seem so obvious to you that they don’t require explicit explanation for your students. Ambrose et. al. point out that one important way in which expert and novice learners differ is in the density of connections among concepts, facts and skills. Research on learning has shown that students retain more of what they learn and for longer if they connect new information with prior knowledge and have an organizational schema, or knowledge structure within which to fit new information.

In addition to giving students a framework within which to fit new information, Advance Organizers help students focus during lectures, whether given in-person or delivered in a pre-recorded format. The duration of a student’s attention span will vary for a variety of reasons, but research suggests that we should anticipate attention lapses and intersperse listening to videos or lectures with frequent activities to bring attention back to the content, integrate low stakes opportunities for students to self-assess understanding and highlight key points and takeaways.

Types and Examples

Expository organizers are used primarily for new and unfamiliar learning material. For example, Dee Ann Gillies tested the use of expository organizers in the form of short statements outlining unifying concepts related to information presented during lectures with students enrolled in an introductory course in medical surgical nursing. The study found improved learning and short-term retention of material.

Narrative organizers use a story structure to convey information. A story might illustrate something about course content through analogy or by turning key concepts into characters and building detail and narrative to help students recall key information (see, for example, Sketchy’s study guides sketchy.com).

Know-Wonder-Learn (KWL) is an advanced organizer that helps both instructor and student access background knowledge. Students are asked to complete the first two columns prior to instruction and to share what they have written. The final column can be filled in for reflection or submission.

(https://teaching.vt.edu/content/dam/teaching_vt_edu/resources/Advance%20Organizers.pdf)

Graphic organizers constitute the biggest category of Advance Organizers and can take a wide variety of forms, including tables, venn diagrams, flow charts and concept maps. They can be used to help students see connections between concepts, the steps in a process or sequence of events or simply to organize students’ thinking before, during or after instruction.

Examples:

(Barkley, Elizabeth F.. Interactive Lecturing: A Handbook for College Faculty, John Wiley & Sons, Incorporated, 2018: 242. ProQuest Ebook Central, https://ebookcentral.proquest.com/lib/templeuniv-ebooks/detail.action?docID=5254245.)

Implementation

Advance Organizers can be distributed in completed form or as blank documents that students complete before, during or after instruction. The SmartArt options in PowerPoint are helpful for creating graphic organizers that suit your specific needs. You might assign students to work in pairs or small groups to complete the Advance Organizer or compare notes. They also can be used in your syllabus or Canvas course to provide students with an overview of the material to come. For additional information on the use of graphic organizers in syllabi, review the CAT workshop recording of The Interactive and Graphic Syllabus here.

Additional Resources

Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M., Norman, M. K., & Mayer, R. E. (2010). How Learning Works: Seven Research-Based Principles for Smart Teaching. Jon Wiley & Sons, Incorporated.
Barkley, E. F. (2018). Interactive Lecturing: A Handbook for College Faculty. John Wiley & Sons, Incorporated.
Center for Excellence in Teaching and Learning, Virginia Tech. Advance Organizers. https://teaching.vt.edu/content/dam/teaching_vt_edu/resources/Advance%20Organizers.pdf
College Star. Advance Organizers: Preparing Students for Learning. https://www.collegestar.org/modules/advance-organizers#jafari-2012
Costley, J., Mik, F., Lange, C., & Baldwin, M. (2021). The effects of video lecture viewing strategies on cognitive load. Journal of Computing in Higher Education, 33(1), 19-38. http://dx.doi.org/10.1007/s12528-020-09254-y
Creately. The Ultimate List of Graphic Organizers for Teachers and Students. https://creately.com/blog/diagrams/types-of-graphic-organizers/
Gillies, D.A. (1984). Effect of advance organizers on learning medical surgical nursing content by baccalaureate nursing students. Research in Nursing and Health, 7, 173-180. https://doi-org.libproxy.temple.edu/10.1002/nur.4770070305
Jafari, K. & Hashim, F. (2012) The effects of using advance organizers on improving EFL learners’ listening comprehension: A mixed method study. System, 40(2), 270-281. https://doi.org/10.1016/j.system.2012.04.009.
Learnet. What is an Advanced Organizer? http://www.projectlearnet.org/tutorials/advance_organizers.html
Luiten, J., Ames, W., & Ackerson, G. (1980). A meta-analysis of the effects of advance organizers on learning and retention. American Educational Research Journal, 17(2), 211–218. https://doi.org/10.2307/1162483
Patricia Cross Academy. Learning Technique 29: Advance Organizers. https://kpcrossacademy.org/techniques/advance-organizers/

Dana Dawson is Associate Director of Temple University’s Center for the Advancement of Teaching.

A Game-based Approach to Teaching Calculus: Implications of the Research for STEM Courses

Cliff Rouder

On March 16, we were honored to have Dr. André Thomas from Texas A&M University as this year’s STEM Educators Lecture guest speaker. Dr. Thomas is an Associate Professor of Practice in the College of Architecture and is the Director of the LIVE lab at Texas A&M. He is also the CEO of Triseum, a small spin-off company of the LIVE lab.

Dr. Thomas shared his experience with the process of creating and testing a video game to help calculus students master a conceptual understanding of limits–a fundamental concept students often struggle with early on and that is essential for success in the course. Calculus is required for several STEM majors, but with a national failure rate (defined as a grade of D, F, or Withdrawl) of 34% for Calculus 1, it is often a barrier to students’ success in their intended major and career.

Dr. Thomas began by reminding us that when we are very young, we learn by playing games, but this element of play disappears as the educational approach shifts to sitting in rows and listening to lectures. Games are a medium for learning just like books, and thus they do belong in the classroom and are indeed currently being used in STEM courses nationally and internationally. However, completing the game isn’t what’s most meaningful; rather, it’s enabling students to take a deeper dive–to spend more time on task–through a medium that this generation of students has grown up playing and enjoying.

The process of designing a video game to teach calculus began for Dr. Thomas and his team by pinpointing the really complex and challenging subject matter that a game might help students master and identifying specific learning objectives. Many iterations later, with input from an interdisciplinary team of students, subject matter experts, game designers, instructional designers, and assessment experts, they developed the game Variant Limits. Receiving feedback from students was essential; successful game creation requires a thorough understanding of what students look for and find pleasurable in a video game experience.

Variant Limits puts students in the role of a female explorer in a 3-D world who lands on an abandoned planet and has to figure out what happened using their conceptual understanding of limits to overcome certain obstacles in that world. At the end of each segment of play, there’s an assessment for students to reflect on what they did and how it worked. To learn about the effects of Variant Limits on students’ grades, we invite you to listen to the Zoom recording of Dr. Thomas’ talk in our CAT Workshop archive.

The process of designing the game and researching its impact on student learning taught Dr. Thomas two key lessons:

STEM faculty have to get out of the mindset that their mission is to weed out students who are not performing well. Rather, they need to embrace the mission of helping more students succeed.
Helping our students recover from suboptimal performance and failure is crucial. STEM faculty are conditioned to shy away from failure, but we should be encouraging students to learn from the failure and try again rather than to quit. A video game like Variant Limits does just that. It gives students the opportunity to productively struggle, fail with no real world consequences, and then learn, try again, and get to 100% mastery.

Creating and maintaining a video game is a large undertaking that requires time, as well as significant collaboration and funding, and may not be feasible or even advisable for faculty to attempt. As an alternative, he suggested the use of existing games, for example, repurposing TIC-TAC-TOE from a competitive game to a collaborative game that might be useful in teaching a concept in their course. Even better, students might develop a game themselves in small groups, and then ask another group to improve upon it. This reinforces the real-world practice of collaborating and building on another’s idea rather than throwing out that idea and starting from scratch.

If game-based learning (GBL) has piqued your interest, be sure to sign up for our 3-part workshop beginning Wednesday the 23rd titled, Level Up! Learning with Games and Gamification. GBL can be a useful pedagogical strategy for any course!

If you’d like to keep the GBL discussion going, post a comment on our new Faculty Teaching Commons. What are your thoughts about–and experiences with–GBL? Inquiring minds want to know!

If you’d like assistance incorporating GBL into your courses, one of our faculty developers or educational technology specialists are ready to help. Make an appointment here or email a CAT staff member directly.

Feedback that Nourishes Mind and Heart

Cliff Rouder

I’ll never forget the feedback I received from two professors when I was an undergraduate (and here, I’m quoting them): “You missed the point of this question–badly!” and “Pretty good about what Sewall thought; totally baffling about why.” The first gem was on a practice question to help me prepare for the big essay exam. We call this formative feedback, where there’s an opportunity to put into practice the feedback we’re given. The second gem was feedback on a big essay; we call this summative feedback coming at the end of a unit or instruction. It’s designed to be evaluative and typically counts for a significant portion of the course grade.

Did either of these gems accomplish anything except deflate my motivation? The answer is a resounding NO for those of you playing at home. Now fast forward a couple of decades. As a conscientious instructor early on in my career, I’ll never forget returning students’ papers on which I must have spent 50 hours providing massive amounts of feedback, only to watch a few students crumple up their papers right in front of me and shoot a 3-pointer into the trash can. Woah!

The feedback we give to students–what it focuses on, how it’s expressed, how much we give, and how frequently we give it–can have a positive (or in my case above, negative) impact on motivation, and thus, on learning. Do I remember getting feedback that was motivating? Absolutely. Try this one on for size and take a moment to reflect on how it differs from the less than helpful feedback above:

“I like where you’re going in this paragraph because you’re leading me to your main argument. What I’d like you to think about is how to better connect it to the prior paragraph.”

Okay, now here’s something I can work with. I was on track with the second paragraph but needed to make a better connection between the first and second paragraph. I could do that.

L. Dee Fink, a noted teaching and faculty development consultant, uses the acronym FIDeLity to describe characteristics of useful feedback.

F=Frequent

I=Immediate

D=Discriminating

L=Loving

The last two are not self-evident, so let me explain what he means by “discriminating” and “loving.” “Discriminating” feedback targets the most important areas for improvement. By “loving,” Fink refers to the spirit in which the feedback is expressed, i.e., wanting your students to improve rather than wanting to demoralize or discourage (see again the two pieces of feedback above). While the feedback I gave in my early teaching career was meant to be loving, the sea of red ink calling out every little thing I found positive and negative was hardly that–nor was it discriminating. Lesson learned.

At this point you may be thinking, “Yeah, yeah, this all sounds good, but in the real world, I teach large lectures and don’t have endless hours to give FIDeLity feedback. Besides, students don’t seem to use my feedback anyway, so nice try.” Both are legitimate concerns, so let’s look at strategies for giving FIDeLity feedback in the real world.

Rubrics to the rescue! Providing students with a grading rubric along with the assignment gives them a roadmap for success. It also saves you time on the back end by streamlining the grading process and minimizing the crafting of individualized feedback. There are different types of rubrics you can create, and you can even create and use them directly in Canvas. And to kick it up a notch, consider providing examples of a strong and weak assignment along with the grading rubric, and let THEM practice grading!
Peer review! You don’t always have to be the one giving feedback. Incorporate opportunities in class and outside of class for peer review. By assessing others’ work, students improve their ability to diagnose issues in their own work. However, keep in mind that you will need to provide guidance for students on how to understand your rubric’s criteria and give constructive and “loving” feedback.
Tech can help! Providing audio or audiovisual feedback in Canvas Speedgrader is quick, more personalized, and helps students zero in on the two or three most important areas to work on. Creating polling questions in Canvas and Poll Everywhere can help students self-assess, and enable you to identify gaps in knowledge and give global feedback to the whole class.
Look for patterns! There are usually themes among the errors students make and in what they did well on a particular assessment. Share those with the class rather than commenting on them to each student individually.
Ask for accountability! Have students demonstrate they’ve used the feedback you do provide. Have them include in a brief statement how they incorporated your feedback the next time a draft or similar assignment is due.

Okay, your turn! Let’s keep this conversation going by posting to our new Faculty Teaching Commons. Go to TUportal, click on your Faculty Tools tab, then click on the Faculty Teaching Commons link. How “FIDeLity” is your feedback? What are the feedback techniques you’ve found effective? Inquiring minds want to know!

As always, if you’d like assistance with giving feedback to your students, one of our faculty developers or educational technology specialists are ready to help. Make an appointment here or email a CAT staff member directly.