Using Online Student Learning Evaluations to Improve Instruction

Steven Bell, Associate University Librarian, Temple University

My first experience teaching an online course occurred in the late 1990s when Drexel University’s College of Information Studies asked me to convert my face-to-face digital research course to an online format. My  primary focus  was just getting things to work. Quality learning experiences were an afterthought.

Fast forward to 2017 and after a ten-year gap I agree to teach online again, this time for San Jose State University’s (SJSU) iSchool program for aspiring librarians. Despite my prior experience teaching online, I took a mandatory 25-hour mini-course in online instruction. It reinforced much of what I learned in other professional development experiences, but also introduced me to Canvas.

Less anticipated was the robust expectation to achieve continuous learning. SJSU takes online education seriously and the institution requires that all online courses adhere to the Quality Matters ™ framework. Their high-quality standards require instructors to complete continuous learning modules in any semester they teach. One of these sessions got me thinking differently about student evaluations. Instead of perceiving them as a necessary evil, I discovered they can serve as both motivator and shaper of quality instruction.

In the online session “SOTES Strategies and Lessons” I learned that student evaluations, when analyzed in aggregate, help instructors to modify their educational methods to align with student learning needs. SOTES, (Student Opinion of Teaching Effectiveness) is SJSU’s student evaluation system. No doubt every instructor reviews course evaluations seeking clues that lead to substantive learning improvement. Instead of tweaks to assignments or syllabi, SOTES can lead to fundamental pedagogical change.

The SOTES session gave twelve tips – focusing on ways to anticipate what most helps students learn and succeed – to influence how educators can design a better learning experience.

Be Practical to Demonstrate Relevance

Theory is important but these mostly adult learners, often working in the field, want concrete examples and anecdotes. Use practical assignments and explain how students will benefit later.

Create Assignments to Enhance Learning
Students want assignments directly relevant to what’s in the lecture. Their top two requests: avoid giving busy work; structure assignments into smaller blocks that build on each other.

Emphasize What’s Important
Provide a verbal or written summarization of the top takeaways from a learning module’s content. Keep reiterating key concepts. Share your observations on the best ideas from course discussions. At the start of each week, I summarize what students should know from the prior week, where they demonstrated competency, where improvement is needed, and how it will apply to new content.

Respond, Respond, Respond

We are told to quickly and comprehensively respond to students’ questions and comments, in discussions or emails. The SOTES clearly confirm that. Respond in a variety of communications formats (audio, video, email, etc.) and keep it cordial and respectful. Avoid putting students on the spot or making them feel regret over their question or comment.

Set the Atmosphere
Online learning is about more than content and assignments. Students expect a patient, positive and encouraging instructor who is present, well organized and gives students opportunities to interact and learn from each other. The right classroom culture facilitates learning.

Easy to Approach
If the atmosphere is right, students will feel they can easily contact and communicate with their instructor. My course was asynchronous, but I offered a weekly synchronous office hour (varying days/times) in order to create connection and elevate students’ comfort level in reaching out to me.

Appreciate Student Diversity
Students respond poorly to an instructor who plays favorites with respect to differences in age, gender, race or work experience. Pay attention to differing backgrounds in student’s self-introductions and commit to treating students equally.

Passion Speaks Loudly
Set the tone by being the champion for the course topic. Instructor enthusiasm is contagious. If the instructor lacks excitement for the topic, students will too.

Challenge Them Intellectually
Students want “make us think” activities that require more than answers to rote questions. Encourage creative thinking by allowing students to develop multimedia projects in which they apply course concepts to their own experience.

Fair Grading Matters
Creating clear assignment instructions and rubrics to guide students is no easy task, but taking time in advance to think through the details of assignment grading will minimize grading disputes or claims of unfairness.

Make the Complex Understandable
Students expect their instructor to make the abstract concrete through the use of realistic examples. Communicate your personal experience in lectures, assignment instructions and discussions.

Provide Meaningful Feedback
Give praise for work well done as well as constructive criticism. Be specific in pointing out what students get right and where they need to improve. Add comments to graded assignments and return them to students while they can benefit from feedback.

Whatever personal opinions online instructors hold about student evaluations, it feels much better when they reflect a uniformly successful learning experience. Learning those factors that lead students to judge a course as successful and an instructor as competent, organized and responsive, greatly shift the odds of a positive outcome in the instructor’s favor. The SOTES session changed how I think about student evaluations. I now see them as a valuable resource for course design, not simply an after-the-fact measure of “How’d I do?”. Though mostly applicable to online learning, these dozen tips could work equally well for face-to-face courses. I encourage you to put them into your practice.

Creating a Student-Centered User Experience in Your LMS Course

Ariel Siegelman, Sr. Instructional Technology Specialist, CAT

New websites or applications often become successful in part because their user interfaces need no explanation: A user can visit the website and use it for its intended purpose without needing much assistance. However, if that experience is difficult, it can lead to a lot of frustration, and the user might give up on what they were trying to do in the first place.
 

It is easy to forget that setting up a course on a learning management system (LMS) is essentially creating a website for students, and that is an important component of creating a student-centered learning experience. Research shows that student-centered teaching, in which an instructor seeks to understand the students and shape the learning experience based on their needs, leads to higher student performance. To that end, a well-structured LMS course designed with students in mind can help students stay on track of their tasks, engage with the course material, communicate with other students and the instructor, and receive timely, helpful feedback. However, an LMS course has just as much potential to hurt student performance as it does to help it. For example, if students need to spend more time trying to figure out where materials are located in the course or what their required tasks are for the week, it wastes precious time that they could otherwise be dedicating to learning. Additionally, instructors may need to devote a significant amount of time responding to frantic emails from students who do not know what to do.
 

Below are three strategies for creating a more student-centered LMS course:   
 

1. Make all of your content accessible in one place and organize it sequentially.

One of the easiest ways for students to become frustrated while navigating within a course is if the location of materials is unclear. A way to make this straightforward and also mobile-friendly is to make all course content–including readings, PowerPoints, assignments, and quizzes–easily accessible from one place. A best-practice to further organize this content is to present it in a sequential order, usually by week or unit.
 

A first instinct might be to organize it by type, such as placing all of the readings or quizzes together, but chronological order makes it simple for students to know which materials they need and when. The Modules feature inside of Canvas, for example, makes it very easy to accomplish this. This organization method also allows students to see the progression of the course content and how each piece of material fits into the overarching themes of the course. Additionally, here at Temple University, professors who present content in this way often tell us that they receive less emails from students asking what they have to do for the week and where it’s located in the course. Ultimately, it’s a time-saver for both the instructor and the students.
 

2. Make all important dates, including due dates, easy to find and accurate.

Inevitably, a few due dates may change over the course of the semester because of circumstances such as inclement weather or shifting lesson plans. If due dates are listed in multiple places across a course, it can be difficult to ensure that they are all up to date, especially if they change. To prevent potentially contradictory dates in a course, it’s a good idea to limit due dates to being listed in just one or two easy-to-find places, or to use features built into the LMS that will update the due dates in multiple places automatically. For example, in Canvas there are a few features that help students stay on top of important dates: The Course Summary on the Syllabus page, which lists all due dates and events created within the course, and the Calendar feature, which can sync all events and due dates seamlessly with a student’s external calendar, such as Google Calendar, iCal, or Outlook. Additionally, after the due date of an assignment or quiz is changed within its respective settings, the due date will automatically update in both the Course Summary and the course’s calendar. This can help ensure that all listed due dates are accurate, and thus students will be able to easily stay on top of their schedule.
 

3. Update the gradebook regularly to provide quick and helpful feedback.

Research shows that one of the best ways instructors can help students be successful in a course is to give them helpful feedback on their performance so they can keep track of their progress in the class and work to maintain or improve their grades accordingly. Many learning management systems, including Canvas, have features that allow instructors to provide frequent, immediate, and instructive feedback, as supported by the research-based “FIDeLity” feedback model. For example, posting grades in the LMS gradebook as soon as possible allows students to confirm their performance so far and note where they need to make improvements. An LMS often has additional quick feedback features, including automatic grading for online quizzes and exams, as well as tools such as Canvas’s SpeedGrader that allow you to make comments and suggestions within students’ submitted essays and projects for students to review as soon as they’re graded. Canvas also allows you to sort graded assignments into groups, and the gradebook will show each student how they are doing in each assignment group. This allows the students to monitor their overall progress in assignment categories such as exams, homeworks, and projects. Rubric features are also useful to communicate assignment criteria to students, to quickly and effectively grade assignments, and they can assist in ensuring that the grading criteria is consistent if there are multiple graders.
 

Ultimately, building an LMS course that is well-organized, straightforward, and provides quality feedback is a fantastic way of creating a positive and supportive learning environment for students. It is one vital step that instructors can take in order to increase the likelihood of their students’ success.
 

Let’s Exchange EDVice!

What strategies do you use to create a student-centered LMS website in your course?

Make Your LMS Your BFF

Stephanie Fiore, Senior Director, CAT

If you teach at one of the many universities that are switching from one Learning Management System (LMS) to another or are simply pushing faculty to explore more fully what they can do with their LMS, you may be saying, “Why all the fuss?” Instructors do not need an LMS to be effective, so why bother with all of the work needed to get a good course site on the LMS up and running? Perhaps you see its usefulness as a repository of documents, but all of those other LMS features seem to be more trouble than they’re worth. Or, you see the necessity of an LMS for online courses, but you teach in a bricks-and-mortar environment, so the LMS is not crucial to the work you do with students. I get it. I’ve been teaching since the dawn of time (well, not exactly, but close) and I think I was a pretty great teacher without an LMS at my disposal. I led class discussions without a discussion board, assigned papers without a plagiarism detection tool, gave paper and pencil quizzes, and assigned group projects to students, fully expecting them to figure out how to collaborate outside of class on the project’s completion.
 

But here’s the thing. If you use it to its full advantage, an LMS can serve an integral role in student-centered teaching and learning practices and support robust learning outcomes for all students. How so, you say? Your LMS can “afford a plethora of teaching and learning possibilities around communication, interaction, collaboration, ‘real-world’ or authentic learning, independent learning, feedback and flexibility.” In our new LMS, for instance, students working in groups have a whole suite of collaboration tools available to them so that they can work as a group more effectively and efficiently. They can collaborate on a document together and edit each other’s work, talk to each other to plan the project, and invite the professor to eavesdrop on these interactions, all without having to arrange meetings in person. There simply are no more excuses for why group work can’t get done as the flexibility these kinds of remote collaborations afford is unparalleled.

Other powerful tools in the LMS are the calendar and syllabus features that keep students abreast of changes in due dates or assignments and so keep students on track. One student in a focus group at our university claimed that he did more homework because of the calendar notifications feature. That made sense to me. I always say that if it’s not on my Google calendar, it doesn’t exist. Without my alerts telling me what’s coming up, I would probably miss half of my meetings! Students are no different. And, of course, there is the convenience and flexibility factor. Students can access course materials from all devices, including their mobile devices, whether they are on or off campus. Winter storm interfering with your ability to hold class? Communicate with your students through the LMS, upload work for them to do, and keep your semester on track.

In both bricks-and-mortar and online classes, the LMS allows for enhanced student engagement with the instructor and with peers, and customized student pathways through the curriculum. A well-organized LMS site leads students through the course in a logical way, framing the work that needs to be done in units that show the overall structure for the course and the scaffolding of assignments and assessments for each unit. It also allows for the delivery of supplemental instructional opportunities, such as video lectures, virtual discussions, and automatically-graded quizzes that can help prepare students for team-based and collaborative activities that deepen learning and clarify complex concepts. A recent study of LMS users found that “the use of the LMS, and more importantly specific tools within the LMS, are significantly related to student achievement.”

I invite you to explore your LMS with an instructional technology specialist on your campus. If you’re new to using an LMS, I think you’ll find it more intuitive and easy-to-use than you think. If you’re already familiar with the LMS basics, think about how you can use the powerful tools available to you to supplement or to directly deliver your already great teaching. You have the tool; just use it!

N.B. Parts of this blog post are taken from work done by the author and Nicole Westrick, Associate Vice Provost, on the LMS Evaluation Report written as part of the evaluation of the Canvas LMS at Temple University.

Pick Me! Choosing the Right Technology Tools for Your Courses

Ariel Siegelman, Sr. Instructional Technology Specialist, CAT

All of us at one point or another have fallen victim to “Shiny New Toy Syndrome.” We hear about a really cool new tech toy, gadget, or app and we have to have it because it looks flashy, or is the latest and greatest fad that makes us look “hip.” But soon after we buy it, we realize that it’s not exactly what we were expecting—maybe it doesn’t work as well as promised, or maybe it’s not really so useful after all. And so, after the initial excitement wears off, it sits forgotten and unused.

Having thousands of new education technology tools and apps available, it can be easy for “Shiny New Toy Syndrome” to strike when searching for resources to use in your courses. You might think, “My students will think my class is so cool if I use this!” or “Everyone is using this tool right now, I should too!” But before grabbing the first trendy-looking tool that comes up on Google or Twitter, slow down for a moment and consider: how is this tool really going to help my students learn? Even though a fancy new tool may seem like a sure-fire way to increase student engagement, if students think that the tool is ineffective or a waste of time, it could actually lead them to be less invested in your course.


With so many new products and applications, how it is possible to figure out which tools to use? After conducting a series of several studies that examined a myriad of digital and mobile applications and how efficient they were at enhancing student learning, Kearney, et al., a group of researchers based out of the Centre for Research in Learning and Change at the University of Technology in Sydney, Australia, published a framework describing three essential features for a digital tool to provide a quality learning experience. While this framework was developed to evaluate mobile tools, it can absolutely be applied to any digital tool out there.

Personalization

First, students must be able to personalize the tool to fit their own needs and interests. This can range from allowing students to customize the way that they study the information, such as choosing whether to read an article or watch a video to learn about a topic, to having students create a personal profile or avatar of themselves that collects awards or achievements as they progress through the content. This aligns with research that shows students learn more effectively if they have more control over their learning experience. Additionally, adding a personal touch to an assignment or educational tool can help students feel more connected to and invested in their work. In other words, each student should not have the exact same experience when using a digital tool–they should be able to curate the tool so the experience is as enjoyable and as effective as possible for them.

Authenticity

Another important aspect to consider is if the tool can assist in providing students the opportunity to connect the experience and content to real-life situations. Research shows that students are more engaged with content when they can identify how it will assist them in their future, thus improving their chances of valuing and understanding information and concepts. If integrated with an assignment or lesson plan that simulates real-world scenarios, a digital tool is worth using if it can help students make meaningful connections.
 

Collaboration

A final feature to consider is a digital tool that allows students to work together to create and discuss content. This provides the opportunity for students to receive and give feedback about their understanding of the information, which is important to ensure that students don’t walk away with any misconceptions or gaps in knowledge. Additionally, in a fully-online class it allows students to connect with one another and establish a greater sense of community. Research supports the idea that when students build social connections while learning new information, they are more likely to become invested in that information.

It can be tempting to use technology just for the sake of it, but by establishing that the tools we use in our classes are well-rounded and effective, we can ensure that the activities our students participate in are worthwhile and enhance their overall learning experience.

Let’s Exchange EDvice!

What kinds of tools do you use, or have you heard about that meet all of these criteria? How have you integrated them into your courses? Or, after learning about this framework, are there any technology tools that you will no longer use in your courses?

Creating presence in an online course… without working around the clock

Laurie Friedman, Instructor and Online MSW Coordinator, School of Social Work

In transitioning to teaching an online course for my first time four years ago, I was initially drawn by the promise of flexibility. Yes! I could work in my pajamas, from anywhere with internet access. I could work around my toddler’s schedule; teaching online afforded me the opportunity to balance work with parenting. While I found that these benefits did exist, what I didn’t expect was feeling burnt out at the end of the semester. I was working around the clock, dutifully responding to students’ questions, actively participating in discussion boards and providing detailed and timely feedback—all the important elements of online teaching I had agreed to when assigned the class. I have since spoken with colleagues who share similar experiences, namely that they are “always working” and that teaching online is “more work” than teaching in the traditional brick and mortar classroom.

Since 2012, our knowledge of best practices in online teaching has expanded, as has my experience. Creating a presence is noted as one of the most important aspects of course delivery; there are elements of course design that contribute directly to course delivery, making planning vitally important. An online presence consists of the relationships we build with and among students to create the learning environment, the role we play as instructional guides, and the personality traits and interests we bring to the course. Research on online classes shows that “students rate contact with faculty as more important than contact with other students.”  Sheridan and Kelly also found that among factors related to course presence, students were most interested in how clearly the instructor conveyed the course requirements and information, as well as the timeliness and quality of their feedback. Creating a presence in an online environment is an intentional process, below are some tips on how I have maintained an online presence while creating boundaries so that I don’t burn myself out.

  1. Start with a welcome video. First impressions matter, and help alleviate students’ anxiety about a new course. Introductory videos should be 3-5 minutes long, include our personal and professional interests and give an overview of the course with tips for success. I’ve received feedback from students that my enthusiasm for the course’s content positively impacted their experience in the course. I also know instructors who choose to include a picture of their pet, favorite vacation spot or family member to begin to create a relationship, which helps to personalize the experience (discussed more below).
  2. Create a course tour. A brief 2 or 3 minute video tour of our online classroom space shows students where to find key information (i.e. assignment information, policies, syllabus, course materials and grades) and lets them know we care, are present and available.
  3. Personalize the experience. Each week, I record a 10-15 minute “lecturette” where I review the major points and questions from the previous week and orient students to the goals and activities of the current week. In the video, I may utilize specific comments from students and discuss how current events (including weather!) relate to course materials. Another way to personalize the experience is to use individual e-mails intentionally. For example, if a student mentions she is particularly interested in a topic that is not the focus of our course, I will share news articles and resources with her if I find them. This lets students know I am paying attention to who they are as individuals.
  4. Post regular announcements. Announcements (using text, video and/or pictures) are key to saving time and replacing individual e-mails. One of the benefits of announcements is that they remain on the course site unless you delete them, with the most recent post on top. Send the announcement via e-mail if it’s a priority message and schedule times when other announcements are posted. This leaves us in control of when we are working and allows us to stagger messages without having to log in repeatedly.  
  5. Set up a “Water Cooler” discussion board. Creating a discussion board, sometimes dubbed the “Water Cooler,” for students to ask general questions pertaining to the course helps to create a sense of community and streamline communication. It is important that from the beginning we socialize students to use this forum, and post responses to individual e-mails in this forum.
  6. Use rubrics. The rubric tool embedded within the learning management system can be used for grading course discussions and other assignments. I absolutely love this feature! I used to send each student an individualized e-mail with feedback on their discussion board participation. To save time and maintain presence, I now embed a discussion rubric within the learning management system (we use Blackboard) and simply check off boxes to assign points. I aim to give each student at least one qualitative comment each week, specifically mentioning a post they had and why it stood out.
  7. Establish online office hours. Online office hours are imperative to creating presence. Even if no students show up, the perception and knowledge that faculty are available has a positive impact on students. It also alleviates my need to instantaneously respond to questions if I have let students know ahead of time when office hours are and my response time for emails.
     

So to return to my original question, how do we do all of this without burning ourselves out? For me the key is transparency, communication and boundaries. We need to follow the same advice we give our students taking an online course: check the course site daily at a time that works for them and carve out time for this class as we would any other class, adding specific times to our calendar when we plan to complete activities (for us it’s grade, record videos, check discussion boards and respond to messages). This helps us get into a rhythm, similar to one that develops when a class meets weekly in a brick and mortar classroom. Creating a presence does not mean we are available 24-7 or that students need immediate responses. In fact, this can inhibit the learning process as they become dependent on us for answers. In case you were wondering, yes, I am still tired at the end of my courses, but I no longer feel burnt out as I did four years ago. My online courses, synchronous and asynchronous, “meet” at predetermined times that fit my needs and those of my students.

Let’s Exchange EDvice!

How else have you set up boundaries to define the classroom space in an online environment? What other strategies do you use to create presence in an online class?

6 Tips for Creating Engaging Video Lectures That Students Will Actually Watch

Simuelle Myers, Instructional Designer, TLC

Image for Creating Engaging Video Lectures That Students Will Actually Watch

How do I get started creating video lectures? How do I engage students from a distance? How do I know if they understand the concepts in my lecture if I can’t see their faces? How do I know students are even paying attention? These are common questions asked by faculty when preparing to create a video lecture. As both online and flipped classroom formats grow in popularity, the number of faculty creating video lectures is increasing. However, many of these videos are recorded as straight lecture, limiting students only to the role of observer.  The six tips below will help you create videos that also engage students in active learning, while giving you information to assess their understanding of course concepts.  

1. Keep it Short!

Break lessons into segments of about 7-10 minutes. This allows students to digest every part of the lesson,  quickly revisit what they may not have understood and provides a meaningful place to pause if they need to return to the lesson later. This can also become invaluable if you need to update a video later. It is much easier to re-record 10 minutes than 50 minutes!

Tip for implementation: If you are used to longer lectures, review your lessons and identify where the natural breaks in the material might be.  Use these as a guides to decide where every recording should begin and end.

2. Use visuals, images and animations

While students greatly value being able to see and connect with their instructor, a lecture consisting only of a “talking head” can be hard to follow.  Visuals can enhance your presentation and make material more accessible. Screencasting software (e.g. Camtasia Relay) allows you to share your screen with students so they can see your presentation, graphs, figures, drawings  and your face all at the same time. These tools can also be used to create video demonstrations for students in your brick and mortar classes.

Tip for implementation: Text-heavy slides can make it difficult to pay attention to what a speaker is saying.  Mix it up and try slides using a single large image. This creates a need for students to listen more to what you are saying instead of just reading the words behind you. This also gives students a reason to take more detailed notes.

3. Create guided or embedded questions

Pause to ask students a question, provide a worksheet that they need to complete as they watch the lecture, or create a task for them to do in between videos. There are also several programs you can use to create questions that are embedded directly into a video that students must answer before they continue watching. Similar to an in-class activity, these allow students to work with the material in the midst of the lesson and add variety to help keep them engaged.

Tip for implementation: How do you make sure that students complete guided questions or worksheets? Have them submit their answers as an assignment. This can also help you assess your students’ understanding of the material.   

4. Test knowledge with quizzes and self-assessment

Frequent, low stakes quizzes encourage students to pay closer attention to video lectures and allow you to assess their knowledge. Self-assessments are typically ungraded, but provide students with diagnostic feedback that  encourages them to re-visit areas of the lesson based on  questions they may have missed. Both methods give students quick feedback so they can gauge early on which concepts or problems they may need help with.

Tip for implementation: Ask students what they are having trouble with. At the end of a unit, have students assess themselves by asking what their “muddiest point” is or what they would like to learn more about and have them submit their responses via the Learning Management System (LMS) as a private journal entry.

5. Use pre-existing videos

You do not always need to create original videos.  Many great videos exist that already do a good job of explaining specific topics. This also creates more variety in students’ learning experience and can be less time intensive for the instructor.

Tip for implementation: Explore what resources are available before you begin recording your own videos to gain an understanding of what currently exists and what you need to do yourself.

6. Be Yourself!

Lastly, remember that this is not a Hollywood production! One of the most important things to do in a video is to be yourself and act natural. It is okay to stumble over a word or quickly correct yourself when you make a mistake. This allows students to truly see your personality and connect with what makes you unique as an instructor.

Tips for implementation: Record a test video, then go back and watch it (bonus points if you have someone else watch it too!). Evaluate what you do well and what needs improvement. Feel free to experiment with environment and style until you feel that you are able to convey yourself in a way that is comfortable and genuine.

Let’s Exchange EDvice!

Are you already creating video lectures? What strategies do you use to keep students engaged?

_____________________________________________________________________________________________________

Additional Resources:

Ho, Yvonne. “Seven Steps to Creating Screencast Videos for Online Learning.” Faculty Focus. Magna, 15 Mar. 2013. Web.

Mayer, Richard E. Multi-Media Learning. Cambridge: Cambridge UP, 2001. Print.