Monitoring Progress and Measuring Outcomes
Systematically monitoring student progress and assessing implementation of SEL strategies during implementation allows for course adaptations and student supports to happen in real time. This is particularly critical to ensure that course rigor and student engagement are comparable to in-person courses.
Measuring the outcomes of this course model ensures the courses have the intended influence on student outcomes and generates ongoing support from administrators, educators, and partners for implementation and growth.
Monitor implementation for continuous improvement
Monitoring the implementation of SEL strategies requires instructors to engage in a regular, reflective practice on at least a biweekly basis to reflect on the SEL strategies they implemented that week, how it went (e.g., student engagement with activities, facilitation, or technical challenges with an activity), and plans for adaptation.
College leaders, or the identified champion, should check in with instructors at least two times per semester to discuss progress, challenges, and lessons with implementation. This includes checking in on learning across instructors to understand how they collaborate to share real-time adaptations and lessons.
Together, these strategies promote ongoing learning and documentation to inform adaptations for future semesters and expansion into other courses.
Measure outcomes
If colleges want to research how SEL strategies are influencing student outcomes, they can build on the approach established by the ICAN project. There are four key areas that colleges might focus on to measure the impact of the courses:
- Students’ skill development and course experience: The college should distribute surveys at the start and end of each course to monitor students’ skill development, shifts in academic identity, and perceptions of instructor support. Colleges can leverage the sample student baseline and follow-up surveys (PDF) developed as part of the ICAN project.
- Course engagement and success: Student engagement in online courses can be measured in several ways, depending on the delivery mode (synchronous, asynchronous, or hybrid). The college should identify at least one engagement measure to collect based on the delivery approach, such as student attendance at live lectures or the time spent in the course’s learning management system.
- Academic achievement: Tracking course pass rates and grade point averages for students in courses that integrate SEL can help the college understand the outcomes of these courses compared with those of students in similar dual enrollment courses.
- Future course-taking and matriculation: To understand the potential longer-term benefits of offering this course model, colleges should track participating students’ future enrollment in dual enrollment courses and compare them with a comparison group of students. Additionally, tracking students’ matriculation to the college after high school graduation can serve as an indicator of their positive experience with the college.
Selecting a comparison group
To measure the impact of SEL strategies on student outcomes, it is necessary to compare the outcomes of students who receive the intervention with those of students taking similar dual enrollment courses but not receiving the intervention. There are a few things to consider when selecting a comparison group:
- Instructor quality should be similar: It is likely that instructors who choose to implement SEL strategies are already effective instructors who strive to personalize learning for students. To make fair comparisons of student outcomes, it will be important to choose comparison instructors with similar practices, instructional approaches, and teaching tenure.
- Course content should be similar: Because course grades and longer-term student outcomes will likely differ across subject areas (for example, sociology versus college algebra), it will be important to select comparison courses in the same (or a similar) subject area.
- Consider differences before and after implementing the course: It may be difficult to be confident in measuring the impact of the course if there are other potential differences across a course with SEL and comparison courses. In addition to instructor quality and subject area, other examples include the characteristics of students enrolling in the courses or the “level” of the course (e.g., whether there are prerequisites or placement exams). To account for these differences, it would be helpful to measure student outcomes in selected courses before and after implementing SEL strategies and observe whether student outcomes improve more in ICAN courses than in comparison courses.
To learn more about the formal evaluation of the ICAN project, visit the American Institutes for Research’s project website.