EXECUTIVE SUMMARY: Evaluation Guide for Effective SEL implementation

Overview

Palo aims to spark every child’s zest for life by providing a bite-size and hyper-relevant social-emotional learning (SEL) to middle schools. Covid-19 separated students from their social lives in school for almost two years. With the internet, students could learn remotely, but meeting and hanging out with friends was not the same. In 2020, many people thought that going back to school and returning to “normal” would be seamless and quick. However, although school learning continued, the isolation caused by Covid-19 held back social-emotional learning for two years. After extensive research and discussions with more than 30 schools and more than 50 counselors, we learned the critical role of SEL at this desperate time, and more importantly, that the current SEL needs a total makeover.

We conclude with six challenges that schools face in their SEL implementation, and provide an evaluation guide for identifying the right SEL for each school.

Problems

After the Covid-19 pandemic, students were more vulnerable to adverse emotional and mental health issues, and the interest in Social Emotional Learning (SEL) for

The SEL evaluation guide by our research considers four attributes that promise a successful implementation:

1. Stakeholder Buy-In and Competencies: This refers to the level of commitment and involvement of all stakeholders affected by the implementation of SEL. Ensuring that all stakeholders are on board and have the necessary skills and competencies is crucial for the success of the program.
2. Framework Integration: An effective SEL program not only teaches social-emotional skills, but also equips students with life skills that can lead to more positive outcomes and an increase in overall well-being.
3. Learning Design with an Approach to Skill Development: The learning design of an SEL program should be tailored to the developmental stages and interests of the students, taking into account the specific age group.
4. Effective Use of Media: In the modern age, the use of media can be a powerful tool for learning. An SEL program should make effective use of media to enhance the learning experience.

Key to Success Implementation

As more and more schools are seeing the positive effects of successful SEL implementation, school leaders are increasingly looking for ways to make SEL work in their schools. However, poorly implemented SEL can burden teachers with additional responsibilities and students with boring and irrelevant learning materials, leading to decreased teacher morale and student interest.

young students increased. Research shows that the benefits of SEL for youth include better academic performance, improved classroom behaviors, stronger resilience to

difficult experiences, and better teacher- student relationships. Long-term benefits

of SEL include stronger social-emotional assets and higher levels of well-being in adulthood.

Despite the promising benefits of SEL, schools face six challenges in implementing it:

1. Lack of resources for teacher-facilitated SEL
2. Lack of measures for Successful SEL
3. School’s Need for SEL partners, not vendors
4. Inadequate Budget
5. Administration turmoil after Covid-19
6. Little consideration for stakeholders

Implementation Consideration

The implementation of an SEL program can be challenging, but addressing the six challenges mentioned earlier can lead to successful outcomes. Our research presents an SEL evaluation guide to help leaders evaluate different SEL programs based on four key attributes:

An effective SEL implementation must consider the needs and perspectives of all stakeholders and address the six challenges mentioned earlier. Our evaluation guide can help schools assess different SEL programs based on their key attributes and elements you may see in a DEMO. However, some schools and stakeholders may have specific considerations that are particularly important to them. In that case, they should focus more on their specific needs when evaluating an SEL program.

To start looking for an SEL program that will work for your school, download the instruction guide and evaluation guide.

Case Study: Character Strong vs Second Step vs Palo

In our whitepaper, we evaluate three SEL programs: Character Strong, Second Step, and Palo. Character Strong does not include an Equity, Diversity, and Inclusion framework and has limited incorporation of students’ developmental stages and interests in the curriculum. Second Step does not provide a post-assessment to measure the outcomes of SEL implementation, and offers little support for implementation beyond an eLearning platform.

Palo meets most of the evaluation attributes, but does not provide SEL for adult stakeholders. However, it does offer support materials such as newsletters and toolkits for teachers to infuse SEL into their classrooms, while Palo does most of the teaching. According to Palo’s website, Palo emphasizes SEL integration and creates minimal disruption to the bell schedule and classroom time.

PROBLEM STATEMENT

With the surge of classroom disruption and student dysregulation after covid, schools couldn’t continue with the outdated and ineffective SEL solutions and let SEL be just “OK”. Instead, schools need to adopt a school-wide implementation that prioritizes teacher-student relationships, helps alleviates the back-to-school process, and promotes student mental well-being.

According to our conversation with educators from close to 50 middle schools nationwide, they struggled to find an SEL implementation that infuses SEL into students’ daily life without burdening their teachers.

SEL is difficult to teach because it is not as simple as passing on knowledge; it requires an encouraging environment and continuous practice. It requires students to be a scientist of themselves and master SEL skills in their personal contexts. Students then need to reflect on the

results of practicing these skills. Thus, it’s important to have teachers create the environment by infusing SEL into their teaching.

These schools have used outdated and ineffective SEL. They are now looking for a working solution and implementation because SEL is significant to students’ well-being, and communities demand it. It is no longer trivial.

SEL IMPLEMENTATION CHALLENGES

According to the Collaborative for Academic, Social, and Emotional Learning (CASEL), there are currently 86 approved Social-Emotional Learning (SEL) programs for various grade levels (“CASEL,” 2022). Besides CASEL-aligned SEL, counselors can easily find hundreds of SEL solutions and curricula by simply searching for SEL on the Internet.

The benefits of evidence-based SEL programs for students are vast and varied. Students who participate in these programs have reported improved attitudes, decreased behavioral issues, and increased academic performance (CASEL, 2022). Furthermore, the positive impact of SEL extends beyond the school setting, with effects on risky behaviors and future employment opportunities (Yang Bear, 2018; CASEL, 2020). Studies have shown

that students who participate in SEL-based programs have demonstrated an average 11% increase in academic achievements. By the age of 25, there has been a recorded 10% reduction in behavioral, psychological, and substance abuse problems among these students. Additionally, according to the US Teacher Survey of 2021, 42% of respondents reported a reduction in the likelihood of physical aggression in schools as a result of the implementation of SEL programs in their schools.

With the growing interest in SEL programs, it is important for schools to choose evidence-based programs that align with frameworks such as CASEL and ASCA. By investing in SEL programs, schools can make a significant and positive impact on the lives of their students, both in and out of the classroom.

SCHOOLS LOOKING FOR ALTERNATIVES

We wanted to understand the challenges that schools are facing with their current SEL solutions and what they think it would be a successful SEL program so we can design an effective social and emotional learning (SEL) program for the post-COVID era.

To gather this information, we conducted research by interviewing over 50 middle schools across the country. We asked three questions:

• What are the SEL challenges or needs at your school?
• What are you currently using to address these?
• How do you measure the success of your SEL program?

Most of the schools we interviewed are Title 1 schools serving more than 40% disadvantaged students or having a diverse student population (e.g., one school reported that 22 languages are spoken in their middle school). Most of the interviews were attended by counselors

The most common challenge is that SEL is not always integrated into daily teaching, and teacher buy-in may be mixed. Without sustainable effort, they find it hard to realize SEL’s promises. For example, Hampton City School has received praise for SEL from school leadership, but implementation has been difficult, with teachers struggling to develop a daily habit of incorporating SEL into their teaching. Additionally, the

Palo SEL, stated that their previous SEL solution provided no feedback to teachers who were eager to know more about their students and get involved in conversations about mental well-being. Second Step and Character Strong are some of the legacy SEL vendors, but they do not provide such data, and educators have to rely on their insights from observation. Even schools

that use PanoramaEd to assess SEL progress stated that they had very little idea of what to do with the assessment results. “It’s cool to look at, but what do we do with it?” Many of them shared when we brought up PanoramaEd, the most prominent SEL assessment provider in the market. Despite previous experiences of mixed results in teacher buy-in and lack of teacher support for SEL in the classroom, most schools are still actively seeking an SEL solution that works for them. However, many of the SEL products currently available in the market are one-way streets, with little to no visibility into student insights for teachers. This lack of teacher visibility can make it difficult for them to identify which students are struggling and in need of additional support, as well as to monitor the progress of all students in the program. Therefore it is crucial for schools to choose an SEL program that provides teachers with visibility and insights into their students’ progress. By having access to this information, teachers can make informed decisions about the type and amount of support their students need and adjust their teaching strategies accordingly.

For students, it is crucial that the SEL content is relevant and up-to-date for the student learning experience. Unfortunately, many of the current SEL programs have outdated and irrelevant

content that fails to address post-COVID student challenges. Middle school counselor Wanda Beniquez-Moore shared her experience of struggling to teach irrelevant content to students, resulting in no students showing up for her SEL workshops. This highlights the urgent

latency of SEL benefits on academics may cause teachers to be hesitant to reduce their core subject teaching time in order to implement SEL. At the same time, teachers may not feel prepared or equipped to teach SEL, as it is typically the responsibility of trained counselors or licensed social workers. That is due to a shortage of counselors in schools, with a recommended ratio of 250-1 according to ACAS, compared to a national average of 400-1 compared to a national average of 400-1 according to State Nonfiscal Public Elementary/Secondary Education

Survey 2019-20. The COVID-19 pandemic has also disrupted student life and led to an increase in behavior problems in schools, further increasing the demand for mental health support and contributing to teacher burnout and quitting. In light of these challenges, schools are seeking SEL programs that can be integrated into their existing curricula so that SEL can be infused into daily teacher teaching and student learning experiences rather than adding a new teaching agenda to all teachers. Roosevelt Union Free School, for example, wants SEL to focus on skill training such as self-advocacy and character strengths, like grit, and to encourage students to take responsibility for their own well-being and develop strong character instead of content-feeding.

Schools also want visibility into students’ social-emotional growth or, at a minimum, engagement with SEL. They understand that SEL is a process and it is important to monitor students’ progress so the effort is sustainable. Uncharted Shores Academy, who started their pilot program with

need for SEL products with more relevant curricula that can effectively address the current challenges students are facing. It is important to remember that SEL is a practice, and for students to practice it, it must be relevant and engaging to them. If the content is not relevant, students are unlikely to engage with it, and the program will be ineffective. For example, although Second Step is a great reference point for SEL teaching, many students find its videos and visuals “cheesy” or “cringe.” Therefore, schools need to choose an SEL program that provides relevant and up-to-date content that resonates with students. Lastly, schools want more equality, diversity, and inclusiveness in SEL, so every student receives this important education without compromise. For example, Winooski Middle School reported that it is hard to find an SEL product that can equally be distributed and be understood by their diversified students, as it is a school with approximately 22 different languages spoken by their students. In addition, SEL needs to consider the diverse student population, especially in Title 1 schools. When an SEL program does not consider EDI, schools report that they find it hard for students to truly connect with the SEL curriculum, resulting in unsuccessful outcomes and compromised SEL promises. In terms of accessibility, most of the SEL products are only offered at school and homeschooled students should have the same experience as the on-campus students.

Not all SEL programs are created equal and have different implementation methods. According to the Wallace Foundation, after they examined various available SEL programs, they found out that even high-quality, evidence-based programs can falter or even have negative effects if not implemented effectively (Jones et al., 2017). However, there is no compromised way of implementing SEL, and as a result, we conclude the current SEL implementation faces six challenges:

Challenge 1: Lack of Resources for Teachers Leading SEL in Classrooms

It has been found that teachers are one of the occupations that suffer from high stress compared to other occupations (Gallup Education, 2014). Even though teachers are compensated with far less money when compared to other jobs that require a similar educational level, there are many duties that are expected of teachers that have become a burden on them and caused many teachers to leave the school system (Allegretto Mi shel, 2019; Robison et al., 2022). Some of the duties expected of teachers are -providing academic lessons, being involved with administrational work, keeping up with the latest trends in education and technologies, and managing relationships with the students and their parents(Stauffer Mason, 2013). These duties have become a burden on them due to a lack of resources including the teacher shortage (Garcia Weiss, 2019a; Stauffer Mason, 2013). Because there are not enough teachers in schools, the duties of remaining teachers are heavily emphasized while only a small amount of compensation is given to them. Also, the lack of other resources, like not enough staff and administration and time for adapting to new educational materials make teachers stressed out even more. The list of lack of resources for teachers is extensive, and as most of the SEL programs are administered to students by the teachers,

requiring teachers to lead SEL will put another heavy burden on them. Because they will feel exhausted by this increased burden, teachers will not be able to deliver as effective SEL to students, which will minimize the positive effects of SEL for students. In addition, teachers do not have extensive knowledge about SEL, so with out extensive training on how to lead the curriculum, teachers will not be able to lead SEL in the most effective way (Jones Bouffard, 2012). However, even if the teachers are provided with the extensive training on SEL, it might decrease their time on improving their academic teaching for students, which will create more stress for teachers. Adding improving their academic teaching for students, which will create more stress for teacher. Adding stress for teachers. Adding to this list of problems, it has been suggested that teacher–facilitated SEL programs might have some limitations (YMCA, 2021). For example, the teachers’ implicit biases can cause some problems during teaching SEL due to cultural differences in the class room of students who have diverse backgrounds(YMCA, 2021). These problems arise from misunderstanding and miscommunication between the teachers and the students which can result in academic failure, exclusionary disciplinary practices, and disengagement from the students (YMCA,2021).

Challenge 2: No Measures for Success

Current SEL products in the market provide no meaningful feedback after the implementation of the programs (Durlaket al., 2011). Most SEL programs do not provide any feedback after implementation or couple with an analysis of the impact on both the school and home (Durlak et al.,2011). Providing feedback and analysis of the impact of the SEL program is essential to evaluate its effectiveness and identify areas for improvement. This information is also critical for spreading the use of SEL programs and encouraging buy-in from teachers and administrators (Durlak et al.,2011). Therefore, schools need to choose an SEL program that includes feedback and analysis of program’s impact. By doing so, schools can evaluate the effectiveness of the program, identify areas for improvement, and make data-driven decisions to support students’ social-emotional development effectively.

Externally, It is essential to have feedback loops that measure SEL effectiveness and its impact on the lives of students and teachers. In today’s fast-changing world, it

is crucial to keep pace with the changing learning culture and continuously adapt to maximize the effectiveness of education for the youth. As Carnegie Corporation of New York (2020) recognized, today’s youth are living in a different world than their parents grew up in, and they need to learn new ways to navigate them selves for success. Additionally, the impact ofCOVID-19 on education highlighted the need for education to keep pace with the fast-changing and unpredictable world by introducing innovations and changes(Zhao Watters on, 2021). Therefore, it is important to gather feedback to identify areas for improvement and update the curriculum to meet the changing needs of today’s students and to ensure that the skills taught in SEL programs are still applicable in students’ lives and have future utility. It also allows SEL to be equipped with not only SEL skills but also new life skills, such as creativity, competence, and critical thinking, that are required for success in the future globalized world(Zhao Watters on, 2021).

Challenge 3: Schools Need SEL Partners, Not Vendors

Many SEL programs in the market are insufficient, and there is a lack of research and evidence on their effectiveness(Durlak et al., 2011). While research suggests positive effects of SEL, such as increased self-regulation and social skills, there is not enough evidence of improved academic performance (Durlaket al., 2011). Although students in SEL programs have shown an 11 percentile-point gain in academic achievement, there is no evidence on specific academic are as or subjects where students can improve(CASEL, 2022). This lack of research and evidence highlights the need for newSEL products that can expand research

and show specific improvements among specific improvements among students.Very often, SEL vendors sell products without any self-assessment. They mostly rely on limited quasi-experiments conducted in one school or community and no follow-up studies after scaling their business. Therefore, partnerships betweenSEL providers and schools are necessary for SEL success. Schools need to partner with SEL providers to not only implementSEL but also conduct proper research and assessment to ensure the effectiveness of their products. Partnerships between SEL providers and school is a must for SEL success.

Challenge 4: Inadequate Budget

School districts receive funding from the U.S. government each year, but it is of tenin sufficient to carry out the implementation of Social-Emotional Learning (SEL)programs. For instance, the San MarinoUnified school district receives only $6 per student for spending on SEL programs, as it receives $16,930 per year from the

U.S. Department of Education for StudentSupport and Academic Enrichment Grant for its 2,821 students. The cost associated with implementing SEL programs, such as setting up teams to handle SEL programs and providing professional development to teachers, is significantly higher.

Challenge 5: Covid-cause administrative turmoil

The emergence of Covid-19 in 2021brought forth a multitude of challenges that necessitated the development of new Social-Emotional Learning (SEL) products. The pandemic exacerbated existing issues in education and created new obstacles. For instance, teachers hort ages in K-12 schools, which we real ready prevalent before Covid-19, be came more widespread, with two in three school districts experiencing shortages (Garcia Weiss, 2019; Frontier Education, 2021). Covid-19 restrictions, budget cuts, stress, and balancing academic curriculum with SEL were contributing factors to this shortage (Sambursky, 2021).

Traditional SEL products that required teacher instruction were no longer feasible due to Covid-19 restrictions, such as stay-at-home orders and online teaching platforms, further highlighting the need for SEL products that did not require teacher involvement. The pandemic also result edin an increase in negative child behavior, such as hyperactivity and aggression, emphasizing the need for appropriate SEL products (Sun et al., 2022). At the same time, research shows that children with high social-emotional skills exhibited fewer behavior problems during the pandemic than those with low social-emotional skills (Sun et al., 2022), emphasizing the importance of SEL for student well-being and resilience.

Challenge 6: Non-sustainable teaching and no SEL-support environment

The lack of stakeholder involvement and support ultimately impacts the willing ness to try something new or additional such as SEL. Therefore, leadership plays a critical role in ensuring stakeholder involvement and support for SEL programs. In order to promote SEL, leadership must show support and provide stakeholders with feedback on the progress of the program.At the same time, leadership must also receive feedback from stakeholders, such as teachers and counselors, to address their needs and solve any problems that arise. Stakeholders in SEL programs include teachers, counselors, principals, and students. Teachers seek SEL programs to teach life skills like social awareness and relationship skills to embrace differences in their diverse classrooms (NationalUniversity, 2022). However, they also need SEL programs that do not add to the already burdensome workload of teaching main subjects in the classroom. Counselors seek a proactive model of SEL that helps them cultivate a positive school

The post-Covid urgency for SEL and inconsistency in available SEL products has led us to look towards CASEL’s guidelines for selecting SEL products.However, while CASEL evaluates the effectiveness and scientific evidence of SEL programs, it does not assess the quality of delivery. This means that schools are unable to determine which SEL program is best for their students and community. To implement SEL successfully, it is essential to consider the needs of all major stakeholders in a school: the principal, teachers, counselors, and students. Each stakeholder has their own priorities and views on the relevance of SEL to their role in the school:

• Principal wants visibility of teachers feedback and classroom improvement.
• Teacher wants to voice their concerns and have SEL to reduce disruptive behaviors and increase teaching time.
• Counselor wants to do more SEL and provide support to students who are in need of more SEL
• Student wants engaging and relevant SEL experiences.

climate, work with struggling students, and implement preventative programs that provide progress reports to district superiors (Hecht-Weber, 2021; Zyromskiet al., 2018; Savitz-Romer et al., 2021). Similarly, principals prioritize a positive school climate as the key to achieving success through SEL, while students desirea better middle school experience through learning SEL in a positive environment. The current market of SEL products mostly requires teachers to teach the contents, which creates a significant burden on teachers who are already struggling with work overload, classroom time management, and disciplinary challenges (Dias-Lacy Guirguis, 2017). Further more, the increased number of adverse childhood experiences due to the Covid-19 pandemich as made it difficult for counselors to help teachers and meet with all students who require one-on-one attention, let alon efacilitate classroom SEL curricula and small group conversations (Savitz-Romer et al.,2021).

It is important to recognize that there is no one “best” SEL program as each school has its unique needs and circumstances. While the CASEL framework provides guidance on the skills to be taught in SEL, the challenge lies in implementing SEL effectively. Therefore, it is critical for leadership to show support for SEL, solicit feedback from stakeholders, and address their concerns to ensure successful implementation.

Implementation Evaluation

Currently, the available evaluation tool for the effectiveness of SEL that is widely known is called the CASEL 5, provided by CASEL. CASEL 5 is the five main broad categories of competence that can be applied to development alstages and across diverse cultures: self-awareness, self-management, responsible decision-making, relationship skills, and social awareness (CASEL, 2022). It is recommended that for an SEL to maximize its effectiveness, SEL should include all five competencies in its program. However, this evaluation tool provided by CASEL only focuses on one dimension, which is the content of the SEL. As the effectiveness of SEL is affected by various other factors, such as the school climate and the embracement of diverse cultures, disregarding these external factors when designing and implementing SEL can lead to poor outcomes (Berg et al., 2017;Torres, 2019). Even though CASEL 5 isa widely known tool for evaluating the effectiveness of SEL, it is still incomplete as it does not inform what it means to be a successful SEL, especially regarding its implementation.

We have developed an evaluation guide to assist middle schools in selecting the most effective SEL for their students. Our guide takes into account not only the content of the SEL, but also the competencies of stakeholders and existing framework integrations, providing a holistic evaluation of SEL from the perspective of eLearning. The guide proposes four essential attributes and associated elements that have been carefully chosen through at horough series of literature review son effective SEL. We also considered promoting collective effort and community buy-in. We are proposing these essential attributes to ensure that everyone is involved in creating a supportive environment and implementing SEL at the same level.

The four essential attributes for an effective SEL implementation are:

• Stakeholder Buy-In and Competencies
• Framework Integration
• Learning Design with an approach to Skill Development
• Effective Use of Media

Stakeholder Buy-In and Competencies:

This attribute focuses on the involvement and support of stakeholders such as counselors, teachers, and principals in choosing and implementing an appropriateSEL program. The elements under this attribute include a supportive context, adult competencies support, and data for informed decision-making.

Skill Development:

This attribute considers both middle-school materials and practices aimed to develop the skills of young adolescents the middle school structure. The elements under this attribute incorporate the cognitive and socio-emotional developmental stages of these populations.

Framework Integration:

This attribute emphasizes the integration of existing evidence-based frameworks to create a frictionless add-on to regular school agendas and enhance key performances of middle schools. The SEL program should align with the school’s mission and vision and consider the social and emotional needs of students.

Equity, Diversity, and Inclusion (EDI):

This attribute promotes positive youth development through assistance with SEL for all students, regardless of their background. It ensures that all students have an equal chance to learn SEL and that the program is culturally responsive and inclusive.

Effective Use of Media:

This attribute focuses on the effective use of media to teach SEL most effectively. It ensures that media is used in an entertaining and effective way while ensuring that students can easily retain the information and perform the skills taught.

Case Study

The effectiveness of SEL programs (Palo, Character Strong, and Second Step) was evaluated using the 4 essential attributes for successful SEL learning. These case studies were used as examples of how to evaluate a SEL program based on the key attributes. The evaluation process involved examining the demos or information provided on the product’s website to determine whether they included the elements under the key attributes.

Results Discussion

Limitation

Reference