Non-cognitive skills are a fundamental aspect of any student’s ability to succeed in high school and in navigating the college access landscape. Non-cognitive skills are equally important in the post-high school world, playing an important role in college completion and workforce readiness (Heckman & Rubinstein 2001). Therefore, iMentor has determined that non-cognitive skills are one of four short term outcomes of our college success model as defined by our Core Metrics, making it important that our students develop and grow these skills. However, analyzing the development and growth of non-cognitive skills in our students is not as simple as just calculating a percentage. This article will focus on some of the nuances involved with developing, measuring and analyzing non-cognitive skills, so it may also be helpful to review an overview of non-cognitive skills and some research around non-cognitive skills to familiarize yourself with the topic.

The methodology used to assess non-cognitive skill development is aligned with best practices developed by leading researchers of non-cognitive skills. Through extensive research and careful reflection, we have identified and implemented effective ways of measuring student development and growth in non-cognitive skills through survey administration (Barkman & Machtmes 2002). And while we are confident in our survey design and administration, there are still challenges present in actually developing non-cognitive skills in our students as well as in measuring that development. Further, due to the nature of developing and assessing non-cognitive skills, we measure non-cognitive growth in the aggregate rather than on the individual level. We look at non-cognitive skill development with the perspective not of which students in the program showed growth, but rather did the students in the program show growth. Please see below for more detail on some of the nuances to the development and measurement of non-cognitive skills in our program: 

• Non-cognitive representation in the program: Our curriculum contains some units and lessons that explicitly teach non-cognitive skills to students while also focusing on incorporating non-cognitive skills into the conversation between mentees and mentors. Other units and lessons are more implicit in how they incorporate non-cognitive skills into the curriculum content and conversation. Further, we teach different non-cognitive skills at different times throughout the four-year program (see more details here). However, we evaluate program-level change in all six non-cognitive skills each year. Therefore, some students may not have received explicit programming on a specific non-cognitive skill by the time we evaluate them on that skill; measurement of non-cognitive skills using the survey may not always be congruent with how students experienced the curriculum and program depending on their age/year. This essentially means that it’s normal for some students to show less, no or even negative growth in certain non-cognitive skills at certain times, based on what they were taught that year.
• Sample size: As with any survey administration, having a large enough sample size is an important prerequisite to drawing conclusive findings. Depending on the number of mentee survey responses that partners receive, it may not be possible to draw solid conclusions about non-cognitive skill development and growth.
• Program dosage and non-cognitive growth: Some research we are doing now indicates that the program “dosage,” or frequency in which a student engages with the curriculum, is a factor in development and growth for certain non-cognitive skills. For example, when pairs hit our communication benchmarks, those students are more likely to show improvement in certain non-cognitive skills than pairs who miss those benchmarks. However, it can still be difficult to draw conclusions about program dosage and non-cognitive growth for smaller or newer programs (see “Sample size” above).
• Program length and non-cognitive growth: Our internal iMentor research illuminates that certain non-cognitive skills take longer to develop than others. When we look at survey responses from the beginning of the year to the end of the year, we may miss some non-cognitive growth that takes more than a year to develop. For example, self-advocacy doesn’t change much in one year, but does have cumulative growth from the beginning of one year to the end of the next; though our annual Year End Reports don’t show that growth because they examine outcomes cross-sectionally (within one year) rather than longitudinally (over multiple years).