Self-Assessment
This study aims to examine the available empirical evidence regarding the accuracy of self-assessment and its effects, as well as to identify potential factors, commonly referred to as moderators, that may influence these effects.
The Importance of Student Self-Assessment
Student self-assessment most generally involves a wide variety of mechanisms and techniques through which students describe (i.e. assess) and possibly assign merit or worth to (i.e. evaluate) the qualities of their own learning processes and products. For example, self-assessment could self-grading approaches and peer feedback before self-assessment.
There were two previous meta-studies on self-assessment. The first study focused on the accuracy of self assessment and found that students and teachers had matching assessments of work an average of 64.1% of the time. The second study focused on the relationship of self-assessment to academic achievement, effects of self-assessment on self-regulation processes, student perceptions of self-assessment, and accuracy of self- assessment. Regarding the effects of age and ability, these tended to be confounded, but younger students tended to present overscoring while older ones were closer to the teachers’ scores. Regarding ability, abler students were more accurate. Regarding task features, activities that were familiar and predictable for the students seem to produce more accurate self-assessment. Regarding methods of self-assessment, the more specific and concrete the standards and criteria, the higher the accuracy. Finally, they also reported on other variables that could influence accuracy but with less certainty such as gender, ethnic culture, personality, or training.
Student Self-Assessment Should be Combined With Teacher Feedback
The authors conducted a meta-analysis using 160 papers. The search was limited to peer reviewed English language articles that had been published after 1989. The authors had three research questions.
Question 1: What are the main characteristics of Self-assessment accuracy studies?
Answer: Most studies focused on higher education, and are centered around student’s scoring accuracy in their self-assessment
Question 2: What is the average student’s self assessment accuracy?
Answer: There was a medium/high correlation between scores issued by the students on their self-assessment and the scores given by the expert/instructor. There was a slight overestimation of the students in their scoring when compared with the expert/instructor.
Question 3: Do the moderators influence students’ self-assessment accuracy?
Answer: It was found that when self-assessment is done without feedback, there is a tendency for students to greatly overestimate their score. The use of a self-assessment rubric did not show much of an improvement.
There is a General Overlap Between a Student’s Self-Assessment and Teacher Feedback
The authors noted that researchers investigating self-assessment should not limit themselves to scoring accuracy alone, but should also investigate content accuracy.
The authors note a reasonable level of reassurance that there is general overlap between what students self-assess and what their instructor would give them. As most studies were focused on higher education, it was found that the more content knowledge students had, the easier it was for them to self-assess.
The authors note that the highest leverage practices teachers have control over in relation to increasing the accuracy of student self-assessment is providing feedback.
Notable Quotes:
1. “We found that when self-assessment was performed in the absence of feedback, a significant overestimation was found”
2: “Having knowledge or mastery about the topic to be self-assessed showed a significantly increased precision though it did not reach perfect precision either”
3: “In future research, it would be interesting to explore whether, while keeping the emphasis in the formative purposes, students would also receive the message that scoring accuracy is important if they anchor their score to the learning goals, something that we have not found in the included studies. “
Personal Takeaway:
This study reinforced the importance of feedback and developing content knowledge in order for students to assess their knowledge. As the authors mentioned, most of the studies they included were from higher education, not K-12. —Matt Browne
León, S. P., Panadero, E., & García-Martínez, I. (2023). How accurate are our students? A meta-analytic systematic review on self-assessment scoring accuracy. Educational Psychology Review, 35(4), 106.
Key Takeaway
Competency in social communication can be an indicator of how socially desirable one is when meeting new people. For people with autism spectrum disorder (ASD), their conversational fluidity can be predictive of friendships and subsequent social and emotional success in early adulthood. In order to address the lack of conversation fluidity among the ASD population, video feedback intervention is one evidence-based strategy that can make a difference in their verbal interaction. —Michael Ho
Video Feedback Intervention
Tagavi, Koegel, Koegel, and Vernon (2021) examined the efficacy of a video feedback intervention to improve conversational fluidity in young adults with ASD. Specifically, the authors aimed to determine whether a video-feedback intervention would improve conversational fluidity, question-asking, and overall social conversational desirability in young adults with ASD. In addition, the participants self-reported their confidence in social communication and their application of what they learned in the intervention to various natural settings.
The following research questions were addressed:
- Will a video-feedback intervention decrease the number of long, awkward pauses young adults make in a conversation with a typically developing (TD) peer?
- Will a video-feedback intervention increase the number of on-topic questions young adults make in a conversation with a TD peer?
- Will participation in this intervention lead to an increase in peer ratings of social desirability for these individuals?
- Will these individuals increase their confidence in their own social communication skills as well as find the intervention acceptable and enjoyable?
Here are the major takeaways from the article:
The Need for Conversation Skills for Adults with ASD
- Tagavi et al. (2021) refer to Sasson et al. (2017)1 – “Starting in adolescence, social conversation skills become increasingly important, as there seem to be strict, yet unspoken, communication norms that determine whether an exchange is successful or not.” The success of one’s social communication can contribute to initial social impressions and can determine the desire for future interactions or a sustained relationship.
- “Social challenges affect individuals with autism spectrum disorder (ASD) of all ages and developmental levels.”2 These individuals typically have fewer frequencies of successful peer interaction; lower levels of self-esteem; and higher rates of loneliness, anxiety, and depression.
- Video feedback intervention, which is a type of video modeling that involves the viewing and evaluating of an individual’s previously filmed performance, is known to be useful for individuals with ASD because of their strong visual perception skills, tendency to think concretely, and ability to apply the skill in multiple contexts over a prolonged period of time.
Increased Conversational Fluidity for Three Adults
- Three adults with ASD, with an average age of 23 years, participated in this study. All intervention sessions were conducted in a clinic room at the University Autism Center. Three baseline sessions over 10 weeks were conducted before the interventions in order to collect data on their current performance in conversational fluidity.
- In response to the first research question, all three participants improved in their ability to use questions, filler words, and follow-up statements to fill in gaps in conversations. This indicates a decrease in the number of long, awkward pauses and an increase in conversational fluidity.
- In response to the second research question, all three participants increased their use of on-topic questions to a rate comparable to their TD peer. On-topic questions were questions the participants asked that were connected to the main topic discussed in the conversation.
- In response to the third research question, raters, who were blind to this study, scored all participants as being more socially desirable during and after the intervention.
- In response to the fourth research question, the participants’ self-reports showed that all participants reported improvements in the confidence to communicate and the ability to ask questions following the intervention. They also reported that they found the interventions acceptable and enjoyable.
- “Participants were able to learn and utilize skills with a variety of peers, indicating that conversational skills learned through video feedback are generalizable.” It is evident that video feedback is not only a powerful tool to increase conversational fluidity but also serves as a gateway to successful social communication across multiple contexts.
Limitations
There are several limitations in this study. First, in addition to conversational fluidity, there are other conversational skills that could also be targeted. Moreover, this study did not compare video feedback to other types of interventions on conversational fluidity. Another limitation of this study is the limited diversity of participants, who were all Caucasion males in their early adult years. Finally, there is limited research on the generalization technique to teach social communication skills to young adults; testing generalization more explicitly is recommended in future studies.
Summarized Article:
Tagavi, D., Koegel, L., Koegel, R., & Vernon, T. (2021). Improving Conversational Fluidity in Young Adults With Autism Spectrum Disorder Using a Video-Feedback Intervention. Journal of Positive Behavior Interventions, 23(4), 245–256. https://doi.org/10.1177/1098300720939969
Summary by: Michael Ho—Michael supports the MARIO Framework because it empowers learners to take full control of their personalized learning journey, ensuring an impactful and meaningful experience.
Additional References:
- Sasson, N. J., Faso, D. J., Nugent, J., Lovell, S., Kennedy, D. P., & Grossman, R. B. (2017). Neurotypical peers are less willing to interact with those with autism based on thin slice judgments. Scientific Reports, 7, Article 40700.
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).
Key Takeaway: When students determine their own goals and design their own rubrics for measuring outcomes, goal attainment is dependent on whether the student’s rating or the teacher’s rating is utilized as the outcome measure. Further research is needed to determine why student and teacher ratings can diverge and to what degree these can be used to draw conclusions on the efficacy of interventions. —Akane Yoshida
The Self-Determined Learning Model of Instruction (SDLMI) is “a teaching model implemented by teachers to enable their students to self-direct and self-regulate their actions in pursuit of goals” (Shogren et al., 2017). In this 2021 article, Shogren et al. analyze their findings from a 3-year trial in which they examined the impact of different educator supports for the implementation of SDLMI in inclusive, secondary core content classes. Students participating in the study were supported in creating their own Goal Attainment Scaling (GAS) rubrics to rate themselves and to allow their teachers to rate them.
The authors’ analysis sought to determine the following:
- How much agreement is there between student and teacher ratings of student goal attainment?
- Does the impact of different educator supports for the implementation of SDLMI on goal attainment outcomes vary across student and teacher ratings?
- Does the impact of student disability status on goal attainment outcomes vary across student and teacher ratings?
In relation to research question 1, the authors found that there was “only a fair amount” of agreement on student goal attainment outcomes, and those discrepancies were most pronounced where ratings suggested that outcomes were far less or far more than expected. Of this, the authors suggest:
“Could it be that students and teachers are providing different perspectives, likely influenced by unique contextual factors, of goal attainment outcomes? Furthermore, how do student and teacher ratings correspond to actual student skills and use of these skills in general education classrooms? Are student or teacher ratings more aligned with actual performance?”
In relation to research question 2, the authors conclude thus:
“[T]he findings suggest a critical need to attend to the role of differing perceptions of outcomes in the analysis of intervention efficacy. When examining the impact of student versus teacher ratings in estimating the effect of teacher implementation supports, student ratings of goal attainment suggested a larger impact as teachers received more intensive supports for implementation.”
With regards to research question 3, the impact of student disability status on differences in goal attainment ratings between students and teachers, the authors found that student disability status led to teachers giving substantially lower ratings of goal attainment compared to students, prompting them to wonder:
“[D]o students overestimate their strengths while teachers identify areas of additional instructional needs and supports? Or are teachers’ expectations of students’ capacities shaped by students having an identified disability…Or are both meaningful yet independent, self-perceptions of outcomes that could predict observed outcomes in distinct ways?”
The authors state the need for ongoing research to be conducted, to further explore how students with disabilities in inclusive settings can be supported to successfully engage in goal setting, as well as determine to what extent student ratings and teacher ratings are aligned with actual behavior in the classroom.
Summarized Article:
Shogren, K. A., Hicks, T. A., Raley, S. K., Pace, J. R., Rifenbark, G. G., & Lane, K. L. (2021). Student and teacher perceptions of goal attainment during intervention with the Self-Determined Learning Model of Instruction. The Journal of Special Education, 55(2), 101-112.
Summary by: Akane Yoshida — Akane believes that developing supportive and nurturing relationships with students is key to helping them to attain their personal benchmarks for success. She loves how the MARIO Framework operationalizes this process and utilizes systematic measurement of student learning and teacher effectiveness to guide interventions.
Additional References:
- Shogren, K. A., Wehmeyer, M. L., Burke, K. M., & Palmer, S. B. (2017). The Self-Determination Learning Model of Instruction: Teacher’s Guide. Lawrence, KS: Kansas University Center on Developmental Disabilities.
Article Abstract
This research analyzed the network of psycho-social influences through which efficacy beliefs affect academic achievement. Parents’ sense of academic efficacy and aspirations for their children were linked to their children’s scholastic achievement through their perceived academic capabilities and aspirations. Children’s beliefs in their efficacy to regulate their own learning and academic attainments, in turn, contributed to scholastic achievement both independently and by promoting high academic aspirations and prosocial behavior and reducing vulnerability to feelings of futility and depression. Children’s perceived social efficacy and efficacy to manage peer pressure for detrimental conduct also contributed to academic attainments but through partially different paths of affective and self-regulatory influence. The impact of perceived social efficacy was mediated through academic aspirations and a low level of depression. Perceived self-regulatory efficacy was related to academic achievement both directly and through adherence to moral self-sanctions for detrimental conduct and problem behavior that can subvert academic pursuits. Familial socioeconomic status was linked to children’s academic achievement only indirectly through its effects on parental aspirations and children’s prosocialness. The full set of self-efficacy, aspirational, and psychosocial factors accounted for a sizable share of the variance in academic achievement.
MARIO Connections
Bandura et al.’s study of the connection between external influences, self-efficacy, and academic achievement informs how MARIO prepares the educator and parent to support the learner’s development of self-efficacy. Aspects of this discussion are also incorporated into MARIO diagnostic tools because understanding the power of a student’s perception of self-efficacy is imperative to the work we do.