Item Analysis

Summary

In this lesson by Teacher Clare, the focus is on the process of item analysis in educational assessments. The lesson starts by explaining the importance of item analysis in evaluating test items, discussing desirable and undesirable characteristics of test items, and providing steps to perform an item analysis. Teacher Clare elaborates on key criteria for determining these characteristics: difficulty index, discrimination index, and the effectiveness of alternatives. Through the use of formulas and tables, she guides students on how to calculate the difficulty and discrimination indices to assess the items' effectiveness. Finally, the lesson touches on the analysis of distractors and presents exercises for further practice.

Highlights

• Teacher Clare explains item analysis and its importance in assessing learning. 📘
• Defines criteria for evaluating test items: difficulty, discrimination, and alternatives. 🧮
• Walks through calculating difficulty and discrimination indices with examples. 📊
• Discusses the interpretation of indices, stating that a positive discrimination index is ideal. 👍
• Highlights the importance of distractors analysis to improve assessment quality. 😎
• Encourages understanding through exercises included in the lesson. 🧩

Key Takeaways

• Understanding the importance of item analysis in assessments. 🎯
• Learning the steps for performing item analysis. 📝
• Grasping the concept of difficulty and discrimination indices. 🤔
• Seeing how to calculate these indices and interpret the values. 📊
• Exploring distractor effectiveness in test items. 🚀
• Engaging exercises to deepen understanding. 💪

Overview

Teacher Clare opens the lesson by revisiting the concept of item analysis, emphasizing its role in evaluating students' response patterns on test items. The assessment's goal is to identify both well-constructed items with desirable characteristics and those requiring revision or rejection due to flaws. She explains the step-by-step process of item analysis, including ranking scores and identifying high and low performers, which sets the foundation for performing detailed analysis.

The lesson continues by delving into the three main criteria that dictate an item's desirability: the difficulty index, the discrimination index, and the effectiveness of alternatives. Teacher Clare explains how these indices are calculated and interpreted, providing examples to illustrate their application. For instance, she shows that an ideal difficulty index is above 0.8, and a positive discrimination index differentiates high-achieving students from the rest.

Concluding the lesson, Teacher Clare discusses the significance of analyzing distractors, which are incorrect but tempting answers meant to evaluate students' understanding thoroughly. The presence of effective distractors differentiates a good test item from a poor one. To reinforce the learning, she presents exercises designed to test students' comprehension of the material. Teacher Clare reminds viewers of the downloadable resources available for further study, ending the session with an invitation to the next lesson.

Chapters

• 00:00 - 00:30: Introduction to Item Analysis Introduction to Item Analysis
• 00:30 - 01:30: Preliminary Steps in Item Analysis The chapter discusses the process of item analysis, focusing on identifying items with desirable and undesirable characteristics. It outlines the preliminary steps in item analysis, starting with ranking student scores from highest to lowest. Following this, the top 27% of performers and the bottom performers are identified for further analysis. Items with desirable traits are retained, while those falling short may be revised or rejected.
• 01:30 - 02:00: Understanding Criteria for Item Analysis The chapter 'Understanding Criteria for Item Analysis' provides a detailed discussion on item analysis, focusing specifically on how to analyze student performance data. The key points include setting aside a portion of total test takers for analysis, dividing them into upper and lower groups, and then using these groups to assess item choices. The learning objective is to develop skills for calculating index of difficulty and discrimination through the given data.
• 02:00 - 02:30: Item Difficulty Index The chapter begins by setting the context for item analysis and highlights three essential criteria for assessing the desirability of test items: the item's difficulty, its discriminating power, and the effectiveness of its alternatives. The focus then shifts specifically to the difficulty index, which measures how challenging a test item is for a group of test-takers and serves as a crucial factor in evaluating item quality.
• 02:30 - 03:00: Calculating Difficulty Index The chapter titled 'Calculating Difficulty Index' explains the concept of the difficulty index in educational assessments. It begins by defining the difficulty index as the proportion of students in both upper and lower groups who answered an item correctly. The chapter emphasizes that a lower difficulty index indicates a higher level of difficulty for the item. It notes that the ideal difficulty index should not be lower than 0.8. The chapter also introduces a problem where the reader is tasked to calculate the difficulty index of an item as part of the learning process.
• 03:00 - 03:30: Discrimination Index Overview The chapter gives an overview of the Discrimination Index (DI), providing the formula DI = R/N where R is the number of students who correctly answered a question, and N is the total number of students who took the test. An example computation is given with R = 45 and N = 75, resulting in a DI of 0.6. This DI indicates that the item in question has a moderate level of difficulty according to the referenced table.
• 03:30 - 04:00: Calculating Discrimination Index The chapter titled "Calculating Discrimination Index" explains the concept and importance of the discrimination index in assessments. The discrimination index (denoted as 'd'), is calculated as the difference in the proportion of students in the upper group who answered correctly and the proportion of students in the lower group who did the same. The key takeaway is that a test item is said to have discrimination when it effectively differentiates between high and low performing students. This is essential for educators to ensure that the test can accurately reflect students' abilities.
• 04:00 - 04:30: Interpreting Discrimination Results This chapter discusses the calculation of the discrimination index, which is a measure used to interpret the effectiveness of test items in differentiating between higher and lower group performances. The formula used is DS = (Pu - Pl) / N, where Pu represents the proportion of students in the upper group who answered correctly, Pl represents the proportion in the lower group, and N is the total number of test takers. The chapter also includes an examination of a table with a focus on interpreting the correct option noted as letter C.
• 04:30 - 05:00: Example Calculation for Discrimination Index The chapter discusses the concept of the discrimination index in educational assessments. It explains that a higher proportion of students in the upper group (UG) getting an item right compared to the lower group suggests a positive discrimination index. For an item to be considered good, it must have a positive discrimination index, indicating that students with more knowledge are more likely to select the correct answer.
• 05:00 - 05:30: Index of Difficulty Calculation This chapter discusses the calculation of the index of difficulty, focusing on item discrimination among students. A negative discrimination index indicates poor item validity, suggesting that students in the upper group find certain test items misleading due to ambiguity, necessitating the rejection of such items.
• 05:30 - 06:30: Interpreting Difficulty and Discrimination The chapter begins by discussing the concept of discrimination in educational assessments. It explains how an item with zero discrimination is characterized by an equal number of high and low achievers making the correct response, suggesting that the item is either too easy or ambiguous. The chapter then proceeds to solve for the index of discrimination by substituting values into the relevant formula.
• 06:30 - 07:00: Effectiveness of Alternatives This chapter discusses the effectiveness of alternatives in relation to a specific test item. The analysis measures the discrimination power of test items by looking at the difference in the number of students with correct responses between upper and lower groups. In the given example, the discrimination index is calculated as (students with correct responses from the upper group minus 15) divided by 56 (total number of test takers from both groups). A result of 0.20 indicates that the test item is non-discriminating or fair, suggesting a need for revision.
• 07:00 - 09:00: Exercises and Conclusion The chapter 'Exercises and Conclusion' introduces the formula for calculating the index of difficulty, which is represented as I = R + N. It explains the process of adding correct responses from two groups — the upper group and the lower group, resulting in a sum of 41 correct responses. This number is then divided by the total, 56, yielding an answer of 0.73.

Item Analysis Transcription

• 00:00 - 00:30 [Music] good day everyone my name is teacher claire welcome to another lesson on assessment of learning in our previous lesson we talked about item analysis given the raw data we also learned that item analysis refers to the process of investigating the student's response to each item of the test
• 00:30 - 01:00 an item that has desirable characteristics can be retained and those with undesirable characteristics can either be revised or rejected likewise i presented the various preliminary steps in doing item analysis first we rank the scores of the students from highest to lowest score second we select and identify the top 27 percent performing tube and the bottom
• 01:00 - 01:30 27 performing from the total takers third we set aside the remaining 46 percent because they will not be used for item analysis fourth we tabulate the number of students in the upper group and the lower group based on the selected choice per item today we will be discussing the item analysis based on the given table the learning objective for this lesson is to demonstrate skills on how to compute the index difficulty and discrimination in
• 01:30 - 02:00 wix before we will discuss how item analysis is done we need to understand the three criteria that determines the desirability and undesirability of an item based from perception 2016 which are as follows the difficulty of an item discriminating power of an item and effectiveness of alternatives first let us discuss item difficulty or difficulty index
• 02:00 - 02:30 this pertains to the proportion of the number of students in the upper and lower group who answered an item correctly by simply looking at the table the lower the difficulty index the higher is its level of difficulty take note that a desired index of difficulty must not be lower than 0.8 for instance in this problem we are tasked to solve the difficulty index of an item to solve
• 02:30 - 03:00 this we have the formula d i equals to r divided by n where r is the total number of students who got the correct response and then is the total number of students who took the test substituting the value we have 45 divided by 75 which is equal to 0.6 so based on our table the item has a moderate level of difficulty next
• 03:00 - 03:30 we have the discrimination index denoted as ds discrimination index is the difference of the proportion between the students in the upper group who got the correct response and the proportion of the students in a lower group so when we say an item must discriminate it simply means that a test can differentiate between low and high performing students to solve
• 03:30 - 04:00 for the indiscrimination index we use the formula ds equals pu minus pl over n where pu is the proportion of the number of students in the upper group who got the correct response pl is the proportion of the number of students in the lower group who got the correct response and n is the total number of takers so let's examine this table in the table our correct option is letter c
• 04:00 - 04:30 looking at the result we can see that the proportion of students who got an item right in the ug or upper group is higher than the proportion of the lower group this indicates a positive discrimination for an item to be considered as good item it must have a positive discrimination index this suggests that the knowledge in by the students enable them to select the correct answer
• 04:30 - 05:00 on the contrary if the proportion of the students who got an item right in the lower group is greater than the students from the upper group then we have a negative discrimination index this means that it has a poor item validity and that the upper group is misled by item ambiguity in this case we must reject the item
• 05:00 - 05:30 however if the same number of high and low achievers make the correct response we say that it has a zero discrimination this suggests that the item is either too easy or the item is ambiguous so let us solve for the index of discrimination now substituting the values to the formula we have 26 which is the total number of
• 05:30 - 06:00 students who get the correct response from the upper group minus 15 we divide it to 56 which is the sum of the test takers from ug or upper group and lowered group which is 28 plus 28 that makes it 56 hence the answer is 0.20 looking at the table the item is considered as non-discriminating or fair item that needs revision
• 06:00 - 06:30 for the index of difficulty here's the formula the i is equal to r plus n as mentioned earlier r is the total number of correct response so we can we add the correct responses from the upper group and lower group that is 26 plus 15 equals 41 then we divide it by 56 which is the total date first the answer for this is 0.73
• 06:30 - 07:00 this time allow me to present the table from reganit 2008 so if our index of difficulty is ac and rds is not discriminating then the item is considered poor and must be rejected the decision is to reject the item with the justification that it is unreasonable to retain an easy item
• 07:00 - 07:30 that cannot differentiate high from low performing students finally we have the last which is the effectiveness of alternatives others call it as destructor analysis this is usually done to find out which destructors or incorrect but plausible options were effective or ineffective in a particular item it must be noted that a destructor which no one chooses means
• 07:30 - 08:00 it's obviously an incorrect answer hence to consider an item good the item must have destructors that attract more people from the lower group than that of the upper group otherwise it is an ambiguous item this time we will be showing you the different exercises to test your level of understanding
• 08:00 - 08:30 some of this will be included in your final exam for our first set of questions we have this here's the second set of questions and our last set of exercises so here's my references so that's all for our lesson on item
• 08:30 - 09:00 analysis if you want to download all answers and its corresponding solutions just click the link in this video description thank you for watching and see you in my next video once again this is teacher claire [Music]
• 09:00 - 09:30 you