1. Some educational psychologists argue that the construct of teacher efficacy is composed of three dimensions: instructional strategies, engagement, and class management. Thus, to measure a teacher’s overall level of teaching efficacy, one must assess the teacher’s confidence in developing and using various instructional strategies, confidence in engaging (or motivating) students in the content presented, and confidence in managing well a potentially disruptive classroom. Given this three-pronged theory of teacher efficacy, you now face the task of developing a very brief measure of teaching efficacy. Your measure can contain no more than three items.
Rather than develop three items, you reference the following work by Ross and Bruce:
Which three items of teaching efficacy presented in Ross and Bruce will you select? Explain why you selected those three items. (8 points)
2. Zapf, Castell, Morawietz, and Karch (2016; Measuring inter-rater reliability…, BMC Medical Research Methodology) evaluate several measures of rater agreement. For one example, they provided a case study in which four German doctors were recruited to evaluate 50 samples of breast cancer biopsies. With each sample the doctors were asked to rate the percentage of tumor cells reacting to antibodies. The percentages were grouped into the following five-point scale:
0 = no cells reacting
1 = less than 10% reacting
2 = 10 to 50% reacting
3 = 51 to 80% reacting
4 = more than 80% reacting
(a) Calculate and report the numeric value for the level of inter-rater agreement among these four doctors. (8 points)
(b) Also, explain which measure of agreement was used to calculate the value reported, and whether the data were treated as nominal, ordinal, interval, or ratio for assessing agreement (only one acceptable). (6 points)
Excel data file:
3. Two radiologists were tasked with reviewing and interpreting xeromammograms with one of four outcomes: normal, benign, suspected cancer, or cancer. The table below reports their interpretations. These classifications should be treated as nominal data.
Assessments of Xeromammograms.
| Radiologist 2 | ||||
| Radiologist 1 | Normal | Benign | Suspected Cancer | Cancer |
| Normal | 15 | 7 | 0 | 0 |
| Benign | 3 | 10 | 1 | 0 |
| Suspected Cancer | 2 | 6 | 12 | 1 |
| Cancer | 0 | 0 | 0 | 2 |
Source: Boyd, N. F., Wolfson, C., Moskowitz, M., Carlile, T., Petitclerc, M., Ferri, H. A., … & Simor, I. S. (1982). Observer variation in the interpretation of xeromammograms. Journal of the National Cancer Institute, 68(3), 357-363.
(a) Calculate and report the numeric value(s) for the level of agreement among these two radiologists. (8 points)
(b) Also, explain which measure of agreement was used to calculate the value reported, and whether the data were treated as nominal, ordinal, interval, or ratio for assessing agreement (only one acceptable). (6 points)
