Lecture 1 Statistics

• Mean score of a question is the average score for all students on the question. It is best to have a range of questions with different mean scores on the test, with some easy (mean score above 90%) and some hard (mean score below 50%).
• Discrimination of a question is the correlation coefficient between the scores on the question and the total assessment scores. Discrimination values should be above 20%, unless a question is very easy (mean score above 95%), in which case it is acceptable to have lower discriminations. It is always better to have higher discriminations for all questions, and a range of discriminations is not desired.
• Auto-graded Attempts for a question is the average number of auto-graded attempts made per student at the question.
• Quintiles shows the average scores on the question for students in the lowest 20% of the class, the next 20%, etc, where the quintiles are determined by total assessment score. Good questions should have very low scores for the lowest quintile (the left-most), and very high scores for the highest quintile (the right-most). This is essentially a graphical representation of the discrimination.

• Mean (μ): Mean score of a question is the average score for all students on the question.
• Median: Median score of a question is the score which separates the lower half and the upper half of students scores.
• SD (σ): This is the standard deviation of student scores on this question.
• Discrim.: Discrimination of a question is the correlation coefficient between the scores on the question and the total assessment scores.
• Some sub. (%): (some submission percentage): The percentage of students that submitted a valid, auto-gradable answer.
• Some perfect sub. (%): (some perfect submission percentage): The percentage of students that submitted an answer that got full auto-graded credit.
• Some nonzero sub. (%): (some nonzero submission percentage): The percentage of students that submitted some answer that got some auto-graded credit.
• μ_{First Sub. Score}: (first submission score average): The average auto-graded score on the first submission over students that had at least one submission.
• σ_{First Sub. Score}: (first submission score standard deviation): The standard deviation of first submission auto-graded scores.
• First Sub. Score Hist.: (first submission score histogram): The histogram of first submission auto-graded scores.
• μ_{Last Sub. Score}: (last submission score average): The average auto-graded score on last submission over students that had at least one submission.
• σ_{Last Sub. Score}: (last submission score standard deviation): The standard deviation of last submission auto-graded scores.
• Last Sub. Score Hist.: (last submission score histogram): The histogram of last submission auto-graded scores.
• μ_{Max Sub. Score}: (max submission score average): The average best-submission score over students that had at least one submission.
• σ_{Max Sub. Score}: (max submission score standard deviation): The standard deviation of best-submission auto-graded scores.
• Max Sub. Score Hist.: (max submission score histogram): The histogram of best-submission auto-graded scores.
• μ_{Avg. Sub. Score}: (average of submission score averages): The average of average submission auto-graded scores over students that had at least one submission.
• σ_{Avg. Sub. Score}: (variance of submission score averages): The variance of average submission auto-graded scores over students that had at least one submission.
• Avg. Sub. Score Hist.: (submission score averages histogram): The histogram of average submission auto-graded scores over students that had at least one submission.
• μ_{Sub. Score Array}: (submission score array): The average submission auto-graded scores (over students that had at least one submission) for the 1st submission, 2nd submission, etc. Submission score arrays are padded with zeros when some students have more submissions than others.
• μ_{Incr. Sub. Score Array}: (incremental submission score array): The average incremental submission auto-graded score gain (over students that had at least one submission) for the 1st submission, 2nd submission, etc. arr[n] = The incremental score gain from submitting the nth submission.
• μ_{Incr. Sub. Points Array}: (incremental submission points array): The average incremental submission auto-graded points gain (over students that had at least one submission) for the 1st submission, 2nd submission, etc. arr[n] = The incremental points gained by submitting the nth submission. Only available for exams.
• μ_{Num. Sub.}: (average number of submissions): The average number of auto-graded submissions.
• σ_{Num. Sub.}: (number of submissions standard deviation): The standard deviation of the number of auto-graded submissions.
• Num. Sub. Hist.: (number of submissions histogram): The histogram of the number of auto-graded submissions.
• Quintile Scores: Quintiles show the average auto-graded scores on the question for students in the lowest 20% of the class, the next 20%, etc, where the quintiles are determined by total assessment score.

In the case that a student takes this assessment multiple times (e.g., if this assessment is a practice exam), we are calculating the above statistics by first averaging over all assessment instances for each student, then averaging over students.