Please Note: The term "Confidence Level" - which was used when this study was performed - has since been changed to "Exposure Level" for use on Moogie on the Net. We feel this term more appropriately describes the concept of the measurement and removes potential confusion over its meaning. However, the term "Confidence Level" is used in this study.
Since
1997, Moogie Math software has been helping
students prepare for state math proficiency
tests. Students who use this software are
given calculated Scores for each of the learning
strands in which they work. In addition, they
are also given a "Confidence Level" which
indicates the percentage of database questions
they have attempted in the given strand. This
study was designed to determine the degree
of correlation between the software-generated
Scores and students' abilities as measured
by actual state test scores. This study then
also examined the effect that the Confidence
Levels had on this correlation.
Two
batteries of tests were performed to compare
students' Moogie Scores with their actual
state test scores. Data were collected from
Douglas High School in Minden, Nevada, where
36 students used Moogie Math and then took
the Nevada 11th Grade Exit Exam.
The data were then entered into seven ASP
databases for analysis seeking to reject the
null hypothesis at the .05 level of significance.
The
first battery of tests employed a series of
simple regressions. While 85% of the resulting
data failed to reject the null hypothesis,
there was an unmistakable indication that
the Scores of students who had attained a
Confidence Level of at least 50% began to
reliably predict the students' actual abilities
with that strand. The second battery used
the method of weighted least squares, where
the Confidence Level was the weighted variable.
These tests rejected the null hypothesis across
the board, showing almost perfect correlation,
until the number of cases available for testing
became insufficient.
In
both batteries, the lack of students with
higher Levels of Confidence made absolute
testing impossible; more data is needed to
fully complete this study. However, the trends
observed with the existing data clearly revealed
that the Confidence Levels are an essential
part of the Moogie scoring algorithm. It was
also found that Moogie Scores effectively
measured students' abilities when Confidence
Levels were above 50%. The conclusion reached
is that the scoring mechanism of Moogie Math
correctly gauges students' actual abilities
on state math proficiency tests.
