Imagine reading something like this:
Yesterday, every school in Metro Atlanta experienced a 100% drop in student suspensions. (This comes as schools have released for the summer holiday last week.)
No one would write this and then believe that schools had done anything in particular to effect such an outcome.
Yet it seems that in education, we often omit key information from our data analyses in order to “fix” our collective problems in a surface manner. Instead of actually improving education, districts, schools, and states manipulate data in order to make their students look better on paper, while failing to improve the true outcomes for our kids.
Unfortunately, it is often the most vulnerable students who lose the most. As the education system tries to make their schools look like they’re making strides towards closing the achievement gap, they waste effort that could be spent on improving the substance of their education.
One of the ways that states and districts have tried to demonstrate this “improvement” has been with the high school graduation rate.
2009-2010: Georgia Graduation Rates Dropped 10%
Before 2009, states in the U.S. could use a variety of methods to report their graduation rates. Some did things like discounting certain subsets of students, and some counted students who took six years to graduate.
During the 2010-2011 school year, the U.S. Department of Education required all states to begin using a uniform high school graduation rate.
Below is an excerpt from the U.S. Department of Education FAQs regarding its move to the new four-year adjusted cohort rate:
Q. Why is a new graduation rate being reported?
A. The U.S. Department of Education is requiring all states to begin publicly reporting comparable high school graduation rates using its new four-year adjusted cohort rate calculation method. In October 2008, a regulation by the U.S. Department of Education [section 1111(h) of ESEA] was amended, which included a requirement for all states and local educational agencies (LEAs) receiving Title I funds (money for schools with a certain percentage of low-income students) to begin calculating and reporting the more uniform rate beginning with 2010-2011 data.
Historically, states have calculated graduation rates using varying methods, creating inconsistent data from one state to the next. The transition to a uniform high school graduation rate requires all states to report the percentage of freshmen students who graduate in four years with a regular high school diploma. This rate will reflect a uniform method for reliable comparisons among states.
Q.How does the “new” graduation rate calculation differ with Georgia’s current rate?
A. The primary difference is defining the cohort. The four-year high school graduation rate defines the cohort when the student first becomes a freshman, and the rate is calculated using the number of students who graduate within four years. The current graduation rate defines the cohort upon graduation, which includes students who take more than four years to graduate from high school.
The Test That Increased Georgia’s Graduation Rate – By Going Away
After schools in Georgia had to deal with the 10% decrease in reported graduation rate, they began to evaluate ways that they could “improve” this.
The year that the GHSGT was ended, graduation rates increased by double digits across many districts in the state, and the state’s graduation rates increased by 6% overall. The largest and most notable increase came for “economically disadvantaged” students, whose graduation rates increased by 12% between the 2014-2015 and 2015-2016 school years.
Generally, every set of data require that the researcher try to control the variables in order to assess the true effect of any one thing. However, when it came to high school graduation rate data, it seemed that no report (except for a slight mention in the Atlanta Journal-Constitution and a couple blogs) placed much weight on the fact that the Georgia High School Graduation Test had been dropped as one of the requirements for graduating.
Some counties reported an almost double-digit graduation rate increase for their entire district.
Below is a table with the graduation rates for DeKalb County Schools (just as a point of reference). Notice the increases between the 2014 and 2015 school years:
Interesting to note, many (not all) of the schools with lower numbers of students who qualified for free-or-reduced lunch (FRL) saw little to no improvement (Dunwoody High School, Lakeside High School, DeKalb School of the Arts, Chamblee Charter High School).
However, many schools with significant percentages of students qualifying for FRL saw double-digit improvements: Gateway to College Academy (22 percentage points), Towers High School (16 percentage points), Miller Grove High School (20 percentage points), Destiny Achievers Academy (17 percentage points), Columbia High School (14 percentage points), and Tucker High School (19 percentage points).
I find it peculiar that the state, districts, and local schools touted these rising graduation rates as an indicator of their increased ability to educate students. Please don’t get me wrong: I don’t want to take away from the enormous efforts and achievements these teachers and students made. Each of these schools made substantial gains, and I’m not trying to diminish those real and notable achievements.
However, the data and facts seem to indicate that the elimination of the GHSGT played a significant role in the dramatic increase in graduation rates. This must be addressed if we want to make meaningful evaluations of what’s working in schools and what’s not.
It’s Not Just Georgia
During the past decade, the U.S. has continued to see graduation rates increase.
However, the U.S. continues to be a low-to-average performer on the OECD’s PISA test. In recent years (2012 & 2015), the U.S. decreased its score in Math and maintained its score in Reading. On average, though, its scores have declined.
And in 2011, Georgia was ranked 40th on individual state performance on the PISA.
So not only is the U.S. performing worse internationally, Georgia also continues to be one of the worst performers in the nation in math, reading, science, and writing. It’s hard to explain how our state continues to do poorly and how our nation continues to do poorly, and how – all the while – Georgia and U.S. graduation rates continue to soar (also here).
The point is this: even though there is great work occurring in many of our schools, it appears that many of our schools, districts, and states are manipulating numbers – not necessarily in an “illegal” manner. I view the dropping of the GHSGT as a completely legal decision; however, the year the state dropped the GHSGT, the graduation rates increased significantly (6% overall and 12% for economically disadvantaged students) — an increase never before seen in the state.
I’m not saying the GHSGT was perfect, but it was at least an attempt to ensure a basic standard of competency that could be applied across the state, regardless of district or school. Instead of improving our wrap-around services or strategically increasing funding or significantly changing the method of teaching, it seems the state took the easy way out in order to help more students graduate.
(Another, more lengthy discussion needs to take place regarding the over-emphasis on standardized tests and the punitive use of data in our education system – but until that conversation happens, we need to at least make sure that the data we are using are reliable, free from manipulation.)
Data Show Our Students Are Unprepared for University
Our students, unfortunately, are the ones who are paying the price. A significant portion of high school graduates from Georgia high schools have to enroll in remedial college math and English classes (1 out of 4 students in 2010 and roughly half of all students in the University System of Georgia in 2013) because their high schools didn’t prepare them adequately for post-secondary study.
If such a high percentage of our students are needing remedial courses in Georgia colleges, then we are certainly failing our students. By manipulating the numbers so that we look better on paper, we are robbing our students of an honest evaluation of their academic achievement and our educational performance.
The Emperor Hasn’t Got Anything On
If we are using data to drive our practice, we need to use solid numbers that allow practitioners to identify and isolate individual variables so that we can at least know what the issue is. Let’s actually close the achievement gap, not just pretend that we are.
The way we are handling this issue (and many like it) is akin to the way the crowd responded in The Emperor’s New Clothes:
So off went the Emperor in procession under his splendid canopy. Everyone in the streets and the windows said, “Oh, how fine are the Emperor’s new clothes! Don’t they fit him to perfection? And see his long train!” Nobody would confess that he couldn’t see anything, for that would prove him either unfit for his position, or a fool. No costume the Emperor had worn before was ever such a complete success.
“But he hasn’t got anything on,” a little child said.
“Did you ever hear such innocent prattle?” said its father. And one person whispered to another what the child had said, “He hasn’t anything on. A child says he hasn’t anything on.”
“But he hasn’t got anything on!” the whole town cried out at last.
The Emperor shivered, for he suspected they were right. But he thought, “This procession has got to go on.” So he walked more proudly than ever, as his noblemen held high the train that wasn’t there at all.
https://www2.ed.gov/admins/lead/account/consolidated/index.html#csp2005 (all years except 2009-2010)
2009-2010 — https://nces.ed.gov/pubs2013/2013309.pdf