How many times per year does a gun go off in an American school?
We should know. But we don't.
This spring the U.S. Education Department reported that in the 2015-2016 school year, "nearly 240 schools ... reported at least 1 incident involving a school-related shooting." The number is far higher than most other estimates.
But NPR reached out to every one of those schools repeatedly over the course of three months and found that more than two-thirds of these reported incidents never happened. Child Trends, a nonpartisan nonprofit research organization, assisted NPR in analyzing data from the government's Civil Rights Data Collection.
We were able to confirm just 11 reported incidents, either directly with schools or through media reports.
In 161 cases, schools or districts attested that no incident took place or couldn't confirm one. In at least four cases, we found, something did happen, but it didn't meet the government's parameters for a shooting. About a quarter of schools didn't respond to our inquiries.
"When we're talking about such an important and rare event, [this] amount of data error could be very meaningful," says Deborah Temkin, a researcher and program director at Child Trends.
The Education Department, asked for comment on our reporting, noted that it relies on school districts to provide accurate information in the survey responses and says it will update some of these data later this fall. But, officials added, the department has no plans to republish the existing publication.
This confusion comes at a time when the need for clear data on school violence has never been more pressing. Students around the country are heading back to school this month under a cloud of fear stemming from the most recent mass shootings in Parkland, Fla., and Santa Fe, Texas.
At least 53 new school safety laws were passed in states in 2018. Districts are spending millions of dollars to "harden" schools with new security measures and equipment. A blue-ribbon federal school safety commission led by Education Secretary Betsy DeVos is holding public events around the country, including one in Alabama Tuesday. Children are spending class time on active-shooter drills and their parents are buying bulletproof backpacks.
Our reporting highlights just how difficult it can be to track school-related shootings and how researchers, educators and policymakers are hindered by a lack of data on gun violence.
"I think someone pushed the wrong button"
The Civil Rights Data Collection for 2018 required every public school — more than 96,000 — to answer questions on a wide range of issues.
It asked what sounded like a simple question:
In the 2015-2016 school year, "Has there been at least one incident at your school that involved a shooting (regardless of whether anyone was hurt)?"
The answer — "nearly 240 schools (0.2 percent of all schools)" — was published this spring.
The government's definition included any discharge of a weapon at school-sponsored events or on school buses. Even so, that would be a rate of shootings, and a level of violence, much higher than anyone else had ever found.
For comparison, the Everytown for Gun Safety database, citing media reports, listed just 29 shootings at K-12 schools between mid-August 2015 and June 2016. There is little overlap between this list and the government's, with only eight schools appearing on both.
A separate investigation by the ACLU of Southern California also was able to confirm fewer than a dozen of the incidents in the government's report, while 59 percent were confirmed errors.
The Civil Rights Data Collection dates to 1968. The Education Department's Office for Civil Rights administers the survey every two years. Every public school is required by law to complete it. These findings often drive public conversations.
For example, the CRDC was the source of recent reports that black students were suspended from school at rates much higher than whites — information that inspired changes in discipline policy across the country.
The survey has dozens of items, ranging from how many middle schoolers passed algebra I to how many students with disabilities were restrained or secluded. It can be completed by filling out an online form or uploading data.
One item, about "Firearm Use," was required for the first time for all schools in the most recent data collection.
Most of the school leaders NPR reached had little idea of how shootings got recorded for their schools.
For example, the CRDC reports 26 shootings within the Ventura Unified School District in Southern California.
"I think someone pushed the wrong button," said Jeff Davis, an assistant superintendent there. The outgoing superintendent, Joe Richards, "has been here for almost 30 years and he doesn't remember any shooting," Davis added. "We are in this weird vortex of what's on this screen and what reality is."
"We got wind of it and nipped it in the bud"
In other cases, something may have happened, but not the firearm discharge the survey asked about.
The biggest discrepancy in sheer numbers was the 37 incidents listed in the CRDC for the Cleveland Metropolitan School District. Roseann Canfora, the district's chief communications officer, told us that, in fact, 37 schools reported "possession of a knife or a firearm," which is the previous question on the form.
The number 37, then, was apparently entered on the wrong line.
Similarly, the CRDC lists four shootings among the 16 schools of the Santa Monica-Malibu Unified School District in California. Gail Pinsker, spokeswoman for the district, says that "going back 20-plus years," no one can remember any incident involving a firearm. Their best guess, she says, is that there was some kind of mistake in coding, where an incident involving something like a pair of scissors (California Education Code 48915[c]), for example, got inflated into one involving a firearm (48915[c]).
Ray Poole, the chief of legal services for the Nassau County School District in Florida, told us that at one school where a shooting was reported, Callahan Middle School, on Nov. 21, 2015, a Saturday, a student took a picture of himself at home holding a gun and posted it to social media. "We got wind of it and nipped it in the bud." No shooting.
The CRDC shows seven shootings in DeKalb County, Ga. Police reports provided to us by that district give a sense of more of the many, many ways the data collection may have gone wrong.
At Redan Middle School, there is a report of a toy cap gun fired on a school bus — not a shooting.
And district officials provided a police report showing that there was a shooting after a McNair High School football game — in August 2016, after the time period covered in the survey.
The Education Department's Office for Civil Rights received complaints about the wording and administration of this survey even before it went out.
A June 2014 research report commissioned to improve the CRDC as a whole noted that in previous data collections, districts had experienced "unacceptable levels of reporting burden." They complained that the CRDC asks them to report information that is similar to what states already collect, but in a different format, or at a level of specificity that they don't currently track.
Also at issue, the internal report says, was a "lack of clarity in the definitions of key terms." When it came to "Offenses," the group of questions including firearm use, districts "indicated dissatisfaction with the categories provided, specifically that the CRDC categories did not align with the categories used in state reporting, other federal reporting, and/or their own district databases."
As an example of this lack of alignment, the federal Gun-Free Schools Act requires schools in states that receive federal funds to expel students who bring a gun to school and requires districts in those states to report the circumstances of such expulsions to the state — regardless of whether a gun goes off.
The state of Florida asks schools to report "weapons possession," excluding pocketknives. California asks schools to report suspensions and expulsions resulting from "possession, sale, furnishing of a firearm" or "imitation firearm."
And so on.
There's also potential for confusion within the CRDC itself. While this particular item refers clearly to "a shooting," the previous item asks about a long list of incidents, some involving "a firearm or explosive device" and others involving "a weapon."
Temkin at Child Trends, who has long studied bullying and school climate, says this wording "could cause confusion."
"Best practices in data collection are not to include double-barreled items," she says, such as asking about a "firearm or explosive device" in the same question. An explosive device could be something like a pipe bomb or even a firecracker.
NPR submitted a Freedom of Information Act request to learn more about problems with the data collection, and we received emails that schools and districts sent as they grappled with this kind of confusion. For example, the Omro school district in Wisconsin wanted to know whether a consensual paintball-gun fight involving several students should be considered an "attack with a weapon" or a "possession of a firearm."
Another reason the shooting data may show these kinds of problems, Temkin adds, is that the item is so new. "Because this was the first year this was asked of all schools, they may not have been as prepared to respond to this item."
And there's another factor at work as well: the law of really, really big numbers. Temkin notes that "240 schools is less than half of 1 percent," of the schools in the survey. "It's in the margin of error."
Liz Hill, an Education Department spokeswoman, told NPR that "at least five districts have submitted requests to OCR to amend the school-related shootings data that they submitted for the 2015-16 CRDC." The plan is to issue what is called "errata" to update the data, but the original document will not be republished, Hill said.
Hill made the point that any "misreporting" is the schools' responsibility, not the department's: "As always, data reported by recipients is self-reported and self-certified."
After we contacted the Santa Monica-Malibu Unified district about the four reported shootings, the district emailed the Office for Civil Rights to try to correct the information. No shootings happened, officials said.
The Office for Civil Rights responded on July 25:
"The CRDC accepts correction requests for up to one year from the moment the submission period opens. For the 2015-16 collection, the corrections period closed on June 30, 2018, and for this reason your data correction request cannot be accepted. However, a data note will be included on the data file to ensure users are aware of the errors you are reporting."
A previous version of the graphic about uncertainty in numbers referred to one school where a shooting took place as Madison Junior High, as it was identified in the CRDC. It's actually Madison Junior/Senior High.
DAVID GREENE, HOST:
This back-to-school season, gun violence is on the minds of so many educators and parents. The Federal School Safety Commission led by the secretary of education, Betsy DeVos, is holding its final public listening session today in Montgomery, Ala. This commission was formed in the wake of the Marjory Stoneman Douglas school shooting in Florida that left 17 people dead. And one question at the heart of the debate over school shootings is just how often they happen. Earlier this year for the first time, the federal government published what it said was a comprehensive count based on a survey of every single public school in the United States. But an NPR investigation has found that the actual number may be far lower than the government is reporting. Anya Kamenetz of NPR's Ed team joins us now to talk about this. Hi, Anya.
ANYA KAMENETZ, BYLINE: Morning, David.
GREENE: All right. So let's start with the official number from the government that we're dealing with here.
KAMENETZ: Two hundred and thirty-five. The Education Department's Office for Civil Rights asked every school this question, and they actually made it mandatory for the first time last year. They asked, in the 2015-2016 school year, has there been at least one incident at your school that involved a shooting, whether - regardless of whether anyone was hurt? And that included going to and from school and school-sponsored events. And they published the answer this past spring.
GREENE: OK, so 235 in one school year - that's - what? - like, almost one for every school day. Am I right?
KAMENETZ: Right. So it seems really high to us. And so the NPR Ed team - this was some intern magic - spent the summer calling around to every single one of those schools. And David, we confirmed just 11 of the incidents.
GREENE: That's astonishing. I mean, that's not just, like, a mistake here or there.
KAMENETZ: Right. So we should say, you know, a quarter of the schools did not get back to us, it being the summer. But in 161 of those 235 cases, we did learn districts or schools told us nothing at all happened. And the biggest chunk of those overall was the Cleveland Municipal School District, reporting 37 shootings. Their best guess is that that number, 37, was placed on the wrong line of the survey.
GREENE: It's just incredible that you have schools saying nothing at all happened, and they're being counted by the federal government as places where there were shootings.
KAMENETZ: Right. So it's a really important question. And it's the first time that they've asked it. And we heard the same kind of errors all over the place. So in the Santa Monica-Malibu Unified School District, among 16 very affluent schools, four shootings are listed. And Gail Pinsker, a district spokeswoman, said no one remembers any guns being fired going back 20-plus years. But maybe, she says, there was a coding mistake.
GAIL PINSKER: There was a code that was selected for a student brandishing a pair of scissors.
KAMENETZ: And somehow that got inflated into a gun going off.
GREENE: That - OK, so maybe the school is just reporting that that an incident happened, and it gets counted by the federal government as a gun going off. But, I mean, mistakes happen, but this is - these are important mistakes.
KAMENETZ: Yeah. I mean - you know, so we partnered with Child Trends, a nonprofit, for this investigation. And Deborah Temkin, one of their program officers, said, you know, on the one hand, there's 96,000 public schools, and so this is really within the margin of error for a massive data collection. But on the other hand, she said...
DEBORAH TEMKIN: I think the challenge, of course, is when we're talking about such important and rare event like school shootings, that amount of data error could be very meaningful.
GREENE: OK, Anya, so what is the Department of Education saying about your investigation?
KAMENETZ: So Liz Hill, a department spokeswoman, said first of all, you know, they rely on schools to self-report and self-certify all of this data. They did say that at least five districts have contacted them and asked to revise their data submissions on the shootings. We know that some of those districts got in touch with the department after talking to us. The Education Department says they'll publish an update called an errata, but they do not plan to republish the original document, which of course has been out for several months now.
GREENE: Which makes me wonder, do we really know anything about the number of school shootings in this country?
KAMENETZ: Yeah, that's the question. I mean, I think the one thing we can say that's good news is that they are, by any measure, extremely rare. But I wish that we knew more. I mean, having - you know, having a kid going back to school myself and seeing, you know, active-shooter drills, seeing districts spending so much money on safety and all kind of focusing on this idea of shootings being something that is, you know, really common, I just wish that we had better information.
GREENE: Anya Kamenetz from the NPR Ed team. Thanks a lot, Anya.
KAMENETZ: Thanks, David. Transcript provided by NPR, Copyright NPR.