Experts criticize methodology of report that called it "unlikely" Spokane Police exhibit racial bias

click to enlarge In 2019, the Inlander asked Spokane Police Chief Craig Meidl about racial disparities in policing. At the time, Meidl suggested that at least part of the reason minorities are overrepresented in policing data is because some demographics simply commit crimes at higher rates.
In 2019, the Inlander asked Spokane Police Chief Craig Meidl about racial disparities in policing. At the time, Meidl suggested that at least part of the reason minorities are overrepresented in policing data is because some demographics simply commit crimes at higher rates.

Last week, the Spokane Police Department released a sprawling, 307-page report looking at disparities in policing by age, sex and race. It contained plenty of headline-grabbing points, among them being this: Officers were 22 percent more likely to use force against Black subjects, and 49 percent more likely against Native Americans.

But the report also found that all races were "equally likely" to be stopped by police, under the report's methodology.

It also
claimed this: "The findings show that it is unlikely that Spokane Police officers are engaged in systemic biased practices against any particular demographic group."

To some who have studied racial disparities in law enforcement, the conclusions in the report are puzzling. For these types of reports, SPD had previously tapped Ed Byrnes, an Eastern Washington University professor. He's found that Black and Native American people are more likely to be stopped, searched and arrested by police.

This time, SPD reached out to a Washington-based company called Police Strategies, which was paid $10,000 for the study. 

So why did this latest report seem to contradict Byrnes' earlier findings? Byrnes says it's because Police Strategies chose to use a methodology that would make it more likely to not find racial disparities. 

"That fundamental assumption drives pretty much everything else they're doing. And it's a problematic assumption," Byrnes says.

EWU professor Ed Byrnes
EWU professor Ed Byrnes

For example, take a look at the report's claim that all races were "equally likely" to be stopped by police. That's not actually true if you compare police stops to the population of minority groups in Spokane; using the population as a baseline would show disproportionality. But the Police Strategies report doesn't use population as the baseline. Instead, it compares police stops to reported crime numbers. When you compare those, the disparity disappears.

In other words, it suggests there's no officer bias as long as police suspect minorities to be criminals as much as the community does. And that assumption — that minorities commit crimes at higher rates — is problematic and has been around for a while, Byrnes says.

"That's basically taking an old narrative of minority criminality and wrapping it in a different veneer," Byrnes tells the Inlander.

Lois James, a researcher with Washington State University College of Nursing in Spokane, says she was also skeptical of the Police Strategies report.

"I have conducted a study with this same population where we found strong evidence of implicit bias," says James.                

Bob Scales, CEO of Police Strategies, admits there is "no perfect benchmark" to analyze racial disparities. Scales is a former King County deputy prosecutor and special assistant U.S. Attorney. His company conducts these sorts of studies for police departments across the country.

Yet while Scales is willing to say their way of analyzing police data may not be the definitive way to do it, he does believe that using reported crimes is a better benchmark than using population.

He uses an example like this: Population-based benchmarks may find that police use force five times more often against minorities than against White people. But if you compare uses of force to the people who are actually arrested — what he calls an "activity-based" benchmark — then the disparity may be much smaller or nonexistent, since minorities may also make up more arrests, proportionally.

Similarly, if he used population-based benchmarks in this report, he says he "would conclude that the department and its officers have a systemic bias against Black and Native American subjects that they come into contact with." But there would also be other disparities by age and sex, then, that couldn't be ignored.

"We would also have to conclude that officers are biased against males and those between the ages of 18 and 49," Scales says.

In arriving at the overall conclusion that it's "unlikely" that Spokane officers exhibit racial bias, he points to data on police actions taken once officers do stop each racial group. These are what he calls "discretionary" actions. And when it's up to officer discretion to issue a ticket or let a driver off without warning, they do so less often than he would expect.

But the analysis doesn't consider the stop itself a discretionary action, only what happens after. If it did consider stops a discretionary action — as Byrnes would — it would find that police are more than twice as likely to stop Black and Native American people. So what the data actually indicates is that Spokane officers are still more likely to stop Black people than other racial groups, but less likely to issue an infraction once they do.

You could argue, then, that officers are racially profiling people if they're stopping them without a valid reason. Scales, however, tries to swat down that argument by saying Blacks and Native Americans are equally likely to be stopped by police if compared to reported crimes. But again,  that's only using his methodology comparing stops to reported crimes, even though it's police who are choosing to make those stops.

Think of it this way: Black people make up about 4 percent of the population, according to the report. Black people make up around 11 percent of police stops, meaning they are nearly three times more likely to be stopped by police than the rest of the population. That could suggest officer bias. But the Police Strategies methodology compares police stops to reported crimes, and Black people are suspects in about 12.5 percent of reported crimes. So the report uses that to conclude they are are "equally likely" to be stopped by police.

This is a big reason the report concluded it was "unlikely" Spokane police were engaged in systemic racial bias. And using that baseline — which Police Strategies applies in reports for other departments as well — all but guarantees that disparities will appear lower or nonexistent.

"I have not conducted or seen any study using activity-based benchmarks that suggests a likelihood that systemic biased practices are occurring against any particular demographic group," Scales says.

But what about the disparities the company did find in this report? It showed, for example, that use of force occurred more often against Black and Native American people. And it showed that there were high disparities in which racial groups were searched by officers. There, Scales says that it's too small of a sample size to say that the disparities were due to systemic practices.

James, the WSU researcher, tells the Inlander she hasn't read through the entirety of the Police Strategies report. But her own research seemingly came to the opposite conclusion as this report. She's found that implicit bias exists among White officers, but also that they were less likely to shoot unarmed Black suspects in a laboratory setting. Similar findings have been found elsewhere. A study conducted by a Harvard professor found that racial disparities exist in non-lethal police uses of force, but not in fatal shootings by police.

She rejects any suggestion that research looking at discretionary police behavior elsewhere hasn't found evidence of officer bias.

"If they believe there isn't evidence of disparate treatment of minority groups by police, they're sorely mistaken," James says. "There's a great deal of evidence for that."

In 2019, the Inlander asked Spokane Police Chief Craig Meidl about racial disparities in policing. At the time, Meidl suggested that at least part of the reason minorities are overrepresented in policing data is because some demographics simply commit crimes at higher rates.

"When you're looking at violent crime in the city of Spokane, over 30 percent of our homicides in the last six years have been committed by African Americans. They make up 2.5 percent of the population," Meidl said at the time. "But 17 percent, over the last two years, of my aggravated assaults, as reported by the community, have been committed by African Americans. Over 10 percent of my rapes, 10 percent of my robberies."

The Police Strategies report, while not coming to definitive conclusions, would seem to support this point of view that disparities by population are caused by minorities committing crimes at higher rates.

Spokane City Council President Breean Beggs also takes issue with the methodology used. He says he's heard Meidl say that the reason they arrest more Black people is that Black people commit more crimes, an assertion that Beggs says is refuted by other studies. In Police Strategies, he says SPD found "a methodology to advance that narrative."

"My concern with the whole process is they should have had that conversation about what methodology with the community before they started," Beggs says.

But Byrnes calls that an "old dog whistle" narrative. Byrnes completed two previous reports for SPD — one in 2015 and one in 2017. He says he wasn't paid for it and worked together with SPD Capt. Brad Arleth to put them together.

Unlike the Police Strategies report, those reports considered officer-initiated contact as a discretionary action. While not measuring "bias" necessarily, Byrnes uncovered that disproportionality was "absolutely in place" when compared to population, he says.

"Even when controlling for some other factors, there was no getting way from it. There was no getting around it," he says.

Using a population-based benchmark is the only thing that makes sense in looking for disparities in police stops, he says, because anybody in the population can be subject to an officer-initiated contact. Comparing officer-initiated contacts to reported crimes, he says, would only make sense if you were only studying how police responded to calls for service.

It's a huge leap, Byrnes says, to apply it beyond that. Yet that assumption drives much of the Police Strategies report.

Byrnes worries it could be an effort to change the conversation on policing "away from social justice."

"We would all love to live in a community where there was no disproportionality," Byrnes says. "But when folks start saying, 'We looked at it this way and got this other result that people like better,' well, the result doesn't justify the methodology. I would just caution anybody to beware of being enamored with the result and therefore justifying the methodology for it."

Staff writer Daniel Walters contributed to this article.

Women Helping Women Fund: An Iconic Night at the Fox @ Martin Woldson Theater at The Fox

Tue., May 17, 4-6 p.m.
  • or

About The Author

Wilson Criscione

Wilson Criscione is the Inlander’s news editor. Aside from writing and editing investigative news stories, he enjoys hiking, watching basketball and spending time with his wife and cat.