Can analytics fix the Rooney Rule?

“We have about one-third of the coaches in the National Football League are from the minority communities. That’s really not a bad pipeline. And so, the question is, why aren’t more of those people getting interviews?”

– Pittsburgh Steelers owner Art Rooney II

The answer to Art Rooney’s question is an important one — NFL coaches and executives are watching, and so is the nation. Over the last 17 years, companies from Amazon to Uber have taken a page from the league’s playbook and implemented their own version of the Rooney Rule, which mandates teams interview at least one minority candidate for any head coach or senior football operations job. But today, the NFL is grappling with the same problem corporate America finds itself struggling with: underrepresentation in top leadership. Seventeen years after the implementation of the Rooney Rule, the numbers are bad for the league and the assessments are blunt.

“When you look at the demographics, it’s embarrassing,” said Troy Vincent, NFL executive vice president of football operations.

Said Pittsburgh Steelers owner Art Rooney II: “I think where we are right now, is not where we want to be, not where we need to be.”

Rod Graves, who is chairman of the Fritz Pollard Alliance, an advocacy group that encourages the adoption of rules and practices that foster diversity within NFL teams, agreed with Rooney and Vincent.

“For all the hoopla that football has become in this country, that kind of progress, or lack of, is shameful,” Graves said.

Diversity is trending downward in a league where being “diverse” is the norm as 70% of NFL players are nonwhite. There are three black NFL coaches, the same number as when the Rooney Rule was adopted in 2003. To put it in context: The XFL, composed of only eight teams, has three black coaches.

To researchers who study hiring practices across many industries, including the NFL, these results aren’t just unsurprising — they were entirely predictable. One of those researchers is Stefanie Johnson, an associate professor at the University of Colorado, Boulder’s Leeds School of Business who is best known for her work on diversity and unconscious bias. A 2016 study she co-authored in the Harvard Business Review on hiring minorities and strategies for mitigating bias was downloaded more than 200,000 times, and she followed that up with a 2018 paper that examined the Rooney Rule. What follows is a conversation with Johnson.

I have to ask, you’ve studied the Rooney Rule, but are you a football fan?

You know, not really. I’m actually a psychologist by training but I think sports is a fascinating microcosm of what’s going on in other areas of business and society. In athletics, it’s just so much more visible. I find it really fascinating from a research side. I really love this topic. I don’t know if I can be passionate about the Rooney Rule, but it’s something that I really think is important in the history of U.S. and so I’m always superexcited to talk about it.

I think we’re living in a real inflection point for a lot of these issues. I read a summary of one of your studies about minority hiring and unconscious bias a few years ago, and thought of that research again when I happened on a quote from Panthers owner David Tepper about what drew him to his new head coaching hire, Matt Rhule. Here’s the quote: ‘He dresses like [expletive] and sweats all over himself. He dresses like me, so I have to love the guy! I was a short-order cook, he was a short-order cook. Nobody gave him anything, nobody [gave] me anything. He had to work hard for everything he got.’

Yeah, totally. I mean, I think that’s exactly how we hire people, right? It’s called a similar-to-me effect. Because essentially, if you have a healthy self-esteem as an individual, you think you’re great. And so someone who shares a similar history to you, being white if you’re white, being a woman if you’re a woman, having played for the same team, having gone to the same school, played the same position, all of those things cause you to evaluate people more favorably. That’s a great example. He really nailed it. It reminds me of … There was a really similar quote. Let’s see, I think it was Max Levchin, one of the founders of PayPal.

… He said he disqualified an applicant from PayPal because he liked to play basketball, which suggested he wouldn’t fit in with the nerdy culture at PayPal. Levchin said in Forbes back in 2007, ‘All of this is about self-selecting people just like you. He thinks like me, he’s just as geeky and he doesn’t get laid very often. Great hire. We’ll get along perfectly.’ So that’s the same thing, right?

I guess you could argue it probably brings a great degree of comfort to these decision-makers or NFL owners who feel like, ‘Well, I can get along with this guy.’

It’s totally absurd … really the goal is to get people who don’t share your same knowledge set. Because if you already have that background and experience, why do you need someone else who has the same background and experience? You actually want someone who’s different, and then you’re less likely to make mistakes if you have different viewpoints on the table. So I’d say that’s a very self-destructive way to hire people. But that’s what people do.

… It’s called the wisdom of the crowd. It’s a mathematical formula that shows the more similar you are to other people who are making decisions, the more similar your errors and decision-making are going to be, and that means you’re actually more likely to make a mistake. But if you’re really different than other people, you’re still going to make errors but your errors will be in a very different way. So they’re nonsystematic errors, and that still leads you to the correct response. That’s pretty cool.

In other words, in homogenous decision structures, the mistakes are amplified.

Yes, exactly. Because everyone will agree with you because they have the same blind spots.

There’s a real business case for representation in leadership.

For sure — there’s study after study that shows that having greater diversity in the leadership team translates to greater economic benefit. Diversity does pay in myriad other ways. It’s not just the bottom line. Winning, yes. But also things like culture and probably the team’s ability to attract and retain players who aren’t under contract … people of color want to be on a team where they believe that people of color are valued.

Three black coaches in 2003, three in 2020. Why isn’t just putting one minority candidate in front of a team’s decision makers working?

The Rooney Rule doesn’t work if you interview one black coach. This isn’t at all surprising. I could’ve statistically predicted that they would hire somewhere between zero and 5% black coaches if they interviewed only one black coach. Just one out of 22 times where teams interviewed one black candidate resulted in a hire [between 2013-2017] – so that’s 4%, 5%.

And increasing the minority candidates to two, how does it —

We found when [teams] interviewed at least two black coaches, that ended up being four [hires] out of 12, so they hired a black coach 33% of the time.

That’s a significant increase.

It is a significant increase and it’s a meaningful increase, because we’re talking about a small number of coaches. And so it’s essentially saying that in the NFL, most of the black coaches, almost all … were hired when a team interviewed two [black candidates]. And it just made so much sense. I think you go in for a coaching interview and you know there’s a Rooney Rule candidate and I’ve read … where black coaching candidates basically are like, ‘Yeah, I was a Rooney Rule candidate. I don’t stand a chance of that job.’

But when they’re interviewing two, even as the interviewee, I think you’re going in with a different perspective. You know you’re not the Rooney Rule candidate because they already interviewed the Rooney Rule candidate. You might have more confidence. And we know that changes the way people interview as well. And from the owner’s side, they’re clearly not doing a check-the-box.

How is it that two candidates change the calculus for decision-makers?

When you’re making a decision, there’s a natural tendency to choose between the more similar candidates. So, even if it’s just three candidates, if two of them are white and one of them’s black, you’re more likely to focus on the two who are more similar. Or if two of them were former quarterbacks and one was a linebacker, it sets a norm in your mind for what must be the most desirable characteristic, if that makes sense. Like if all of the candidates are white and there’s one black candidate, there must be something that’s less qualified about the black candidates, especially if you look at the base rate of black and white players in the NFL, right? Like there’s more black players than white players even though there’s almost zero black coaches.

And that’s called the Decoy Effect. It’s a marketing concept … Stores use this to sell products with things like price. So, if you go to buy a TV and two TVs have priced at $2,000 and there’s a third TV that’s like $800, people won’t buy the $800 TV. Because it seems like it must be really chintz y… At the same time, if there’s two $800 TVs and one $2,000 TV, why would you buy the $2,000 TV?

…. It seems absurd. Essentially people are bad decision-makers and our goal is to use as little cognitive effort as possible in making a decision. It’s easier to compare between two similar things and see which one’s better of these two and knock off the random candidates. So, I think that’s part of the explanation. But the other thing that we found, and it’s not in either of the papers, but it’s kind of interesting, is when there’s only one minority candidate … people believed that candidate was there based on their race alone. And I think that might be at the heart of what’s happening with the Rooney Rule, that you know there’s a Rooney Rule and so when you interview the one black coach, you know he’s the Rooney Rule candidate. I think the inference is that he’s only there because of his race rather than he’s there because he is a qualified candidate. I don’t think they’re giving those candidates a fair shake.

On the one hand you said if there’s just one minority candidate and people know that they’re interviewing them because the Rooney Rule mandates it as such, it undermines their qualifications because people think they are only there because of race.

But on the other hand, one finding from your 2016 research was that if you just increase the number of minorities that you interview to just two, it mitigates, I guess, some of the other bias that hiring managers might show. But if the league mandated two minority candidates, aren’t you back in the same place?

I know, right. Circular logic, right? So then if you mandated two, then people assume that those two are there not because they deserve to be, but because they have to be. So is there a way to fix this? I would say just still mandate two. Who cares? Because at least when you have two black candidates, I think the natural human instinct is to compare them to each other. And then when you’re comparing say, Lovie [Smith] to Vance [Joseph], then you will start to see their credentials and what they’ve achieved. Whereas when you just have one, it’s really easy to not even look at what they have done in the past.

If you were in charge of devising a Rooney Rule 2.0 and the desired outcome is to increase diversity among head coaches and general managers, the most powerful positions on a team, you would clearly add another minority candidate to the interview process. Is there anything else you would do?

Oh, there’s lots more. I think the Rooney Rule, yes. Let’s do Rooney Rule 2.0. Let’s call it that because it’s two [minority candidates]. I think that’s step one. But there’s so many things, I call it the D-C-B-A principle, which is defining criteria before assessing people. Basically, once you interview someone and then you’re like, ‘Oh, that person was great because of this,’ and your mind just fills in the blanks and whoever’s the most stereotypical or prototypical candidate somehow magically seems like the best candidate. And you can fill in a very long list of reasons why that is … our minds are supergood at filling in the blanks, why we’ve made a decision, even though the decision is just our gut instinct of who is more similar to us, or who fits the prototype that you’re looking for. But if you set up the criteria ahead of time … [for example] someone who has a history of a turnaround, or we’re really looking for someone who’s specifically good at coaching the offensive line. If you set those criteria and then judge the candidates against the criteria, the data shows you’re more likely to get diverse candidates.

That’s interesting.

You’ve made a better decision because you’ve actually hired the person who really did fit your needs rather than just hiring the person who was the most personable, I guess. I think having more diversity on interview panels … You’re interviewing with a few people, the owner, the general manager, the assistant GM, and all those people are white too. So we know the rapport during an interview strongly impacts the reactions of the interviewers, so much so that you can change someone’s success in an interview by increasing or decreasing rapport. So if you diversify the interview panel as well, then you’re more likely to hire diverse people. So we could do that.

What’s the biggest X factor with companies that you have studied that have managed to reach their goals as far as diversity is concerned. Is there a through line? Is it simply will?

It’s all about leadership, right? If the CEO’s not buying into this, then it’s not going to happen. But when top leadership says, ‘This is something that’s important and we’re going to do it,’ people will do it. Maybe that speaks to where the problems are in the NFL.

People who are working under leadership, generally like to keep their jobs and will do it. I guess it would be the commissioner, but would also take a plurality of NFL owners, because that’s where the real power lies.

Yeah. That’s a good point. On the flip side some diversity initiatives are achieved from a grassroots way method. Salesforce’s CEO, one of the people I interviewed said, ‘This is what employees are telling me, this is what they want. They want a diverse and inclusive workplace. And as the leader …’ he’s going to do what they want. And if NFL players are like, ‘This is what we want.’ Then maybe that can have some power. If NFL fans are saying it, maybe that can have some power. Maybe if put those two things together, a top-down influence and a bottom-up excitement around increasing diversity, then maybe something could change. I mean, the NFL would fall apart if there were no black players.

Yeah, there would be no NFL.

So I mean, there’s something to the power. I think people have a lot more power than we realize, especially in an age of social media where anyone can have their story heard by billions of people. It’s kind of like the #MeToo movement I think brought just tons of awareness. The #MeToo movement has dramatically changed how organizations function and how they pay attention to sexual harassment and gender equality, right? It really has changed. I have data, it’s changed. And so I don’t see why that couldn’t be the same for sports. I think it’s possible.

Are there any other points that you wanted to make as it relates to sort of the NFL and the situation it finds itself in now, which is they had good intentions with the Rooney Rule but it hasn’t increased minority hiring to influential positions in a meaningful way?

I think we shouldn’t underestimate the power that sports have to change the rest of society. And so when we’re talking to these top-level execs at the NFL, or coaches or owners at the NFL, they’re thinking about their team and their success and their fans, or whatever. But I think what they’re doing is bigger than that, because people watch and follow sports as an example of what we should do in the rest of society. So I feel like it’s a call to action to the leaders at the NFL. If they believe that equality is something that’s important to them more broadly, then I feel like that’s just a huge responsibility that they should take action on, at least on their own team.