How merit is in the beholder’s eye
Great news! A January 21, 2025 Executive Order has declared that we’re going to be a meritocracy! The President’s E.O., titled Ending Illegal Discrimination and Restoring Merit-Based Opportunity, blames governmental and private sector DEI programs for “undermin[ing] our national unity, as they deny, discredit, and undermine the traditional American values of hard work, excellence, and individual achievement in favor of an unlawful, corrosive, and pernicious identity-based spoils system.” According to the E.O., “[h]ardworking Americans who deserve a shot at the American Dream should not be stigmatized, demeaned, or shut out of opportunities because of their race or sex.”
I had no idea that DEI programs are so insidiously powerful that they can cause deadly wildfires, make aircraft tragically collide, and also destroy the American Dream. But phew! Thanks to this E.O., being merit-based will solve all our problems, right?
Here’s a useful look at “merit” in practice. Dr. Arin Reeves conducted a study to test whether and how law firm partners were influenced by race when reviewing associates’ work. First, a team of lawyers drafted a research memorandum from a hypothetical third year litigation associate. The memo was seeded with 22 errors, including minor spelling and grammar errors, substantive technical writing errors, errors in reciting the facts underlying the memo, and errors in analyzing the facts.
Next, the memo was sent to 60 different partners, at 22 law firms, who’d agreed to participate in a study of “writing competencies of young attorneys.” Each reviewing partner received exactly the same memo, with its supporting research materials, presented as written by “Thomas Meyer,” a law firm 3rd year associate and graduate of NYU Law School. The reviewing partners were asked to edit the memo for all factual, technical and substantive errors, and also to rate the memo’s overall quality from 1 (extremely poorly written) to 5 (extremely well written).
The only difference? Half of the reviewing partners were told that Thomas Meyer was Black, and the other half were told that Thomas Meyer was White.
The results? The overall quality of this identical memo was rated at an average of 3.2 of 5 for Black Thomas Meyer and 4.1 of 5 for White Thomas Meyer.
- An average of 5.8 of the spelling and grammatical errors were noted in Black Thomas Meyers’ memo, but only 2.9 for White Thomas Meyers.
- An average of 4.9 of the technical writing errors were identified for Black Thomas Meyer, compared to 4.1 for White Thomas Meyers.
- An average of 3.9 of the factual errors were found in Black Thomas Meyers’ memo, versus 3.2 for White Thomas Meyers.
- Black Thomas Meyer’s memo received 29 comments on how the memo’s formatting should be improved, while White Thomas Meyer’s memo received 11.
- While it was not feasible to quantify the reactions to the analysis errors, White Thomas Meyer’s memo was evaluated as better at analyzing the facts and had substantively fewer critical comments on its analysis.
And the reviewers’ overall comments were consistently more positive for White Thomas Meyer (“generally good writer but needs to work on…”; “has potential”; and “good analytical skills”) than for Black Thomas Meyer (“needs lots of work”; “average at best“: and, my personal favorite, “can’t believe he went to NYU”).
So Thomas Meyer clearly has more merit than … Thomas Meyer.
As shown in study after study (for examples, see here, and here, and here), “merit” is subjective, is in the eye of the beholder, and can be influenced by a myriad of factors, often in unconscious ways. If you want a dose of humility on this, take one or more of the Harvard Implicit Association Tests.
The point of such studies is not to serve up a “gotcha” moment, branding individuals as racist, or misogynistic, or otherwise bigoted. We’re all just people. We house a host of different biases that are deeply ingrained and can drive our decision-making. And we’re often confused about just how often we react with our automatic, “fast” thinking and then, in hindsight, clean things up by rationalizing our actions with our logical, “slow” thinking. Such as (ahem) when we make an instinctive decision about hiring, advancement, or who belongs, and then rationalize it as “merit-based.”
And apparently it’s still necessary to reiterate that diversity, equity, and inclusion are needed to help ensure people of different lived experiences, who have merit, will also have a fair and equal opportunity to succeed, and to belong.
So, how about a pop quiz? Picture in your mind’s eye the law firm partner from the study above, reviewing the Thomas Meyer memo disparately based upon race, and take a mental screenshot.
Is it an image of an older White guy (like me)? Not necessarily so. In fact, the study’s cohort of participating partners was more diverse than most law firms’ partnership ranks: 23 women, 37 men, 21 “racial/ethnic minorities,” and 39 White.
And here’s the kicker. Among the study’s findings was this:
There was no significant correlation between a partner’s race/ethnicity and the differentiated patterns of errors found between the two memos. There was also no significant correlation between a partner’s gender and the differentiated patterns of errors found between the two memos.
In other words, the race, ethnicity, and gender of the reviewing partners had no significant impact on the different evaluation results for Black Thomas Meyer’s and White Thomas Meyer’s identical memo.
How could this be? We’ll dig into that next time.