One thing I learned only after being on the other side (i.e., being on hiring committees) is that sorting through candidates is a staggering amount of work. We had 65 or so candidates for our most recent position, which isn't even that much compared to what other institutions receive. If I spend on average 10 minutes (10!) for each candidate that's over 10 hours of work. And 10 minutes is, like, fine. It's ok. It's enough to do a cursory review and to sort on rough-grained differences. But it's obviously not enough time to read all materials, look over people's work, do some Googling, etc.
This is all to say that hiring decisions are made in a low-information environment out of necessity. And in that context, people sort based on dubious indicators of fit and competence (e.g., What do teaching evals really show? That you grade easy?). But, honestly, as some other people are expressing, I'm not sure there's a better way, especially not with everything else we all have to do.
Like, yes, I want to hire someone good, both because it benefits me and because of principle (of honoring the job ad and some semblance of meritocracy), but what am I supposed to do that I am not doing already? What is the indicator that is "fair"? How are we to make these decisions?