So your department resists the idea of intentionally making a minority hire. They think the most just approach is to hire the best of the candidates. However, that recommendation assumes we can achieve a success that might be even more difficult than recent work on biases may make us aware.
One factor is that people in a particular ingroup tend to remember the excellencies achieved by members of that ingroup than those of members of an outgroup. (Gaertner, S. L., & Dovidio, J. F. (2000). Reducing intergroup bias : the common ingroup identity model. Philadelphia, PA: Psychology Press.) Supposing this is true, what will the effect be when a white male philosophers is asked to assess overall the quality of student contributions to a seminar he gave? He will have found the white male students, e.g., were sometimes particularly outstanding, while the white female students and minority members were often enough good, but not as outstanding as some of the others.
Perhaps much worse, when looking back over the job candidates, the white males will be more outstanding to the white males on the selection committee, other things being equal. A woman can end up arguing that a female candidate’s cv is as good as a male candidates. One unfortunate consequence is that the female candidate’s qualifications are clarified in an argumentative situation. It all can indeed turn quickly into who is in favor of which gender.
At the risk of making this seem to be more about me than I want to, let me recount what may be an instance of the bias I am describing. About 60 days after my tenure as faculty senate president ended, I discovered that a group of the faculty senate guys had gotten together to nominate our administrator for an award. I asked why I had not been included in the project. I was told that the group just included the present faculty senate president and past presidents. It may of course been that their association with “past faculty senate presidents’ were wholly male. Or it may have been that I was already forgotten. The hypothesis we have looked at here would make sense of the latter possibility.