The failure of pre-filtering


A few years ago, when I’d just joined Mozilla and become Dr. Belshaw, I ran an experiment. At that point, I was attending, on average, an event per week, so I was meeting lots of people for the first time. I had three variables in play:

  1. My ideas
  2. The fact I worked for Mozilla
  3. Mention of doctorate

Sometimes when I talked to people for the first time, I’d not mention who I worked for, nor give people my business card (which featured my title). I’d just talk about my ideas. People would listen and give me their ideas on the subject. That was great, but they’d usually done zero work or reading on the subject. Often, people would say what I was trying to do (Web Literacy Map, Open Badges) ‘just wouldn’t work’ for various reasons.

Interestingly, I found that just adding in one of the other two variables changed that. My status as a ‘doctor’ would often change the conversation. While people weren’t deferential, there was certainly a noticeable switch to “oh, this guy must know what he’s talking about.” Similarly, if I mentioned that I worked for Mozilla, all of a sudden what I was talking about was eminently possible. Taken together, people would often scrabble to think of ways we could work together.

Funny, isn’t it? The way that I’ve come to rationalise this that people recognise and value ‘pre-filtering’. We all recognise working for certain organisations and having particular qualifications as a kind of shorthand. These days, as a consultant, I get similar results if I mention some of the clients I’ve worked with, such as Creative Commons or City & Guilds, along with my academic qualifications. I’d like not to be just my LinkedIn profile, but it’s what people respond to.

All of this brings me to Seth Godin, who recently wrote:

The SAT is a poor indicator of college performance, but most colleges use it anyway.

Famous colleges aren’t correlated with lifetime success or happiness, but we push our kids to to seek them out.

And all that time on social networks still hasn’t taught us not to judge people by their profile photos…

Most of all, we now know that easy-to-measure skills aren’t nearly as important as the real skills that matter.

Everyone believes that other people are terrible at judging us and our potential, but we go ahead and proudly judge others on the basis of a short interview (or worse, a long one), even though the people we’re selecting aren’t being hired for their ability to be interviewed.

The first step in getting better at pre-judging is to stop pre-judging.

This takes guts, because it feels like giving up control, but we never really had control in the first place. Not if we’ve been obsessively measuring the wrong things all along.

When I talk to people about Open Badges, there’s still plenty of them who are being introduced to alternative credentialing for the first time. The idea that the way we hire people or recognise their knowledge, skills, and behaviours might be problematic, comes as a surprise. In effect, I’m saying that the large filters we use for everyone just don’t work that well.

This reminds me of Clay Shirky’s presentation almost a decade ago, the title of which is now also a well-known quotation of his:

It’s not information overload. It’s filter failure.

Right now, there’s plenty of jobs for which HR managers receive hundreds, if not thousands, of applications. I don’t know in every case how these are boiled down to decide who comes for interview, but I can’t help but think that they’re probably using the same lenses, time and time again. Although it’s not the only reason my interest in Open Badges remains strong, I do think more granular credentials can’t help but provide smaller, additional lenses that lead to greater diversity, and better person/organisation fit.

Image CC BY-NC-ND Nadar