Matching Algorithm

Because I was the Acting CTO for eHarmony at it's start, I quite often get introduced to people who have an idea for a startup company that is based on some kind of matching algorithm.  They describe the company as the eHarmony of careers, clothes, jobs, college, tutoring services, doctors, service companies, investments, etc.  In fact, you can get a good idea of these various things by just searching on Google for "eHarmony of" startup.

Each of these startup ideas has at it's core a matching algorithm that reduces the friction between a consumer and some need.  In eHarmony's case, it was the friction of finding a compatible marriage partner.  In the case of college matching, it's the complexity of what college matches best with the person's needs.

As I've had more than a hundred conversations with entrepreneurs who are planning to build a company around a matching algorithm, I thought it would be worth capturing a few thoughts that commonly come up.

Margin in Mystery

An Angel investor that used to be on a startup CEO roundtable with me, always had a lot of great phrases that would help startups.  One of my favorites was, "There's margin in mystery."  What he meant by this is that anything that's too obvious to the consumer can be easily evaluated for it's value and often then suffers from low margins.  The flip side is that if you can offer something that's not at all clear how you are doing it, then people perceive greater value.

An example that relates directly to creating a matching algorithm, is the classic Myers-Briggs Type Inventory.  The experience is often that after you've answered a whole bunch of questions, it comes back with a description of you that seems eerily accurate.  For example, after taking a similar personality test, the person who came to give me feedback walked into the room with "Tony, you like going to bookstores don't you."  And I certainly do, but the test never asked me anything about bookstores.  The feedback had lots of other items that somewhat "nailed me."

Certainly eHarmony relies on this.  They come back with their free personality profile that nails you.  It gives you confidence that they understand you and what would make a good potential marriage partner.  And what they tell you about how you will be in a relationship do not directly come from any questions they asked you.  That feels powerful.  It's a bit mysterious.

Now, if the assessment that I took had asked me – "Do you like to go to bookstores?"  and the assessment echoed what I had answered to that question, then the algorithm is obvious.  There's no mystery.  And I will perceive lower value.

This is really important when it comes to a matching algorithm, because I'm often presented a matching algorithm that really isn't a matching algorithm at all.  It's really just a simple filtered search.  For example, if you are going to be building the best matching algorithm for high school students looking to find the right college, but it is based on criteria that are a search (geographical location, majors offered), no one is going to ascribe greater value.  You may have a perfectly fine business, but it's not going to be differentiated based on that simple algorithm.

Instead, what you need to have is something like leveraging personality characteristics of students who have been successful at that college, or maybe common life ambitions of students who say they are happy there, or ???

Requires Data

You will notice that the suggestions I made on how to increase the mystery in a matching algorithm also just created a need for data.  In order to successfully match students to colleges where they will like it and be successful, you probably should have a lot of data from students who have already attended that college, whether they liked it and were successful, and their personality profile, or life ambition or whatever you plan to use to match people.

A lot of people I talk to about their matching algorithm don't know that eHarmony (more specifically Neil Clark Warren) had years of scientific research that were the basis of his dimensions of compatibility.  These dimensions related to personality, values, likes/dislikes, etc.  And related to each of these dimensions they had been doing years of research to determine what combinations produced happy/unhappy marriages as well as how long the marriages lasted.  It was a lot of data that they was distilled down into a fairly complex matching algorithm.

Many of the startups that I talk to don't have any of that kind of data.  You can maybe find a researcher or something to use as a proxy.  You can make educated guesses.  You can start to collect the data as part of the system.  But without the foundation, you are likely going to have trouble creating something that will truly have mystery.

That said, I will admit that there have been a few ingenious entrepreneurs who had a matching algorithm that had mystery and no data yet appeared to be fairly valid and valuable.

Broad Value and Appeal

My guess is that if I discussed this with fellow computer scientists, they would argue that many of the ideas I'm calling a matching algorithm really are not strictly a matching algorithm.  While strictly speaking that's true, I think it's misses the broad value and appeal that this kind of algorithm provides.  Whether or not it's truly a matching algorithm, they have broad value and appeal because they apply anywhere that there's a fairly large sets of options, complexity to those options, and friction in reducing the set and dealing with the complexity.

This is why there are so many "eHarmony of" companies. 

And I actually think this is going to grow significantly!

If you think about what's going on with the web, we've reached a point where everyone and everything is connected, it's represented online, it's creating content.  The numbers are growing rapidly.  The amount of data we have about it is growing rapidly.  Our choices are growing.

Yet most of us are much happier with fewer choices.  Actually, that's not strictly accurate.  We are happier with a smaller list of well vetted, reasonable choices.

A matching algorithm is at the heart of how you deal with scale and complexity.

It comes up all the time. 

• Who should I meet here in Los Angeles professionally that will be an interesting conversation and a potentially valuable contact?
• What entrepreneurs / startup companies would benefit from talking to me?
• Who should read my blog?  Who's blog should I read?

And, again, while the line is fuzzy, this also all relates to issues like social filtering and Curator Editor Research Opportunities on eLearning Learning.  The challenge is how you can help a consumer make sense of a large complex space.  This is only going to get more interesting.

And, because of the web, there are all kinds of new sources of the data that a matching algorithm can use.