Preface

John and I met while working together as consultants for OpenSource Connections (OSC) solving tough search problems for clients. Sometimes we triaged performance (make it go faster!). Other times we helped build out a search application. All of these projects had simple-to-measure success metrics. Did it go faster? Is the application complete?

Search relevance, though, doesn’t play by these rules. And users, raised in the age of Google, won’t tolerate “good enough” search. They want “damn smart” search. They want search to prioritize criteria they care about, not what the search engine often idiotically guesses relevant.

Like moths attracted to a flame, we both felt drawn to this hard problem. And just like said moths, we often found ourselves burned. Through these painful lessons, we persevered and grew, succeeding at tasks we initially considered too difficult.

During this time, we also found our voices on OSC’s blog. We realized that little was being written about search relevance problems. We developed ideas such as testdriven relevancy. We documented our headaches, our problems, and our triumphs. Together we experimented with machine learning approaches, like latent semantic analysis. We dove into Lucene’s guts and explored techniques for building custom search components to solve problems. We began exploring information retrieval research. As we learned more techniques to solve hard problems, we continued to write about them.