concept search in category solr

This is an excerpt from Manning's book Relevant Search: With applications for Solr and Elasticsearch.
When it comes to relevance ranking, a search engine can seem like a mystical black box. It’s tempting to ignore relevance problems—turning the focus away from search and toward other, less mystical parts of the application such as performance or the UI. Unfortunately, the work of search relevance ranking can’t be avoided. Users increasingly need to work with large amounts of content in today’s applications. Whether this means products, books, log messages, emails, vacation rentals, or medical articles—the search box is the first place your users go to explore and find answers. Without intuitive search to answer questions in human terms, they’ll be hopelessly lost. Thus, despite the maddening, seemingly mystical nature of search, you have to find solutions.
Relevant Search demystifies relevance. What exactly is relevance? It’s at the root of the search engine’s value proposition. Relevance is the art of ranking content for a search based on how much that content satisfies the needs of the user and the business. The devil is completely in the details. Ranking search results for what content? (Tweets? Products? Beanie Babies?) For what sorts of users? (Doctors? Tech-savvy shoppers?) For what types of searches? (Written in Japanese? Full of grocery brands? Filled with legal jargon?) What do those users expect? (A shopping experience? A library card catalog?) And what does your employer hope to get out of this interaction? (Money? Page views? Goodwill?) Search has become such a ubiquitous part of our applications, creeping in inch by inch without much fanfare. Answering these questions (getting relevance right) means the difference between an engaging user experience and one that disappoints.
Listing 8.12. Partial response for a search for “Star Trek” with highlighting enabled
{ '_id': '193', '_index': 'tmdb', '_score': 5.0279865, '_type': 'movie', 'fields': { 'overview': ['Captain Jean-Luc Picard ...'], 'title': ['Star Trek: Generations']}, 'highlight': { 'overview': ['renegade scientist Soran who is destroying entire <em class="hlt1">star</em> systems. Only one man can help Picard stop Soran\'s'], 'title': ['<em class="hlt1">Star</em> <em class="hlt2">Trek</em>: Generations']}}, {'_id': '201', '_index': 'tmdb', '_score': 5.0279865, '_type': 'movie', 'fields': { 'overview': ['En route to the honeymoon of William Riker ...'], 'title': ['Star Trek: Nemesis']}, 'highlight': { 'overview': ['En route to the honeymoon of William Riker to Deanna Troi on her home planet of Betazed, Captain Jean-Luc'], 'title': ['<em class="hlt1">Star</em> <em class="hlt2">Trek</em>: Nemesis']}}
Throughout this book, we’ve emphasized that, at its core, a search engine is a sophisticated token-matching and document-ranking system. We discussed techniques to ensure that search matches and ranks documents to reflect your notion of relevance. As we move on to personalized search, the fundamental nature of a search engine doesn’t change. No special magic or hidden feature makes personalization possible. Search is still about crafting useful signals, and modeling them with the search engine through queries and analysis. The main difference is that instead of drawing information exclusively from documents, you’ll look to the users themselves as a new source of information.
Maybe search is not the application you should be building. Maybe you should be building recommendations.
Consider what happens if there’s no explicit search in a personalized, concept-based search application. What if the search box is left empty and filters left unchecked? Can the application still be put to use? Yes! Even without immediate input from the user, the application has a significant amount of context that can be used to make rich recommendations. For instance, if the user looks at an item detail page, then the application can use methods discussed in this chapter to recommend related items. If the user interacted with the application in the past, the recommendations can incorporate the user’s behavioral and conceptual information so that the recommendations will be personalized. So you see, recommendation is something that can still exist without an explicit query from the user. As we’ll show in the following paragraphs, it may even be useful to think of search as a subset of recommendation.
Let’s dig farther. Think about the analogues to search and recommendation that exist in real life. When viewed in its worst possible light, a basic search application can be like a gum-chewing, disinterested, teenage store clerk. You say, “I need a shirt,” and the clerk points to a wall of shirts. There are hundreds of shirts—a mixture of all styles, sizes, and prices imaginable. It’s too much to process, so you attempt to filter the search: “Yeah, but do you have anything in size M?” The clerk (still smacking gum) glances up and points to the bottom rack. There are still a lot of shirts to choose from—a mixture of various styles and prices—but you need a new shirt, so you walk over and start looking through the shirts in your size.
Even though we’ve couched this story in terms of a search for a new shirt, the store clerk is effectively making recommendations to the customer. They’re just not particularly good recommendations. The clerk ignores the other personalization and conceptual context clues that could help direct customers to just the item they’re looking for.

This is an excerpt from Manning's book Solr in Action.
Although these are the four main characteristics of data that search engines like Solr handle efficiently, you should think of them as rough guidelines, not strict rules. Let’s dig into each to see why they’re important for search. For now, we’ll focus on the high-level concepts; we’ll get into the “how” in later chapters.
Many languages present challenges to text searching—some contain various forms of the same word that should all return in a search, some (such as French and Spanish) contain special accents, some (German and Dutch, for example) contain compound words made up of subwords, and some (such as Chinese, Japanese, and Korean) may have no whitespace between words to distinguish them from one another. Linguistic analysis can help overcome these complexities within your text by performing analysis using language-specific rules associated with the language(s) represented within your documents. Although Solr provides many preconfigured field types in its example schema.xml, these examples are designed to be used with only one language at a time. This chapter will cover the available language libraries in Solr and will also show how you can use them to create a powerful multilingual search experience that can even handle mixed-language content within a single document or field.
Multilingual documents may require dual-storing content per language field, increasing index size and subsequently slowing down searches.