Searching parts of words with elasticsearch

I figured out how to search parts of words with elasticsearch. If you are not familiar with the terms used in the documentation this can be quite a challenge. I will show you the key parts of the configuration you need.

First, you need to know a little about the search engine. When you store documents, the text is indexed. This is done by tokenizing the text, basically chopping it up and then running those chops through one or more filters. When searching the same happens to the search terms (chop / filter / used for search).

The tokenization and filtering for indexing and searching can be configured independently. Most of the time these configurations are to be kept the same, as illustrated by this case that uses a lowercase filter:

• text to be indexed: Lorem Ipsum,
• tokenized: 'Lorem', 'Ipsum'
• filtered: 'lorem', 'ipsum'

If you search for this text (using the lowercase filter again):

• text you search for: 'Lorem'
• tokenized: 'Lorem'
• filtered: 'lorem'

Notice that both the indexed text and the search term are lowercase and will thus match. If you would not configure a lowercase search filter this would not return a result.

In the case where you want to search for parts of words however, the filters used for index time differ from the filters used for search. Here is the elasticsearch configuration for creating an index, that defines the filters to be used for indexing and for searching:

The numbers correspond to the critical parts of the configuration (explained below).

1. At index time, we use the translation filter, that is of type ngram, declared at (3).
2. Notice that for the search we DO NOT use the ngram filter
3. The declaration of the ngram filter.