Raw Value Searching and Sorting

Note that this only applies to searching within a single app and not for full site search.

By default when a field is indexed in the search index the individual words in the field value are tokenized and indexed separately. For example, say that we have a field called "Book Title" and it has a value of "This Book Has No Words". When that field is indexed in the search index the value will be broken out into it's individual words and each word is stored separately for that value. (Technically a lot more complicated stuff happens but that's a simple description). 

There are very good reasons for indexing values this way, however, depending on how you are searching or sorting by that field you can sometimes get inaccurate results. 

To fix this we also store the raw, un-tokenized value for each field in the search index. This value can be accessed using this format: APP_ATTRIBUTE_LAYOUT_KEY.raw

If our "Book Title" field has a layout key of "bookTitle" it's value in the search index can be accessed using it's layout key of "bookTitle". The raw, un-tokenized value can be accessed using "bookTitle.raw".

Most of the time you will not need to use the raw value however here are a few cases when you might:

  • if you need an exact match on a field whose value can be more than one word (i.e. it's a open ended text field like a product name)
  • if you need to sort by a field whose value can be more than one word

Search Example

Lets say that we want to search store products using the product name. We could have this for a field in our search form:

<input type="text" name="field[productName]">

If we did a search for "Cars" using that field then we would get all products that have "cars" in the product name.

However, if we wanted to find products whose name matches exactly to a search term then we would want to use "productName.raw".

<input type="text" name="field[productName.raw]">

Again, if we did a search for "Cars" using this field then we would get all products whose name is "Cars". 

Sort Example

Using the search example above lets say that we're going to search for products whose name contains the search terms (i.e. not an exact match) and we want to sort the results in ascending order by the product name. We could have the following fields in our store search form.

<input type="text" name="field[productName]">
<input type="hidden" name="sort[productName]" value="asc">

However if we did that search then we'll notice that our results aren't sorted in alphabetical order. That is because by default fields are tokenized into individual words so there isn't the whole product name phrase to sort by. To get that we need to use the raw value: "productName.raw".

<input type="text" name="field[productName]">
<input type="hidden" name="sort[productName.raw]" value="asc">

Now if we did a search using those fields the search results are returned in alphabetical order by the product name.