Your Search Ends Here. What is the best Search provider for Sitecore – Part I - Internal Indexes

Sitecore is all about websites. Ideally, great websites. And there is nothing worse to a user than not being able to find what he/she is looking for in a site. Clearly, a good search experience is a central point for any website which aims to call itself a great one. And that is when the hard questions start popping in the head of any good solution architect: Which search provider should we use? Why this one and not that one? What provides the best cost benefit value?  

Don’t worry, I am here to help. I’ve spent the last five years of my career focusing on how to offer a great search experience to my Sitecore costumers. It’s time to share everything I’ve learned with the Sitecore community. Sit back, pour a cup of coffee, and let's ensure you set your customers up with the best search provider for their needs. 

Internal vs Client-facing Search 

The first thing we need to understand when it comes to search in Sitecore is that it is actually two separate things. We need to discuss them individually, since a good search option for one is not necessarily the best one for the other, and vice-versa.  

The first one is the internal search, the one which the Content Authors and Sitecore Admin users are going to be using while editing content or managing Sitecore itself. This search provider powers the internal search and is a prerequisite when dealing with Sitecore. We must have an internal search provider, otherwise Sitecore won’t work at all. It is a very basic search feature, mostly based on keywords search with some possible facets to filter results. And, it is a major help to Sitecore admin users, allowing them to find what they need when they need it, especially on big website implementations with thousands of items. 

The second one is the client-facing search experience. Different from the first one, there is no pre-defined specification here. We can get as fancy as we want. We can use keywords, facets, tabs, sort by, listing pages, whatever you need to make sure the users have the best navigation experience and find what they are looking for. And please, even though it is not a prerequisite, it’s totally against the law (mine, at least) to have an enterprise-level website without search. 

This blogpost is the first in a series where we will cover both, starting with the internal indexes first. After all, what is the best choice for you? 

Azure Search 

Sitecore started supporting Azure Search when the company released Sitecore XP 8.2 update 1, back to November 2016. Not only that, but Azure Search was the default search provider on every Sitecore Cloud instance deployed using Azure SDK. In 2021, Sitecore announced that this option is going to be discontinued soon. I know, it’s hard to understand that, but even though it used to sound like a good option, Sitecore is right about it. 

Azure Search looked like an amazing option at the first glance. The world was going full Azure back to 2016, so having to manage an external search provider while the entire application was hosted and managed on Azure did not make a lot of sense. Sitecore acted fast and brought this new feature along quickly. Everybody was happy and excited. That is, until they started implementing their first projects using it… 

While being able to handle search with a native Azure feature was great, Azure Search has technical limitations. A few limitations were just minor hindrances but others quickly became huge project blockers.  

Here is a summary of a few of them: 

• Number of fields limited to 1000.  
• Number of indexes limited to 15 (on lowest-priced options) 
• Can’t query on path fields 
• Cost can go up to a few thousand/month 

With those limitations and not being cheap, more and more people stopped using Azure Search and went back to the basics. 

Lucene 

Before Sitecore released the XP 7.0 version, Lucene used to be the way to go. Lucene is indeed a very powerful search provider, containing pretty much every feature anyone would need to provide a good search experience to the users. When it comes to Sitecore, it is no different. There are no feature limitations or even hard limitations, such as number of fields or indexes. Lucene works like a charm. 

The problem with Lucene starts when we talk about scaling Sitecore. As you may know, a Sitecore production environment consists of multiple Sitecore instances, each one playing a different role and sometimes with redundance. It’s normal to have multiple Content Delivery instances or even multiple Content Management. Lucene has to be installed on each server that plans on sending queries. That’s the problem. If each instance has its own index version, how can instance B be aware of the changes performed by instance A, and vice-versa? 

It is technically possible to create this kind of data integrity in Lucene, but it is not trivial and it is not even officially supported by Sitecore. That’s why Sitecore stopped supporting Lucene since XP 9.3. Again, a right move in my opinion. 

Solr 

Starting with Sitecore 7.0 (2014), Sitecore and Solr have had a lovely relationship. Solr brings everything positive about Lucene but also adds a lot of power with its API. We can almost look at it like a Lucene-As-A-Service, since Solr is a web application which offers search related endpoints and allows applications to query and manage the index with HTTP requests. 

Solr is more than just a wrapper around Lucene, as it brings innovations and increases the performance when it comes to search. Sitecore, for instance, suggests that Solr should always be used when dealing with more than 50k indexed items.  In my opinion, this is not a substantial number. 

From a developer perspective, Solr is like a plug-and-play solution. You still need to install it first - and there are a bunch of scripts which can automate this for you - But after you do this, Sitecore keeps the complexity behind the scenes, and everything is 100% transparent for you. You just keep doing what you are used to, creating templates and content items, and - like magic - everything is going to be searchable by any Sitecore instance through this centralized search endpoint. As easy as it can be. 

Bonus Track: Managed Solr 

As I just wrote, Solr is almost a plug-and-play solution. You still must install and maintain it. You will still need to manage scheduled backups and disaster recovery. If you are like me, all that sounds like a waste of time. There is no business value being delivered when performing those tasks. And let us be honest, it is just boring as hell. 

That is why I have been pushing a managed Solr solution on every project I work on. Ideally, the Sitecore developers on my team should be focused on creating new features and making the customer happy. If we can pay a few hundred bucks a month to not worry about everything I mentioned, it sounds like sweet music to me. 

We have been using SearchStax (https://www.searchstax.com/solutions/sitecore-solr/) to manage our Solr instances in a few projects; and so far, so good. The company offers a PowerShell script which automates the integration between Sitecore and their managed solution. It takes 5 minutes to switch from a hosted Solr to their managed one. After that, it is in their court. We can then get back to delivering business value and make the life of our customers easier. 

Conclusion 

In theory there are three options on the table, but, there is really only one. Solr is the way to go when it comes to a Sitecore internal search provider. Azure Search is deprecated, and Lucene is no longer supported, so the message is clear from them: Solr for the win. 

If you want to make it even better, go with a managed Solr solution. The price is usually super accessible and the development time you usually save pays for it. The development team is going to be happy and the customer is going to be happy, it is a win-win. 

But what about the search providers options for client-facing search? What is the best provider in the market? Stay tuned, that is going to be the subject of our next blog post.  Consult Konabos for the Search Functionality on your website