Selecting a Search Provider for Jamstack

Transcript

Note: The following is the transcription of the video produced by an automated transcription system.

Kamruz Jaman: Hey, Akshay. Good to see you again. I want to talk to you about selecting search providers for Jamstack. Now you and I have both come from a traditional monolithic CMS background in various guises. And we don't usually think too much or too hard about search. The search is whatever is built into the platform is whatever is built into the product, and we would usually just extend that. And maybe you have one or two additional extension options for those for those traditional platforms. But with the Jamstack approach this this changes, right? Because you're using a headless CMS, which just provides content and then you're piecing it together with other services that you may need. Right. And so how do we go about selecting a search provider in the Jamstack world? And what are the what are some of the gotchas and what are some of the things that we should be should be watching out for?

Akshay Sura: Yeah, and I mean, like you said with the monolith, you can add any provider you want, but it won't be easy because you have to write the integration yourself, right? But when you're thinking about from a headless Jamstack perspective, the world is your oyster. Everything is API based, so you index what you want. And one of the biggest things I think is you need to find a vendor. And I didn't put it on this slide I should have is the API. So how easy it is to do the integration and I do have integration in there. But like you said,

Kamruz Jaman: We always want to be looking for. Jamstack compatible right to the MACH, the MACH compatible services, when we are integrating our API first services together just to make our own lives easier, so if we stick with everything which is fully API driven, we can usually make those things work together, right?

Akshay Sura: Yeah, absolutely. And then the best part is you can index whatever you want. You can use whatever you want. All of these vendors in the space for Jamstack, MACH Alliance based providers API first, they all have very similar features. So we just wanted to go over a few things with you guys. So in terms of. Finding the right provider for you is really based on what your needs are, but I know usually when our departments are looking for procurement, you're not just looking at your current needs, but future needs and what you could use. So most of them, like I said, provide similar features. But to be able to visualize your data, have a nice dashboard where non-technical people can look at what's happening, the analytics reporting part to see which keywords are doing well, which needs more work, how many of the results are coming up with no results and try to focus on that from a technical perspective, you want to keep track of the availability and the security. Being able to provide geo based searches, alerts and monitoring integrations is a big, big deal. So being able to integrate with other systems is super critical, especially in the Headless space. And a lot of the providers do provide mobile and web searches. So in order for you to render them, highlight the search queries, tune your relevance tuning. And again, a lot of them are SaaS based, but there are some options where you could self hosted. So if you're kind of like an org which has restrictions depending on the sector you're working in, you might have to host it locally in a specific data center in a specific country. So all of those and then you have bells and whistles like workflows and machine learning in order to predict what your customer is trying to search for and provide features, according to that. So really, the features are what exactly you're looking for now and the future of your company.

Kamruz Jaman: Yeah, that hosting piece is an important one, right? Because we when we started looking, we kind of flipped between SaaS based and then self-hosted and back to a SaaS solution. In the end, it, as you said, it depends on the industry, the company's policies around security and some of those licensing agreements as well. But yet I think it can. That's the flexibility of having this Jamstack approach as you can pick self-hosted, fully managed. You can pick different data center locations and you can pick completely different technologies as to who your search provider is, right? You know you're not tied to one specific provider at all. You can use some or all of these features and then build on them over time as well.

Akshay Sura: Yeah, and the beauty of that is most of these providers give you because it's API. First, they already give you a widget for facets and for dropdown search like type of head searching and stuff. And depending on like they even give you a pre-built libraries for Node. So all the all the React based or Vue based, you could straightaway integrate or a .NET based app. So that's really awesome. One of the. So, you know, looking at the drawbacks based on our experience. So this kind of caught us off guard a little bit. So when we were looking at the number of most of these providers restrict you by requests. So a search request is one or more search operations, which is done in like a single call. So if you're trying to search for, for instance, a product wipe or glove or whatever that is, and then you have a bunch of restrictions and you send it to the search provider. And so if you get me everything and that's construed as one request. So if your customers on your sites are doing a lot of search requests and there's a lot of traffic on your site, this will get bumped up.

Akshay Sura: Where we got kind of screwed with this is that we in our implementation new search, not just for site search, which is a web crawl, but also in terms of rendering of the product listing pages and the search results page. And we were doing about three to five million queries a day and it extremely becomes cost restricted. But you're talking about like 50 million or so in a week and then you end the month. And when you go from the cost pricing for that, the request per month goes through the roof and it's a big, big no-no. So in this case, for that specific customer, we ended up choosing a fully managed search instance which did not restrict us in terms of, you know, the amount of requests we made, we could make as many. There was no network bandwidth restriction. There was no request for a month restriction. So it's an interesting gotcha. But you got to lean into it and choose the right provider who doesn't, you know, based on your site needs, you have to work on it.

Kamruz Jaman: And so does the request, so when you does a request, per month also count towards when you're indexing that data to begin with as well.

Akshay Sura: No, thankfully. So that we can go to the next one so I can explain both at the same time. So yes and no, so what they do and some do this, they do either or either it's the record store per month or it's the amount of search queries you make. And that's kind of like a double edged sword because typically based on your org, the record stored are probably not that much a hundred thousand two hundred thousand, maybe even a million two million records. But if a site's taking a lot of traffic, the request for one goes up. So it's either or. And the kind of thing which was kind of difficult for us is predictability of which one was going to be the net usage for that particular month, right? So from a client perspective, you have to figure out what's it going to cost us and that's the next slide, unfortunately. But the costs are you need a predictable cost. So if you're pretty straightforward, you kind of know everything in advance, OK, I have a million records stored and I'm probably going to do two million or so search requests per month. You're pretty set. It's pretty good. But in terms of growing traffic, SEO getting better. Predictability gets to be difficult. So pick a provider where you can predict the cost because operating costs are something which is really, really important. Initial costs, not so much. You know, getting into any of these providers, it's really, really cheap. Some even offer a free plan to get into and then you can judge what you want to do and get into the appropriate tier. But predictability of cost is the biggest thing in this space.

Kamruz Jaman: Yeah, I think one of the good things about these API based providers is you can play around with them, as you said, that you know that a lot of them will provide you free accounts so that you can just go in and have a create POC. And even as you're doing that initial development and trying to figure things out because it is API based, they're fairly easy to switch out. I'm not going to say they're really easy, but they're fairly easy to switch up because they all follow a similar kind of pattern with the APIs.

Akshay Sura: Yeah, no, absolutely. So yeah, just be careful. Try doing as much work, so maybe you can look at your existing system to see, oh, how many queries are running, and most often than not, it's kind of hard for you to come up with that number because depending on the monolith or the system you're currently in use, you might have to add additional code or features or look through logs or something like that to figure out how many am I running and it might surprise you, is what all I can say.

Kamruz Jaman: Yeah, absolutely.

Akshay Sura: All right. Hope this helped Kamruz.

Kamruz Jaman: Yeah, definitely. There's a lot of good information in there.

Akshay Sura: All right. Have a good rest of your day.

Kamruz Jaman: Your week, guys.

If you have any questions, please get in touch with me. @akshaysura13 on Twitter or on Slack.