Monday, June 29, 2020

How to configure Dynamic Facets in Adobe Search and Promote?

Dynamic Facets

The dynamic facet is used to create new range selections automatically at the time of search. The facets are included dynamically based on the search result.

The Dynamic Facets feature is not enabled in Adobe Search&Promote, by default. Contact Technical Support to activate the feature for your use.

In our previous tutorial, we have seen how to enable static facet for the search results. Refer the below link for the details, the steps are going to be same for Dynamic facet with small changes.

Facets that are sparsely populated across your website and only appear for a subset of searches are good candidates to make dynamic.

In our example, the product with type “Watch” will have an additional attribute with name “size” and the facet associated with “size” is applicable only for the searches with keyword “Watch”.

The search with key word “Watch” will shows two facets “productType”(static) and “size”(dynamic) but the search with “Book” will shows only “productType”(static) facet.

adobe-search-and-promote-dynamic-facet

Configuring Dynamic Facet

Some additional configuration required to enable Dynamic Facet compared to Static Facet(Refer Static facet tutorial for basic configurations)

Enabled additional product attribute to the feed file based on the product Type — “size”, “size” attribute is applicable only for the productType “Watch”

<feed xmlns:xs="http://www.w3.org/2001/XMLSchema" version="2.0">
<channel>
<title>Product Feed</title>
<Item>
<link>https://www.example.com/product-title/p/prod1</link>
<title>
<![CDATA[Book Prod1 Title]]>
</title>
<description>
<![CDATA[<p>Prod1 description</p>]]>
</description>
<productType>Book</productType>
<ProductId>prod1</ProductId>
<imageUrl>/content/dam/Images/product/prod1.jpg</imageUrl>
</Item>
<Item>
<link>https://www.example.com/product-title/p/prod2</link>
<title>
<![CDATA[Book Prod2 title]]>
</title>
<description>
<![CDATA[<p>Prod2 description</p>]]>
</description>
<productType>Book</productType>
<ProductId>prod2</ProductId>
<imageUrl>/content/dam/Images/product/prod2.jpg</imageUrl>
</Item>
<Item>
<link>https://www.example.com/product-title/p/prod3</link>
<title>
<![CDATA[Watch Prod3 Title]]>
</title>
<description>
<![CDATA[<p>Prod3 description</p>]]>
</description>
<productType>Watch</productType>
<size>10</size>

<ProductId>prod3</ProductId>
<imageUrl>/content/dam/Images/product/prod3.jpg</imageUrl>
</Item>
<Item>
<link>https://www.example.com/product-title/p/prod4</link>
<title>
<![CDATA[Watch Prod4 Title]]>
</title>
<description>
<![CDATA[<p>Prod4 description</p>]]>
</description>
<productType>Watch</productType>
<size>20</size>

<ProductId>prod4</ProductId>
<imageUrl>/content/dam/Images/product/prod4.jpg</imageUrl>
</Item>
</channel>
</feed>

Create a new meta data definition for “size” field, Settings → Metadata →Definitions

Enable Dynamic Facet option for the metadata

Update the IndexConnector configurations with new meta data field “size”, Settings → Crawling →Index Connector

adobe-search-and-promote-dynamic-facet

Configure facet with name “size” — there is no “Dynamic Facet” setting in Facet, only the configuration is in the underlying metadata configuration(already enabled), Design →Navigation →Facets

adobe-search-and-promote-dynamic-facet

Configure a new Query Cleaning rule, Rules → Query Cleaning , to set the backend parameter “sp_sfvl_df_count”, the sp_sfvl_df_count parameter
determines the total number of dynamic facet fields to return.

adobe-search-and-promote-dynamic-facet

Update the back end transport with Dynamic Facet Support


<search-content-type-header charset="UTF-8">
{ 
	"general": {
        "query" : "<search-query />",		
        "total" : "<search-total />",
        "lower" : "<search-lower />",
        "upper" : "<search-upper />"

	},
	"facets" : [
        {
            "name" : "productType",
            "values" : [<search-field-value-list name="productType" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name="productType" quotes="no"  data="results" sortby="values" />]
        }<search-dynamic-facet-fields>,
        {
            "name" : "<search-dynamic-facet-field-name>",
            "dynamic-facet" : 1,
            "values" : 
                    [<search-field-value-list quotes="yes" commas="yes" data="values" sortby="values" encoding="json" />],
                    "counts" : [<search-field-value-list quotes="yes" commas="yes" data="results" sortby="values" />]
        }</search-dynamic-facet-fields>
    ],

	"results" : [
		<search-results>
		{
			"fields" :
			[
				{
					"name" : "mdi",
					"value" : "<search-display-field name="mdi" length="500"  encoding="json" />"
				},

				{
					"name" : "title",
					"value" : "<search-display-field name="title" encoding="json" />"
				},
				{
					"name" : "productType",
					"value" : "<search-display-field name="productType" encoding="json" />"
				},
				{
					"name" : "size",
					"value" : "<search-display-field name="size" encoding="json" />"
				}
            ]		
        } 
        <search-if-not-last>,</search-if-not-last>
		</search-results>
	]
}

The configurations are ready, let us now run a Stage indexing (note the configuration are not pushed to live yet)

Index →Full Index →Staged Index →Run Full Index

adobe-search-and-promote-dynamic-facet

Access http://stage-xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_staged=1&sp_q=Watch

xxxxxxxxxxx — Search and Promote account number

This will respond with JSON data which contains the required Static, Dynamic facet and meta data details.

{
"general": {
"query": "",
"total": "2",
"page_lower": "1",
"page_upper": "2",
"page_total": "1"
},
"facets": [
{
"label": "productType",
"long": false,
"values": [
{
"value": "Watch",
"selected": "false",
"count": "2",
"link": "?do=json;i=1;q1=Watch;sp_q=Watch;sp_staged=1;x1=productType",
"undolink": "",
"threshold": false
}
]
},
{
"label": "size",
"long": false,
"values": [
{
"value": "10",
"selected": "false",
"count": "1",
"link": "?do=json;i=1;q1=10;sp_q=Watch;sp_staged=1;x1=size",
"undolink": "",
"threshold": false
},
{
"value": "20",
"selected": "false",
"count": "1",
"link": "?do=json;i=1;q1=20;sp_q=Watch;sp_staged=1;x1=size",
"undolink": "",
"threshold": false
}
]
}
]
,
"results": [
{
"index": "",
"title": "Watch Prod3 Title",
"productType": "Watch",
"size": "10"
},
{
"index": "",
"title": "Watch Prod4 Title",
"productType": "Watch",
"size": "20"
}
]
}

The configuration can be pushed live after successful validation and run a live index →Full Index →Live Index →Run Full Index

adobe-search-and-promote-dynamic-facet

The URL to access live data http://xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_q=Watch

The facet data in the response can be used to present the filtering options to users to narrow down the website search. The Dynamic Facet option enables the facets based on the search data.

Sunday, June 28, 2020

How to implement autocompletion and search suggestion in AEM through Lucene | Predictive Search in AEM | AEM Search Suggestions

How to implement autocompletion and search suggestion in AEM through Lucene | Predictive Search in AEM | AEM Search Suggestions


This tutorial explain the approach to implement autocompletion and search suggestion in AEM through Lucene.

When you start typing something in search form most of the applications helps you by suggesting the data matching to your search term.

aem-autocompletion-search-suggestion

The purpose of autocomplete is to resolve a partial query , i.e., to search within a controlled vocabulary for items matching a given character string.

Starting from AEM 6.1 the feature of suggestion is available through the suggest module of Lucene. Prior to AEM 6.1, all the possible combination of the words needs to be indexed to support the autocompletion.

The Lucene Suggest module provides a dedicated and optimized data structure allows the engine to give autocompletion and suggestion feature without indexing all the possible combination of a word. 

There is a specific analyzer (AnalyzingInfixSuggester) used that loads the completion values from the indexed data and then build the optimized structure in memory for a fast lookup. 

In order to implements the autosuggestion, feature you need to define an index of type Lucene and for each property X of nodes that you are indexing you can add a specific property useInSuggest to tell to the engine to use X for suggesting query to the user.

Refer the following URL for details on enabling custom index - https://www.albinsblog.com/2020/04/oak-lucene-index-improve-query-in-aem-configure-lucene-index.html  

I have already enabled a custom Lucene index(testindex) for the property "id", add a property "useInSuggest" to tell the engine to use id for suggesting query to the user.

aem-autocompletion-search-suggestion


An additional property suggestUpdateFrequencyMinutes define the frequency of updating the indexed suggestions - useful to mitigate performance issues that can arise if indexed properties are frequently updated by the users of your application. The default value is 10 minutes but the values can be modified as required.

To enable the property "suggestUpdateFrequencyMinutes ", create a node with name "suggest" of type "nt:unstructured" under "testindex" and update the value as required

aem-autocompletion-search-suggestion


 In order to use Lucene index to perform search suggestions, the index definition node (the one of type oak:QueryIndexDefinition) needs to have the compatVersion set to 2. 

aem-autocompletion-search-suggestion


Let us now execute the query to find the suggestions - either one of the below query can be used.

The testindex was defined for the content path "/content/sampledata" so the query will be executed based on the "testindex" but the index name is explicitly defined in the first query.

aem-autocompletion-search-suggestion


SELECT [rep:suggest()] FROM [nt:unstructured] WHERE SUGGEST('te') OPTION(INDEX NAME [testindex]) /* oak-internal */ 

SELECT [rep:suggest()] FROM [nt:unstructured] WHERE SUGGEST('te') AND ISDESCENDANTNODE('/content/sampledata')

The above query uses path restriction to filter the data, it requires evaluatePathRestrictions property should enabled as true on index definition.

aem-autocompletion-search-suggestion


The Query tool shows the total number of unique suggestions matching with the search data but it wont displays the matching node details

aem-autocompletion-search-suggestion


The below Servlet can be used to fetch the suggestion data through QueryManager API
import java.io.IOException;

import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.jcr.query.InvalidQueryException;
import javax.jcr.query.Query;
import javax.jcr.query.QueryManager;
import javax.jcr.query.QueryResult;
import javax.jcr.query.Row;
import javax.jcr.query.RowIterator;
import javax.servlet.Servlet;
import javax.servlet.ServletException;

import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.servlets.HttpConstants;
import org.apache.sling.api.servlets.SlingSafeMethodsServlet;
import org.json.JSONArray;
import org.osgi.framework.Constants;
import org.osgi.service.component.annotations.Component;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component(immediate = true, service = Servlet.class, property = {
		Constants.SERVICE_DESCRIPTION + "=Custom Root Mapping", "sling.servlet.methods=" + HttpConstants.METHOD_GET,
		"sling.servlet.paths=" + "/bin/getSuggestions", "service.ranking=" + 100001 })
public class SuggestionData extends SlingSafeMethodsServlet {

	/**
	* 
	*/
	private static final long serialVersionUID = 1L;

	protected void doGet(final SlingHttpServletRequest req, final SlingHttpServletResponse resp)
			throws ServletException, IOException {

		Logger logger = LoggerFactory.getLogger(this.getClass());
		logger.error("inside custom servlet");

		final Session session = req.getResourceResolver().adaptTo(Session.class);

		final JSONArray suggestions = new JSONArray();

		String queryString = "SELECT [rep:suggest()]  FROM [nt:unstructured] WHERE "
							 +"SUGGEST('te') OPTION(INDEX NAME [testindex]) /* oak-internal */ ";

		try {
			QueryManager queryManager = session.getWorkspace().getQueryManager();
			Query query = queryManager.createQuery(queryString, Query.JCR_SQL2);
			QueryResult result = query.execute();
			RowIterator rows = result.getRows();

			while (rows.hasNext()) {
				suggestions.put(((Row) rows.next()).getValue("rep:suggest()").getString());
			}

		} catch (InvalidQueryException e) { // TODO Auto-generated catch block
			e.printStackTrace();
		} catch (RepositoryException e) { // TODO Auto-generated
			e.printStackTrace();
		} finally {
			session.logout();

		}

		resp.setContentType("application/json");
		resp.getWriter().write(suggestions.toString());

	}

}

aem-autocompletion-search-suggestion


This suggestion data cab be used to display  the search autocompletion/suggest data to the website users.



Tuesday, June 23, 2020

How to configure Navigation Facet in Adobe Search and Promote?

How to configure Navigation Facet in Adobe Search and Promote?

The Facets can be used to customize your presentation layer and provide your users with a Guided Search that lets them drill down into their search results.
This tutorial explains the approach to configure the facets in Adobe Search & Promote.
The website users can narrow down their search results with enabled facets, there are two types of facets in Search and Promote — Static and Dynamic, we are going to use Static facets for this tutorial.
Refer the below screen shot for example on the facets, the users can narrow down the search based on the brands they interested.



Facet Configuration

The meta data’s can be managed from Settings →Metadata →Definitions

adobe-search-and-promote-facet

adobe-search-and-promote-fccet

The metadata crawled and indexed through URL entry points — URL entry point can be a website URL or feed data through Index Connector, using Index Connector for this tutorial.
The met data is ready now let us configure a facet based on the meta data defined “productType”
Navigate to Design →Navigation →Facets

adobe-search-and-promote-facet

Add a new Facet


adobe-search-and-promote-facet

Facet Name — Select the meta data filed based on which the facet should be defined.
Display Label — Enter display label
Behavior — there are different behaviors like Normal, Category, Category Multi-Select, Sticky and Multi-Select, I am going with Normal behavior for this tutorial
Normal — When a customer clicks a facet, whose behavior is set to Normal, it drills into the search results for that item. From there, the customer can further refine and narrow the number of search results.
Category — Category facets act like navigational elements. These facets are top-level facets that customers typically drill through before revealing facets with attribute options. Category facets do not narrow when other facets are selected and remain open. Clicking a different value within a category facet deselects all other facets on the page except for that category facet’s parents.
Category Multi-Select — facets are category facets that support the selection of multiple items from the facet where the items are “ORed” together.
Sticky — When a customer clicks a facet, whose behavior is set to Sticky, the facet with the selected option remains open during the drill-down. This option is useful when you want to let a customer changes a previous choice.
Multi-Select — Allows the selection of multiple items from a facet, where the items within the facet are “ORed” together. This option is useful for a facet that may show a minor attribute such as colors and you want to let the customer have the ability to build a query that lets them “show shoes in my size that are red or black”.
Show Always — For a normal or sticky facet, sets the facet to remain visible to the customer at all times. This option is only available if you selected Normal, Category, or Sticky from the Behavior drop-down list.
Refer the UI for details on other fields, I am only configuring the Facet Name and Facet Label for this tutorial, enable other fields as required.
Click Add, now the facet is enabled based on the meta data field “Product Type”


adobe-search-and-promote-facet

JSON Transport Template

Created a JSON template(custom_backend_json.tpl) with minimal data to support the search data
The new template can be added by navigating to Designs à Templates à Add Template
Select Template Type as Transport


adobe-search-and-promote-facet

Sample Template Content with minimal configuration– modify based on your requirement, even you can use the existing Transport Template with required customization
<search-content-type-header charset="UTF-8">
{
"general": {
"query" : "<search-query />",
"total" : "<search-total />",
"lower" : "<search-lower />",
"upper" : "<search-upper />"
},
"facets" : [
{
"name" : "productType",
"values" : [<search-field-value-list name="productType" quotes="yes" data="values" sortby="values" encoding="json" />],
"counts" : [<search-field-value-list name="productType" quotes="no" data="results" sortby="values" />]
}
],
"results" : [
<search-results>
{
"fields" :
[
{
"name" : "mdi",
"value" : "<search-display-field name="mdi" length="500" encoding="json" />"
},
{
"name" : "title",
"value" : "<search-display-field name="title" encoding="json" />"
},
{
"name" : "productType",
"value" : "<search-display-field name="productType" encoding="json" />"
}
]
}
<search-if-not-last>,</search-if-not-last>
</search-results>
]
}
Enabled with a single facet — update the template with additional required facets, also add the additional meta data’s and other configurations e.g. breadcrumbs, menus etc
e.g
"facets" : [
        {
            "name" : "productType",
            "values" : [<search-field-value-list name="productType" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name="productType" quotes="no"  data="results" sortby="values" />]
        },{"name" : "facet2",
            "values" : [<search-field-value-list name=" facet2" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name=" facet2" quotes="no"  data="results" sortby="values" />]
        }
    ],

JSON Presentation Template

Created a JSON template(custom_presentaion_json.tmpl) with minimal data to present the search data to JSON
<guided-content-type-header content="application/json" />
<guided-if-query-param-defined gsname="callback" /><guided-query-param gsname="callback" />(</guided-if-query-param-defined>
 {"general" :{
            "query" : "<guided-query-param gsname='q' />",
            "total" : "<guided-results-total />",
            "page_lower" : "<guided-results-lower>",
            "page_upper" : "<guided-results-upper>",
            "page_total": "<guided-page-total/>" 
        },  
        "facets" :
        [<guided-zone gsname="Facets_Default"><guided-facet  gsname="productType" >{ 
                        "label" : "<guided-facet-display-name gsname="productType" />",
                        "long" : <guided-if-facet-long>true<guided-else-facet-long>false</guided-if-facet-long>,
                        <guided-if-facet-selected>
                        "selected" : true,
                        "undolink" : "<guided-facet-undo-path gsname="productType" />",
                        </guided-if-facet-selected>
                        "values" : 
                        [
                            <guided-facet-values>
                            {
                                "value" : "<guided-facet-value  escape="ijson">",
                                "selected" : "<guided-if-facet-value-selected>true<guided-else-facet-value-selected>false</guided-if-facet-value-selected>",
                                "count" : "<guided-facet-count>",
                                "link" : "<guided-facet-value-path>",
                                "undolink" : "<guided-facet-value-undo-path>",
                                "threshold" : <guided-if-facet-value-equals-length-threshold>true<guided-else-facet-value-equals-length-threshold>false</guided-if-facet-value-equals-length-threshold>
                            }<guided-if-not-last>,</guided-if-not-last>
                            </guided-facet-values>
                        ] 
                    }
                    </guided-facet>
            </guided-zone>
        ],
        "results" :
        [ 
            <guided-results gsname="default">
            {
                "index" : "<guided-result-index />",
                "title" : "<guided-result-field gsname="title" escape="ijson" />",
                "productType" : "<guided-result-field gsname="productType" escape="ijson" />"
            }<guided-if-not-last>,</guided-if-not-last>
            </guided-results>
        ]
    }
<guided-if-query-param-defined gsname="callback">)</guided-if-query-param-defined>

Pre-Search Rule

Define a pre-search rule to configure the presentation and Transport template for the search, the rule is execute when the search request has a parameter “do=json”(the rule can be fired for every search request also)


adobe-search-and-promote-facet
adobe-search-and-promote-facet

The configurations are ready, let us now run a Stage indexing (note the configuration are not pushed to live yet)
Index →Full Index →Staged Index →Run Full Index


adobe-search-and-promote-facet

Access http://stage-xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_staged=1&sp_q=*
xxxxxxxxxxx — Search and Promote account number
This will respond with JSON data which contains the required facet and meta data details.
{
   "general": {
      "query": "",
      "total": "3",
      "page_lower": "1",
      "page_upper": "3",
      "page_total": "1"
   },
   "facets": [
      {
         "label": "productType",
         "long": false,
         "values": [
            {
               "value": "Sample1",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample1;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            },
            {
               "value": "Sample2",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample2;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            },
            {
               "value": "Sample3",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample3;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            }
         ]
      }
   ],
   "results": [
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample1"
      },
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample2"
      },
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample3"
      }
   ]
}
The configuration can be pushed live after successful validation and run a live index →Full Index →Live Index →Run Full Index


adobe-search-and-promote-facet

The URL to access live data http://xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_q=*
The facet data in the response can be used to present the filtering options to users to narrow down the website search.