Dynamic Facets

The dynamic facet is used to create new range selections automatically at the time of search. The facets are included dynamically based on the search result.

The Dynamic Facets feature is not enabled in Adobe Search&Promote, by default. Contact Technical Support to activate the feature for your use.

In our previous tutorial, we have seen how to enable static facet for the search results. Refer the below link for the details, the steps are going to be same for Dynamic facet with small changes.

Facets that are sparsely populated across your website and only appear for a subset of searches are good candidates to make dynamic.

In our example, the product with type “Watch” will have an additional attribute with name “size” and the facet associated with “size” is applicable only for the searches with keyword “Watch”.

The search with key word “Watch” will shows two facets “productType”(static) and “size”(dynamic) but the search with “Book” will shows only “productType”(static) facet.

Configuring Dynamic Facet

Some additional configuration required to enable Dynamic Facet compared to Static Facet(Refer Static facet tutorial for basic configurations)

Enabled additional product attribute to the feed file based on the product Type — “size”, “size” attribute is applicable only for the productType “Watch”

<feed xmlns:xs="http://www.w3.org/2001/XMLSchema" version="2.0">
 <channel>
  <title>Product Feed</title>
  <Item>
   <link>https://www.example.com/product-title/p/prod1</link>
   <title>
   <![CDATA[Book Prod1 Title]]>
   </title>
   <description>
   <![CDATA[<p>Prod1 description</p>]]>
   </description>
   <productType>Book</productType>
   <ProductId>prod1</ProductId>
   <imageUrl>/content/dam/Images/product/prod1.jpg</imageUrl>
  </Item>
  <Item>
   <link>https://www.example.com/product-title/p/prod2</link> 
   <title>
   <![CDATA[Book Prod2 title]]>
   </title>
   <description>
   <![CDATA[<p>Prod2 description</p>]]>
   </description>
   <productType>Book</productType>
   <ProductId>prod2</ProductId>
   <imageUrl>/content/dam/Images/product/prod2.jpg</imageUrl>
  </Item><Item>
   <link>https://www.example.com/product-title/p/prod3</link>
   <title>
   <![CDATA[Watch Prod3 Title]]>
   </title>
   <description>
   <![CDATA[<p>Prod3 description</p>]]>
   </description>
   <productType>Watch</productType>
    <size>10</size>
   <ProductId>prod3</ProductId>
   <imageUrl>/content/dam/Images/product/prod3.jpg</imageUrl>
  </Item>
  <Item>
   <link>https://www.example.com/product-title/p/prod4</link>
   <title>
   <![CDATA[Watch Prod4 Title]]>
   </title>
   <description>
   <![CDATA[<p>Prod4 description</p>]]>
   </description>
   <productType>Watch</productType>
    <size>20</size>
   <ProductId>prod4</ProductId>
   <imageUrl>/content/dam/Images/product/prod4.jpg</imageUrl>
  </Item>
  </channel>
</feed>

Create a new meta data definition for “size” field, Settings → Metadata →Definitions

Enable Dynamic Facet option for the metadata

Update the IndexConnector configurations with new meta data field “size”, Settings → Crawling →Index Connector

Configure facet with name “size” — there is no “Dynamic Facet” setting in Facet, only the configuration is in the underlying metadata configuration(already enabled), Design →Navigation →Facets

Configure a new Query Cleaning rule, Rules → Query Cleaning , to set the backend parameter “sp_sfvl_df_count”, the sp_sfvl_df_count parameter
determines the total number of dynamic facet fields to return.

Update the back end transport with Dynamic Facet Support

<search-content-type-header charset="UTF-8">
{ 
	"general": {
        "query" : "<search-query />",		
        "total" : "<search-total />",
        "lower" : "<search-lower />",
        "upper" : "<search-upper />"

	},
	"facets" : [
        {
            "name" : "productType",
            "values" : [<search-field-value-list name="productType" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name="productType" quotes="no"  data="results" sortby="values" />]
        }<search-dynamic-facet-fields>,
        {
            "name" : "<search-dynamic-facet-field-name>",
            "dynamic-facet" : 1,
            "values" : 
                    [<search-field-value-list quotes="yes" commas="yes" data="values" sortby="values" encoding="json" />],
                    "counts" : [<search-field-value-list quotes="yes" commas="yes" data="results" sortby="values" />]
        }</search-dynamic-facet-fields>
    ],

	"results" : [
		<search-results>
		{
			"fields" :
			[
				{
					"name" : "mdi",
					"value" : "<search-display-field name="mdi" length="500"  encoding="json" />"
				},

				{
					"name" : "title",
					"value" : "<search-display-field name="title" encoding="json" />"
				},
				{
					"name" : "productType",
					"value" : "<search-display-field name="productType" encoding="json" />"
				},
				{
					"name" : "size",
					"value" : "<search-display-field name="size" encoding="json" />"
				}
            ]		
        } 
        <search-if-not-last>,</search-if-not-last>
		</search-results>
	]
}

The configurations are ready, let us now run a Stage indexing (note the configuration are not pushed to live yet)

Index →Full Index →Staged Index →Run Full Index

Access http://stage-xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_staged=1&sp_q=Watch

xxxxxxxxxxx — Search and Promote account number

This will respond with JSON data which contains the required Static, Dynamic facet and meta data details.

{
  "general": {
    "query": "",
    "total": "2",
    "page_lower": "1",
    "page_upper": "2",
    "page_total": "1"
  },
  "facets": [
    {
      "label": "productType",
      "long": false,
      "values": [
        {
          "value": "Watch",
          "selected": "false",
          "count": "2",
          "link": "?do=json;i=1;q1=Watch;sp_q=Watch;sp_staged=1;x1=productType",
          "undolink": "",
          "threshold": false
        }
      ]
    },
    {
      "label": "size",
      "long": false,
      "values": [
        {
          "value": "10",
          "selected": "false",
          "count": "1",
          "link": "?do=json;i=1;q1=10;sp_q=Watch;sp_staged=1;x1=size",
          "undolink": "",
          "threshold": false
        },
        {
          "value": "20",
          "selected": "false",
          "count": "1",
          "link": "?do=json;i=1;q1=20;sp_q=Watch;sp_staged=1;x1=size",
          "undolink": "",
          "threshold": false
        }
      ]
    }
  ],
  "results": [
    {
      "index": "",
      "title": "Watch Prod3 Title",
      "productType": "Watch",
      "size": "10"
    },
    {
      "index": "",
      "title": "Watch Prod4 Title",
      "productType": "Watch",
      "size": "20"
    }
  ]
}

The configuration can be pushed live after successful validation and run a live index →Full Index →Live Index →Run Full Index

The URL to access live data http://xxxxxxxxxxx.guided.ss-omtrdc.net/do=json&sp_q=Watch

The facet data in the response can be used to present the filtering options to users to narrow down the website search. The Dynamic Facet option enables the facets based on the search data.

How to implement autocompletion and search suggestion in AEM through Lucene | Predictive Search in AEM | AEM Search Suggestions

This tutorial explain the approach to implement autocompletion and search suggestion in AEM through Lucene.

When you start typing something in search form most of the applications helps you by suggesting the data matching to your search term.

The purpose of autocomplete is to resolve a partial query , i.e., to search within a controlled vocabulary for items matching a given character string.

Starting from AEM 6.1 the feature of suggestion is available through the suggest module of Lucene. Prior to AEM 6.1, all the possible combination of the words needs to be indexed to support the autocompletion.

The Lucene Suggest module provides a dedicated and optimized data structure allows the engine to give autocompletion and suggestion feature without indexing all the possible combination of a word. 

There is a specific analyzer (AnalyzingInfixSuggester) used that loads the completion values from the indexed data and then build the optimized structure in memory for a fast lookup. 

In order to implements the autosuggestion, feature you need to define an index of type Lucene and for each property X of nodes that you are indexing you can add a specific property useInSuggest to tell to the engine to use X for suggesting query to the user.

An additional property suggestUpdateFrequencyMinutes define the frequency of updating the indexed suggestions - useful to mitigate performance issues that can arise if indexed properties are frequently updated by the users of your application. The default value is 10 minutes but the values can be modified as required.

To enable the property "suggestUpdateFrequencyMinutes ", create a node with name "suggest" of type "nt:unstructured" under "testindex" and update the value as required

 In order to use Lucene index to perform search suggestions, the index definition node (the one of type oak:QueryIndexDefinition) needs to have the compatVersion set to 2. 

Let us now execute the query to find the suggestions - either one of the below query can be used.

The testindex was defined for the content path "/content/sampledata" so the query will be executed based on the "testindex" but the index name is explicitly defined in the first query.

SELECT [rep:suggest()] FROM [nt:unstructured] WHERE SUGGEST('te') OPTION(INDEX NAME [testindex]) /* oak-internal */ 

SELECT [rep:suggest()] FROM [nt:unstructured] WHERE SUGGEST('te') AND ISDESCENDANTNODE('/content/sampledata')

The above query uses path restriction to filter the data, it requires evaluatePathRestrictions property should enabled as true on index definition.

The Query tool shows the total number of unique suggestions matching with the search data but it wont displays the matching node details

The below Servlet can be used to fetch the suggestion data through QueryManager API

import java.io.IOException;

import javax.jcr.RepositoryException;
import javax.jcr.Session;
import javax.jcr.query.InvalidQueryException;
import javax.jcr.query.Query;
import javax.jcr.query.QueryManager;
import javax.jcr.query.QueryResult;
import javax.jcr.query.Row;
import javax.jcr.query.RowIterator;
import javax.servlet.Servlet;
import javax.servlet.ServletException;

import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.servlets.HttpConstants;
import org.apache.sling.api.servlets.SlingSafeMethodsServlet;
import org.json.JSONArray;
import org.osgi.framework.Constants;
import org.osgi.service.component.annotations.Component;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Component(immediate = true, service = Servlet.class, property = {
		Constants.SERVICE_DESCRIPTION + "=Custom Root Mapping", "sling.servlet.methods=" + HttpConstants.METHOD_GET,
		"sling.servlet.paths=" + "/bin/getSuggestions", "service.ranking=" + 100001 })
public class SuggestionData extends SlingSafeMethodsServlet {

	/**
	* 
	*/
	private static final long serialVersionUID = 1L;

	protected void doGet(final SlingHttpServletRequest req, final SlingHttpServletResponse resp)
			throws ServletException, IOException {

		Logger logger = LoggerFactory.getLogger(this.getClass());
		logger.error("inside custom servlet");

		final Session session = req.getResourceResolver().adaptTo(Session.class);

		final JSONArray suggestions = new JSONArray();

		String queryString = "SELECT [rep:suggest()]  FROM [nt:unstructured] WHERE "
							 +"SUGGEST('te') OPTION(INDEX NAME [testindex]) /* oak-internal */ ";

		try {
			QueryManager queryManager = session.getWorkspace().getQueryManager();
			Query query = queryManager.createQuery(queryString, Query.JCR_SQL2);
			QueryResult result = query.execute();
			RowIterator rows = result.getRows();

			while (rows.hasNext()) {
				suggestions.put(((Row) rows.next()).getValue("rep:suggest()").getString());
			}

		} catch (InvalidQueryException e) { // TODO Auto-generated catch block
			e.printStackTrace();
		} catch (RepositoryException e) { // TODO Auto-generated
			e.printStackTrace();
		} finally {
			session.logout();

		}

		resp.setContentType("application/json");
		resp.getWriter().write(suggestions.toString());

	}

}

This suggestion data cab be used to display  the search autocompletion/suggest data to the website users.

Sample Configuration - https://github.com/techforum-repo/youttubedata/tree/master/lucene

How to configure Navigation Facet in Adobe Search and Promote?

Facet Configuration

JSON Transport Template

<search-content-type-header charset="UTF-8">
{
"general": {
"query" : "<search-query />",
"total" : "<search-total />",
"lower" : "<search-lower />",
"upper" : "<search-upper />"
},
"facets" : [
{
"name" : "productType",
"values" : [<search-field-value-list name="productType" quotes="yes" data="values" sortby="values" encoding="json" />],
"counts" : [<search-field-value-list name="productType" quotes="no" data="results" sortby="values" />]
}
],
"results" : [
<search-results>
{
"fields" :
[
{
"name" : "mdi",
"value" : "<search-display-field name="mdi" length="500" encoding="json" />"
},
{
"name" : "title",
"value" : "<search-display-field name="title" encoding="json" />"
},
{
"name" : "productType",
"value" : "<search-display-field name="productType" encoding="json" />"
}
]
}
<search-if-not-last>,</search-if-not-last>
</search-results>
]
}

"facets" : [
        {
            "name" : "productType",
            "values" : [<search-field-value-list name="productType" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name="productType" quotes="no"  data="results" sortby="values" />]
        },{"name" : "facet2",
            "values" : [<search-field-value-list name=" facet2" quotes="yes"  data="values" sortby="values" encoding="json" />],
            "counts" : [<search-field-value-list name=" facet2" quotes="no"  data="results" sortby="values" />]
        }
    ],

JSON Presentation Template

<guided-content-type-header content="application/json" />
<guided-if-query-param-defined gsname="callback" /><guided-query-param gsname="callback" />(</guided-if-query-param-defined>
 {"general" :{
            "query" : "<guided-query-param gsname='q' />",
            "total" : "<guided-results-total />",
            "page_lower" : "<guided-results-lower>",
            "page_upper" : "<guided-results-upper>",
            "page_total": "<guided-page-total/>" 
        },  
        "facets" :
        [<guided-zone gsname="Facets_Default"><guided-facet  gsname="productType" >{ 
                        "label" : "<guided-facet-display-name gsname="productType" />",
                        "long" : <guided-if-facet-long>true<guided-else-facet-long>false</guided-if-facet-long>,
                        <guided-if-facet-selected>
                        "selected" : true,
                        "undolink" : "<guided-facet-undo-path gsname="productType" />",
                        </guided-if-facet-selected>
                        "values" : 
                        [
                            <guided-facet-values>
                            {
                                "value" : "<guided-facet-value  escape="ijson">",
                                "selected" : "<guided-if-facet-value-selected>true<guided-else-facet-value-selected>false</guided-if-facet-value-selected>",
                                "count" : "<guided-facet-count>",
                                "link" : "<guided-facet-value-path>",
                                "undolink" : "<guided-facet-value-undo-path>",
                                "threshold" : <guided-if-facet-value-equals-length-threshold>true<guided-else-facet-value-equals-length-threshold>false</guided-if-facet-value-equals-length-threshold>
                            }<guided-if-not-last>,</guided-if-not-last>
                            </guided-facet-values>
                        ] 
                    }
                    </guided-facet>
            </guided-zone>
        ],
        "results" :
        [ 
            <guided-results gsname="default">
            {
                "index" : "<guided-result-index />",
                "title" : "<guided-result-field gsname="title" escape="ijson" />",
                "productType" : "<guided-result-field gsname="productType" escape="ijson" />"
            }<guided-if-not-last>,</guided-if-not-last>
            </guided-results>
        ]
    }
<guided-if-query-param-defined gsname="callback">)</guided-if-query-param-defined>

Pre-Search Rule

{
   "general": {
      "query": "",
      "total": "3",
      "page_lower": "1",
      "page_upper": "3",
      "page_total": "1"
   },
   "facets": [
      {
         "label": "productType",
         "long": false,
         "values": [
            {
               "value": "Sample1",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample1;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            },
            {
               "value": "Sample2",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample2;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            },
            {
               "value": "Sample3",
               "selected": "false",
               "count": "1",
               "link": "?do=json;i=1;q1=Sample3;sp_q=*;sp_staged=1;x1=productType",
               "undolink": "",
               "threshold": false
            }
         ]
      }
   ],
   "results": [
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample1"
      },
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample2"
      },
      {
         "index": "",
         "title": "product-title",
         "productType": "Sample3"
      }
   ]
}