Serving the static resources through different domains(Cookie less Domains) - Adobe Experience Manager(AEM)
This post will explain the approach to improve the page load time of the websites through serving static resources via multiple cookie less domain.
The page load time of the website directly related to the number of resources requested to render the page in browser, most of the cases the browser makes multiple calls to receive the related resources.
The browser is restricted with number of default simultaneous connections per server, if the number of resources requested in the same domain is more then that will delay the page load time - the resources are loaded sequentially(this restriction is more for HTTP 1.1 but HTTP 2 protocol support parallel downloading). This issue can be addressed by distributing the static resources across multiple domains that will intern increase the parallel download of resources.
The static resource to be on a cookie-less domain that makes the content load faster. The Cookies are uploaded with every request on a domain although they are only required on dynamic pages. This will add additional overhead while requesting the static resources through a cookie aware domain and increases the page load time - loading the cookie to request and download the cookie from the response.
To avoid this problem as discussed above define multiple domains and those are cookie less.
To define a cookie less domain, create a multiple domains e.g static1.example.com, static2.example.com etc and CNAME points to the parent domain - Make sure the cookie is set in the server only specific to the parent domain.
In Adobe Experience Manager(AEM) this can be achieved in the below two approaches
Component level:
Changing the static resource URL's in individual components with newly defined domains based on the resource type - may be separate domain for images, scripts, css etc. This will required more effort and also every component level changes to assign the specific URL's for the static resources. There is a possibly the developer will not implement this in all the components and the resources will be served from main domain.
Sling Rewriter to change the static resource URL's:
Define a Static Resource Transformer that will rewrite the resource URL's with static domains defined - multiple domains can be used
The ACS Static Reference Rewriter can be used to change the static resource URL's - https://adobe-consulting-services.github.io/acs-aem-commons/features/utils-and-apis/static-reference-rewriter/index.html
I have defined a Static Resource rewriter based on the above one to add some additional functionalities to match our requirements, thought of sharing as this may help someone with same requirement.
- Exclude the rewrite based on attribute name and values
- Exclude the rewrite for external resources
- Exclude based on the complete path and URL prefix
- Rewrite the URL's for all srcset URL's
- The Rewriter is invoked only for specific content path.
StaticResourceTransformFactory.java
import java.io.IOException;
import java.util.Dictionary;
import java.util.HashMap;
import java.util.Map;
import org.apache.commons.lang3.ArrayUtils;
import org.apache.felix.scr.annotations.Activate;
import org.apache.felix.scr.annotations.Component;
import org.apache.felix.scr.annotations.Modified;
import org.apache.felix.scr.annotations.Properties;
import org.apache.felix.scr.annotations.Property;
import org.apache.felix.scr.annotations.PropertyUnbounded;
import org.apache.felix.scr.annotations.Service;
import org.apache.sling.commons.osgi.PropertiesUtil;
import org.apache.sling.rewriter.ProcessingComponentConfiguration;
import org.apache.sling.rewriter.ProcessingContext;
import org.apache.sling.rewriter.Transformer;
import org.apache.sling.rewriter.TransformerFactory;
import org.osgi.service.component.ComponentContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.AttributesImpl;
import org.apache.felix.scr.annotations.ConfigurationPolicy;
@Component(
label = "Static Resources Transformer Factory",
description = "Static Resources Transformer Factory",
metatype = true,policy = ConfigurationPolicy.REQUIRE)
@Service
@Properties({
@Property(
name = "pipeline.type", label = "Static Resources Transformer Pipeline Type",description ="Static Resources Transformer Pipeline Type"),
@Property(
name = "webconsole.configurationFactory.nameHint",
value = "Static Resources Transformer: {pipeline.type}")
})
public final class StaticResourceTransformFactory implements TransformerFactory {
public final class StaticResourceTransformer extends org.apache.cocoon.xml.sax.AbstractSAXPipe
implements org.apache.sling.rewriter.Transformer {
@Override
public void startElement(String uri, String localName, String qname, Attributes attr) throws SAXException {
final String[] includeElementAttributesArray =includeElementAttributes.get(localName);
AttributesImpl attrs = new AttributesImpl(attr);
boolean excludeAttributeValue=false;
for (int i = 0; i < attrs.getLength(); i++) {
final String[] excludeAttributeValuesArray = excludeAttributeValues.get(attrs.getLocalName(i));
excludeAttributeValue=ArrayUtils.contains(excludeAttributeValuesArray, attrs.getValue(i));
LOG.info("excludeAttributeValue:"+excludeAttributeValue);
if(excludeAttributeValue)
{
break;
}
}
for (int i = 0; i < attrs.getLength(); i++) {
String name = attrs.getLocalName(i);
String value = attrs.getValue(i);
if (ArrayUtils.contains(includeElementAttributesArray, name)
&& !ArrayUtils.contains(excludePath, value) && !isExcludedPrefix(excludePrefix,value)
&&!(value.startsWith("https") || value.startsWith("http") || value.startsWith("//")) && !excludeAttributeValue) {
{
if(name.equals("srcset"))
{
String[] srcset=value.split(",");
String srcsetValue="";
for(int j=0;j<srcset.length;j++)
{
if(!(value.startsWith("https") || value.startsWith("http") || value.startsWith("//")))
{
srcsetValue=!srcsetValue.equals("")?srcsetValue+","+ prependHostName(srcset[j].trim()):srcsetValue+prependHostName(srcset[j].trim());
}else
{
srcsetValue=srcsetValue+","+srcset[j];
}
}
attrs.setValue(i, srcsetValue);
}else
{
attrs.setValue(i, prependHostName(value));
}
}
}
}
super.startElement(uri, localName, qname, attrs);
}
@Override
public void dispose() {
// TODO Auto-generated method stub
}
@Override
public void init(ProcessingContext arg0, ProcessingComponentConfiguration arg1) throws IOException {
// TODO Auto-generated method stub
}
}
private static final int DEFAULT_HOST_COUNT = 1;
private static final Logger LOG = LoggerFactory.getLogger(StaticResourceTransformFactory.class);
private static final String[] DEFAULT_ATTRIBUTES = new String[] { "img:src,srcset", "link:href", "script:src" };
@Property(unbounded = PropertyUnbounded.ARRAY,label = "Rewrite Elements - Attributes to Include", description = "List of element/attribute pairs to rewrite", value = {"img:srcset,src", "link:href", "script:src" })
private static final String PROP_INCLUDE_ELEMENT_ATTRIBUTES = "includeElementAttributes";
@Property(unbounded = PropertyUnbounded.ARRAY, label = "Exclude path", description = "List of paths to be excluded", value = {})
private static final String PROP_EXCLUDE_PATH = "excludePath";
@Property(unbounded = PropertyUnbounded.ARRAY,label = "Rewrite Attributes - values to Exclude", description = "List of Attribute/value pairs to exclude", value = {"rel:alternate,canonical" })
private static final String PROP_EXCLUDE_ATTRIBUTES_VALUES = "excludeAttributeValues";
@Property(unbounded = PropertyUnbounded.ARRAY, label = "Exclude path prefix", description = "List of prefix path to exclude", value = {})
private static final String PROP_EXCLUDE_PREFIX = "excludePrefix";
@Property(intValue = DEFAULT_HOST_COUNT, label = "Static Host Count",description = "Number of static hosts available.")
private static final String PROP_HOST_COUNT = "host.count";
@Property(label = "Static Host Pattern", description = "Pattern for generating static host domain names. "+ "'{}' will be replaced with the host number. If more than one is provided, the host count is ignored.", unbounded = PropertyUnbounded.ARRAY)
private static final String PROP_HOST_NAME_PATTERN = "host.pattern";
private Map<String, String[]> includeElementAttributes;
private Map<String, String[]> excludeAttributeValues;
private String[] excludePath;
private String[] excludePrefix;
private int staticHostCount;
private String[] staticHostPattern;
public Transformer createTransformer() {
return new StaticResourceTransformer();
}
@Activate
protected void activate(ComponentContext componentContext) {
final Dictionary<?, ?> properties = componentContext.getProperties();
this.includeElementAttributes = convertoMap(PropertiesUtil.toStringArray(properties.get(PROP_INCLUDE_ELEMENT_ATTRIBUTES), DEFAULT_ATTRIBUTES));
this.excludeAttributeValues = convertoMap(PropertiesUtil.toStringArray(properties.get(PROP_EXCLUDE_ATTRIBUTES_VALUES)));
this.excludePath = PropertiesUtil.toStringArray(properties.get(PROP_EXCLUDE_PATH));
this.excludePrefix = PropertiesUtil.toStringArray(properties.get(PROP_EXCLUDE_PREFIX));
this.staticHostPattern = PropertiesUtil.toStringArray(properties.get(PROP_HOST_NAME_PATTERN), null);
this.staticHostCount = PropertiesUtil.toInteger(properties.get(PROP_HOST_COUNT), DEFAULT_HOST_COUNT);
}
@Modified
protected void modified(ComponentContext newComponentContext) {
this.activate(newComponentContext);
}
private String prependHostName(String value) {
if (staticHostPattern != null && staticHostPattern.length > 0) {
final String host;
if (staticHostPattern.length == 1) {
final String hostNum = getShardValue(value, staticHostCount, toStringShardNameProvider);
host = staticHostPattern[0].replace("{}", hostNum);
} else {
host = getShardValue(value, staticHostPattern.length, lookupShardNameProvider);
}
return String.format("//%s%s", host, value);
} else {
return value;
}
}
private static String getShardValue(final String filePath, final int shardCount, final ShardNameProvider sharder) {
int result = 1;
if (shardCount > 1) {
final int fileHash = ((filePath.hashCode() & Integer.MAX_VALUE) % shardCount) + 1;
String hostNumberString = Integer.toString(fileHash);
if (hostNumberString.length() >= 2) {
// get the 2nd digit as the 1st digit will not contain "0"
Character c = hostNumberString.charAt(1);
hostNumberString = c.toString();
// If there are more than 10 hosts, convert it back to base10
// so we do not have alpha
hostNumberString = Integer.toString(Integer.parseInt(hostNumberString, shardCount));
result = Integer.parseInt(hostNumberString) + 1;
} else {
result = fileHash;
}
}
return sharder.lookup(result);
}
private Map<String, String[]> convertoMap(String[] inputMapping) {
Map<String, String[]> outputMap = new HashMap<String, String[]>();
for (int i = 0; i < inputMapping.length; i++) {
String inputString = inputMapping[i];
String[] split = inputString.split(":");
String key = split[0];
String[] value = split[1].split(",");
outputMap.put(key, value);
}
return outputMap;
}
private boolean isExcludedPrefix(String[] prefix, String value)
{
boolean isExcludedPrefix=false;
for(int i=0;i<prefix.length;i++)
{
if(!prefix[i].equals("") && value.startsWith(prefix[i]))
{
isExcludedPrefix=true;
break;
}
}
return isExcludedPrefix;
}
private interface ShardNameProvider {
String lookup(int idx);
}
private static final ShardNameProvider toStringShardNameProvider = new ShardNameProvider() {
@Override
public String lookup(int idx) {
return Integer.toString(idx);
}
};
private ShardNameProvider lookupShardNameProvider = new ShardNameProvider() {
@Override
public String lookup(int idx) {
return staticHostPattern[idx - 1];
}
};
}
Add the below dependencies in pom.xml
<dependency>
<groupId>org.apache.cocoon</groupId>
<artifactId>cocoon-xml</artifactId>
<version>2.0.2</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.2</version>
<scope>provided</scope>
</dependency>
Configure the Transformer Factory:
Login to http://localhost:4502/system/console/configMgr
Search for - "Static Resources Transformer Factory" and provide the required configurations.
Static Resources Transformer Pipeline Type - Enter some unique name for Pipeline Type, the same value should be configured in rewrite pipeline.
Rewrite Elements - Attributes to Include - Configure the element and attributes - the attributes for which the URL should be rewritten based on the element type. Format - elmentname:attributename1,attributename2. Multiple elements along with attributes can be configured.
e.g
img:srcset,src
script:src
Exclude path - Complete path that should be excluded from rewriting.
e.g.
/etc/designs/geometrixx-outdoors/clientlibs_desktop_v1.js
Rewrite Attributes - values to Exclude - Attribute and values based on that the URL rewriting should be excluded.
e.g
rel:alternate,canonical
class:test-class1,test-class2
Exclude path prefix - The prefix path for that the URL rewriting should be excluded.
e.g - /etc/designs/geometrixx-outdoors
Static Host Count - Number of static hosts available
Static Host Pattern - static{}.resources.com, {} - will be replaced within nuber 1-3 as Static host count is configured with 3.makse the the three cookieless domains are configured - static1.resources.com, static2.resources.com and static3.resources.com
Final configuration - com.packagegenerator.core.StaticResourceTransformFactory.config
# Configuration created by Apache Sling JCR Installer
host.pattern=["static{}.resources.com"]
host.count=I"3"
excludePath=[""]
excludePrefix=[""]
includeElementAttributes=["img:srcset,src","link:href","script:src"]
pipeline.type="staticresourcerewriter"
excludeAttributeValues=["rel:alternate,canonical"]
Rewriter pipeline configuration:
Copy /libs/cq/config/rewriter/default to /apps/myapp/config/rewriter (create the missing folders)
Remove the child nodes of default node
Rename the default node to friendly name - rewriter-sites
Configure the required values
Make sure the order is changed to 1
Add "staticresourcerewriter"(defined in earlier step) as part of transformerTypes
Path - add the paths to which this rewriter should be applied(the rewriter is applided to the current node and the child nodes). This configuration will help us to enable the rewriter for particular sites or path.
e.g
/content/geometrixx-outdoors/en/men/shorts
/content/geometrixx-outdoors/en/men/shirts
Final configuration -
<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
jcr:primaryType="nt:unstructured"
contentTypes="[text/html]"
enabled="{Boolean}true"
generatorType="htmlparser"
order="1"
paths="[/content/geometrixx-outdoors/en/men/shorts,/content/geometrixx-outdoors/en/men/shirts]"
serializerType="htmlwriter"
transformerTypes="[linkchecker,staticresourcerewriter]"/>
Verify the configuration Status:
Login to http://localhost:4502/system/console/status-slingrewriter
Access the pages configured in the pipeline rewriter or the child nodes (/content/geometrixx-outdoors/en/men/shorts,/content/geometrixx-outdoors/en/men/shirts]), the static resources will be rewritten to the new domains based on the configuration provided.
The highlighted URL is not rewriten as the URL is specified with hostname(complete URL and considered as external URL) and other URl's are rewritten to new host names(distributed between 3 hosts)
The highlighted URL's are excluded from rewriting as the attribute rel with the value alternate is configured for exclusion.
No comments:
Post a Comment