Wednesday, November 3, 2021

Adobe Experience Manager(AEM) Content Translation - Deep Dive(Part2)

Adobe Experience Manager(AEM) Content Translation

In the previous blog - Adobe Experience Manager(AEM) Content Translation - Deep Dive(Part1), we have seen the different configurations and details on AEM Content Translation Framework. In this blog, let us see how to use AEM Content Transalation Framework to perform Human/Machine Translation. 

Let us start with the Translation Project

Translation Project:

Translation projects enable you to manage the translation of AEM content. A translation project is a type of AEM project that contains resources that are to be translated into other languages. These resources are the pages and assets of the language copies that are created from the language master.

When resources are added to a translation project, a translation job is created for them. Jobs provide commands and status information that you use to manage the human translation and machine translation workflows that execute on the resources.

The Translation Project can be created through multiple channels, the content can be added to the existing Translation Project, multiple translation jobs can be enabled to a translation project.

Project Console - the Translation Project can be created through the project console

AEM Content Transaltion

AEM Content Translation

If required you can create new master projects that can be used as templates to create a new project.

AEM Content Translation


Add the basic details like Title, Description, Start Date, End Date, Users to manage the project, etc

AEM Content Translation

In the Advanced tab, select the Source Language, select the required Target Languages - you can select multiple target languages, separate Translation Job will be created for every Target language.

Select the Cloud configuration, leave it blank if the default configurations should be used(libs or global conf), if not select the site-specific configuration.

AEM Content Translation

Select the -

Translation Type - Machine Translation or Human Translation

Translation Provider for Machine Translation, also select the content category

Other options based on the project need - Automatically Promote Translation Launches, Delete Launch After Promotion, Automatically Approve Translation - this configuration helps us to automatically approve and promote the translation content.

You should be able to schedule the translations, Create Job for repeated translation(Daily, Weekly, Monthly, and Yearly)

AEM Content Translation

Now the project is created with a translation project for the selected target languages and you will be able to add the required content for translation.

AEM Content Translation

You can add pages, assets, content/experience fragments, and tags 

AEM Content Translation


AEM Content Translation

While selecting the Assets/Pages the corresponding tags and i18 strings included

You should be able to add additional Translation Jobs to the project

AEM Content Translation

AEM Content Translation

Site Console - Create Language Copy

The translation project can be created while creating a new language copy

AEM Content Translation

Add the required pages to be part of the language copy/translation

AEM Content Translation

Specify some of the basic translation configurations - Target Languages

AEM Content Translation

Create Translation Project or Create Multi-Language Translation Project(translation to multiple languages)


AEM Content Translation

The translation project is now created and the selected content is added to the project.


Content Reference Panel

Go to the site console, select the language master, reference panel, and select language copies

AEM Content Translation

Select the project type and click on Update, the translation project will be created whenever there is a change in the language master.

AEM Content Translation

Now the translation project is created with required translation jobs

AEM Content Translation

Human Translation:


Let us now see, how to initiate the human translation flow

In human translation flow, the content is exported in any one of the forms - XML, XLIF, or JSON and sent to the translator for offline translation, the translated content is imported back to the AEM

Create a Translation Project and specify the Translation method as Human Translation, also select the target languages.

AEM Content Translation

 The current status of the Translation Job is Draft, add the required pages/assets to the translation project, while adding the content select "Create Language Copy" or "Update the Language Master" (if u maintain language to master for every language) options.

AEM Content Translation

AEM Content Translation


Now you should be able to export the content for translation

AEM Content Translation

Download the exported zip file and share it with the translator - the downloaded zip file contains the translation content in different files with the extension configured e.g XML, XLIF, or JSON, the default is XML.

AEM Content Translation

AEM Content Translation

The translator can translate the content to the target language and send back the zip file.

AEM Content Translation

The file can be imported back to AEM through translation Jobs

AEM Content Translation


Now the Job status is changed to "Translation In Progress", Go to the content panel

AEM Content Translation

Considering the "Auto Approval" was selected, the translation could have auto-approved and the Job status changed to "Approved", due to the issue with i18 translation - no i18 available for the selected content the translation is not auto-approved.

AEM Content Translation

For the Jobs without auto-approval configuration, you should be able to review and accept the translation here. In this case, the job is auto-approved after deleting the "Component Strings" job, even the same can be deleted while adding the content for translation.

AEM Content Translation

Now you can complete the Job

AEM Content Translation

AEM Content Translation

When you translate modified content for the same language and the same "Create Language Option" is selected, the launch is created to avoid the override of the actual content.

AEM Content Translation


If the "Automatically Promote Translation Launches" option is not selected, the lanches should be manually promoted after the translation is completed through the launch console.

AEM Content Translation

Any time new translation e.g pages, assets, tags, i18, and content/experience fragments are required, a new Job can be added to the existing project or a new project can be created based on the need.

Machine Translation:


Let us now how to use Machine Translation, most of the translators provide connector to integrate the TMS system with  the AEM Tranlation Framework to orchestrate the translation process, AEM OOB box provides Microsoft Machine translation connector and tria license, that can be used for your trial purpose.

The TMS specific connector should be installed and configured to orchestrate the transaltion process.

I am going to use Microsoft Translation Connector with free tier license - the trail license can be right away used without any additional configurations

Login to Azure Portal and create new Translation Service

AEM Content Translation

AEM Content Translation

Ensure the region is selected as global, refer to the following URL to understand more on the issue while the region is selected other than global - https://www.albinsblog.com/2021/10/microsoft-translator-service-issue-aem.html

Select the pricing tier based on your need - I am going with free tier

AEM Content Translation

You should be able to copy the keys once the configuration is completed

AEM Content Translation

There are two keys, you can copy either one of them

AEM Content Translation

Let us now configure the Microsoft Translation Connector in AEM

Create new Translation Cloud Service

AEM Content Translation

Select the appropriate configuration context, I am enabling under global context

AEM Content Translation

Select "Microsoft Translator" from the dropdown and provide a Title and Name

AEM Content Translation

In the next screen, enter the key copied from Azure Translation service

AEM Content Translation

Verify the connection and Save the configuration

AEM Content Translation

Now the connector is ready, as discussed ealier configure the Translation Prject with the Microsoft translator.

AEM Content Translation

Now you can add the required content into the translation job and start

AEM Content Translation

Now the job status changed to Approved as the "Automatically Aprove Translations" option was selected.

AEM Content Translation

You can mark the Job as Completed

AEM Content Translation

Now the content is translated to the target language

AEM Content Translation

AEM Content Translation

Translation Memory Update:


Manual edits of translated content can be synchronized back to the Translation Management System (TMS) to train its Translation Memory. if we edit any of the translated content(the content was not translated corerctly) those details can be be reported back to TMS system so that TMS system can learn from corrected data.

Selected the edited content from site console and click on "Update Translation Memory"

AEM Content Translation

Update the Memory for selected edits(shoing for demo)

AEM Content Translation

The Connetor should support the Update Translation Memory functionality to send those content edit details

AEM Translation Framework Bootstrap Connector:


The Bootstrap connector is built using the Translation Framework AP, the purpose of the connector is to
  • Provide sample code to Translation partners and Service providers to start building their connector
  • Provide best practices for building/packaging the connector
  • Highlight the coding standards for the Connector certification process
  • Serve as a reference implementation of the Translation API
Refer to https://github.com/Adobe-Marketing-Cloud/aem-translation-framework-bootstrap-connector for more details and source code. The bootstrap connector can be be updated based on your need and installed to AEM to orchestrate the tranlation process flow specific to you.


I was getting the below exception while deploying the bootstrap conenctor

[INFO] --- filevault-package-maven-plugin:1.2.2:validate-package (default-validate-package) @ bootstrap-connector.ui.content ---
[INFO] Overriding wrong default value for validator option 'additionalJcrInstallerFileNodePathRegex' (see https://issues.apache.org/jira/browse/JCRVLT-564)
[INFO] Start validating package 'target\bootstrap-connector.ui.content-2.0-SNAPSHOT.zip'...
[WARNING] Using default nodetypes, consider specifying the nodetypes from the repository you use!
[INFO] Register node types from jar:file:/C:/Users/issaca1/.m2/repository/org/apache/jackrabbit/vault/vault-validation/3.5.4/vault-validation-3.5.4.jar!/default-nodetypes.cnd
[INFO] Using 10 validators for package of type CONTENT: jackrabbit-accesscontrol (org.apache.jackrabbit.vault.validation.spi.impl.AccessControlValidator), jackrabbit-filter (org.apache.jackrabbit.vault.validation.spi.impl.AdvancedFilterValidator), jackrabbit-properties (org.apache.jackrabbit.vault.validation.spi.impl.AdvancedPropertiesValidator), jackrabbit-docviewparser (org.apache.jackrabbit.vault.validation.spi.impl.DocumentViewParserValidator), jackrabbit-dependencies (org.apache.jackrabbit.vault.validation.spi.impl.DependencyValidator), jackrabbit-emptyelements (org.apache.jackrabbit.vault.validation.spi.impl.EmptyElementsValidator), jackrabbit-mergelimitations (org.apache.jackrabbit.vault.validation.spi.impl.MergeLimitationsValidator), jackrabbit-oakindex (org.apache.jackrabbit.vault.validation.spi.impl.OakIndexDefinitionValidator), jackrabbit-packagetype (org.apache.jackrabbit.vault.validation.spi.impl.PackageTypeValidator), jackrabbit-nodetypes (org.apache.jackrabbit.vault.validation.spi.impl.nodetype.NodeTypeValidator)
[WARNING] ValidationViolation: "jackrabbit-filter: Filter root's ancestor '/home/users' is not covered by any of the specified dependencies nor a valid root.", filePath=META-INF\vault\filter.xml
[WARNING] ValidationViolation: "jackrabbit-nodetypes: Invalid primary type 'sling:Folder': sling: is not a registered namespace prefix. Skip validation of nodes with that type/name", filePath=jcr_root\var\bootstrap-tms\.content.xml, nodePath=/var/bootstrap-tms, line=4, column=37
[ERROR] ValidationViolation: "jackrabbit-accesscontrol: Found no access control list, but there is supposed to be one contained as the property 'acHandling' is set to 'MERGE_PRESERVE'!"
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Bootstrap Translation Connector 2.0-SNAPSHOT:
[INFO]
[INFO] Bootstrap Translation Connector .................... SUCCESS [  0.748 s]
[INFO] Bootstrap Translation Connector - Core ............. SUCCESS [  6.551 s]
[INFO] Bootstrap Translation Connector - UI apps .......... SUCCESS [  2.592 s]
[INFO] Bootstrap Translation Connector - UI contents ...... FAILURE [  2.872 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.699 s
[INFO] Finished at: 2021-11-03T14:55:51-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.jackrabbit:filevault-package-maven-plugin:1.2.2:validate-package (default-validate-package) on project bootstrap-connector.ui.content: Found 1 violation(s) (with severity=ERROR). Check above errors for details -> [Help 1]

AEM Content Translation

Used the below maven command to make the deployment success(skipped the package validation - the actual issue should be addressed on the package)

mvn clean install -PautoInstallPackage -Dvault.skipValidation

This enables Bootstrap Translation Service OSGI configuration

AEM Content Translation


Now addional option will be added while creating the Tranlation Cloud Configuration(Bootstrap Connector)

AEM Content Translation

AEM Content Translation

Bootstrap framework supports both Human and Machine translation

AEM Content Translation


Now you can perform the import/export of translated content

AEM Content Translation


The bootstrap connector can be modified to support both Machine and Human translation based on your process flow.

Scoping a Translation Job:


Scope a translation job to obtain an estimate of the cost of translation from your translation service provider. When you scope a job, source files are submitted to the translation vendor who compares the text to their pool of stored translations (translation memory). Typically, the scope is the number of words that require translating.

Scoping is optional. You can start a translation job without scoping. Refer to https://experienceleague.adobe.com/docs/experience-manager-64/administering/introduction/tc-manage.html?lang=en#scoping-a-translation-job for more details on scoping.

The scoping should be supported by conenctor, if enabled you can "Request for Scope" - the option is enabled for Bootstrap Connector

AEM Content Translation

Automating the translation process through AEM Translation Framework and the TMS connector is critical for the success of the multi lingual AEM projects, autmated transaltion process helps to launch the multi language websites quickly.

References:






Monday, October 25, 2021

Adobe Experience Manager(AEM) Content Translation - Deep Dive(Part1)

The website translation is the process of taking your website content in its original language(e.g en) and adapting it into other languages e.g es, to make it is accessible and useable across global customers. Content Translation allows you to create an initial version of a page based on an existing version from a different language. Automate the translation of page content, assets, etc to create and maintain multilingual websites through the translation process.

The Adobe Experience Manager Translation Framework enables website content managers to easily submit content for professional translation and automatically receive translations back in AEM. All that tedious manual effort is replaced by a streamlined process that delivers personalized customer experiences.

Most of these contents are already part Adobe documents, thought of sharing my learning for reference and also if it helps someone with the same need.

There are two types of content translations

Human Translation:


Content is sent to your translation provider and translated by professional translators. When complete, the translated content is returned and imported into AEM. When your translation provider is integrated with AEM(translation provider enables the connector to integrate AEM and TMS(Translation Management System) system to automate the translation process), content is automatically sent between AEM and the translation provider.

Machine Translation:


The machine translation service immediately translates your content. The content is shared with Translation Management Systems through connectors (APIs) and the translated content is immediately sent back to AEM through the same connector API. There is no Human step involved the end-to-end translation process is automated through the connector. The translation time is very less and the end-to-end process completes quickly.

Machine translation is generally used where the quality-level requirement is not as stringent as where the human translation is required. Use machine translation where the quantity of content, speed of content creation, and budget make it impossible to use human translation.

Translation Integration Framework:


AEM provides Translation Integration Framework to orchestrate the translation process between AEM and the TMS system, the Translation Integration Framework integrates with third-party translation services to orchestrate the translation of AEM content. The Translation Integration Framework can be configured based on our translation needs, also TIF helps AEM to integrate with the TMS system to translate the content/assets.

By default, AEM TIF supports 

Human Translation
  • Export the content for translation 
  • Translate the content
  • Import the translated content back to AEM
The TIF framework supports different content formats for importing/exporting the content -  XML, XLIF 1.2, XLIF 2.0, and JSON. The connector specific to vendors can provide the support for Human translations - import/export the content based on the custom TMS needs

Machine Translation

AEM includes the capability to connect to Microsoft Translator(Trial) by default. Multiple other vendors provide support(Connectors) to translate the AEM content.

Refer to the below diagram for overall all orchestrated translation process flow

AEM Content Translation

TIF Configurations:


The default TIF configuration specifies how to translate your content. The configuration includes the following information:
  • Which translation service provider to use.
  • Whether human or machine translation is to be performed.
  • Whether to translate other content that is associated with a page or asset, such as tags.
Tools -- Cloud Services -- Translation Cloud Services

AEM Content Translation

libs -- Translation Integration -- Default Configuration

AEM Content Translation

Modify the site/asset specific configurations(the editable configuration is editable only for the first time after same should be managed through /con/global)

AEM Content Translation


AEM Content Translation


AEM Content Translation

AEM Content Translation

Unselect "Translate Assets" to stop translating asset binaries.

On save, the new framework configuration is enabled under /con/global - applicable for all contexts

AEM Content Translation

 Context-specific configurations can be enabled under the required configuration folder - override for a specific context

AEM Content Translation


Refer to https://experienceleague.adobe.com/docs/experience-manager-cloud-service/sites/administering/reusing-content/translation/integration-framework.html?lang=en to understand more on the specific configurations. (the translation configurations are applied based on Context-aware configuration)

Export formats:


As we discussed by default the Translation Integration Framework export the content in XML format but the framework supports XLIF 1.2, XLIF 2.0, and JSON formats. The export format can be configured through http://localhost:4502/system/console/configMgr/com.adobe.cq.wcm.translation.impl.TranslationPlatformConfigurationImpl configuration.

AEM Content Translation

Change the configuration based on the need - the OSGI configuration should be enabled through code, the translation content will be exported in a format based on this configuration.

The XLIF export will not be supported by default, you will be receiving the below exception while enabling XLIF transport.

Caused by: com.day.cq.workflow.WorkflowException: Error while translating language copy for /content/projects/test/jcr:content/dashboard/gadgets/translationjob.
at com.adobe.cq.wcm.translation.impl.process.SyncTranslationProcess.execute(SyncTranslationProcess.java:136) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.day.cq.workflow.compatibility.CQWorkflowProcessRunner.execute(CQWorkflowProcessRunner.java:93) [com.day.cq.workflow.cq-workflow-impl:6.3.16]
... 9 common frames omitted
Caused by: com.adobe.granite.translation.api.TranslationException: Failed to perform :START_EXPORT
at com.adobe.cq.wcm.translation.impl.TranslationPodImpl.syncTranslation(TranslationPodImpl.java:2566) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.adobe.cq.wcm.translation.impl.process.SyncTranslationProcess.syncTranslationJob(SyncTranslationProcess.java:171) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.adobe.cq.wcm.translation.impl.process.SyncTranslationProcess.execute(SyncTranslationProcess.java:131) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
... 10 common frames omitted
Caused by: com.adobe.granite.translation.api.xliff.TranslationXLIFFServiceException: XLIFF service is not available
at com.adobe.cq.wcm.translation.impl.TranslationObjectImpl.exportToXLIFFString(TranslationObjectImpl.java:883) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.adobe.cq.wcm.translation.impl.TranslationObjectImpl.exportFileForTranslation(TranslationObjectImpl.java:944) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.adobe.cq.wcm.translation.impl.TranslationPodImpl.startExportNow(TranslationPodImpl.java:948) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
at com.adobe.cq.wcm.translation.impl.TranslationPodImpl.syncTranslation(TranslationPodImpl.java:2512) [com.day.cq.wcm.cq-wcm-translation:1.6.102]
... 12 common frames omitted


Translation Integration Configuration:


AEM provides Translation Configuration UI to manage the content translation rules to control the properties and references that will get translated. Translation rules identify content in AEM to be extracted for translation.  Out-of-the-box translation rules cover common use cases such as Text components and alt text for Image components. Depending on a project's translation requirements additional rules may be needed. In general translation rules allow users to specify:
  • Properties that should be translated based on path and/or resource type
  • Filters for properties that should NOT be translated
  • Referenced content that should be translated (i.e Images or Content Fragments)
The default rules cover the out-of-the-box components and properties but custom rule configuration is required to handle the translation for the custom components.

The UI can be accessed through, Tools -- General -- Translation Configuration

AEM Content Translation

 
AEM Content Translation


The configuration rules are stored as an XML file, the default file is available under /libs/settings/translation/rules/translation_rules.xml, the file will be stored under /conf/global/settings/translation/rules/translation_rules.xml on the first modification(the file under /libs/ should not be directly modified)

AEM Content Translation

For the versions prior to 6.3, the file is stored under /etc/workflow/models/translation/translation_rules.xml - the UI is not available for the AEM versions prior to AEM 6.3

Let us now see how the custom components properties can be enabled for translation - the custom components properties(nonstandard) are not considered by Translation Framework for sending to translation.

I have a custom component with some standard properties and some custom, the standard properties e.g text are by default considered for translation but the custom properties e.g testProp not considered for translation.
AEM Content Translation


AEM Content Translation


AEM Content Translation

Under "General"

AEM Content Translation


Now enable the configuration for custom component properties, the filters and references can be added if required

Click on "Add Component", enter component resource path "mysite/components/helloworld"

AEM Content Translation


AEM Content Translation

Now add the new property under the custom component section, add the new property and save the rule

AEM Content Translation

Now the new properties will be exported for translation. The other contexts(/content/dam and /content/forms) can be based on the need, also a new context can be added if required.

AEM Content Translation


Additional Languages for Translation:


The original list of supported languages for translation in AEM under  /libs/wcm/core/resources/languages 

AEM Content Translation

Overlay /libs/wcm/core/resources/languages to /apps(/apps/wcm/core/resources/languages) (manage the changes through code)

AEM Content Translation

Copy the language node from /libs/wcm/core/resources/languages and paste to /apps/wcm/core/resources/languages(remove the old node and rename the new node to language)
 
Enable the additional languages(copy-paste the individual node and modify the values)

AEM Content Translation

Update the OSGi configurations -Day CQ WCM Language Manager with the new location

AEM Content Translation

Restart the AEM instance, Now the new language will be available while creating a translation project to translate the content(let us see more details on Translation Projects in upcoming blogs)

AEM Content Translation



Let us see in the next blog how to use the Translation Framework to translate the content. - Adobe Experience Manager(AEM) Content Translation - Deep Dive(Part2)


References: