Internationalization

Introduction

Every web application should be internationalized, or at least be ready for it. Internationalization (i18n) means that the content of the site will change according to the locale of the user visiting it, where locale stands for "language and regional settings".

These are the aspects of i18n that should be considered:

language user choice
search engine indexing
static text
database text fields
date and time
enum values

Language user choice

Browsers send the user system language to the server at each request. Actually, some browsers allow users to override the system language. Still, that’s a very inconvenient way to choose language. You should also give the user a way to choose a different language for your site using something like a dropdown menu.

Search engine indexing

The chosen language should be reflected in the url for a number of reasons:

better search engine indexing
bookmarking

One way to put the language in the url would be to add a request parameter like ?lang=fr; another option is to add the language code in the servlet path, like example.com/fr/home. The Yada Framework supports both.

You can also use a full Locale code instead of the language code, like fr_CA for french in Canada.

Immutable (static) text

A typical web page usually shows some text that is statically typed, i.e. not added via a CMS backend but typed directly into the HTML code. This may be a title, a menu entry, an image caption, or a description. All this statically typed text can’t be used in a i18n application: the text must change according to the user language.

Database text fields

Traditional applications with just one language store a Java String attribute into a single table column in the database. With i18n applications, where a Java attribute can have as many values as the languages that are allowed, the single-column approach can’t work anymore. There are many solutions to this problem. The Yada Framework stores a localized String attribute into a dedicated table, where a row holds the value for a single instance in a single language.

For example, consider a Product with a non-localized "name" and a localized "description". The Database model could be like the following:

Table 1. Product
ID	NAME
1	Mercury Ball
2	Jupiter Ball

Table 2. Product_description
PRODUCT_ID	DESCRIPTION	LOCALE
1	Small inflatable ball	en_GB
1	Piccola palla gonfiabile	it_IT
2	Big inflatable ball	en_GB
2	Grande palla gonfiabile	it_IT

Date and time

Date and time are represented differently in different regions, for example by writing the day before the month or the opposite.

Most importantly, every user lives in a specific time zone in the world, e.g. "Europe/Rome" or "Asia/Tokyo", and a time like "2024-09-19@11:37" is a different moment in different time zones. This means that the code should take the time zone of the user into consideration when receiving, showing or using a date/time value. This also applies to scheduled tasks that must run at a specific time of day if the time has an impact on the user.

Enum values

TO BE CONTINUED...

Initial language choice

When a user loads the site for the first time, the initial language will be chosen from the accept-language header sent by the browser. This is implemented in Spring’s CookieLocaleResolver but overridden in YadaWebConfig so that all values from the header are considered in order to find the first language chosen by the user that has also been configured in the webapp. If no values from the header are acceptable, the default configured language will be used. Configuration example:

	<i18n localePathVariable="true">
      <locale default="true">en</locale>
      <locale>ar</locale>
      <locale>de</locale>
      <locale>fr</locale>
   	</i18n>

On the following requests, the standard Spring cookie is used to determine the previously chosen locale.

Configuring "language in the path"

URL Example:

http://www.example.com/it/myHome

Java code

If your project does not use YadaSecurity, change WebApplicationInitializer in order to add a servlet filter:

@Override
protected Filter[] getServletFilters() {
      // Locale in the path
      // See https://stackoverflow.com/a/23847484/587641
      return new Filter[] { new DelegatingFilterProxy("yadaLocalePathVariableFilter") };
}

The above adds a filter to the Spring servlet engine. It is not needed when using YadaSecurity because the same is already done in net.yadaframework.security.SecurityWebApplicationInitializer.

The "language in the path" functionality is implemented in YadaLocalePathVariableFilter, YadaLocalePathChangeInterceptor, YadaLinkBuilder

Application configuration

Edit the conf.webapp.prod.xml configuration file (were 'prod' is the environment code for "production") adding a section like the following:

<i18n localePathVariable="true">
      <locale>it</locale>
      <locale default="true">en</locale>
      <locale>de</locale>
      <locale>es</locale>
      <locale>fr</locale>
      <locale>ru</locale>
</i18n>

Note the use of the "default" attribute, that selects the default language as explained later.

Other than just the language, you can use a full locale code though this is rarely needed:

<i18n localePathVariable="true">
      <locale>it_IT</locale>
      <locale default="true">en_GB</locale>
      <locale>en_US</locale>
      <locale>es_ES</locale>
      <locale>fr_FR</locale>
      <locale>fr_CA</locale>
</i18n>

You can also configure a country to be added to the locale after the request has been received. This way you can still use just the language code in the url but receive a full Locale in the java @Controller:

<i18n localePathVariable="true">
      <locale country="IT">it</locale>
      <locale country="GB" default="true">en</locale>
      <locale country="DE">de</locale>
      <locale country="ES">es</locale>
      <locale country="FR">fr</locale>
      <locale country="RU">ru</locale>
</i18n>

Using "language in the path"

Java

Language on redirect

When returning a redirect string, the language path should be present: /fr/products. The method YadaWebUtil.redirectString() can add the needed language to the url, and also any parameters (see javadoc):

return YadaWebUtil.redirectString("/products", locale, "id", "172");

The YadaWebUtil.redirectString() returns the "redirect:" prefix too. In order to create a string without that prefix, use YadaWebUtil.enhanceUrl().

URL with no language

The default language is also needed when someone types just the server address without path from a browser in a language that is not in the configuration. In such case, the default language should be used:

@RequestMapping("/")
public String home(Model model, HttpServletRequest request, Locale locale) {
        if (YadaLocalePathChangeInterceptor.localePathRequested(request)) {
                // Language was in the url
                return home(model, request);
        }
        // Language was not in the url
        String currentLanguage = locale.getLanguage();
        if (!config.getLocaleStrings().contains(currentLanguage)) {
                // Not a configured locale - use the default one
                Locale defaultLocale = config.getDefaultLocale();
                if (defaultLocale==null) {
                        // Default locale was not configured - use english
                        defaultLocale = Locale.ENGLISH;
                }
                currentLanguage = defaultLocale.getLanguage();
        }
        return "redirect:/" + currentLanguage + "/home"; // Moved temporarily

The default language redirect should be implemented in YadaLocalePathVariableFilter

HTML

The standard Thymeleaf @{url} syntax has been retrofitted to automatically handle language in the path: the current locale will be added at the start of every url, so @{/home} becomes /de/home for example.

A language menu can be easily implemented with code like this:

<select id="langmenu">
      <option value="en" th:selected="${#locale.language=='en'}">EN</option>
      <option value="it" th:selected="${#locale.language=='it'}">IT</option>
</select>

Javascript

The language in the path variable can be changed via javascript using

yada.changeLanguagePathVariable(locale);

where "locale" is the ISO2 locale code. This code could be called when choosing from the list of languages:

$("#langmenu").change(function() {
      var locale = $(this).val();
      yada.changeLanguagePathVariable(locale);
});

Configuring "language request parameter"

URL Example:

http://www.example.com/myHome?lang=it

This is easier to configure because you don’t need to change the Java code. The application configuration is the same but you need to set localePathVariable="false".

Check that this stil works and what it does. I think YadaWebUtil.enhanceUrl() doesn't work properly

Coding with i18n text

Static text

To implement localized static text just use the standard Spring "MessageSource" concept: store all text in different message.properties files, indexed by a key.

The Yada Framework expects message source files to be in the WEB-INF/messages folder, with a file name in the messages[_<lang>].properties format. Example:

messages_de.properties
messages_fr.properties
messages_ru.properties
messages.properties

Each file stores the text of a different language. You don’t need to add them all immediately: start from the default language in messages.properties then add the translations when they become needed. The default language can be any language that you consider to be the "base" language: all keys that are not found in a specific language are searched in the default one; when not found, the key is shown as text.

The content of the file is in the standard Java "MessageFormat" format:

Example:

validation.empty = This value can't be empty
validation.password.length = Password can''t be shorter than {0} characters and longer than {1}
wait.time = Wait {0} {0, choice, 0#minutes|1#minute|1<minutes} and retry
files.total = There {0, choice, 0#are no files|1#is one file|1<are {0,number,integer} files}.

In particular:

{0}, {1} … are ways of passing parameters (there can be any)
when a parameter is specified, any single quote must be escaped by another single quote
there’s a powerful way of specifying variations like singular/plural (choice format):
- | separates choices
- # is an exact match, so 1# means "when 1 equals the parameter"
- < is "less than" where the subject is the number, so <1 means "when 1 is less than the parameter"
- > is "greater than" where the subject is the number, so >1 means "when 1 is greater than the parameter"
- each choice can contain text and/or parameters like {1} or {2,number,integer}

In production, files are reloaded every 600 seconds (10 minutes) to pick up changes.

The Message Source configuration is implemented in YadaAppConfig.messageSource()

Usage with Thymeleaf

The syntax to show a localized string in Thymeleaf is #{<key>}. Example:

<p th:text="#{validation.empty}">Any placeholder text here will be overwritten</p>

See the Thymeleaf docs for more details.

Emails

Email templates can use the same message properties of HTML files, or be saved in separate files, one per language. See Internationalization in the email chapter.

Usage in Java

To get the localized text in java you first autowire a MessageSource bean, then use the getMessage() method:

@Autowired private MessageSource messageSource;

public String someMethod(Locale locale) {
  String msg1 = messageSource.getMessage("validation.empty", null, locale);
  String msg2 = messageSource.getMessage("validation.password.length", new Object[]{5, 10}, locale);

Database fields

The Yada Framework uses the table-per-attribute approach to multivalue string attributes. An @Entity with a localized string attribute can be defined with a Map<Locale, String> so that values are related to their locale:

@ElementCollection
@Column(length=8192)
@MapKeyColumn(name="locale", length=32)
@CollectionTable(
	uniqueConstraints = @UniqueConstraint(columnNames={"MyEntityName_id", "locale"})
)
private Map<Locale, String> description = new HashMap<>();

The uniqueContratints (optional) annotation ensures that there can’t be two values for a given locale. The "MyEntityName_id" value should be the actual column name in the ElementCollection table: it usually is the name of the Entity followed by "_id" with a first capital letter.

To retrieve the value in a specific locale, use YadaUtil.getLocalValue(). This will return the value in the specified locale or null. If a default locale has been configured (see Application configuration above) then the default locale will be tried before returning null. This is useful when all locales have the same value and you only want to set it once: the value for the default language will be "inherited" by all current and future configured languages.

String productDesc = YadaUtil.getLocalValue(product.getDescription(), locale);
String productDesc = YadaUtil.getLocalValue(product.getDescription()); // Use current locale

It can be very convenient to add to the entity a method that retrieves the value in the current locale (the locale of the current request):

@Entity
public class Product {
  ...
  @ElementCollection
  @Column(length=8192)
  @MapKeyColumn(name="locale", length=32)
  @CollectionTable(
    uniqueConstraints = @UniqueConstraint(columnNames={"Product_id", "locale"})
  )
  private Map<Locale, String> description = new HashMap<>();

  ...
  public String getDescriptionLocal() {
    return YadaUtil.getLocalValue(description);
  }

This allows for a simple use in Thymeleaf:

<p th:text="${product.descriptionLocal}">Some description</p>

Be careful that Maps are lazy by default, so the localized value won’t be returned outside of a transaction. The solution is to either prefetch the map in the DAO (most efficient solution) or to eagerly load it (simpler implementation).

Prefetching in the DAO can be done by simply calling a .size() or by using YadaUtil.prefetchLocalizedStrings() and similar methods:

public Product findProduct(Long id) {
	Product product = em.find(Product.class, id);
	// Either call .size()
	product.getDescription().size();
	// Or prefetch all localized strings via reflection
	YadaUtil.prefetchLocalizedStrings(product, Product.class);
	return product;
}

In order to eagerly load the attribute, use FetchType.EAGER together with FetchMode.SELECT:

@ElementCollection(fetch = FetchType.EAGER)
@Fetch(FetchMode.SELECT)
@Column(length=8192)
@MapKeyColumn(name="locale", length=32)
@CollectionTable(
	uniqueConstraints = @UniqueConstraint(columnNames={"Product_id", "locale"})
)
private Map<Locale, String> description = new HashMap<>();

not using FetchMode.SELECT may result in a cross join that loads a huge amount of values into memory, possibly causing an OutOfMemory exception!

Considering the user time zone

Setting the user time zone

The user time zone is automatically retrieved on page load by yada.js to what the browser reports and sent to the server once per session. The server stores this value in the user session with the YadaConstants.SESSION_USER_TIMEZONE key. This session attribute is read after user authentication and stored in the database in the YadaUserProfile table under the timezone column. This value will then be available to the application via YadaUserProfile.getTimezone().

The application should offer users to change their time zone manually. In such case, the code setting the time zone should also set the YadaUserProfile.timezoneSetByUser flag in order to stop the above automatic change at each login:

Figure 1. User Time Zone Form

if (!userProfileForm.getTimezone().equals(userProfile.getTimezone().getID())) {
	// If the timezone is different from before, set the flag
	userProfile.setTimezoneSetByUser(true);
}
userProfile.setTimezone(TimeZone.getTimeZone(userProfileForm.getTimezone()));

The time zone should also be set on user registration. When receiving the registration request in the "/signup" controller, the automatically retrieved time zone should be set in the YadaRegistrationRequest:

TimeZone userTimezone = null;
HttpSession session = request.getSession(false);
if (session!=null) {
	userTimezone = (TimeZone) session.getAttribute(YadaConstants.SESSION_USER_TIMEZONE);
}
yadaRegistrationRequest.setTimezone(userTimezone);

When creating the new user after confirmation, the user time zone should be set in the user profile:

TimeZone userTimezone = registrationRequest.getTimezone();
if (userTimezone==null) {
	userTimezone = TimeZone.getTimeZone("Europe/Rome"); // Default
}
userProfile.setTimezone(userTimezone);

Working with date and time

TO BE CONTINUED...