How to Internationalize a Package

Tip

For multilingual websites we recommend using the UTF8 charset. In order for AOLserver to use utf8 you need to set the config parameters OutputCharset and URLCharset to utf-8 in your AOLserver config file (use the etc/config.tcl template file). This is the default for OpenACS 5.1 and later. For sites running on Oracle you need to make sure that AOLserver is running with the NLS_LANG environment variable set to .UTF8. You should set this variable in the nsd-oracle run script (use the acs-core-docs/www/files/nds-oracle.txt template file).

  1. Replace all text with temporary message tags. From/acs-admin/apm/, select a package and then click on Internationalization, then Convert ADP, Tcl, and SQL files to using the message catalog.. This pass only changes the adp files; it does not affect catalog files or the catalog in the database.

    You will now be walked through all of the selected adp pages. The UI shows you the intended changes and lets you edit or cancel them key by key.

    • The automatic process can easily get confused by tags within message texts, so that it tries to create two or three message keys for one long string with a tag in the middle. In these cases, uncheck those keys during the conversion and then edit the files directly. For example, this code:

        <p class="form-help-text"><b>Invitations</b> are sent,
          when this wizard is completed and casting begins.</p>

      has a bold tag which confuses the converter into thinking there are two message keys for the text beginning "Invitations ..." where there should be one:

      Instead, we cancel those keys, edit the file manually, and put in a single temporary message tag:

        <p class="form-help-text"> <#Invitations_are_sent <b>Invitations</b> are sent,
          when this wizard is completed and casting begins.#>
        </p>
    • Complex if statements may produce convoluted message keys that are very hard to localize. Rewrite these if statements. For example:

      Select which case <if @simulation.casting_type@ eq "open">and
      role</if> to join, or create a new case for yourself.  If you do not
      select a case <if @simulation.casting_type@ eq "open">and role</if>
      to join, you will be automatically assigned to a case <if
      @simulation.casting_type@ eq "open">and role</if> when the
      simulation begins.

      ... can be rewritten:

      <if @simulation.casting_type@ eq "open">
      
      Select which case and role to join, or create a new case for
      yourself.  If you do not select a case and role to join, you will
      be automatically assigned to a case and role when the simulation
      begins.
      
      </if>
      <else>
      
      Select which case to join, or create a new case for
      yourself.  If you do not select a case to join, you will
      be automatically assigned to a case when the simulation
      begins.
      
      </else>
    • Check for duplicate keys. For common words such as Yes and No, you can use a library of keys at acs-kernel. For example, instead of using myfirstpackage.Yes, you can use acs-kernel.Yes. You can also use the Message Key Search facility to find duplicates. Be careful, however, building up sentences from keys because grammar and other elements may not be consistent across different locales.

      Additional discussion: Re: Bug 961 ("Control Panel" displayed instead of "Administer"), Translation server upgraded, and Localization questions.

  2. Replace the temporary message tags in ADP files. From the same Convert ADP ... page in /acs-admin/apm as in the last step, repeat the process but deselect Find human language text ... and select Replace <# ... #> tags ... and click OK. This step replaces all of the temporary tags with "short" message lookups, inserts the message keys into the database message catalog, and then writes that catalog out to an xml file.

  3. Replace human-readable text in TCL files with temporary tags. Examine all of the tcl files in the packages for human-readable text and replace it with temporary tags. The temporary tags in TCL are slightly different from those in ADP. If the first character in the temporary tag is an underscore (_), then the message keys will be auto-generated from the original message text. Here is an unmodified tcl file:

    set title "Messages for $a(name) in $b(label)"
    set context [list [list . "SimPlay"] \
                      [list [export_vars -base case-admin { case_id }] \ 
                        "Administer $a(name)"] \
                      "Messages for $a(name)"]
    

    ... and here is the same file after temporary message tags have been manually added:

    set title <#admin_title Messages for %a.name% in %b.label%#>
    set context [list [list . <#_ SimPlay#>] \
                      [list [export_vars -base case-admin { case_id }] \
                        <#_ Administer %a.name%#>] \
                      <#_ Messages for %a.name%#>]
    

    Note that the message key case_admin_page_title was manually selected, because an autogenerated key for this text, with its substitute variables, would have been very confusing

  4. Replace the temporary message tags in TCL files. Repeat step 2 for tcl files. Here is the example TCL file after conversion:

    set title [_ simulation.admin_title]
    set context [list [list . [_ simulation.SimPlay]] \
                      [list [export_vars -base case-admin { case_id }] \
                        [_ simulation.lt_Administer_name_gt]] \
                      [_ simulation.lt_Messages_for_role_pre]]
    
  5. Internationalize SQL Code. If there is any user-visible TCL code in the .sql or .xql files, internationalize that the same way as for the TCL files.

  6. Internationalize Package Parameters.  See Multilingual APM Parameters

  7. Internationalize Date and Time queries. 

    1. Find datetime in .xql files. Use command line tools to find suspect SQL code:

      grep -r "to_char.*H" *
      grep -r "to_date.*H" *
      
    2. In SQL statements, replace the format string with the ANSI standard format, YYYY-MM-DD HH24:MI:SS and change the field name to *_ansi so that it cannot be confused with previous, improperly formatting fields. For example,

      to_char(timestamp,'MM/DD/YYYY HH:MI:SS') as foo_date_pretty

      becomes

      to_char(timestamp,'YYYY-MM-DD HH24:MI:SS') as foo_date_ansi
    3. In TCL files where the date fields are used, convert the datetime from local server timezone, which is how it's stored in the database, to the user's timezone for display. Do this with the localizing function lc_time_system_to_conn:

      set foo_date_ansi [lc_time_system_to_conn $foo_date_ansi]

      When a datetime will be written to the database, first convert it from the user's local time to the server's timezone with lc_time_conn_to_system.

    4. When a datetime field will be displayed, format it using the localizing function lc_time_fmt. lc_time_fmt takes two parameters, datetime and format code. Several format codes are usable for localization; they are placeholders that format dates with the appropriate codes for the user's locale. These codes are: %x, %X, %q, %Q, and %c.

      set foo_date_pretty [lc_time_fmt $foo_date_ansi "%x %X"]

      Use the _pretty version in your ADP page.

      • %c: Long date and time (Mon November 18, 2002 12:00 AM)

      • %x: Short date (11/18/02)

      • %X: Time (12:00 AM)

      • %q: Long date without weekday (November 18, 2002)

      • %Q: Long date with weekday (Monday November 18, 2002)

      The "q" format strings are OpenACS additions; the rest follow unix standards (see man strftime).

  8. Internationalize Numbers.  To internationalize numbers, use lc_numeric $value, which formats the number using the appropriate decimal point and thousand separator for the locale.

  9. Internationalizing Forms. When coding forms, remember to use message keys for each piece of text that is user-visible, including form option labels and button labels.

  10. Checking the Consistency of Catalog Files.  This section describes how to check that the set of keys used in message lookups in tcl, adp, and info files and the set of keys in the catalog file are identical. The scripts below assume that message lookups in adp and info files are on the format \#package_key.message_key\#, and that message lookups in tcl files are always is done with one of the valid lookups described above. The script further assumes that you have perl installed and in your path. Run the script like this:

    acs-lang/bin/check-catalog.sh package_key

    where package_key is the key of the package that you want to test. If you don't provide the package_key argument then all packages with catalog files will be checked. The script will run its checks primarily on en_US xml catalog files.

View comments on this page at openacs.org