Modern javascript localization with c-3po and the good old gettext
-
Upload
alexander-mostovenko -
Category
Education
-
view
93 -
download
1
Transcript of Modern javascript localization with c-3po and the good old gettext
Mostovenko Alexander
Company: EVO company
Twitter: @MostovenkoA
Github: github.com/AlexMostMe
About this talk
About this talk
Localization process in general and more about gettext format.
About this talk
Localization process in general and more about gettext format.
What is c-3po and how it helped us to improve existing process?
About this talk
Localization process in general and more about gettext format.
What is c-3po and how it helped us to improve existing process?
Continuous translations.
Our initial tasks
Our initial tasks
1. Localize frontend on several projects:
(cabinet, company sites)
Bigl.ua
others...
Our initial tasks
1. Localize frontend on several projects:
(cabinet, company sites)
Bigl.ua
others ...
2. Setup translation process when devs and translators can work independently.
Gettext (.po)
ICU
...Localization
formats
ICU - International Components for Unicodehttp://userguide.icu-project.org/
ICU - International Components for Unicodehttp://userguide.icu-project.org/
ICU features >>> gettext features
ICU is cool, but ...
ICU is cool, but ...
1. Poor tooling (editors for translators e.t.c).
ICU is cool, but ...
1. Poor tooling (editors for translators e.t.c).
2. Our backend already used gettext.
ICU - cool but ..."{gender_of_host, select, " "female {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to her party.}" "=2 {{host} invites {guest} and one other person to her party.}" "other {{host} invites {guest} and # other people to her party.}}}" "male {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to his party.}" "=2 {{host} invites {guest} and one other person to his party.}" "other {{host} invites {guest} and # other people to his party.}}}" "other {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to their party.}" "=2 {{host} invites {guest} and one other person to their party.}" "other {{host} invites {guest} and # other people to their party.}}}}"
GNU gettexthttps://www.gnu.org/software/gettext/
GNU gettexthttps://www.gnu.org/software/gettext/
1. Simple format.
GNU gettexthttps://www.gnu.org/software/gettext/
1. Simple format.
2. Has good ecosystem for translation process.
gettext idea
‘Hello world’
Source code
gettext idea
gettext(‘Hello world’)
Source code
gettext idea
gettext(‘Hello world’)
Source code
‘Hello world’
.pot file
gettext idea
gettext(‘Hello world’)
Source code
msgid: ‘Hello world’msgstr: ‘’
.po file
‘Hello world’
.pot file
gettext idea
gettext(‘Hello world’)
Source code
msgid: ‘Hello world’msgstr: ‘Здоровеньки були’
.po file
‘Hello world’
.pot file
gettext idea
gettext(‘Hello world’)
Source code
msgid: ‘Hello world’msgstr: ‘Здоровеньки були’
.po file
‘Hello world’
.pot file
gettext idea
gettext workflow
Code .pot Extracting translation strings from code (AST traverse)extract
gettext workflow
Code .pot Extracting translation strings from code (AST traverse)
.pot .po Merge new extracted translations from .pot file with existing. msgmerge
extract
merge .po.po
gettext workflow
Code .pot Extracting translation strings from code (AST traverse)
.pot .po Merge new extracted translations from .pot file with existing. msgmerge
extract
merge
Translator adds translationstranslation
.po.po
.po.po.po.po.po.po
gettext workflow
Code .pot Extracting translation strings from code (AST traverse)
.pot .po Merge new extracted translations from .pot file with existing. msgmerge
extract
merge
Translator adds translationstranslation
.po.po
.po.po.po.po.po.po
resolve CodePlacing translations back to the code (by locale).po.po.po
.po file structure
Example .po file
msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-start\n"
"PO-Revision-Date: 2017-02-01 22:22+0200\n"
"Language-Team: Ukrainian\n"
"Language: uk\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n!=1);\n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"
msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-start\n"
"PO-Revision-Date: 2017-02-01 22:22+0200\n"
"Language-Team: Ukrainian\n"
"Language: uk\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n!=1);\n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"
Headers
msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-start\n"
"PO-Revision-Date: 2017-02-01 22:22+0200\n"
"Language-Team: Ukrainian\n"
"Language: uk\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n!=1);\n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"
Translations
"Plural-Forms:
nplurals=2;
plural=(n!=1);\n"
Plural-forms header
"Plural-Forms:
nplurals=2;
plural=(n!=1);\n"
Plural-forms header
Plurals count
"Plural-Forms:
nplurals=2;
plural=(n!=1);\n"
Plural-forms header
Plurals formula
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
1 to 1 translations
#: app.js:10
msgid "${ n } second"
msgid_plural "${ n } seconds"
msgstr[0] "${ n } секунда"
msgstr[1] "${ n } секунди"
msgstr[2] "${ n } секунд"Plural forms example
#. Notes for the translator
#: app.js:6
#, flag
msgid "Choose locale"
msgstr "Оберіть локаль"
Comments
msgctx "email"
msgid "Hello"
msgstr "Привіт"
msgctx "main page"
msgid "Hello"
msgstr "Вітаємо"
Context example
c-3pohttps://c-3po.js.org/
c-3po
Code .pot Extracting translation strings from code (AST traverse)extract
resolve CodePlacing translations back to the code (by locale).po.po.po
Library:
https://github.com/c-3po-org/c-3po
Core functionality.
Translation functions (tags):
t, ngettext, gettext, jt
Babel-plugin:
https://github.com/c-3po-org/babel-plugin-c-3po
Extract, Resolve, Validation
Key ideas
Key idea 1. Tagged template strings for formatting
c-3po translations are based on tagged template strings
import { t } from 'c-3po'
const name = 'Mike';
console.log(t`Hello ${name}`);
What is a template string?
What is template string?
console.log('Hello ' + name)
console.log(`Hello ${name}`)
What is a tagged template string?
const name = ‘Mike’
t`Hello ${name}`
function t(strs, ...exprs) {
// strs -> [‘Hello’, ‘’]
// exprs -> [‘Mike’]
}
Valid javascript
t `Hello ${name}`
t(5)`Hello ${name}`
t(5) `Hello ${name}`
Why just not simple functions but tags?
Why just not simple functions but tags?
c-3po must work without transpilation
Why just not simple functions but tags?
// Without babel transpile !!!
const name = ‘Mike’
t(`Hello ${name}`)
function t(str) {
// strs -> ‘Hello Mike’
// no chance to get ‘Hello ${name}’ msgid here :(
}
Existing solutions
Force you to use sprintf formatting(i18next, jed e.t.c)
sprintf
gettext("Hello %s", user)
gettext("%2$s %3$s a %1$s", "cracker", "Polly", "wants")
gettext("Current timestamp: %d", Date.now)
sprintf
● Alien formatting for js
sprintf
● Alien formatting for js● No reason to use es6 template strings
sprintf
● Alien formatting for js● No reason to use es6 template literals● Extra CPU work on the client (sprintf parsing)
Key idea 2. Precompile translation on a build step
Traditional translations resolve flow
Traditional translations resolve flow
Load assets to the client (browser)
Traditional translations resolve flow
Load assets to the client (browser)
Resolve locale (from cookie, params e.t.c)
Traditional translations resolve flow
Load assets to the client (browser)
Resolve locale (from cookie, params e.t.c)
Fetch locale data
Traditional translations resolve flow
Load assets to the client (browser)
Resolve locale (from cookie, params e.t.c)
Fetch locale data
Apply translations
Wanted translations resolve flow
Wanted translations resolve flowLoad assets to the client
(browser)
Wanted translations resolve flowLoad assets to the client
(browser)
That’s it, they are already localized !!!
Benefits of precompiled translations:
Benefits of precompiled translations:
● Smaller bundle size
Benefits of precompiled translations:
● Smaller bundle size● Less work on the client
Benefits of precompiled translations:
● Smaller bundle size● Less work on the client● No wait until translations are loaded and applied
Key idea 3.Works with and without babel
Demo on jsfiddle
Key idea 4. Can extract and resolve translations from
everything that works with babel (jsx)
Extract and resolve translations with simple config
Extract:
{ "extract": { "output": "extract.pot" } }
Resolve:
{ "resolve": { "translations": "uk.po" } }
https://c-3po.js.org/quick-start.html
Key idea 5. Efficient dev and prod setup
Requirements for the dev setup
Requirements for the dev setup● Faster builds
Requirements for the dev setup● Faster builds● Simple integration
Dev setup with c-3po
Dev setup with c-3po● Use whole c-3po lib (import c-3po).
Dev setup with c-3po
● Use whole c-3po lib (import c-3po).
● Babel plugin only for extraction translations
Requirements for the prod setup
Requirements for the prod setup● Smaller result assets
Requirements for the prod setup
● Smaller result assets
● Faster locale load
Prod setup with c-3po
Prod setup with c-3po● Use c-3po mock lib (alias in webpack e.t.c).
Prod setup with c-3po
● Use c-3po mock lib (alias in webpack e.t.c).
● Babel plugin for transformations.
Prod setup with c-3po
● Use c-3po mock lib (alias in webpack e.t.c).
● Babel plugin for transformations.
● Separate build for each locale.
Key idea 6.Validate translation strings
Validation problem
msgid "http://some.random.domain.link"msgstr ?
msgid ""msgstr ?
msgid "translate ${ test ? result1() : result2() }"msgstr ?
msgid "5"msgstr ?
c-3po validation (empty str)
t``
Module build failed: SyntaxError: Can not translate empty string
5 |
6 |
> 7 | t``
| ^
c-3po validation (only numbers)
t`3243243`
ERROR in Error: Module build failed: SyntaxError: Can not translate '3243243'
2 |
3 |
> 4 | t`3243243`
| ^
c-3po validation (programs inside translations)
t`Translate ${someVar ? callFn1() : callFn2()}`
ERROR in Error: Module build failed: SyntaxError: You can not use
ConditionalExpression '${someVar ? callFn1() : callFn2()}' in localized
strings
2 |
3 |
> 4 | t`Translate ${someVar ? callFn1() : callFn2()}`;
| ^
5 |
c-3po validation (only var wrap)
t`${someVar}`
ERROR in Error: Module build failed: SyntaxError: Can not translate '$
{ someVar }'
2 |
3 |
> 4 | t`${someVar}`;
| ^
msgid:’${someVar}’
Validate not translated strings
Validation (no translated str)
t`non translated str`
ERROR in Error: Module build failed: SyntaxError: No "non translated str" in
"uk.po" file
2 |
> 3 | t`non translated str`;
https://c-3po.js.org/configuration.html#configresolveunresolved-string-one-of-fail-warn-skip
Key idea 7.Can use any default locale in sources
(standard gettext uses only English locale)
The problem
Standard ngettext accepts only 2 plural forms
char * ngettext(const char * msgid1, const char * msgid2, unsigned long int n);
Ukrainian/Russian strings in sources
How to be with plurals?
Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2 plural forms?
Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2 plural forms? What if we have more or less than 2 plural forms?
Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2 plural forms? What if we have more or less than 2 plural forms?
ngettext for other locales ( plural forms > 2 )
ngettext for other locales ( plural forms > 2 )
Doc - https://c-3po.js.org/ngettext.html
Change defaultHeaders setting to:
'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'
ngettext for other locales ( plural forms > 2 )
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html
Change defaultHeaders setting to:
'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'
ngettext(msgid`${n} банан`, `${n} банана`, `${n} бананів`, n)
c-3po tags and functions
t - tag gettext
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html
t - tag gettext
t`Hello ${name}` < Usage
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html
t - tag gettext
t`Hello ${name}`
msgid "Hello ${ name }"msgstr "Привіт ${ name }"
< Usage
< Extract
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html
t - tag gettext
t`Hello ${name}`
msgid "Hello ${ name }"msgstr "Привіт ${ name }"
< Usage
< Extract
`Привіт ${name}` < Resolve
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html
ngettext - plural forms
ngettext - plural forms
ngettext(msgid`${n} time clicked`, `${n} times clicked`, n) < Usage
ngettext - plural forms
ngettext(msgid`${n} time clicked`, `${n} times clicked`, n)
msgid "${ n } time clicked"msgid_plural "${ n } times clicked"msgstr[0] "${ n } time clicked [translated]"msgstr[1] "${ n } times clicked [translated]"
< Usage
< Extract
Forgot msgid ?
ERROR in Error: Module build failed: SyntaxError: First argument must be tagged
template expression. You should use 'msgid' tag
2 |
3 |
> 4 | ngettext(`${hours} hour`, `${hours} hours`, hours);
| ^
5 |
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html
ngettext - resolve
function _tag_ngettext(n, args) {
return args[+(n != 1)];
}
_tag_ngettext(n,
[n + " time clicked [translated]", n + " times clicked [translated]"]));
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html
First contribution !!! jt (jsx-gettext)
First contribution !!! jt (jsx-gettext)
jt - jsx gettext
jt - jsx gettext
const btn = <button key="btn">{ t`me`
}</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage
jt - jsx gettext
const btn = <button key="btn">{ t`me`
}</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage
< Extractmsgid "Click ${ btn }"
msgstr "Click ${ btn } [translated]"
jt - jsx gettext
const btn = <button key="btn">{ t`me`
}</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage
< Extractmsgid "Click ${ btn }"
msgstr "Click ${ btn } [translated]"
['Click ', btn, ' [translated]'] < Resolve
gettext
gettext
gettext('simple gettext'); < Usage
gettext
gettext('simple gettext'); < Usage
< Extractmsgid "simple gettext"
msgstr "simple gettext [translated]"
gettext
gettext('simple gettext'); < Usage
< Extractmsgid "simple gettext"
msgstr "simple gettext [translated]"
'simple gettext [translated]' < Resolve
Why add gettext func if we have ‘t’ tag?
Legacy - *.coffee *.eco (100 K+ LOC)
1. Wrap string in gettext inside *.coffee *.eco
2. Add babel loader after coffee loader to the
webpack pipeline
3. Profit !!!
Even more features
Aliasinghttps://c-3po.js.org/aliasing.html
Babel plugin will extract aliased translation
import { t as i18n } from 'c-3po'
i18n`this translation will work`
Multilinehttps://c-3po.js.org/multiline-strings.html
import { t } from 'c-3po';
function test(name) { return t`multi line string with multiple line breaks and with formatting ${name}`}
Multiline (auto dedent)
Translator comments
Translator comments// translator: some description for the extracted string. t`some string`
Translator comments// translator: some description for the extracted string. t`some string`
#. some description for the extracted stringmsgid "some string"msgstr ""
Summary
Summary● Native js tagged template strings for
formatting
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step● Works with and without babel
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step● Works with and without babel● Extracts and resolves translations from
everything that works with babel (jsx)
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step● Works with and without babel● Extracts and resolves translations from
everything that works with babel (jsx)● Efficient dev and prod setup
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step● Works with and without babel● Extracts and resolves translations from
everything that works with babel (jsx)● Efficient dev and prod setup● Validation for translated strings
Summary● Native js tagged template strings for
formatting● Can precompile translation on a build step● Works with and without babel● Extracts and resolves translations from
everything that works with babel (jsx)● Efficient dev and prod setup● Validation for translated strings● Can use any default locale in sources (not
only English)
Continuous translations process
Devs VCSrepo
Continuous translation process
Translator
Devs VCSrepo
Continuous translation processpush
Translator
Devs VCSrepo
Continuous translation processpush notify
Translator
Devs VCSrepo
Continuous translation processpush
translate
notify
Translator
Devs VCSrepo
Continuous translation processpush
translate
notify
update Translator
Tools for translators
Poeditor - https://poeditor.com/
Transifex - https://www.transifex.com/
Weblate - https://weblate.org
Devs VCSrepo
Continuous translation processpush
push
pull
update Translator
notify
translate
Weblate example
Weblate example
Weblate - https://docs.weblate.org/en/latest/
Links
● c-3po doc - https://c-3po.js.org/
● c-3po quick start - https://c-3po.js.org/quick-start.html
● gettext - https://www.gnu.org/software/gettext/
● ICU - http://userguide.icu-project.org/
● Weblate - https://weblate.org
Thx :)