Post on 15-Aug-2020
INTERNATIONALINFRASTRUCTURE AT TWITTER
Cameron Dutro@camertron
˜
Cameron DutroInternational EngineeringTwitter, Inc@camertron
core value:
Reach every person on the planet.
core value:
Reach every person on the planet.
~7 billion people
Practices
Coding standards
Practices
Coding standards Release process
Practices
Coding standards Release process Twitter Translation Center
Practices
Coding standards Release process Twitter Translation Center Birdshot
Practices
Coding standards Release process Twitter Translation Center Birdshot Pseudo-locales
Practices
Coding standards Release process Twitter Translation Center Birdshot Pseudo-locales
Practices
Libraries
Coding standards Release process Twitter Translation Center Birdshot Pseudo-locales
translation-api
Practices
Libraries
Coding standards Release process Twitter Translation Center Birdshot Pseudo-locales
translation-api international-release
Practices
Libraries
Coding standards Release process Twitter Translation Center Birdshot Pseudo-locales
translation-api international-release twitter-cldr
Practices
Libraries
Practices
Coding OverviewPutting translations in source code
fast_gettext gem _() function multiple backends gettext compatible faster inline ruby only
javascript _() function hash from json dump fast inline
var twttr = { i18n: { ... } };
function _(key) { return twttr.i18n[key] || key;}
mustache {{_i}}{{/i}} tags logic-less views language independent
{ }<%= _("Hello world")%>
RUBY / ERB
{ }function() { _("Hello world");}
JAVASCRIPT
{ }<p> {{_i}}Hello world{{/i}}</p>
MUSTACHE
Release ProcessMaximizing efficiency and scalability
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.json ˜
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.json ˜adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.sv.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.es.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.ko.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.json ˜
asse
t co
mpi
ler
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.sv.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.es.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.ko.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.rbadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.json ˜adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}DE
KO
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
{}AR
asse
t co
mpi
ler
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.sv.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.es.json
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.ko.json
twitter.com
adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.xml ˜adsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.ja.xmladsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.jsonadsf fasdf asdfss asdfasdfasdf asdfasssssdf asdfasdfasfsdf asf asdfasdf
.hu.xml
.strings(UTF-16)
.strings(UTF-16)
.xml(UTF-8)
.strings(UTF-16)
.xml(UTF-8)
.json(UTF-8)
.strings(UTF-16)
.xml(UTF-8)
.json(UTF-8)
mobile.twitter.com.yml
(UTF-8)
full localization solution JSON interchange format crowdsourced platform independent
Twitter Translation Center http://translate.twttr.com
Twitter Translation Center http://translate.twttr.com
800,00017,0002,300,000450,000
volunteersactive phrasestranslationsapproved translations
Twitter Translation Center http://translate.twttr.com
importorganizetranslatemoderateexportlearnglobalize
Birdshot Automated screenshot framework
1,00020%
total screenshotscoverage
Pseudo-locales Localization testing with fake data
Pseudo-locales Localization testing with fake data
Pseudo-locales Localization testing with fake data
Libraries
translation-apiProgrammatic access to the Translation Center
extract phrases from source code import / export json files phrase diffs
TranslationAPI::CanonicalsExtractor.new([ "/path/to/view_file1.js", "/path/to/view_file2.html.erb", "/path/to_view_file3.mustache"])
TranslationAPI::CanonicalsExtractor.new([ "/path/to/view_file1.js", "/path/to/view_file2.html.erb", "/path/to_view_file3.mustache"])
TranslationAPI::API.add_import( phrases, file_name, project_name, tag_id, comment, submitter)
TranslationAPI::CanonicalsExtractor.new([ "/path/to/view_file1.js", "/path/to/view_file2.html.erb", "/path/to_view_file3.mustache"])
TranslationAPI::API.add_import( phrases, file_name, project_name, tag_id, comment, submitter)
TranslationAPI::API.import_diff( phrases, file_type, project_name)
international-release Automated cross-project build tools
automated import / export static analysis
config = YAML.load_file(@options[:config_file])project = InternationalRelease::Base.detect_projectexchanger = InternationalRelease::Base.get_exchanger(project, config)exchanger.run(@options)
AUTOMATED IMPORT / EXPORT
runner = InternationalRelease::StaticAnalysis::Runner.new( Dir.glob(File.join(`pwd`.chomp, "**/**")))
runner.run
STATIC ANALYSIS
twitter-cldr http://github.com/twitter/twitter-cldr-rb
CLDR for Ruby / Rails / JavaScript native ICU for Ruby dates / times / numbers sorting / bidi / utf-8
DATES / TIMES
# 21:46:09 24/04/2012DateTime.now.localize(:es).to_s
DATES / TIMES
# 21:46:09 24/04/2012DateTime.now.localize(:es).to_s
# 21:46:09 UTC -0700 2012. április 24., szerdaDateTime.now.localize(:hu).to_full_s
DATES / TIMES
# 21:46:09 24/04/2012DateTime.now.localize(:es).to_s
# 21:46:09 UTC -0700 2012. április 24., szerdaDateTime.now.localize(:hu).to_full_s
# 21時46分09秒 UTC -07002012年4月24日水曜日DateTime.now.localize(:ja).to_full_s
DATES / TIMES
# 21:46:09 24/04/2012DateTime.now.localize(:es).to_s
dt = TwitterCldr::LocalizedDateTime.new(DateTime.now, :es)dt.to_s # 21:46:09 24/04/2012
# 21:46:09 UTC -0700 2012. április 24., szerdaDateTime.now.localize(:hu).to_full_s
# 21時46分09秒 UTC -07002012年4月24日水曜日DateTime.now.localize(:ja).to_full_s
PLURALS
str = _('there %<{"horse_count": {"one": "is one horse", "other": "are %{horse_count} horses"}}> in the barn').localize
str % { :horse_count => 3 } # there are 3 horses in the barn
PLURALS
str = _('there %<{"horse_count": {"one": "is one horse", "other": "are %{horse_count} horses"}}> in the barn').localize
str % { :horse_count => 3 } # there are 3 horses in the barn
'в амбаре есть %<{"horse_count": {"one": "1 лошадь", "few":"%{horse_count} лошади", "other": "%{horse_count} лошадей"}}>'
PLURALS
str = _('there %<{"horse_count": {"one": "is one horse", "other": "are %{horse_count} horses"}}> in the barn').localize
str % { :horse_count => 3 } # there are 3 horses in the barn
'в амбаре есть %<{"horse_count": {"one": "1 лошадь", "few":"%{horse_count} лошади", "other": "%{horse_count} лошадей"}}>'
str % { :horse_count => 3 } # в амбаре есть 3 лошади
NORMALIZATION
# [101, 115, 112, 97, 241, 111, 108]"español".localize.code_points
NORMALIZATION
# [101, 115, 112, 97, 241, 111, 108]"español".localize.code_points
# [101, 115, 112, 97, 110, 771, 111, 108]"español".localize.normalize.code_points
NORMALIZATION
# [101, 115, 112, 97, 241, 111, 108]"español".localize.code_points
# [101, 115, 112, 97, 110, 771, 111, 108]"español".localize.normalize.code_points
# [101, 115, 112, 97, 241, 111, 108]"español".localize.normalize(:using => :NFC).code_points
COLLATION
firstmanmanymañanamaxinext
firstman
manymaxi
mañananext
Regular Locale-aware
COLLATION
["first", "man" ...].localize.sort.to_a
firstmanmanymañanamaxinext
firstman
manymaxi
mañananext
Regular Locale-aware
Fragen?perguntas?सवाल ?
questions? 質問は？
أسئلة؟질문 있으세요?