Language tools/status

Last update on: 2014-12-monthly

2011-05-16

Alolita Sharma and Erik Möller are currently gathering requirements on this project with the help of possible customers, including the language committee.

2011-06-30

Alolita Sharma published two RfPs to assemble a dedicated team to tackle localization issues from a feature development and outreach perspective. She is currently reviewing applications and contracts.

2011-08-01

Alolita Sharma continued to assemble a team dedicated to localization, and to work on the project definition and priorities.

2011-08-31

Siebrand Mazeland, Niklas Laxström and Gerard Meijssen joined the project in August. Niklas focused on code review for MediaWiki 1.18 regarding internationalization issues; he also introduced more flexible language fallback sequences. Siebrand worked on the product roadmap for 2011-2012, and started to plan hackathons & localization sprints in India in November. The Kiwix offline app was added to translatewiki.net, where it can now be localized.

2011-09-30

Siebrand Mazeland created a list of initial Indic target languages (and their properties) to improve support in MediaWiki for India's 28 official languages. Roan Kattouw deployed the Babel extension and found issues in the Translate extension, which were fixed by Niklas Laxström. Niklas started to rewrite MediaWiki's logging system to better support internationalization. He also added real-time statistics on translation completion, fixed a number of i18n bugs and reviewed code. Santhosh Thottingal implemented interface changes in the Narayam extension (suggested by Trevor Parscal), which was deployed to select Wikimedia wikis. He fixed bugs and added features to the WebFonts extension, and researched new possible fonts and their license. Alolita Sharma continued to plan the India Hackathon with Siebrand, Santhosh, and the local India team. Last, the i18n/l10n team met in person in San Francisco in late September.

2011-10-31

Siebrand Mazeland and the rest of the team continued to plan their work using Mingle (credentials: guest/guest). Santhosh Thottingal added new features, and support for more languages, to the WebFonts extension, as well as new keyboard mappings and languages to the Narayam input method extension. Gerard Meijssen started to build language support teams focusing on localization and internationalization. A major achievement was the deployment of the Translate extension to meta-wiki, by Niklas Laxström and Sam Reed; it will be used to translate content pages, e.g. for the 2011 fundraiser. The localization team ended its first sprint as a team, and presented a development showcase.

2011-11-30

The code of the WebFonts extension was reviewed, and deployment is planned for December. The Narayam input methods extension now remembers the five last used schemes, and a basic version of message group workflows was implemented in the Translate extension. A major focus of the team in November was the India hackathon, during which a lot of work got done. New Narayam keyboard mappings were developed, and WebFonts were tested further.

2011-12-31

The WebFonts extension was deployed to select Indic languages and projects, making it possible to read content in languages using non-Latin fonts without installing fonts manually. The deployment uncovered bugs and issues that were addressed by the team, like cross-site font loading. The team also improved the Narayam and Translate extensions, and the latter was enabled on mediawiki.org to facilitate the translation of software documentation (like the Help:Extension:WebFonts page).

2012-01-31

January 2012 was slow on the code production side because of the code slush preparing for the branching of MediaWiki 1.19. The Localization team invested a lot of time in writing user documentation for translation tools, input methods and web fonts. Thanks go in particular to the volunteers that assisted in writing and proofreading the documentation. Much of the user documentation can be translated using the Translate extension. Amir worked to solve the old bug in the EasyTimeline extension, that prevented it from working with Indic and right-to-left scripts. Another focus was to improve test coverage for WebFonts and Narayam. In February we expect to deliver further improvements for the translation process, notably on Meta, with workflow improvements, the introduction of a translation memory and (subject to delay) notifying translators of newly available translations from inside MediaWiki by e-mail.

2012-02-29

The team is currently in the middle of a Translate extension sprint. They added new translation admin features, and are trying to get translation memory (TMX) ready for deployment to Wikimedia sites. Santhosh Thottingal used the sfntly library to compress some WebFonts even more. The team also deployed WebFonts on the English Wikisource. The team will be moved to the Experimentation & Internationalization team going forward.

2012-03-31

The team started to develop (with UI/UX contractors) the UI for a Universal language selector for desktop and mobile. They also added keymaps for language support to Narayam, added Lohit font updates from upstream to WebFonts, fixed bugs, reviewed code for localization support in MediaWiki 1.19, and discussed language support metrics. Niklas Laxström migrated the Translatewiki.net workflow to reflect the move to git/gerrit.

2012-04

The team has completed the first round of UI designs for a Universal Language Selector (ULS) for desktop and mobile. UI/UX team members (Pau Giner and Arun Ganesh) are now implementing a prototype to showcase the first version of ULS. The team also added keymaps for language support to Narayam, added notification support to Translate, fixed bugs, reviewed code for localization support in MediaWiki 1.19, and discussed language support metrics.

2012-05

The team continued integrating the first round of UI design for the Universal Language Selector (ULS) for desktop and mobile browsers. The prototype to showcase the first version of ULS was completed and demonstrated. The team completed development and deployed enhancements to the Translate extension with notification support, added more language support to the Narayam extension, fixed bugs, reviewed code for i18n support in Mediawiki, and completed a first draft for language impact metrics. The team also participated in IRC office hours with the community.

2012-06

In June, the team:

Completed initial UI design and user experience testing for the Universal Language Selector (ULS)
Developing initial prototype for the Universal Language Selector (ULS)
Developed and deployed Translation Notifications
Added more language input methods to Narayam
Added more language script fonts to Web Fonts
Made progress on integration of Translate functionality on meta for communications and fundraising groups (with integration into CentralNotice)
Started work with Arabic community to increase Arabic language support into i18n/L10n tools

2012-07

The team continued to work on the Universal Language Selector and set up a prototype to test it. Development was completed on Translation memory and CLDR (Common Locale Data Repository) plurals support and code review is pending. User experience testing of the Translate extension is in progress to Translate UX enhancements. The WebFonts and Narayam extensions were deployed on Bengali Wikisource and Punjabi Wikipedia. Development started on Project Milkshake.

2012-08

The team continued to work on the Universal Language Selector (ULS): the display settings dialog was completed and is now able to show and set WebFonts, similarly to the WebFonts extension which will be phased out once the ULS is deployed. The lists of languages were tweaked to emphasize those likely to be chosen by the user, based on their location and past selections. Translation memory was deployed on all Wikimedia sites using the Translate extension, and CLDR (Common Locale Data Repository) plurals support was merged into the core master. User experience testing of the Translate extension is in progress. Initial analysis for i18n metrics was also completed and published. The team conducted its monthly office hours, a bug triage and development showcase.

2012-09-monthly

The team formerly known as the "Localisation team" has been rebranded to "Language Engineering team". The goal of this name change is to communicate its goals more easily, and we were of the opinion that terminology like internationalization and localisation does not illustrate this clearly enough to those not in the know.

2012-11-monthly

In November 2012, the Language Engineering team travelled to India for 10 days together with the Mobile team for 6 events in total: the two-day Language Summit at the Red Hat offices in Pune, a Language Engineering Community Meetup in Pune, the three-day DevCamp 2012 Bangalore, a Language Engineering Community Meetup in Bangalore, a presentation by Erik Moeller on the current state of tech in the Wikimedia Foundation, and Coffee with Arky, a meetup of Mozilla users.

The rest of the month, development time was spent on completing the Universal Language Selector, and getting it to a state where it could be put in maintenance mode for a few months. In April 2013, phase two of the ULS will start, will consist of adding content language selection.

The Language Engineering designers completed the design for the Translation UX project, for which development has commenced end of November, and will continue for 8 sprints of a fortnight, until mid-March 2013.

2012-12-monthly

Development of the new user interface for Translate, as well as the translation editor functionality, continued at full pace throughout the month of December, with iterative feature development and user experience improvements. Santhosh Thottingal and Niklas Laxström are leading development and Pau Giner is focusing on optimizing user experience elements. The team also released the latest version of the MediaWiki Language Extension Bundle. Increased support for language variants, alternate language codes were added to the Universal Language Selector. Alolita Sharma continued to work with Red Hat's localization and internationalization teams to evaluate localization data, translation tools and internationalization tools and technologies.

2013-01-monthly

Development of the new user interface for Translate, as well as the translation editor functionality, continued throughout the month of January. Focus was on back-end work and extending the WebAPI to support the remaining features which are needed to reach feature parity with current editor. The MediaWiki Language Extension Bundle 2013.01 was released. Universal Language Selector was deployed with limited features to a selection of Wikimedia sites projects using the Translate extension. Collaboration projects also continue with Red Hat's language technologies teams, with an upcoming work sprint to complete several projects extending internationalization support for Indic languages. Runa Bhattacharjee kicked off the Language coverage matrix, an attempt to compile a snapshot of our internationalization tools coverage per language for 300 languages.

2013-02-monthly

Translate (TUX) enhancements: Development continues full steam ahead on the new translation editor with proof reading feature by Santhosh and Amir. Niklas continues to enhance and test backend translate infrastructure, including Solr integration and other translation aids.
Plurals support: by Santhosh and Amir to be more consistent with CLDR standards.
Technical Font Specification for Indic scripts: We kicked off this collaborative project between Red Hat and Wikimedia at the Language Summit in February. Santhosh and Runa are contributors to this project.
Language Coverage Matrix: This matrix aims to provide an up-to-date status of language support for all tools that the team is developing and maintaining.
Mediawiki i18n code review: Team continues to support Mediawiki release with i18n code reviews across other features and extensions.
Mediawiki Language Extension Bundle (MLEB): Monthly release of MLEB completed with release notes by Amir.

2013-03-monthly

Highlights for this month's team progress include:

The latest translation interface and translation editor improvements for the Translate extension were completed and deployed. The new interface was made default with MediaWiki 1.22wmf1, later enabled on Meta on 2013-04-03; the old translation editor is planned to be removed completely in a few weeks. The user documentation has already been updated.
The Search improvements were completed as well, with a pilot of the new Special:SearchTranslations (based on a Solr backend) that will be extended to Wikimedia projects in a few days.
Publication of the language coverage matrix.
Development progress for the translatewiki.net homepage redesign.

2013-04-monthly

translatewiki.net home page development continues but was deprioritized due to development efforts around changes to the Universal Language Selector. The Mediawiki Language Engineering Bundle (MLEB) was released on April 30; updates include localization updates to Babel, Translate extension improvements, Xliff file format support, and easy access to message tools menu for the translation editor. Please note that MLEB is no longer compatible with Mediawiki 1.19. A Divehi language web font was also added. Specifications for the Language Coverage Matrix dashboard were designed. An internationalization test strategy was presented to and reviewed by the team.

2013-05-monthly

The last round of major bugs have been fixed in the Universal Language Selector (ULS) before we start phased its deployment over the month of June across hundreds of Wikimedia websites. Communications announcements have been started for the first phase of deployment which includes removal of Narayam and WebFonts from sites before ULS is rolled out with the same integrated feature set. Test scenarios for ULS have been identified in detail. Implementation of automated tests is in progress. A combination of ULS integration and cross browser testing is in progress. Final performance tweaks are in progress before we get ready to launch the redesigned home page of translatewiki.net in June. The monthly version of the Mediawiki Language Extension Bundle (MLEB) was released on May 29.

2013-07-monthly

The language team deployed Universal Language Selector (ULS) to most Wikimedia wikis to provide easier configuration options to readers and contributors. ULS provides a flexible way to configure and deliver language settings like interface language, fonts, and input methods (keyboard mappings). Also, ULS allows users to type text in different languages not directly supported by their keyboard, read content in a script for which fonts are not available locally, or customise the language in which menus are displayed. For more information, please see the FAQ.

2013-08-monthly

The language team continued maintenance of the UniversalLanguageSelector, in particular improving performance and integration testing, and completed its integration with EventLogging, which will provide metrics useful e.g. to choose the best default font for a language. Counts from translatewiki.net are live and a deployment plan for Wikimedia projects is under analysis. The team also released its monthly version of the MediaWiki Language Extension Bundle (MLEB) which is used by third party developers and community members to add language support for their MediaWiki applications.

The team continued mentoring four Google Summer of Code (GSoC) students. Praveen Singh, mentored by Santhosh Thottingal, released a Chrome extension for Wikimedia Input Tools and contributed to the Indic Font Specification, a collaborative open source project. Team members also continued to work with Red Hat on various language initiatives.

The team participated at Wikimania in Hong Kong, which was an opportunity to meet face to face, as well as to interact with Wikipedians and community members to solve a variety of issues, including dealing with Chinese language variants and adding language assets for Javanese. The team also presented various talks on language engineering.

2014-01-monthly

UniversalLanguageSelector was disabled on January 21 2014 in production for all language Wikimedia sites (other than wikidata.org) due to font delivery performance issues. Users can still enable ULS for their language needs by going to their user profile preferences and enabling ULS from the internationalization settings. Development is in progress for a solution to enable ULS when a user logs in and selects their language preferences explicitly to enable webfonts.

David Chan continued his work on language support integration for VisualEditor for phase 5 languages. Niklas Laxström and Santhosh Thottingal participated in the architecture summit in San Francisco in January in RFC discussions and JSONification of i18n support for VisualEditor.

2014-02-monthly

UniversalLanguageSelector was re-enabled with webfonts disabled by default. Research is ongoing to see whether they can and should be re-enabled by default at least for some languages.

More convenient shortcuts were added by Niklas Laxström to the Translate extension.

Kartik Mistry and Amir Aharoni are working on stabilizing the browser tests for all the language extensions and on setting up more robust online staging sites.

2014-03-monthly

MediaWiki's LocalisationUpdate extension was rewritten by Niklas Laxström to modernize its internal architecture to be able to support JSON message file formats. Kartik Mistry released the team's monthly MediaWiki Language Extension Bundle (MLEB 2014.3) with the latest version of LocalisationUpdate (see release notes). Niklas Laxström also started migrating the Translate extension's translation memory and translation search back-end from Solr to ElasticSearch in line with Wikimedia's search migration. David Chan continued his work on input method support for the VisualEditor project.

2014-04-monthly

The team prepared the migration of the translation memory infrastructure from Solr to ElasticSearch.

2014-06-monthly

The Translate extension received numerous bug fixes, including fixing workflow states transitions for fundraising banners.
Changes in input methods: Ludic (lud) transliteration and Tibetian (bo) EWTS layouts added.

2014-07-monthly

CLDR extension was updated to use CLDR 25; this work was mostly done by Ryan Kaldari. The team made various internationalization fixes in core, MobileFrontend, Wikipedia Android app, Flow, VisualEditor and other features. In the Translate extension, Niklas Laxström fixed ElasticSearchTTMServer to provide translation memory suggestions longer than one word; and improved translation memory suggestions for translation units containing variables (bug 67921).

2014-08-monthly

Niklas Laxström (outside his WMF job) completed most of the work needed in Translate to Recover gracefully from session expiration, a known pain point for translators. The PageMigration feature (a GSoC project mentored by Niklas) was released . The team also worked on session expiry checking (to prevent errors in long translations), updated YAML handling, deployed auto-translated screenshots for the VisualEditor user guide (a GSoC project mentored by Amir and done by Vikas Yaligar). They did internationalization testing of the new Android and iOS apps, as well as internationalization testing and bug fixes in VisualEditor, MobileFrontend and Flow.

2014-09-monthly

The CLDR extension was updated to version 26 and entries identical to CLDR were removed from LocalNamesEn.php. The team made RTL fixes in core, Echo and Wikibase, and tested Flow for RTL support. Maintenance of the Translate extension continued, and the performance of translation memory was improved on ElasticSearch with the help of Nik Everett

2014-10-monthly

CLDR version 26 updates that affected plural rules for MediaWiki i18n have been integrated. The affected messages have been updated and translators have been informed. As part of regular maintenance of the Translate extension, many unused configuration variables have been removed and Special:Translate now works better with custom skins. Niklas also fixed a major regression in a refactoring which broke the message group selector.

2014-11-monthly

The team fixed several bugs in UniversalLanguageSelector. A bug was fixed that caused JavaScript errors on a few special pages without headings. The font size for button text was fixed to improve display on Monobook skin for Mozilla Firefox. The team also added support for the WOFF2 webfont format. Experiments revealed significant improvement in overheads. However, there are no WOFF2 webfonts in the font repository yet due to pending issues in WOFF2 font generation.21 new languages are now supported in the language selector, and autonyms for 5 languages were updated.

The team migrated the translation memory service of the Translate extension to ElasticSearch. Thanks to WMF's ElasticSearch cluster, this migration increases the speed and reliability of the service. We have identified one issue with the suggestions, which is being fixed during December. Thanks to Chad and Nik for helping Niklas.

Last, the team also made RTL fixes in MobileFrontend and VisualEditor.

2014-12-monthly

UniversalLanguageSelector:
- Made the width of the ULS panel configurable according to the number of languages.
- Removed the map. The analytics showed that very few people click it.
- Improve the visual design of the Compact Language Links beta feature (done by Niharika Kohli, design and review by Pau Giner and Santhosh Thottingal)
- Improve the performance of language search (done by Thiemo Mättig, review by Santhosh Thottingal)
- Add ability to rely on default configuration when loading messages in jquery.i18n.
- Positioning fixes for the Input Method selector.
Translate
- Multiple fixes to translation memory