Wikitext cleanup is not fully Unicode compatible
Closed, ResolvedPublic2 Estimated Story PointsBUG REPORT
Actions

Assigned To

Authored By

	thiemowmde
	Jul 27 2020, 10:17 AM

Description

It appears like certain regular expressions in the FileImporter wikitext cleanup code are not Unicode-safe. For example, the link [[兵庫県立考古博物館]] doesn't get the interwiki prefix. It should become [[:ja:兵庫県立考古博物館]], but doesn't.

Example: https://commons.wikimedia.org/wiki/Special:Diff/435420995
Reported here: https://www.mediawiki.org/wiki/Topic:Vqwxi9chttpgxdy7

Details

	Subject	Repo	Branch	Lines +/-
	Make some more regex patterns Unicode-aware	mediawiki/extensions/FileImporter	master	+3 -3
	Fix several regex patterns not being Unicode-aware	mediawiki/extensions/FileImporter	master	+23 -8

Customize query in gerrit

Related Objects

Mentioned In: rEFLIba17f02b8373: Make some more regex patterns Unicode-aware
rEFLIeca9978799f9: Fix several regex patterns not being Unicode-aware

Event Timeline

thiemowmde created this task.Jul 27 2020, 10:17 AM

Change 616460 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Fix several regex patterns not being Unicode-aware

https://gerrit.wikimedia.org/r/616460

gerritbot added a project: Patch-For-Review.Jul 27 2020, 10:20 AM

thiemowmde moved this task from Sprint Backlog to Review on the WMDE-QWERTY-Sprint-2020-07-22 board.Jul 27 2020, 10:20 AM

thiemowmde set the point value for this task to 2.

thiemowmde changed the subtype of this task from "Task" to "Bug Report".

thiemowmde moved this task from Backlog to Tickets in sprint on the Move-Files-To-Commons board.

Change 616460 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Fix several regex patterns not being Unicode-aware

https://gerrit.wikimedia.org/r/616460

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.2; 2020-07-28).Jul 27 2020, 11:00 AM

thiemowmde mentioned this in rEFLIeca9978799f9: Fix several regex patterns not being Unicode-aware.Jul 27 2020, 11:03 AM

Maintenance_bot removed a project: Patch-For-Review.Jul 27 2020, 11:10 AM

thiemowmde moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2020-07-22 board.Jul 27 2020, 1:16 PM

Lena_WMDE closed this task as Resolved.Aug 11 2020, 9:16 AM

Lena_WMDE moved this task from Demo to Done on the WMDE-QWERTY-Sprint-2020-07-22 board.

Change 640799 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Make some more regex patterns Unicode-aware

https://gerrit.wikimedia.org/r/640799

gerritbot added a project: Patch-For-Review.Nov 12 2020, 8:39 AM

Change 640799 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Make some more regex patterns Unicode-aware

https://gerrit.wikimedia.org/r/640799

thiemowmde mentioned this in rEFLIba17f02b8373: Make some more regex patterns Unicode-aware.Nov 12 2020, 9:42 AM

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.18; 2020-11-17); removed MW-1.36-notes (1.36.0-wmf.2; 2020-07-28).Nov 12 2020, 10:00 AM

Maintenance_bot removed a project: Patch-For-Review.Nov 12 2020, 10:10 AM