Admin tools development

Group:	Platform / Features
Start:	2012-07-27
End:	2013-02
Team members:	Chris Steipp, Tim Starling

Rationale

A number of powerful anti-disruption and protective tools are provided for our communities on the Wikimedia clusters. These include local tools for each project's sysops or other higher-graded users, and cross-wiki tools co-ordinated on Meta-Wiki that are either for all Meta-Wiki sysops or for stewards.

These tools are crucial for the community to fight against spam, disruptive behaviour and large-scale vandalism, and dealing with private information or other, less-common situations. It is important that the Wikimedia Foundation works on these tools so that the communities we support are not overloaded with work and that they can maintain the quality of the projects.

Sadly, the resources of the Wikimedia Foundation are limited, and we do not wish for this to interfere with our timely delivery of appropriate tools. Thus, we hope that a member of the community can work as the volunteer Product Manager for these admin tools, liaising between the users of these tools and the developers, shaping what the products should look like and prioritising areas of particular concern.

Roadmap

See also the overall Stewards-and-global-tools Phabricator project.

	watch

Active areas

Global renaming of user accounts

Overall status

Deployed

Code tasks

In progress - gerrit:39171 Create a tool that is essentially a global version of the pre-existing RenameUser extension, which allows privileged users (stewards on WMF sites) to rename global accounts.
Adjustment to provide a limit for accounts with fewer than 5,000 edits globally, for performance concerns.
Longer-term, we will need a solution for renaming accounts with 5,000 or more global edits.

Global blocking of user accounts and anonymous IP addresses

Overall status

Code written; to be deployed

Code tasks

Done - In 1.21wmf11 In CentralAuth provide a mass account-locking (200 accounts at a time) tool.
Done - Global (and local) blocking based on XFF headers bugzilla:23343 - Prevent edits that use multiple proxies if one proxy in the chain is blocked

Global CheckUser tool

Overall status: Requirements somewhat gathered; awaiting developer time.

Global AbuseFilter tool

Overall status

Code written; deployed to test2, test, mediawiki

Code tasks

Done - gerrit:17620 Allow defining AbuseFilter rules on a central location (Meta-Wiki) and apply these rules against edits on all Wikimedia wikis. See AbuseFilter Priorities / Design for details.
Done - gerrit:17343 Global AbuseFilter variables/throttling. See AbuseFilter Priorities / Design for details.

SUL Audit

Overall status

Databases are ready for querying, ongoing

Tasks

Done - Procure hardware (db29), slave s7.centralauth, and load all other user tables
In progress - Creating and running queries
In progress - Initial scripts to migrate non-clashing users ("pass0" and "pass1" or their replacements)

Miscellaneous

In progress code review bugzilla:12518 - Fix cross-wiki userrights reflects groups on local wiki, not target wiki - code in gerrit:36330
Done shell request 43913 - Fix all "User@metawiki" entries to just "User" in the meta user rights log; requires just a single SQL query to be run (probably via maintenance/runBatchedQuery.php) by shell users.

Open tasks

Finalise Single User Login (SUL) to forcibly rename all clashing local accounts. (Blocker for Echo and Flow; OAuth etc.; …) - see SUL Audit, above.
Replace Single User Login with simpler system (shared DB?) to pay down technical debt.
Evaluate the effectiveness of the current CAPTCHA system (ConfirmEdit extension) and look into possible new CAPTCHA systems or other alternative ways of verifying that the editor is a human being and not a bot.
This is necessary because nowadays bots can crack most CAPTCHAs easily and still post their spam, whereas the blurry word is a great inconvenience to some human editors (see also bugzilla:5309 about internationalization concerns) and has the potential to raise the barrier of entry for humans.
Some ideas and insights in this e-mail thread: Captcha for non-English speakers II.
Set up a git repository for the Phalanx extension, clean up the code and fix whatever bugs there are, and evaluate the possibility of enabling Phalanx on WMF wikis (with Meta-Wiki as the central database, i.e. the database where its tables will be stored in).
This could address a number of previously identified issues, such as merging the previous ad hoc anti-spam tools into one powerful tool and keeping the lists on database instead of on wiki pages.
See admin tools development/Phalanx for some initial thoughts and concerns.
See also 27988
Done - The e-mail black list (which prevents account registration or sending e-mail using matching addresses) is currently split into a file that only shells can access, or an entirely-public list on Meta-Wiki. A middle-ground tool which could only be accessed by a small number of users would be a significant improvement for privacy purposes.
25000 - Better IP throttling configuration - Allow local sysops to temporary variation of the IP throttles for particular IPs and IP ranges (e.g. so that an event with a lot of expected newbies would not hit the "too many account creations from your IP" issue), which currently can only be solved by a user with shell access, or individually-creating accounts by local sysops. The code hooks for this functionality already exist, and an extension - Extension:ThrottleOverride - written which could be deployed.
13601 - An extension to one-click revert all of a user's actions on RecentChanges - Create an extension which allows to revert all actions in the recentchanges table by a given user. Such a tool would be useful for reverting edits made by vandalism-only accounts and maybe some spambots.
Block known spambot user agents - Extend the AbuseFilter extension to allow preventing edits (and/or other actions) if the user's User-Agent string matches a known spambot UA. The Bad Behavior extension contains a list of spambot or otherwise problematic User-Agent strings.
This is prioritized as a low-priority item because changing the User-Agent is very trivial, but like some of the pre-existing anti-spam methods (such as the SimpleAntiSpam extension, which is enabled on all WMF wikis), it's an easy way to block the dumbest spambots.
3233 - Send a cookie with each block to block IP-changing vandals.
Prioritized as low, because clearing your cookies is pretty easy, just like changing your IP address.
Extend GlobalBlocking to accounts. - Current the extension only allows for global blocking of IPs. However, Stewards are currently fulfilling requests to "block" registered users by locking their accounts in SUL. This workaround is imperfect to say the least. For example, it requires that the user have attached their accounts to work. Regardless of any debate as to whether accounts should be subject to global blocking, it is in fact happening, and the tool being used is not designed for that use case. Previous discussion on bug 15294.
bugzilla:15273 - Split GlobalBlocking action options out like local blocks - Currently, checking "Block anonymous users only" on a global block blocks anonymous users and also prevents them from creating accounts.
bugzilla:42734 - Hide AbuseFilter logs for deleted pages - Automatically hide any AbuseFilter logs for public filters, for pages that were later deleted.
Integrate bits.wikimedia.org/geoiplookup into Admin Tools, e.g. CheckUser, so that users do not have to use a third party tool (bugzilla:16068?)
Enable OpenID (as a provider) for all global Wikimedia accounts, based off users.wikimedia.org.
20189 - Allow suppression using the RevisionDeletion tool of multiple revisions of deleted page (existing functionality in action=history which is needed in Special:Undelete).
60373 - Convert Oversight revisions to (revision deleted) suppressed on Wikimedia wikis.

See below for more details on what these items mean.

List of in-scope extensions and core features

Local project-specific tools

AbuseFilter
Revision suppression and Oversight
ConfirmEdit
CheckUser
BadImageList
MassDelete
File upload cleanness checking (mostly relevant to Commons).

Note: local blocking tools are, at least for now, out of scope.

Metawiki-based tools

GlobalAbuseFilter (not currently extant - see below)
GlobalBlocking
CentralAuth account locking, renaming, merging
TorBlock
AntiBot (currently broken?)
SpamBlacklist
TitleBlacklist

Existing major problems and possible solutions

The number one cross-wiki issue is dealing with spammers and other bot-driven disruptive editing. This is especially a problem on our smaller wikis where the spam can easily swamp the local community. Beyond the original problem, noticing the problem once it's occurred can take some time during which our readers are poorly served, and cleaning up the spam can be arduous.
- To try to prevent the spam from occurring, we are looking to provide a version of the AbuseFilter extension, based on metawiki, that would apply its event rules to act globally on all Wikimedia wikis at once, as directed by stewards. This functionality is currently implemented as an experiment in the existing codebase, but it has never been used, is untested, and likely to be rusty).
- To deal with multiple accounts being created - or multiple edits being made - very rapidly across a number of wikis, we would like to extend AbuseFilter to allow per-IP throttles on events ('only 5 edits a minute' and similar) to apply across all of our wikis globally, rather than be implemented on a per-wiki basis.
One recurring Wikimedia Foundation issue is the performance of the systems that we provide, and in particular the scalability of existing systems to work as the load grows.
- In general, the Foundation would like to re-engineer the ad hoc collection of tools into a smaller number of more powerful tools (ideally one), based on the existing AbuseFilter extension. This would take some time and is not the top priority, but is our overall intended direction.
- A particular piece of work in simplifying and speeding up our systems would be to move some of the lists of banned article titles, account names, used images and external links from blocks of wikitext that need to be interpreted by their own extensions into the database.

Documents

User requirements:
Specifications:
Software design document:
Test plan:
Documentation plan:
User interface design docs:
Schedule:
Task management:
Release management plan:
Communications plan:

Communications

Rationale

Roadmap

Active areas

Global renaming of user accounts

Global blocking of user accounts and anonymous IP addresses

Global CheckUser tool

Global AbuseFilter tool

SUL Audit

Miscellaneous

Open tasks

List of in-scope extensions and core features

Existing major problems and possible solutions

Other problems or possible pieces of work

Documents

Communications