Wikipedia:Dispute Resolution Improvement Project/Newsletter
The Olive Branch: A Dispute Resolution Newsletter (Issue #2)
edit- To add your name to the newsletter delivery list, please sign up here
- Background
Until late 2003, Jimmy Wales was the arbiter in all major disputes. After the Mediation Committee and the Arbitration Committee were founded, Wales delegated his roles of dispute resolution to these bodies. In addition to these committees, the community has developed a number of informal processes of dispute resolution. At its peak, over 17 dispute resolution venues existed. Disputes were submitted in each venue in a different way.
Due to the complexity of Wikipedia dispute resolution, members of the community were surveyed in April 2012 about their experiences with dispute resolution. In general, the community believes that dispute resolution is too hard to use and is divided among too many venues. Many respondents also reported their experience with dispute resolution had suffered due to a shortage of volunteers and backlogging, which may be due to the disparate nature of the process.
An evaluation of dispute resolution forums was made in May this year, in which data on response and resolution time, as well as success rates, was collated. This data is here.
- Progress so far
Leading off from the survey in April and the evaluation in May, several changes to the dispute resolution noticeboard (DRN) were proposed. Rather than using a wikitext template to bring disputes to the Dispute Resolutions Noticeboard (DRN), editors used a new javascript form. This form was simpler to use, but also standardised the format of submissions and applied a word limit so that DRN volunteers could more easily review disputes. A template to summarise, and a robot to maintain the noticeboard, were also created.
As a result of these changes, volunteers responded to disputes in a third of the time, and resolved them 60% faster when compared to May. Successful resolution of disputes increased by 17%. Submissions were 25% shorter by word count.[1]
Outside of DRN other simplification has taken place. The Mediation Cabal was closed in August, and Wikiquette assistance was closed in September. Nevertheless, around fifteen different forums still exist for the resolution of Wikipedia disputes.
- Proposed changes
Given the success of the past efforts at Dispute Resolution (DR) reform, the current RFC proposes we implement:
1) A submission gadget for every DR venue tailored to the unique needs of that forum.
- Similar to the one that was deployed, with great success, to the DRN.
- Structured based on the specific issues most commonly dealt with at each forum.
- Designed to improve the quality of requests for DR and the efficiency of DR at that forum.
- Applicable at following noticeboards: Dispute resolution, Neutrality, Reliable Sources, Original Research, Biographies of Living Persons, Notability noticeboard, Fringe theories, Conflict of Interest, Ethnic and cultural conflicts, External links, Third opinion, Mediation Committee, Arbitration Committee.
- Forms will merely fill out any existing templates (such as Arbcom's) and create a markup-free form in line with specific noticeboard practices otherwise.
- Example form fields: What pages are involved? What users are involved? What is the issue? What resolution is desired?
2) A universal dispute resolution wizard, accessible from Wikipedia:Dispute resolution.
- This wizard would ask a series of structured questions about the nature of the dispute.
- It would then determine to which dispute resolution venue a dispute should be sent.
- If the user agrees with the wizard's selection, s/he would then be asked a series of questions about the details of the dispute (for example, the usernames of the involved editors).
- The wizard would then submit a request for dispute resolution to the selected venue, in that venue's required format (using the logic of each venue's specialized form, as in proposal #1). The wizard would not suggest a venue which the user has already identified in answer to a question like "What other steps of dispute resolution have you tried?".
- Similar to the way the DRN request form operates, this would be enabled for all users. A user could still file a request for dispute resolution manually if they so desired.
- Coding such a wizard would be complex, but the DRN gadget would be used as an outline.
- Once the universal request form is ready (coded by those who helped create the DRN request form) the community will be asked to try out and give feedback on the wizard. The wizard's logic in deciding the scope and requirements of each venue would be open to change by the community at any time.
3) Additionally, we're seeking any ideas on how we can attract and retain more dispute resolution volunteers.
Please share your thoughts at the RfC -->
--The Olive Branch
The Olive Branch: A Dispute Resolution Newsletter (Issue #1)
edit| Attention |
| My apology to anyone who felt receiving a message about this newsletter was spam or disruptive in any way. In future, we will not deliver messages in the same way. You can add your name to the opt-in list for future newsletters here. -- Ocaasi t | c 21:05, 4 September 2012 (UTC) |
Welcome to the first edition of The Olive Branch!
Background: Let's start with brief overview of the DR ecosystem. Most disputes begin at an article talk page, and many are settled there. Policy specific questions are typically raised at a noticeboard, such as WP:BLPN (biographies of living persons), WP:RSN (reliable sources), WP:ORN (original research), WP:FRINGE (alternative beliefs/pseudoscience), WP:CP or WP:NFCR or WP:MCQ (copyright), WP:PM (mergers), WP:RM (page moves), WP:ELN (external links), WP:N/N (notability), WP:ANI (administrator's incidents), or WP:AN3 (edit warring). User conduct issues go through Wikiquette noticeboard while more serious situations warrant an WP:RFC/U. Small content disputes can start with the lightweight Third opinion process; more substantial disputes wind up at the Dispute resolution noticeboard; and even more intractable issues arrive at the Mediation Committee. Some complex questions can be resolved with an WP:RFC. Last, WP:ARBCOM often needs to step in and levy a decision. An obvious observation here that there are a lot of places where dispute resolution happens. Some of them are more extensive, better known, or more effective than others.
Research: One of the first steps in improving our DR processes has been getting good data about DR. This has been spearheaded by Steven Zhang whose 2012 Community Fellowship is focused on understanding and improving DR on Wikipedia. One of the key purposes of this newsletter will be to present the best research and ideas we have about which options are working and what we as a community might do about improving the rest of them.
April survey results: In April 2012 Steven Zhang conducted a dispute resolution survey. Among its findings, here were some highlights:
- Over half of all respondents (and 80% of females) were older than 40.
- 94% had requested assistance from a dispute resolution forum at some point, and 62% within the last year. Respondents were generally unhappy with their experiences in dispute resolution – only one in five were satisfied.
- Requests for Comment is the most used dispute resolution forum, with 60% participating in the last year. Mediation Committee proceedings were used the least, only 10% were involved in proceedings.
- Opinions of dispute resolution were overall relatively negative - Arbitration was rated as the best dispute resolution forum by respondents – with one in three respondents rating it as good or better. In contrast, Wikiquette assistance was rated the worst – only 1 in 12 rated it as satisfactory.
- Dispute resolution volunteers do so because they felt the process was critical to Wikipedia functioning, liked helping people or as payback for previous assistance.
- Some respondents haven’t volunteered due to the unpleasantness of disputes, the prolonged nature of dispute resolution, or due to poor past experiences or a lack of knowledge in resolving disputes.
- When asked about their personal experiences with dispute resolution, positive aspects were that their dispute was resolved, the examples set by volunteers and the positive behavior of their fellow participants, while negative aspects included the time it takes to resolve a dispute, and the potential for the processes to become unfair - many citing the source of this unfairness as administrators that became involved in the process.
- The main problems given for dispute resolution are its complexity, its inaccessibility, and that there are too many resolution processes and not enough volunteers. Respondents want stricter action taken against problematic editors, a simplified, more accessible process where closure can be bought to a dispute quickly.
Activity analysis: The following table summarizes activity in several DR forums for the month of May 2012.
| Forum | disputes | participants | volunteers | Average first response | Average resolution | Success rate |
|---|---|---|---|---|---|---|
| Third opinion | 31 | Not assessed | N/A | 25 hours | N/A | 52% |
| Dispute resolution noticeboard | 42 | 207 | 25 | 16.6 hours (21.4% never looked at) | 8.6 days | 47.61% |
| Mediation Cabal | 4 | 17 | 5 | N/A | 28 days | 100% |
| Formal Mediation | 7 | 25 | 4 | N/A | 15 days | 0% |
| Wikiquette assistance | 17 | 67 | Not assessed | 5.3 hours | 45.5 hours | 21.4% |
| Requests for comment | 15 | 339 (212 from one RFC) | N/A | N/A | N/A | N/A |
Of note is that the most active forum of those analyzed was the still relatively new Dispute resolution noticeboard (DRN). Also interesting is that WQA was often the fastest place to yield a response; however, the survey found WQA rarely resulted in satisfactory resolution.
A follow-up analysis in August 2012 compared the DRN to its May data:
| Metric | Result for May | Goals for August | Results for August | Results for August % |
|---|---|---|---|---|
| First response time | 16 hrs 36 mins | <10 hrs | 5h 29 min | 67% reduction in response time |
| # of active volunteers | 25 - 1 to 12 ratio with 207 participants (average 1.47 per thread with 42 threads) | 30+ | 20 - 1 to 11 out of 177 total editors (average 2.85 per thread with 52 threads) | 20% reduction in volunteers |
| Timeframe thread open | 8.6 days | 5 - 7 days | 2d 10 hr - 3 d 11 hr for disputes that were addressed | 60% reduction in discussion time |
| Success rate | 47.61% | 70%+ | 64.29% | 16.6% increase in success |
The results at DRN showed some encouraging stats - a reduction of 67% to first response times, 60% reduction in discussion times, 25% reduction in thread size, an average of 2.85 volunteers to a thread up from 1.5 and a success rate of over 64%. The amount of volunteers decreased by 20%; so from the results it shows that while the disputes were handled in a quicker timeframe by more volunteers per thread, it was from a small core group of volunteers. This emphasises the need for more volunteers - if the existing volunteers burnout, dispute resolution will suffer.
The WQA close proposal is still receiving comments. Initial response was generally sympathetic to concerns about WQA's ineffectiveness, but there was widespread backlash against routing all WQA disputes to AN/I. The number one complaint about WQA was that it just doesn't work, and further that it dilutes the DR pipeline and confuses editors with too many options. Several editors noted that too much blaming goes on there, sometimes even increasing the drama. A common point was that without the admin teeth of AN/I, WQA does little to discourage our most uncivil editors. Opponents to the close noted that WQA at least gets the mess of conduct disputes off of article talk pages and is a lighterweight alternative to RFC/U. Also mentioned was that the threat of AN/I delivered at WQA was sometimes enough. Consensus seemed to form around third opinions (3O) as a suitable replacement for WQA given our current options.
For Discussion: Do you think that the Geopolitical, ethnic, and religious conflicts noticeboard is effective? Should we close or reroute it to the Dispute Resolution Noticeboard? Please share your views...
Coming up: A second Request for comment will be opened soon to discuss the future of dispute resolution.
See Also: This week's Signpost opinion piece on dispute resolution, by Steven Zhang.
--The Olive Branch 23:58, 3 September 2012 (UTC)