Our Ranking Methodology

One of the main tools of Open Doors to track and measure the extent of persecution in the world is the World Watch List (WWL). Open Doors has been monitoring persecution of Christians worldwide since the 1970s. During the 1980s and 1990s, the WWL methodology evolved gradually. In 2012, the methodology of the WWL was comprehensively revised in order to provide greater credibility, transparency, objectivity and scientific quality. In 2013, further refinement of the methodology took place. Below follows a short overview of the main elements of the methodology.

1. Independently audited World Watch List methodology
The World Watch List 2014 will be the independently audited World Watch List. This auditing body is the only institution with academics dedicated to studying the religious liberty of Christians – the International Institute of Religious Freedom (IIRF). It is also the first time the full methodology will be available for scrutiny. This puts our research in a far more defensible position, and we hope, will make its impact greater. 2014 will also see the production of “mini-WWL’s” that will focus on the top 10 places where violence is highest against Christians, or the role of the persecution engines. With our larger questionnaires, the dataset gathered is much larger, and we can continue to share this data throughout the year, and not merely at the beginning of it.

2. The pattern of persecution
A persecution situation presents a complex reality. It is not always clear if and to which extent pressure felt by Christians or even violence against them is directly related to them being Christian. Sometimes, just living in a chaotic world creates substantial amounts of suffering for Christians and others alike. Other times, suffering results from antipathy, hatred or it could simply be ‘double vulnerability’ of Christians in a problematic context, and will be called persecution. The latter is what the WWL methodology tries to monitor and capture.

image002Persecution is when Christians and their communities experience specific pressure and/or violence in a situation of ‘brokenness’ that are related to persecution dynamics prevalent in their environments and are forcing them to comply with the drivers of these dynamics. The WWL methodology regroups these dynamics in three different impulses, fueling eight different persecution engines and being driven by specific actors or drivers of persecution. The diagram shows the relation between ‘brokenness’, impulses, persecution engines and the related drive for exclusive power.

The table below presents the different impulses with the persecution engines emanating from them:

Underlying impulses

Persecution engine

Tribal impulse Islamic extremism
Other religious militancy
Tribal antagonism
Ecclesiastical arrogance
Secular impulse Communist oppression
Aggressive secularism
Exploitative impulse Totalitarian paranoia
Organized corruption

The drivers of persecution engines are people and/or groups embodying the three main impulses. The WWL methodology studies who they are, and which are involved in hostilities against Christians in a particular country.

The WWL methodology distinguishes the following drivers of persecution:

Drivers of persecution

Government Government officials at any level from local to national
Society Ethnic group leaders
Non-Christian religious leaders at any level from local to national
Religious leaders of other churches at any level from local to national
Fanatical movements
Normal citizens (people from the broader society), including mobs
Extended family
Political parties at any level from local to national
Revolutionaries or paramilitary groups
Organized crime cartels or networks
Multilateral organizations

Often more than one driver is active in and around one or more persecution engines.

3. Pressure (squeeze) and violence (smash)
World Watch Research distinguishes two main expressions of persecution: squeeze (the pressure Christians experience in all areas of life) and smash (plain violence). Nevertheless, while it would seem that smash is the most prevalent and invasive expression of persecution, it is often the squeeze that is most prevalent and invasive. The WWL methodology, therefore, negates the idea that the more violence there is against Christians, the more persecution there must be. The WWL methodology also seeks to negate another assumption, which is that the most violent persecutors of the church are its main persecutors.

While smash can be measured and tracked through incidents of violence, squeeze needs to be tracked otherwise. It needs to be tracked by discerning how the act of Christian life and witness itself is being squeezed in all the different areas of life.

image004The WWL methodology has defined the ‘five spheres concept’ to track the expressions of persecution in different areas of life. These five spheres express the squeeze (pressure) in each sphere of life. A sixth building block expresses the smash (plain violence). The sixth block potentially cuts across all five spheres of life.

 

4. Tools for monitoring hostilities against Christians
The team of World Watch Research has designed a system for integral monitoring of hostilities against Christians worldwide. The different components of the system are specified as follows: the Rapid Appraisal Tool (RAPT), combining information from other sources and internet search. High scores on RAPT necessitate further research through the World Watch Survey. The Vulnerability Assessment Tool (VAT) is an alternative to the World Watch Survey. In-depth research is done through the World Watch List (WWL) questionnaire.

The diagram shows steps for monitoring hostilities against Christians:

Screen Shot 2014-01-07 at 12.18.29 PM

Developed by World Watch Research

When RAPT surpasses a specified threshold, it is followed by the World Watch Survey or by VAT. If the results from the World Watch Survey or VAT confirm the need for in-depth investigation, the WWL questionnaire will be used for that country.

5. Scoring countries and the World Watch List ranking
In order to be able to score countries for the World Watch List, a scoring system has been developed.
The scoring grid consists of four categories of ‘Yes’ and ‘No’

 

No

Yes, somewhat or rarely

Yes, significantly

Yes, very significantly

Yes, absolutely

0 points

1 point

2 points

3 points

4 points

(1) Proportion of types of Christianity

None

up to 1/4

>1/4 – 2/4

>2/4 – 3/4

>3/4 – 1

(2) Proportion of inhabited territory

None

up to 1/4

>1/4 – 2/4

>2/4 – 3/4

>3/4 – 1

(3) Intensity

None

Low

Medium

High

Very high

(4) Frequency

None

Sporadic

Quite frequent

Frequent

Permanent

 

The answer to each question in blocks 1 to 5 is the rounded average of the four elements:
1) Proportion of types of Christianity persecuted
2) Proportion of inhabited territory affected
3) Intensity of persecution
4) Frequency of persecution.

As a result of the WWL process, each country gets a specific final score. This final score is used to determine the order of countries from position 1 to 50 on the annual World Watch List. The WWL methodology does not produce ‘absolute scores’. This is because WWL has not established standardized norms and measures against which to score specific situations.

The degree of persecution is characterized by a scale of 0 – 100 points, directly linked to a set of 96 questions covering five spheres of life and the prevalence of violent incidents. For practical purposes the WWL has split up this range 0 – 100 points into six categories.

The six categories are based on scoring intervals of 15 point, with the last group having 25 points. The number of categories and their ranges (15 or 25 point) do not have a special meaning.

image010

Developed by World Watch Research

The range of colours goes from dark to lighter to indicate degrees of intensity. These colours are however not prescriptive.

The WWL ranks countries according to their final scores. The most important reason to rank the countries is to be able to present a complex reality to the broader public. For this to be done properly, the WWL must always be paired with country persecution profiles that explain the particularities of the persecution situation in the country.

The ranks give the possibility to compare the countries on the WWL among each other. They do not permit to compare ranks of the foregoing WWL exercise with the actual one. This is because ranks are relative positions based on a set of country scores that differs from year to year.

It can for instance happen that a country receives a lower rank on the WWL while the scores are higher than the year before. If this happens, other countries have received an even higher score and have, therefore, ended up higher on the WWL.

On comparing country ranks for the same WWL, it must be stated that close scores can cause different ranks. The differences between these ranks are not necessarily very meaningful. This is due to the margin of statistical error inherent to any such exercise.

6. Information streams for the World Watch List
World Watch Research distinguishes two categories of questionnaires:

a) questionnaires belonging to the so-called ‘field-stream’, which are filled out by Open Doors’ field staff and/or key contacts;
b) questionnaires belonging to the so-called ‘non-field-stream’, which are filled out by external experts.

If both streams include multiple respondents, the respective questionnaires are integrated separately first, before integrating the two streams.

If it is not possible for one of the streams to obtain a questionnaire, it is possible to use the World Watch Survey and other narrative sources instead. However, in that case, the persecution analyst of World Watch Research could use the provided information to fill out a WWL questionnaire in order to construct that stream.

7. Time frame for the WWL process
The time frame for the WWL process starts beginning of July and extends till December:

Processing of the WWL questionnaire

Send WWL questionnaire to Field & receive first results July 1 – August 31
Send WWL questionnaire or World Watch Survey to external experts & receive first results August 1 – August 31
First check by World Watch Research September 1 – October 31
Final completion by Field & external experts & check by World Watch Research October 15 – November 22
Overall check & follow-up by World Watch Research November 23 – November 30
Finalization new WWL (scores and ranking) December 1 – December 7
Finalization of other documentation for release of WWL December 1 – December 15

Worst