ArrowArtboardCreated with Sketch.Title ChevronTitle ChevronIcon FacebookIcon LinkedinIcon Mail ContactPath LayerIcon MailPositive ArrowIcon PrintIcon Twitter

Peter Tasker: The flawed 'science' behind democracy rankings

Few Japanese people may have noticed, but their country has just been relegated from the "Premier League" of democratic nations. According to the Economist Intelligence Unit's recently published annual Democracy Index, Japan no longer belongs among the "full democracies." Its score has fallen from 8.08 in 2014 to 7.96 in 2015, which reduces its ranking from 20th place in 2014 to 23rd and pushes it into the inglorious category of "flawed democracies."

     Still, there is some consolation for Japan. It ranks narrowly ahead of Belgium and France, which are also apparently "flawed democracies." All three, however, can look down smugly on Singapore, which languishes in 74th position, barely above Albania. Not a single Asian country is a member of the "full democracy" club and both the U.S. and U.K. are close to demotion. Instead, the top 10 are dominated by northern European social democracies.

     How did the EIU come up with a scoring system that is supposedly accurate to two decimal places? What it did has the semblance of rigor. It asked various experts to answer 60 questions and assigned each reply a numerical value, with the weighted average deciding the ranking. Who are these experts? Nobody knows. Wikipedia dryly notes that the report does not reveal their number, nationality, credentials or even field of expertise.

     Some idea of where they are coming from can be gauged by the report's comments on individual countries. France, we learn, has been defenestrated because of a "deterioration in social cohesion." Those inveterate goodie-goodies, the Swedes, are on the naughty seat because of declining membership in political parties and more social discrimination. An important recent phenomenon, the growth in support for populist politicians, is not seen as a sign of democratic systems responding to shifts in public opinion. Rather, it is evidence of "discontent with democracy" itself and thus to be deplored.

     In other words, despite the appearance of scientific objectivity, the whole exercise of ranking a country's democratic credentials is as much riddled with biases, value judgments and hidden agendas as awarding Oscars to films or Michelin stars to restaurants -- which are also decided by groups of mysterious experts using criteria best known to themselves.

     The same could be said for the Global Competitiveness Index, the brainchild of the World Economic Forum crowd, which offers up a dizzying array of statistical conjuring tricks. Here the news is better for Singapore, the U.S. and Japan, which the most recent report ranks second, third and sixth, respectively. Indeed, if you took the two indexes seriously you could be forgiven for thinking that among developed countries, the less democracy the more economic competitiveness and vice versa.

     Unlike the Democracy Index, the GCI purports to have predictive value in that it ranks countries by their ability to provide "current and medium-term levels of prosperity." As such, it comes into the category of "expert forecast," analyzed to devastating effect by the University of Pennsylvania academic Philip Tetlock, author of "Expert Political Judgment: How Good Is It, How Can We Know?" One of Tetlock's key contentions is that the accuracy of an expert's predictions has an inverse relationship to his or her self-confidence, renown and, beyond a certain point, depth of knowledge. Or more bluntly, most experts have no clue what the future holds, no matter how slickly they package their prognostications.

OOPS ICELAND   This might explain why Iceland came 13th in the GCI pecking order in 2007, above Canada and Austria. Just a few months later the Icelandic economy imploded in the global economic crisis and its currency suffered a 60% collapse. By 2011, Iceland had fallen to 31st place, just one notch above Tunisia, then the poster child of the Arab Spring. In case you are wondering, Tunisia has subsequently plummeted to the 92nd position. To adapt Bob Dylan, the future is already a thing of the past for these two countries.

     If these indexes are not enough, we also have the Heritage Foundation's Index of Economic Freedom (topped by Hong Kong), the Massachusetts Institute of Technology's Economic Complexity Index (topped by Japan) and the Global Liveability Ranking (topped by Melbourne), which is also produced by those compulsive rankers at the EIU. There are many, many more indexes, all offering hierarchies of desirable attributes created by supposedly sound, objective methods.

     How is the confused observer to know which are most reliable? The obvious solution is to create a ranking of the rankers -- to be based, needless to say, on a weighted average of inputs from a group of anonymous experts.

Peter Tasker is an analyst with Tokyo-based Arcus Research.

Sponsored Content

About Sponsored Content This content was commissioned by Nikkei's Global Business Bureau.

You have {{numberArticlesLeft}} free article{{numberArticlesLeft-plural}} left this monthThis is your last free article this month

Stay ahead with our exclusives on Asia;
the most dynamic market in the world.

Stay ahead with our exclusives on Asia

Get trusted insights from experts within Asia itself.

Get trusted insights from experts
within Asia itself.

Try 1 month for $0.99

You have {{numberArticlesLeft}} free article{{numberArticlesLeft-plural}} left this month

This is your last free article this month

Stay ahead with our exclusives on Asia; the most
dynamic market in the world

Get trusted insights from experts
within Asia itself.

Try 3 months for $9

Offer ends October 31st

Your trial period has expired

You need a subscription to...

  • Read all stories with unlimited access
  • Use our mobile and tablet apps
See all offers and subscribe

Your full access to Nikkei Asia has expired

You need a subscription to:

  • Read all stories with unlimited access
  • Use our mobile and tablet apps
See all offers
NAR on print phone, device, and tablet media

Nikkei Asian Review, now known as Nikkei Asia, will be the voice of the Asian Century.

Celebrate our next chapter
Free access for everyone - Sep. 30

Find out more