How World Bank unfairly influenced its own competitiveness rankings

The World Bank's chief economist said he would recalculate national rankings of business competitiveness going back at least four years

The World Bank repeatedly changed the methodology of one of its flagship economic reports over several years in ways it now says were unfair and misleading.

The World Bank’s chief economist, Paul Romer, told The Wall Street Journal on Friday he would correct and recalculate national rankings of business competitiveness in the report called “Doing Business” going back at least four years.

The revisions could be particularly relevant to Chile, whose standings in the rankings have been especially volatile in recent years and potentially tainted by the political motivations of World Bank staff, Mr. Romer said.

The report is one of the World Bank’s most visible initiatives, ranking countries around the world by the competitiveness of their business environment. Countries compete against each other to improve their standings, and the report draws extensive international media coverage.

The former director of the group responsible for the report, Augusto Lopez-Claros, didn’t immediately respond to requests for comment sent to personal and work email addresses listed on his personal website.

A former professor at the University of Chile, Mr. Lopez-Claros is on leave from the World Bank this year while serving as a senior fellow at Georgetown University. The World Bank and Georgetown didn’t immediately provide other contact details for Mr. Lopez-Claros.

A World Bank spokesman said the World Bank had no comment beyond Mr. Romer’s statements.

Over time, World Bank staff put a heavy thumb on the scales of its report by repeatedly changing the methodology that was used to calculate the country rankings, Mr. Romer said.

The focus of the World Bank’s corrections will be changes that had the effect of sharply penalizing the ranking of Chile under the most recent term of Chile’s outgoing president, Michelle Bachelet.

“I want to make a personal apology to Chile, and to any other country where we conveyed the wrong impression,” Mr. Romer said. The problems with the report, he said, were “my fault because we did not make things clear enough.”

Chile’s overall ranking has fluctuated between 25th and 57th since 2006. During that period, the presidency of Chile has alternated between Ms. Bachelet, of Chile’s socialist party, and Sebastián Piñera, a conservative. Under Ms. Bachelet, Chile’s ranking consistently deteriorated, while it consistently climbed under Mr. Piñera.

Mr. Romer said the World Bank is beginning the process of correcting the past reports and republishing what the rankings would have been without the methodology changes. Mr. Romer said he couldn’t defend “the integrity” of the process that led to the methodology changes.

Recalculating the numbers could show significant changes to other countries as well.

“Doing Business” ranks nations on metrics like the number of days it takes to open a business, or the cost of getting construction permits. Countries that make their business environment worse, for instance by drawing out the permitting process, get penalized in the rankings.

The World Bank has updated the methodology over time. For example, in the years covered by Mr. Romer’s review, the World Bank added new components dealing with construction permits, new measures of electricity reliability and tariffs, new measures of the quality of judicial processes for shareholders and a new measure of tax filing, among others.

During Ms. Bachelet’s tenure since 2014, new components had the effect of lowering Chile’s ranking. For example, in the report published in 2015, Chile had been ranked 33rd for ease of paying taxes; in the report published in 2016, the World Bank added a new metric on the amount of time businesses must spend dealing with taxes after having filed them, such as via audits or obtaining refunds on value-added taxes. Chile scored exceptionally poorly on this new index, and once it had been added, its ranking for ease of paying taxes sank from 33rd in the world to 120th.

According to a preliminary analysis by Mr. Romer, over the past four years, Chile’s drop was driven almost entirely by adding new metrics to the index, and not by changes to standing measures of Chile’s business environment. He added that changes to the methodologies used in the rankings had the appearance of being politically motivated.

“Based on the things we were measuring before, business conditions did not get worse in Chile under the Bachelet administration,” Mr. Romer said. “I didn’t do enough due diligence and later realized that I didn’t have confidence in the integrity” of the report’s data.

Mr. Romer raised the concerns with World Bank leadership who supported his decision to correct and recalculate the figures.

Mr. Romer joined the World Bank in October 2016 from New York University. The changes to past reports will date back years before his arrival at the World Bank.

Mr. Romer has clashed with World Bank economic staff before. In May, he published an internal memo he’d written in which he told World Bank economists they should write more clearly and concisely. 

“The problem with vague writing is that it lets an author convey a false impression yet retain plausible deniability when someone tries to verify the claim,” he wrote.

In the memo, Mr. Romer referenced critical research from Stanford University’s Literary Lab that had studied the writing in World Bank reports and concluded the reports were written in “almost another language, in both semantics and grammar” and were “becoming more abstract, more distant from concrete social life; a technical code detached from everyday communication.”

During that 2017 clash, Mr. Romer also published an earlier memo he had written about a budget document he received early in his tenure as chief economist. The budget memo said converting 90 contract workers to full-time workers would “have no impact on the Bank Budget,” but upon questioning the staff and investigating the issue, Mr. Romer determined that the full-time employees would cost considerably more than contractors.

Mr. Romer became alarmed, saying that a group in charge of economic data and empirical research shouldn’t be producing internal documents with misleading data. “If people in the Bank cannot believe everything [the Development Economics Group] writes, they can’t believe anything we write,” he wrote in that internal memo.

Source: The Wall Street Journal