Beyond Benchmarking: Why Countries should Ignore International Rankings
Rather than accepting global rankings and their accompanying policy recommendations, Robyn Klingler-Vidra and Yu Ching Kuo contend that policymakers should either ignore, or challenge, their findings.
In Ranking the World, Alexander Cooley and Jack Snyder explore the rise of benchmarking and rankings of countries. They indicate more than 95 such rankings by the time their book was published in 2016. Today, with the success of country rankings such as the Economic Intelligence Unit’s Democracy Index and the World Economic Forum’s Global Competitiveness Rankings, that number has grown two-fold, as more than 200 rankings systems compare countries for their democratic quality, investor friendliness, economic competitiveness, and more.
But, international benchmarking methods are problematic; they reflect politics, suffer from incomplete coverage, sample size and bias challenges, and institutional bias. Why, then, do countries increasingly rely on them to inform their policymaking? We employ Taiwan, and entrepreneurship rankings, as a lens to explore the accuracy of benchmarking methodologies, and the offer a new way forward. One that is informed by local evidence rather than global rankings, and as such, is better positioned to solve the ecosystem’s pressing challenges.
The problems of benchmarking
As the winding down of the World Bank’s Doing Business ranking last September showed, the methods underpinning country indices are, at best, contestable and non-transparent, and at worst, dubious. The World Bank’s long-touted ranking, for instance, was influenced by politics, with state leaders threatening to reduce their funding. When the international ranking systems are overshadowed by politics, countries can be excluded. Taiwan, for instance, is not included in the World Intellectual Property Organization’s Global Innovation Index since the island is not a recognized member of the United Nations.
Other issues that plague the rankings include the problem of subjectivity and sample sizes for the surveys that constitute the basis of the scoring. In the World Economic Forum’s survey, just 80 executives in each country comprise the survey-based elements for its Global Competitiveness Rankings. This small sample of respondents in each country offer their opinions about their country’s talent force, dynamism of the business environment, and access to finance. One of the problems with such (small) survey-based assessments is that those 80 respondents are meant to represent a mix of large and small firms, and reflect the industrial mix of the country. But, this can mean that just one or two respondents would constitute the sample for their sector, such as agriculture in the United Kingdom.
The Global Entrepreneurship Monitor (GEM), which surveys for entrepreneurial propensity and risk-taking, has a similar sampling challenge. The GEM research team claims that at least 2,000 adults aged 18–64 years in each economy were randomly selected to participate in its Adult Population Survey. However, a Taiwanese research team found that only a few entrepreneurs with busy schedules were available to take the phone interview in the evening when this kind of survey is conducted. With survey sampling involving a wide range of randomly selected individuals, such soft data is inadequate to catch the picture of entrepreneurial dynamics in a country.
There’s also the problem of bias and worldview of the organization itself, such as the Heritage Economic Freedom Index, IMD World, and WEF all reflecting neoliberal biases. If we “follow the money” to see who is funding these ranking organizations, we find a range of Western-domiciled corporations.
How benchmarking scores shape policy agendas
Despite the shortcomings of benchmarking methodologies, states increasingly (1) care about their performance, and even (2) adjust their policies in order to boost their rankings.
Take Taiwan as an example. Especially since 2016, with the creation of the President Tsai administration’s “Asia Silicon Valley Development Plan”, government ministries repeat the refrain that they are not ranking amongst the world’s most entrepreneurial countries, and as a result, need to better promote startups in Taiwan. In particular, since the early 2000s, Taiwan’s National Science and Development Plans, which outline priorities and programs for coming four-year periods, present analyses of the country’s latest performance in the World Economic Forum, IMD, and GEM rankings in order to name, and justify, their policy priorities.
For instance, the launch of several Small and Medium Enterprise Administration, Ministry of Economic Affairs, programs in 2019 cite recent GEM results, which found that in Taiwan there is underdeveloped cognition for entrepreneurial activity and little risk-taking appetite. So, the government explains that funding, tax incentives, regulatory changes, and accelerator programs need to be enacted in order to remedy the country’s lackluster performance. The state needs to promote the development of its “nascent” startup cluster. Aligned with the GEM reports, officials point out that the provision of entrepreneurship education is insufficient. They, for instance, make the case for enhancing entrepreneurial education in order to better equip students with an entrepreneurial mindset and approaches, in a bid to boost future benchmarking results. Several cross-ministry industry enhancement programs, for example, the Digital Nation and Innovative Economic Development Program- 2017-2025 also make specific mention of fostering innovation systems in order to enhance their competitive position in the rankings.
But, is the analysis accurate?
While the survey-based GEM results suggest inadequate entrepreneurial propensity and activity levels in Taiwan, hard data suggests otherwise. A Taiwanese consultant for service-oriented industrial entrepreneurship, Pei-Fen Lee, wrote in the column “Italent” edited by the Industrial Development Bureau of MOEA, that in Taiwan most relevant statistics regarding entrepreneurship only presents a partial story of entrepreneurship, and, often, only illustrates the early stage of entrepreneurship. Lee notes that that according to the 2019 SME White Paper published by the SME Administration in 2018, the number of SMEs in Taiwan was 1.46 million, accounting for 97.64% of all enterprises, an increase of 1.99% over 2017. If the co-financing ratio of nearly 30% is considered, one out of every six working-age population in Taiwan is an owner or a shareholder in an SME. Aligned with Lee’s research, we investigate the 2021 SME White Paper. This updated version shows that the number of SMEs in 2020 was 1.54 million, accounting for 98.93% of all enterprises, while the number of SMEs employed reached 9.31 million, accounting for 80.94% of the total employment in the country. When the co-financing ratio of approximately 47% is considered, one out of every 5.08 people in Taiwan’s working-age population is an owner and/or a shareholder. Hardly a nascent ecosystem.
Benchmarking results also suffer inaccuracies on account of different cultural norms across countries. Cultural norms can explain why Taiwan would rank so low, and contrast the hard-data on the prevalence of entrepreneurial activity. Sarah Kulkofsky and Qi Wang’s research showed cross-cultural differences in a number of cognitive processes “do not simply reflect response biases or impression management concerns, but rather modesty as an important cognitive component of the East Asian self”. In a similar vein, Jennifer An-Hwa Liu wrote that in Taiwan there is “a general cultural tendency toward modesty in which self-deprecation is taught and bragging is sanctioned against”. As Sanjaya Lall suggests, “the phenomenon is too multifaceted and complex to permit easy measurement”.
The GEM findings note that survey respondents say that risk-taking and entrepreneurship are lacking in Taiwanese culture. These responses may reflect cultural humility more than genuine entrepreneurial propensity. For instance, anyone who has spent time in Taiwan will know the popular saying, that it is “better be the head of a cock than the tail of a cow”. There’s widespread understanding that it’s better to be an entrepreneur and lead something small than to work for a large company and be an employee. In Israel, by the way, there is a very similar saying: “Its better to be the head of the foxes than the tale of the lion.”
A locally-informed, benchmark-free way forward
Rather than accepting the global rankings and their accompanying policy recommendations, we contend that policymakers should either ignore, or challenge, the findings. Because the Taiwanese government – like so many countries today – wants to succeed in continuing to attract talent and investment to Formosa Island it engages the rankings, and reflects the findings and suggestions in their policies. They want to show that they are responding to the rankings, in an effort to show their business-friendly nature and their international awareness.
The problem of accepting the benchmarking results is that governments, like Taiwan’s, are using their limited resources to solve the wrong problems. In Taiwan, tech entrepreneurship has been long-established, pre-dating even the creation of Hsinchu Park, as a globally-leading semiconductor cluster in 1980. Also, Taiwan doesn’t need to foster the idea of connections with other startup clusters as something new; as AnnaLee Saxenian and others have meticulously documents, Taiwan has had strong global links for more than a generation, especially with Silicon Valley.
Instead of the deficiencies identified by the benchmarking studies, the Taiwanese government should be empowered to say that it has long been a world leader in startups, that they helped create the model. Taiwan – and so many other states – should challenge the rankings, or maybe better yet, not acknowledge them. Policy should be informed by government statistics and close engagement with entrepreneurs. They should strive to understand what are the real challenges, like talent needs, the patent landscape, and developing new technology clusters, and then craft policy accordingly. Taiwanese government agencies have been surveying various issues, ranging from manpower and social welfare, to national economics and business activities.
Policy should be informed by such research and interactions with local entrepreneurs and established firms, rather than giving agenda-setting power to global rankings firms. The demise of the Doing Business ranking shows that such benchmarking systems do not deserve the power they have wielded so far in the 21th century. Let’s empower innovation policy managers, rather than teams in Davos, to set their own, locally-informed, agendas.
Robyn Klingler-Vidra is Reader in Political Economy at King’s College London. She is the author of The Venture Capital State: The Silicon Valley Model in East Asia (Cornell University Press, 2018).
Yu-Ching Kuo is an independent researcher based in Kaohsiung. She is the co-author of ‘Brexit, Supply Chains and the Contest for Supremacy: The Case of Taiwan and the Semiconductor Industry’ in A New Beginning or More of the Same? The European Union and East Asia After Brexit (Palgrave Macmillan, 2021).
Photo by Susanne Jutzeler
