Empirical observations and methodological challenges of studying populism on Facebook

January 20, 2020 12:00 AM

By Andrej Skolkay*

Populist leaders’ political communication is usually more populist during election campaigns. That is the preliminary conclusion that has stemmed from a roundtable discussion recently held in Brussels** to explore the 2019 European Parliament elections. The discussion brought together members of DEMOS and researchers from all around Europe. It also highlighted a methodological challenge: what kind of methodological tools should be employed to assess the specificities of social media in populism? It is a burning question for any communication scholars who wants to conduct comparative analysis. And it is relevant for multiple reasons. The Slovakian case reveals some of them.

The first observation is that the findings and conclusions of the international literature seem to be less generalizable for Slovakia. Why is that so? Is that related to methodology only or does the conclusion denote some differences in communication and regional specificities/local particularities?

It is interesting to observe that although two Slovak populist parties (We Are Family—Boris Kollár, WAF, and OĽaNO) showed identical share of types of key populist indicators (anti-elitism, people-centrism and othering strategies), their populist rhetoric as well as their political roles are perceived differently in the country by local experts. WAF is seen as “hard” populist (hindering democracy) while OĽaNO is viewed as “light” populist (contributing to democracy). Key WAB representatives, however, are mostly telling the truth based on data analysis, and they are certainly not the least trusted politicians in the country. The key difference appears to be who are their targets or “the others” - these are typically migrants or foreigners for WAF. In other words, statistical data (coding) do not reveal the complete and correct picture.

Mr. Pavel Havlíček, (second from the left; European Endowment for Democracy, EED) and Professor Eglé Butkevičienė (DEMOS, Kaunas Technology University)

The DEMOS study on populist leaders’ Facebook pages exposed another problematic point related to intercoder agreement. Many local coders have coded specific posts differently, which brings up questions about whether the concept of populism is well defined. The coding process in several waves shows how difficult it is to have agreement on very subjective and complex variables, especially ones that have a theoretical background that the coders might not be as aware of as the researchers designing the codebook. The inter-reliability scores showed that an acceptable reliability was reached only for the very simple variables (format of posts: text/video/image; visual representation), and only in some cases for more complex ones.

A colleague pointed out that, despite a well-prepared manual, it is difficult in some cases to resort to unambiguous coding. An interesting example is the case of cooking goulash, a traditional Hungarian, Czech and Slovak food. It’s been featured in several political party events in Slovakia. But it is also offered in multiple local activities, from music festivals to public holidays. Cooking goulash at political party events might be seen as a way for politicians to suggest that they are just common, ordinary people, but coding it for the purposes of scientific results might yield inaccurate and ambiguous answers.

Should we categorize it just as a simple event announcement on social media, or is there a hidden message in the announcement, such as “we are like common people”, “we are like you”, “we have equal values and customs”?

The selection of to-be-coded social media posts may be worth examining in detail. A hypothesis can be that if a Facebook page of a politician is more popular than that of a party/movement, the page is more populist. In this case, the political party loses its significance at the face of dominance of a political leader. To test this hypothesis, we might focus on shares and likes of each post (by the leaders and by political parties) rather than the Facebook pages of leaders or political parties’.

In short, populist communication strategies and their messages reflect local conditions­—they can be identical or they can be totally different. But by definition it will include features of either “anti-elitism”, and/or “people-centredness”, and/or “othering”. These are all key populist features.


Dr Andrej Školkay is a research director of School of Communication and Media, Bratislava, Slovakia. He has published widely on media and politics, especially on political communication, but also on ethics, media regulation, populism, and media law, in Slovakia and abroad. His most recent book is Media Law in Slovakia (Kluwer Publishers, 2016). Dr. Školkay has been a leader of national resarch teams under H2020 Projects COMPACT (CSA - 2017-2021), DEMOS (RIA - 2017-2020), FP7 Projects MEDIADEM (2010-2013) and ANTICORRP (2013-2017), as well as Media Plurality Monitor (2015). Dr. Školkay was member of the Press Council of Slovakia in years 2005-2008. He obtained his PhD in 2000 from Comenius University in Bratislava, Slovakia.

** From roundtable co-organized by the project partner School of Communication and Media in Slovakia (SKAMBA), and European Policy Center in Belgium, and hosted by the  European Digital SME Alliance in Brussels in October, 2019. Read more about the event here.

Comments (0)