Last week, local research house, Ratepop, issued a media release claiming to have conducted the country’s “first social media technology-driven political sentiment” survey.

In it, a number of claims are made about support for the governing African National Congress (ANC) by party faithfuls.

Some of the results include: that 59.2% of ANC supporters should vote “as they see fit” and not tow the party line in the Motion of No Confidence in President Jacob Zuma (which was held) on August 8, 2017; and should President Zuma remain at the helm until 2019, ANC support in the general elections would suffer.

Other results of the survey include:

Current deputy president Cyril Ramphosa is most likely to succeed President Zuma after the 2019 elections with 49.5% support, compared to just 17.9% for his closest rival, Nkosazana Dlamini-Zuma;

Issues such as unemployment, housing, service delivery, Black economic empowerment, crime and corruption remain the most pertinent ills that would motivate ANC partisans vote choice in the 2019 polls.

Details about the survey

Many questions have since been raised about the survey’s results, notably the methodology adopted by Ratepop.

In his communications with SABC Digital News Head, Izak Minnaar, company CEO Socratis Avgitidis confidently defended his organisation’s first-of-its-kind survey.

“We are the first company to use a Facebook messenger bot to conduct extensive political polling research. Voters were randomly sent SMS messages asking them to participate in a political survey without reward. More than 6 000 voters across the spectrum of South African political affiliation agreed to participate and the results have been astonishing,” Ratepop CEO, Socratis Avgitidis affirmed.

Avgitidis further asserted that the views of the approximately 2000 “randomly selected ANC voters” or “members of the electorate who voted for the ANC in the 2016 municipal elections” were solicited, lending more credibility to the survey.

A case of flawed sampling

Despite the reassurances from Avgitidis, several questions have since been raised about the survey’s sampling methods.

Firstly, results were obtained using Facebook messenger through handheld mobile phones, implying that respondents needed to have web-enabled Smart Phones, in addition to the requisite technological access and technological literacy.

More worrying is, of the 2000 respondents surveyed, 704 or 35.5% were sourced from the highly-urbanised Gauteng.

In a 2013 Ipsos survey, ANC partisans were found to be concentrated in KwaZulu-Natal (22%), Gauteng (19%), Eastern Cape (16%) and Limpopo (13%).

These figures immediately set alarm bells ringing: the provincial weightages are flawed from the onset. In the Ratepop methodology, one sees Gauteng is over-represented (35% compared to 19%); and regions such as KwaZulu-Natal, Limpopo and the Eastern Cape are substantially under-represented.

In KwaZulu-Natal for example, Ratepop views from respondents here were just 18.5% whereas ANC support – according to Ipsos – is high as 22%.

The ANC’s own organisational report prepared for the 2012 National Conference corroborates this. KwaZulu-Natal had the biggest membership back then.

Additionally, in both Limpopo and the Eastern Cape ANC partisans amount to 13% and 16% respectively. Yet, Ratepop used just 9.7% respondents in Limpopo and a meagre 5.1% in the Eastern Cape.

A third concern of the Ratepop survey is that 44.7% of respondents were sourced from a “big city”, 36.6% from a “small town” and 18.5% from “rural” South Africa.

The 2013 Iposos voter support survey paints an altogether different picture: the ANC’s support base was mainly rural (48%) with the party enjoying just 37% support in the bigger cities and metros.

Noting that the use of Facebook was “interesting” for this survey, Dr Collette Schulz-Herzenberg jotted down more methodological concerns.

“My major concern unsurprisingly relates to the sampling. As Ratepop admits, the sampling is not remotely representative. According to their demographics, most of the respondents are based in Gauteng (urban), older voters and mostly male. Their respondents show no real resemblance to the wider South African population. How then can we even begin to suppose that these responses reflect anything more than the significant handful of people that self-reported?

Secondly, surely sampling error or margins of error assumes a random, representative sample of the full population in which all respondents have an equal chance of being selected. Only once this prerequisite is met can we talk about sampling error as based on sample size. Thus, I am not sure how Ratepop can claim that their margin of error is so low (despite a high number of respondents of 6000), because the sample is non-representative. I may be wrong because I am not a statistician but this is my understanding of how margins of error work.

Third, the people who are willing to take the time to participate in this type of survey will most likely be voters who are already engaged and interested in politics. Already, this narrows the group down dramatically since we know that this represents a very small proportion of the South African electorate. This group may have very different attitudes and sentiments about the ANC compared to the vast majority of core ANC supporters who may be less cognitively engaged on political issues,” Schulz-Herzenberg explained.

Surveys are just that

Despite some errors in sampling and how respondents were selected by Ratepop, credit must be given to the organisation for embarking on such an ambitious exercise.

There remains a dire shortage of such surveys using sentiment analyses, especially ones employing the wide-reaching capabilities of social media and social networking sites.

One has to remember though that surveys are by no means meant to replace rigorous scientific inquiry; and thus should be treated as random sampling efforts to understand prevailing sentiment.

[Source: SABC]