As Nielsen’s dominance wanes, SurveyAuto puts its disruptive tech to work

On the one hand, we have the dwindling presence of an iconic American data and information firm. On the other, the up and coming project of a former government darling spurned, stripped of his position and forgotten after the coming to power of a new political party despite his stellar performance. 

It seems an innocuous end for legacy market intelligence services. But it has been on the cards for years now, with the Covid-19 pandemic striking the final blow. The Nielsen Corporation, established back in 1929, currently operates in more than 100 countries. It is an iconic name, one most associated with the ‘Nielsen rating’ –  the mode of measuring television ratings that is still used in most of the world today. 

It has also been operating in Pakistan for decades. And on the 30th of September this year, Nielsen announced that they were shutting down their retail measurement unit in Pakistan. Nielsen has rarely trusted the Pakistani market, refusing to enter the television ratings business in the country and always being quick to safeguard itself. This care, however, has not meant that Nielsen has grown slowly in Pakistan. And their current abandoning of ship means they are leaving a large vacuum. 

This is where Dr Umar Saif, the former chairman of the Pakistan Information Technology Board (PITB) and poster-boy of former Punjab Chief Minister Shehbaz Sharif, comes in. When a company like Nielsen contracts or falls, there are many that try to seize the opportunity. These attempts normally begin with dizzying highs and end in abysmal lows. A vacuum means a company is bound to get clients, but it also usually means that the ones seizing the day come in hastily and ill prepared, and end up making fatal mistakes. 

Except this time, the takeover is not a hasty one, and Dr Saif has been preparing his market intelligence company, SurveyAuto, for years. It is simply a happy coincidence that Covid-19 has forced Nielsen to pull back. It is an ethereal moment of the stars aligning perfectly. Because Dr Saif was not looking for a gentle filling of a gap, he was rearing for battle. SurveyAuto was supposed to revolutionise market intelligence in Pakistan. It was preparing to be a technological disruption, one that provides more accurate information than any existing company. They were ready to lock horns with more traditional data intelligence companies. Instead, Dr Saif’s competitors have pulled up their drawbridge even before he made his intentions known. 

Things have fallen in place, and far too easily, far too simply – almost enough to make one nervous and flinch at every little mishap. When will the challenge come? Will it ever? It has been smooth sailing, ridiculously so, but it isn’t quite done and dusted. All SurveyAuto and its prolific CEO have to do is walk the road and take the market intelligence throne for themselves. But companies have fumbled simplet opportunities, and there is a story to be told here, and lessons to be learned.

The market intelligence market

Market intelligence services have been around for more than a century. These are companies that collect, analyse, and provide information to other companies about the trends, competitors, and customers in their markets. And if you’ve been around for that long, even if you have been trying to keep up with the times, it is entirely likely that you are antiquated in many ways. 

The idea of market intelligence companies is that whether you are a steel manufacturing company or make children’s toys, you can walk into the office of a marketing intelligence service, pay them a fee, and in exchange they will research your market for you and tell you where to spend your money. 

And it isn’t just companies that need these services. Government and development agencies require data for effective decision making. Be it to understand availability of medicines or clean drinking water, measuring poverty or in fact all the domains in which the government or development agencies work require actionable intelligence, without which decision making would be flawed and would fail to achieve desired results.

For companies like Nielsen and SurveyAuto, data is a currency. It is more valuable and global than the US dollar, and when you have specific information that a company or a government or an organization needs, it is precious. It is critical for any manufacturer to really know what their actual offtake is from the shop to end consumer. That is what companies like Nielsen do for FMCG companies, that is gather data on sales, stocks and other variables that help the FMCGs make decisions about their brands. 

For example, think about the availability of Milkpak in stores in a Lahore neighborhood. It is critical for Nestle to know if the stock of Milkpak has depleted in the said neighborhood shops and if that needs to be replenished, the scarcity or abundance of stock, how long it takes to locate the said product in a store. For brands selling outside of their home country, it is critical to understand the unique challenges every market poses due to different economic, political and cultural environments.

Companies like Nielsen gather this data for their clients – the aforementioned CPG companies – in what is called ‘Retail Audit’. This is exactly the service that Nielsen has now shutdown in Pakistan. 

Besides retail audit, another dataset that is valuable for manufacturers is consumer insights, the trends in human behaviour that affect a products sales and consequently financial performance of a company that is producing these products. An example of this would be the brand perception of Coca Cola in a Pakistani village. These are services based more on artificial intelligence than on survey data.

Nielsen’s fall 

So where did things go awry for Nielsen? Large FMCGs have trusted Nielsen over the years to provide accurate data and market information that has gotten them results. Was it really just a question of post-Covid realities being too harsh? Nielsen has always been a cautious company. 

Back in 2001, a great push was made for Nielsen to provide television ratings in Pakistan. However, the company expressed distrust in the market and dashed these hopes. They were later proven right, when Pakistan’s television ratings business ended up becoming murky territory. But there are still some that indicate the decision may not have been as voluntarily as Nielsen is trying to portray it as. Is it possible that Nielsen was forced out, and is simply trying to exit respectably?  

In the run up to Nielsen wrapping up its retail audit operations in Pakistan, Profit learnt from various sources, including former Nielsen staffers and high ranking employees at FMCGs that have been clients of Nielsen such as Nestle and Engro Pakistan, that the market intelligence company’s data was being challenged by these companies as it was not representative of what the reality would turn out to be.

The way Nielsen generates market insights is that the company, in its individual capacity, is constantly engaged in market research that it perceives is valuable for big manufacturers. The reports are then furnished to FMCGs that have subscribed to such market research. 

These are general research projects, like retail audits. So if Nielsen researches on the snacks segment, all snacks producers that have subscribed to Nielsen will get the same report, and then it will be up to the different companies what they do with the information. The report will contain information of, for instance brand health and how the competitors are doing in a specific segment. Then there are bespoke surveys carried out by Nielsen and tailored for their clients. These reports are supposed to give the clients, who are naturally paying more, a massive edge compared to their competitors. 

So where did the mistrust come from? Initially, it happened because of small discrepancies in the retail audit reports. For a long time, even large companies have left market research to third parties. Having your in-house research team is expensive to maintain and expensive to set up, and companies like Nielsen have managed to provide results for a long time. However as companies continue to expand, in-house research, while not as detailed, has become more common. 

So, for instance, if an FMCG company, let’s say Nestle, subscribed to the retail audit service of Nielsen, they might see through Nielsen’s data that their sales for Milkpak had increased by 12% over a certain period. However, their own research will show that its sales have grown by 15%. This naturally makes Nestle question where Nielsen is getting their 12% figure from. 

At this point, the FMCG company would start recording observations over a period of time and monitor the trends to see if the data coming from Nielsen had been underreported all along. If this turns out to be the case, it proves that there is something clearly wrong with the methodology being used to collect this data, or the way it was being carried out. Whatever the case, the FMCG would be making strategic errors because of incorrect data.

The other question is, if Nielsen was getting their regular, run of the mill, bread and butter reports wrong, could their tailored surveys for individual clients be trusted? All the surveys, carried out by a fleet of auditors that incurs huge census, pre-survey, actual survey, logistics and personnel costs, data collection and reporting costs, end up being ridiculously expensive. Profit was not able to get specific details about how much the total would run up to for such surveys, but learnt from a source that it is a very costly exercise running in tens of millions of rupees. Multiple variables affect the prices for instance the type of survey and complexities associated with carrying out such a survey.

A source at one of the large CPG companies told Profit that it had been over a year now that Nielsen’s data was being challenged by the company, with data coming from locations where there used to be no shops. So if you are looking at it from a profitability and growth standpoint, both for the FMCG and the market intelligence firm, FMCGs would be reluctant to pay high prices to Nielsen as premium for its services when the data would not even be representative of what was happening on ground. 

And that the competitors to Nielsen, they were offering similar services at a much lower cost. As a consequence, Nielsen lost many of its clients, large clients in the tobacco industry such as Pakistan Tobacco Company, which is perhaps what prompted its withdrawal from Pakistan. 

“They had big issues and that would be a pain for FMCG companies. Manufacturers would hire people specifically from companies who would be working with Nielsen to check the backend quality of the data. So it really becomes a dirty, messy business primarily because there is no credible data,” a source told Profit.

But that does not quite solve the problem. There is a reason why Profit  has chosen to focus only on Nielsen in this feature. It is because Nielsen is the gold-standard in the industry, and its competitors do not even come close to it. So while Nielsen was losing clients to cheaper options, the problem for companies such as Pakistan Tobacco Company, is that the alternatives might be less expensive, but were not providing accurate data either. Complaints like this have also been made about alternatives such as Ipsos and Access Retail. So when even the best is faltering, who do you turn to? In 2020, the answer is always going to be technology. 

That is the vacuum we mentioned in the beginning. The gaping hole that needed filling. It is not clear whether Dr Umar Saif knew Neilsen was in the doldrums, but he has been preparing to become a challenger in the market intelligence industry at least since 2018. This is not a sudden entry. It has been planned, and now helped by fate. This is where SurveyAuto comes in.

Enter SurveyAuto

A little less than two years ago, Profit visited the residence of Dr Umar Saif. The interview was about the prospects of growth of Lahore’s Information Technology University (ITU), and the reasons behind its creation. With an undergraduate degree in Computer Science from the Lahore University of Management Science (LUMS), a doctorate from the University of Cambridge, and the post-doctorate from the Massachusetts Institute for Technology (MIT), he is nothing if not prolific. 

One of the most influential bureaucrats in Shehbaz Sharif’s cabinet between 2013-18, he was at the forefront of the PITB, and served as the Vice Chancellor of ITU. Within a few months of the Pakistan Tehreek-e-Insaf (PTI) government forming a government in Punjab, he was removed from both of these positions. 

At his residence, the reporters were led to a room lined with computers that looked like a high-tech call center. Only two people were in the room, and they too were wrapping up their work. Dr Saif’s residence seemed a relatively normal one, which is why the feeling of having been shuffled into the secret lair of a scientist was all the more potent.  It was some time before Dr Saif arrived, informing us that we had mistakenly been led to this room, and he had been waiting next door the entire time. 

Suave, gracious, and possessing a keen mind, Dr Saif diverted us quickly and jumped into our conversation for the day. The incident was quickly forgotten, and perhaps unwisely, no questions were asked about the mysterious room. 

What we would discover later is that the lab was building artificial intelligence (AI), and a machine learning powered technology platform. This would later be branded as SurveyAuto, the company that is today primed to challenge well established market intelligence companies. It has been a little less than two years since that day. For much of that time, Dr Saif has spent his resources and mind on the system to make it so powerful that it is second to none. For all these years, SurveyAuto, of which Dr Saif is the CEO, has been in stealth mode, improving its technology and systems, and that too purely boot-strapped.

The original purpose of the business, like any other technology business is to disrupt. And disruption is what SurveyAuto wants to create in the data collection/market intelligence industry, that has not been penetrated by technology startups seriously as yet. 

The way a tech-based market intelligence company is run would be starkly different. Business graduates in hoity-toity suits with clipboards and ‘analysis’ skills are replaced by computers, data entry professionals, computer scientists, and software engineers.  

And Nielsen’s withdrawal is only good fortune for SurveyAuto. Even if Nielsen had chosen to invest more into Pakistan, Dr Saif would still have gone forward with his plans. It not only validates that the old brick-and-mortar model is outdated, but also creates a room for technology companies to establish themselves as dominant players.

The Uber-isation of data collection

The central problem, at the end of the day, is the non-availability of reliable data. As times change, data that gets close does not cut it anymore. What companies need is precision, speed, and scientific analysis. People can only do this so far, and the general consensus is that large corporations are tired of human error. A company’s growth is affected by the accuracy of the data because of how heavily companies depend on this data for their growth strategies. Accuracy is what SurveyAuto promises. It is important to state again just how different a technology based company like SurveyAuto will be from a traditional market intelligence service like Nielsen. 

Consider this use-case: a local manufacturing company wants to find out that in a village, how many people smoke its brand of cigarettes. Now, under the traditional data collection model, it would be costly. It would be a manual, labour intensive, and consequently costly and time consuming exercise whereby a survey team would do on-ground mapping of shops or houses to be surveyed.

By being costly, it can not be repeated at a higher frequency, thereby creating an issue of quantity of data, which needs to keep pace with the rapidly evolving market dynamics. Then the process of data collection and analysis is time consuming, sometimes running in months. Meanwhile, if the FMCG runs some discounts on its products, there would be a lag of a few months before they received reports of how it has played out in the market, which would have changed by the time they get to know about it. 

SurveyAuto says that it’s platform, a Software as a Service (SaaS) model, can be operated by the FMCG itself, whose sophisticated machine learning and artificial intelligence algorithms allow geomapping, thereby reducing the need to do expensive census and pre-surveys, that can all be done on the platform, without going there. The SaaS model means self reliance for these companies. They no longer have to trust analysts in suits and wait for months to receive reports riddled with human errors. SurveyAuto will simply design a software for you that you can use and operate yourself. It is essentially equipping in-house market intelligence research with high-tech systems. 

“The company can turn on satellite imagery from the platform and at a 100 metre square granularity, it can tell how many houses there are and how many people live there. The technology will accurately be able to tell how many houses there are in the village,” explains Dr Said. His glee when it comes to technology is palpable. “The platform can also be tweaked to tell which are high density areas and which are not. Using AI and machine learning, the sample can be defined real-time sitting anywhere.”

To be more statistically representative, the AI and machine learning algorithms automatically suggest what would the representative sample out of the total number of households or shops to be surveyed for the cigarette brand. The sample size, all defined by the company itself, thereby minimising the liability of SurveyAuto. And the confidence that SurveyAuto has is that its system is highly accurate because of the algorithms powering it. Once the sample size is defined, the survey can be created and then commissioned to the team that is going to carry out the survey. This is where things get more interesting.

Companies like Nielsen deploy fleets in areas that are to be surveyed, and all the associated costs make it expensive. Instead of this high-cost system, SurveyAuto has created a marketplace of these enumerators – the persons employed to do these surveys. What that means is that anyone can download the SurveyAuto application and sign up as an enumerator. They will then be assigned digitally and will be paid per task completed through mobile banking channels. So when a tobacco company is commissioning the survey to the team of enumerators, it would be carried out by the people that are in the vicinity of the village where the said survey is going to be carried out, eliminating the need to deploy enumerators there manually. The best way Dr Saif can find to describe this system is to call it the ‘Uber-isation’ of data collection.

The traditional model still being used by Nielsen and its competitors are human oriented. Other than the high-cost of this method, the problem is that their enumerators take samples in non-scientific ways. This is because they ask their fleets to do surveys house-to-house. This means that the chances of the sample being representative is not very high. Up until now, at least in Pakistan, companies were aware that this was not the best possible method to achieve their goals. However, it was the best method available. Now, with a service like SurveyAuto available, companies can get more accurate data for cheaper. The only question is, will enough enumerators sign-up to their Uber-esque system?  

“Algorithmically, SurveyAuto’s AI would automatically filter out houses that are truly representative of the sample. It will take more samples from areas where house concentration is greater, and less from where the number of houses are less. This would be randomised and geographically representative,” says Dr Saif. The method is sound, especially since the density of population is measured through satellite images in the SurveyAuto model. 

It is also randomised, which means that it can be changed the next time so that the same sample is not repeated. If this works, the sample will be more representative than anything companies like Nielsen can provide. The system is sophisticated enough that it can drop exact pins on the houses that are to be surveyed, so that the enumerators are assigned houses digitally to collect samples from all of the ones specified by the company, leaving nothing on the enumerator. The team of enumerators can be held accountable to multiple variables under quality controls like time assigned to complete the survey, recording audio to judge whether the right questions were asked in the right manner, and the enumerators can even be tracked in real-time. 

The commissioning client further has the ability to accept or reject surveys real-time, even ones that are completed and can commission them again there and then, all in all creating actionable intelligence that is not shoddy. Since asking the correct questions is on the enumerators, who are close-by, it is easy to do as many surveys as possible since expensive fleets do not have to be dispatched. This means repeating surveys is possible numerous times, making the results more accurate.   

“My idea was to Uber-ise data collection where we get people to install our application on their cellphones and they become a part of our marketplace,” says Dr Saif. “We wanted our platform to be self-reliant. So that it could decide for itself where it was important to collect the data.” This is ambitious, but it is also the hallmark of what AI should be – as intelligent as possible. 

“By statistically representative we mean if the geography is a village, how many houses there are in that village, how many houses out of those in the village are statistically representative, that if you ask these households, you would have a better chance of knowing correctly if the household consumes a certain brand or the other. That is the way to do it.” 

And if the locality where the survey has to be carried out is a city instead of a village, where the number of shops or households are large, SurveyAuto’s AI has the capability to carry out a census of shops digitally, creating clusters in the city where these shops are located in such a way that it would highlight markets out of them, and understand the population dynamics surrounding these markets. Questions like how much population is served by a shop in a particular market and what is their economic class. 

“Essentially we have compartmentalised. So we took the shops, and from the shops we demarcated markets, and from markets we took out the catchment area, and from this be derived how much of the population served by these markets, and from this we determine what the socio-economic class of this population is,” says Dr Saif.  “All this hassle is to achieve statistically representative data, in a controlled environment, through science and technology.” 

“We have people in East African countries like Malawi that have downloaded our application and are working as enumerators. We can track them and keep an eye on exactly what they are doing. Whether they even went to the place or not. We record audio to confirm if the questions were actually asked and answers were received, we look at the light sensor to see if it was an indoor survey or an outdoor survey, the gender of the respondent and other variables,” he adds.

Though all data companies work with government as well as other development agencies, the real meat lies in the commercial world, with big manufacturers like Nestle, Unilever, Procter & Gamble, where hardcore data collection comes into play. These companies are entire economies on their own, and they require data so they can make informed decisions. Data that can make or break companies. Data is the new oil, as they say. And for those in the industry, even new entrants like SurveyAuto, the old mantra remains: data is currency. 

Now SurveyAuto does not go out to the manufacturer to present these results to these companies and leave it to them to analyse. It connects with the secondary sales systems of a distributor, and that shows the sales of that particular manufacturer to SurveyAuto. That enables SurveyAuto to determine what was the sales target for a particular area and compare it with survey results. Since SurveyAuto is integrated into the secondary sales system of distributors, it has access to data pertaining to competitors which enables it to tell its clients how they fare against their competition, creating reliable, actionable intelligence for its clients.

“So if you want to see the shops that have been able to achieve a certain sales target, you can see that on the platform. That is the sales analysis. The platform is also equipped to tell in which areas, there is some population but no shop to serve that population, where the FMCGs distributor has never even reached out. The FMCG has the capability to spot a certain area on the SurveyAuto platform and commission a survey to, let’s say, find out if the FMCGs products are available in the shops in the selected area or not. This is where they trigger our field operations,” Dr Saif says.

The optimisation that SurveyAuto is bringing is this: it identifies the areas from where the data needs to be collected, no need to send fleets of auditors to areas where they are not even needed. It removes inconveniences and inefficiencies. It is time-saving. Surveys can be tracked real-time and by the time surveys are complete, they will be available in real-time for any analysis, with the ability to recommission them again if required. It can also do online surveys on social media platforms such as FaceBook, Twitter, Instagram, and what is most attractive is that these surveys will all be controlled by the manufacturers themselves.

The Unicorn road 

All serious startups, at some point or the other, think about where they are going. And if they are ambitious enough, that goal is becoming a unicorn. Say what you may about it, and the traditionalists in the market intelligence business will indeed have a lot to say, but SurveyAuto definitely is ambitious. While all of its initial operations were bootstrapped, Dr Saif thinks his company has all the right ingredients to become a unicorn: agility, nimbleness, low cost structures, multiple revenue streams, a large market to serve and operations in multiple countries.

The traditional data companies that are global players that SurveyAuto aims to disrupt are all multi-billion dollar companies. A lot of that has to do with having large FMCGs as their clients with operations in multiple countries. For instance, Nielsen came into Pakistan a few decades ago by virtue of its affiliation with clients with global operations. Where these large FMCGs would exist, Nielsen would set up offices in those countries or they would have affiliates in these countries. Local market intelligence company Aftab Associates, which is the pioneer of retail audit in Pakistan, also became Nielsen’s affiliate some 30 years ago, before Nielsen set up its office here. But now, as Nieslen has taken on local identity everywhere, it is the very FMCGs that brought them here that are dissatisfied with their services. 

The only reason why you would have Nielsen is that across geographies, then you would have a comparable thing to share. So if you are talking about Coca Cola, you have comparable data from Nielsen to share with Coca Cola in the Middle East or Africa. If it is a different data company in another geography, the comparables are distorted, it is not an apples to apples comparison anymore. 

This is exactly what is likely to fuel SurveyAuto’s growth and expansion. It has recently entered into agreements with large FMCGs in Pakistan, all negotiated and carried out by global headquarters of these FMCGs. This means that if SurveyAuto is able to prove itself as a reliable data company in Pakistan, these companies might allow SurveyAuto, much like Nielsen and others, to serve these companies in other geographies. And wherever there is Nielsen, there are the inefficiencies that come with this sort of traditional market intelligence service. 

In Nielsen’s case, expanding has not always been in its best interest in the long run because of how expensive it is everywhere it goes. But unlike Nielsen and others, Dr Saif’s assessment is that technology will allow SurveyAuto to keep its costs controlled. They do not need to open large offices or hire a massive workforce when they go to a new territory. Companies would be able to do surveys in different countries sitting in their offices.

As Profit has been informed, SurveyAuto has completed projects in Malawi, Senegal, Afghanistan, India, Nairobi, Liberia, Rwanda, Jordan, Saudi Arabia and Nepal. And that is just the beginning, or at least that is what CEO Dr Saif claims. The company has thus far survived on founder’s capital and donor funding, some of which was provided by the Bill and Melinda Gates Foundation.

As for the revenue model, since it is a SaaS model, companies are charged a license fee and a subsequent cost per type of survey. Each survey takes different time so each survey has different costs. But Dr Saif candidly admits that the company is not making much at the moment. What does look hopeful, however, is that the unit economics is highly favourable, further claiming that the cost structure of SurveyAuto is nearly one-third of what other companies charge, and that would still turn profits. And things are about to change after their entry into the FMCG segment. 

Now the million dollar question: how can a nascent company, that is not even making much, take down companies, some of whom have existed for close to a century now and have multi-billion dollar revenues? 

To keep it classic, all good things come to an end. To keep it real, technologies businesses have gained momentum whereas traditional companies have been slow, have been working in a set way for decades, have millions of people employed and trained to do certain things only. It is a risk for them to disband the set system. All in all, these companies are captives and slaves of their own legacy.

At the same time, SurveyAuto reaching a unicorn status also seems like a far cry and highly speculative for now. There is no parallel in the world of such a startup that has achieved a unicorn status. The data industry has truly remained untouched by technology until now. So the final and obvious question is: Will SurveyAuto be able to disrupt, or would traditional data companies be able to survive this disruption? Whatever happens, this is another case of the new threatening the old, and the new usually wins.