Machine Learning To Enrich Marketing Social Events

Machine Learning to Enrich Marketing Social Events

Prabhash Thakur

Director, Data Science

There are natural events, and there are social events. We have satellites and sensors to detect natural events such as floods and earthquakes. But is there a way to get an early sign of a social event such as new fashion, a movement like me-too or a political event like the Arab Spring?

If you are a marketer, then getting an early idea of a social trend or a celebrity can go a long way in strengthening your brand presence.

Where to look for the data?

If you are hunting for the social event, then you need to analyse the data that appears in social media sites. The usual suspects such as Twitter, Instagram and Facebook are the places to look for the data.

Data from these sites have some peculiarities and getting a good sense of these is crucial in avoiding any false detection. Some of these peculiarities are listed below

Social media data is massive in scale. Twitter gets over 500 million tweets per day!
Not all spike in data is useful. For example, at the beginning of the new year, you would expect a lot of new year greetings, but we do not need AI to learn that
The text in social media is informal to the extent that an NLP program without suitable adaptation would fail

The above means that raw computation power is not sufficient for us to get real-time insights about social events. We must be smart about the algorithms.

Can a supervised machine learning classifier solve this problem?

If you are tempted to throw a deep neural network at this problem, pause for a moment and look at the image below.

A supervised model will work only if the outcome categories are fixed upfront. But the whole idea of social event detection is to find out novel social ideas that impact a large population. So, we will rule out supervised learning.

If it is not supervised learning, then is it unsupervised learning?

Yes, in theory, but not in practice. There are two problems here. First, we need to store data for a period to create clusters and given the scale of social media data that is not possible. Secondly, if something is far from a cluster centre, is it an outlier or an emerging trend?

Let us try a simpler method

This is a classic case where a complex problem deserves a simple solution. We are trying to capture spikes in discussions without storing all the data. The diagram below is a representation of such a scenario.

There are quite a few methods which can be applied to detect social events. At Centelon, we have found n-gram hashing to be the most effective one. Here is a brief approach to solving this problem

Create n-grams for a sample of text
Apply a hash map on them, which means each word or n-gram has a hash bucket assigned to it
Store exponential moving average and exponential moving variance of the n-gram counts in the hash bucket
If their ratio crosses a set threshold, then the words or n-grams represent a social event
To avoid hash collisions, use multiple hash maps

Reach out to us if you would like to know more about social event detection.

Prabhash Thakur

Director, Data Science

November 10, 2020

Written By Prabhash Thakur

Products

Let’s Create Big Stories Together

Expertise Deployments in Salesforce, ERP, CRM, Web & Mobile Developments, Artificial Intelligence, Data Management & Resource Augmentation.

Thanks for showing an interest in our products.

Our team will get back to you at the earliest to book a requested demo call at your preferable time.

Back to Website

Machine Learning to Enrich Marketing Social Events

Prabhash Thakur

Where to look for the data?

Can a supervised machine learning classifier solve this problem?

If it is not supervised learning, then is it unsupervised learning?

Let us try a simpler method

Prabhash Thakur

Written By Prabhash Thakur

Products

Categories

Let’s Create Big Stories Together

Contact Us

Australia

Level 13, 200 Queen Street Melbourne VIC 3000
Australia

India

B Wing, Level 2, Ghule Square DSK Ranwara Road, Bavdhan,
Pune 411-021

Singapore

2 Shenton Way #15-04 SGX Centre 1, Singapore 068804

USA

196 N 3rd Street, Suite 319, San Jose,
CA 95112

Prabhash Thakur

Where to look for the data?

Can a supervised machine learning classifier solve this problem?

If it is not supervised learning, then is it unsupervised learning?

Let us try a simpler method

Prabhash Thakur

Written By Prabhash Thakur

Products

Categories

Tags

Australia

Level 13, 200 Queen Street Melbourne VIC 3000 Australia

India

B Wing, Level 2, Ghule Square DSK Ranwara Road, Bavdhan, Pune 411-021

Singapore

2 Shenton Way #15-04 SGX Centre 1, Singapore 068804

USA

196 N 3rd Street, Suite 319, San Jose, CA 95112

Level 13, 200 Queen Street Melbourne VIC 3000
Australia

B Wing, Level 2, Ghule Square DSK Ranwara Road, Bavdhan,
Pune 411-021

196 N 3rd Street, Suite 319, San Jose,
CA 95112