The fourth generation of machine intelligence, adaptive learning, creates the first truly integrated human and machine learning environment. For text analytics, this has given us the most accurate analytics to date, allowing us to get actionable information in many areas for the first time. In the examples we will share here, we show that adaptive learning is 95% accurate in predicting people’s intention to purchase a car. Adaptive learning correlates with actual sales, unlike any previous approach to Machine Intelligence.

Adaptive learning combines the previous generations of rule-based, simple machine learning, and deep learning approaches to machine intelligence. Human analysts are optimally engaged in making the machine intelligence smarter, faster, and easier to interpret, building on a network of the previous generations of machine intelligence.

robots

Four generations of machine intelligence

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1st generation: Rules

The first generation of machine intelligence meant that people manually created rules. For example, in text analytics someone might create a rule that the word “Ford” followed by “Focus” meant that “Ford” referred to a car, and they would create a separate rule that “Ford” preceded by “Harrison” meant that “Ford” referred to a person.

The rule-based approach is very time consuming and not very accurate. Even after an analyst has exhausted all the words and phrases they can think of, there are always other contexts and new innovations that aren’t captured. For one of our clients, their experts analysts were only able to capture 11% of the documents they wanted to analyze using rules: this clearly is too limited.

2nd generation: Simple machine learning

The dominant form of machine intelligence today is simple machine learning. Simple machine learning uses statistical methods to make decisions about data processing. For example, a sentence might have the word “Ford” labeled as a car, and the machine learning algorithm will learn by itself that the following word “Focus” is evidence that “Ford” is a car in this context.

Simple machine learning can be fast, provided that you already have labeled examples for ‘supervised learning’. It also tends to be more accurate, because statistics are usually better than human intuition in deciding which features (like words and phrases) matter. The major drawback for supervised machine learning is that you need the labeled examples: if you have too few labels or the labels aren’t representative of the entire data set, then the accuracy is low or limited to a specific domain.

3rd generation: Deep learning

There has been a recent rise in the use of machine learning that learns more sophisticated relationships between features, known as deep learning. For example, if you had the sentence “We Will Let Harrison Ford Focus on Star Wars”, there is conflicting evidence between “Harrison” and “Focus” about whether “Ford” is a person or a car.

Deep learning can automatically learn how to use combinations of features when making a decision. For simple machine learning, a human has to tell the algorithm which combination of features to consider. Deep learning often cuts down on the amount of human time needed and typically gets up to 5% more accurate results than simple machine learning for text analytics–although only when applied to data from the same sources as it learned from.

4th generation: Adaptive learning

Adaptive learning brings human analysts into the process at every step. This is in contrast to rule-based, simple machine learning and deep learning approaches, where the humans only create rules and label data at the start of the process. For example, if you had the sentence “We Will Help Tom Ford Escape from New York”, and your system hadn’t seen any examples of “Tom Ford” or “Ford Escape”, you will need human input to build the knowledge.

Adaptive learning systems require the least human effort because they only require human input when it matters most and continually expand their knowledge when new information is encountered. As we show here, they are also the most accurate. They combine the three other types of machine intelligence, adding new types of ‘unsupervised machine learning’ and methods for optimizing the input from multiple, possibly disagreeing, humans.

Example of why it matters

Idibon’s solutions are the fourth generation of machine intelligence. We help organizations extract actionable information from text globally, including the largest manufacturers of consumer electronics, the creator of the world’s most popular games, many financial services companies, and the United Nations. Across all our clients, the information is most useful when it correlates with actual behavior in the world. Our adaptive learning system optimizes many aspects of text analytics with layers of machine learning, optimizing what an analyst sees for review (active learning), how they see it (human-computer interaction) and how that information is applied (data vs. rewriting its own code).

Here, we’ll use an example that everyone can understand: predicting car sales from social media communications. For example, how can we predict sales of Toyota Camrys from people saying things like “Thinking of getting a Camry”? We thank the engineers, data scientists, and statisticians at Edmunds.com, the world’s most visited car information site (and Idibon client), for feedback on an early version of these results.

Here are our results, compared to deep learning and simple machine learning systems from some of the most prominent machine intelligence companies in text analytics:

cars

Accuracy in sentiment analysis about purchasing cars: Idibon and competitor systems

 

 

 

 

 

 

 

 

 

Idibon’s adaptive learning is 95% accurate, compared to 69% from deep learning and 60% from simple machine learning systems. This is a huge difference. To assess this accuracy, we created and manually labeled a few thousand items and then tested them. For Idibon, none of these items were used in the learning phase: our 95% accuracy is over completely novel social media communications. We couldn’t confirm this for our competitors, so it’s possible that some of our competitors had a slight advantage.

What does 95% accuracy get you?

For any analytics to be valuable, the output needs to be actionable. So for 14 car models, we looked at how well we could predict monthly sales from text analytics alone. Predicting sales in advance of the monthly sales figures is much more valuable than simply surfacing a selection of positive and negative tweets. It can help car manufacturers understand which features are resonating with buyers in new models, it can help make logistics decisions about the numbers of cars to ship to dealers each month, and it can help an investor predict how the share price of a car company will move before that company has made their end of month announcement of sales.

Car model Correlation
Beetle 0.69
Cruze 0.55
Elantra 0.50
Passat 0.48
Accord 0.45
Camry 0.44
Focus 0.43
Impala 0.42
Civic 0.39
Jetta 0.36
Malibu 0.25
Altima 0.24
Fusion 0.23
Corolla 0.15

For 10 of the 14 models, Idibon’s 95% accurate analytics for predicting people buying a car correlate with actual sales. These correlations were calculated via Spearman’s rank correlation and confirmed by also analyzing the data using a variety of other statistical tests (e.g., Kendall’s tau).

Conclusion

Adaptive intelligence goes beyond the ~70% accurate sentiment analysis that we’re seeing across text analytics vendors, to a level of accuracy that genuinely predicts human behavior.

Just as importantly for Idibon’s clients, it also minimizes that amount of time that analysts need to spend. Here are the results from our examples about “Ford” above:

Screen Shot 2015-09-28 at 1.20.54 PM

After just tens of minutes of analyst feedback, the system can disambiguate the use of “Ford” in social media more accurately than systems built on hundreds of hours of human intervention.

One downside of adaptive learning is that it’s not easy to package as a application that you can download and run on a single server. To help people who can only run single-server applications and need 70% accuracy, Idibon gives away a 70% accurate system for free. Our Idibon Public product is 69.9% accurate on this same data, equal to the best accuracy of our deep learning competitors who charge for the service.

When you can get to 95% accuracy, you can actually match what’s happening in the real world, and Idibon helps our clients predict changes and events in many market, including the car sales example here. We regularly show how much more accurate a system can be when it takes the humans-in-the-loop seriously.

Humans have always been part of Machine Intelligence systems–they create the unstructured data, they load it into systems, they tune statistical models, they interpret results. Adaptive learning optimizes people’s time and effort–making machines smarter to make people smarter. Really, it’s about making the most of human intelligence.

Rob Munro

Rob drives the company’s vision and strategic direction. He is a leader in applying big data analytics to human communications, having worked in diverse environments, from Sierra Leone, Haiti and the Amazon to London, Sydney and San Francisco. He completed a PhD in Computational Linguistics as a Graduate Fellow at Stanford University. Outside of work, he learned about the world’s diversity by cycling more than 20,000 kilometers across 20 countries.


Next Read
Applied intelligence: AI and the future of advanced data analytics

Last week, I had the pleasure of giving an industry keynote presentation on active learning at the Georgian Partners Annual Conference for Applied Analytics in Toronto. The applied intelligence session of the conference explored developing trends in artificial intelligence and machine learning. Meet Amy, an AI-driven personal assistant Dennis Mortensen, CEO and founder of x.ai,…

Read More
Next Read
Predicting Chevy Impala sales from Twitter

Marketers need to understand people’s awareness and opinions of their brands on social media. Ideally, they will be able to predict sales based on this information and use this to inform marketing campaigns and business decisions. Our solution is to move away from general models, which are often too broad in definition to pick up…

Read More
Next Read
How to sell a car on social media

To write like a car dealer, address your audience directly with “you” and “your”. And use a lot of ampersands.

Read More

2 thoughts on “The fourth generation of machine learning: Adaptive learning

  1. If Adaptive Learning is this much better than Deep Learning, then why are Google and Facebook focused on Deep Learning and not using Adaptive Learning instead?

  2. Rob Munro

    That’s a good question. The short answer is that Google, Facebook, and others, are using something like Adaptive Learning learning already.

    Adaptive Learning means that we are automating more of the machine learning pipeline.

    For every large company using text analytics, the machine learning experts aren’t relying on machine learning that learns only once. Every large company that I’ve seen using text analytics has some form of human-coded rules to improve the machine learning output. Almost every place I’ve seen also obtains additional labeled items for the machine learning training.

    The problem is that it’s typically a manual process to select the additional data to label or the rules to incorporate. There is no integrated feedback to ensure that the extra human input for labeled data optimizes the time and expertise of those people, or that the rules created are optimized to improve the final accuracy.

    With Adaptive Learning, we’re automating more of what was previously a purely manual task for the machine learning experts. The strength of Deep Learning is that it took a partially manual process of selecting the right features to combine and learn from, and automated that process with better overall accuracy as a result. We’re getting better accuracy when automating more of the machine learning pipeline: in the case of Adaptive Learning, it’s the biggest improvement we’ve seen!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>