META AI – MULTI-TASK LEARNING ALGORITHM
The world of Artificial Intelligence has gone through many AI winters and inflection points in its journey of over seven decades. The present AI inflection points seems to most likely to stay, given the humongous amount of data that is being generated, increase in storage capability and the increased power of processing, courtesy Moore’s law. With the variety of Machine Learning and Artificial Intelligence tools in vogue and increasing by the day, the journey towards general artificial intelligence might be closer than what we had imagined even in the best-case scenario. The latest news is very encouraging. Meta’s new learning algorithm can teach AI to multi-task. The silo-based AI learning might just be broken.
The good news which has set the AI world on fire is a single technique for teaching neural networks multiple skills. This advancement is a leap towards general-purpose AI. The present status is if the neural network can recognize the dog in the photo, it would not be able to recognize it by its description. Deep neural networks have become proficient in identifying objects in photos and also conversing in natural language, but the catch is, they cannot do both at the same time. There are any number of AI models which can excel in each of these but not both. This has been the problem statement for quite some time. The catch is that these different models learn different skills using different techniques.
The dire need was having machines which could multi-task and adapt. The lack of it also meant that the advances in deep learning in one skill could not be transferred to others. Meta AI wants to change all that. Researchers there have done the magic by developing a single “algorithm that can be used to train a neural network to recognize images, text and speech.” This can certainly be termed as the most important AI breakthrough in recent years. It will change the very manner researchers view and conduct AI research. ”The algorithm, called Data2vec, not only unifies the learning process but performs at least as well as existing techniques in all the three skills.” Besides the multi-tasking, it shortens the learning curve considerably, as multitasking is starting at a level from where AI maturity has reached in the three areas.
The approach taken is known as self-supervised learning. This is a new approach to the way neural networks are made to learn. ”The neural networks learn to spot patterns in data sets by themselves, without being guided by labelled examples.” This is also the model followed by GPT3 large language models literally scraping unlabelled data through the internet. Data2vec uses two neural networks, a teacher and a student. Teacher having picked up the skill, the student is then ”trained to predict the internal representations of the teacher.” This wards off the need for the algorithm to be tailored to a particular type of input. The big trend in AI of which Data2vc is a part is towards models that learn and understand the world in more than one way. Mark Zuckerberg, already on the Metaverse mode, is dreaming of its potential metaverse applications. We are slowly moving into a totally different and fruitful AI trajectory.
MUTI-TASKING IN AI SKILL LEARNING IS A GIANT LEAP FORWARD.