This week, we dive into the twists and turns of artificial intelligence with Olivier Ezratty, expert in AI and quantum technologies, on the occasion of the publication of his e-book on the uses of AI. We summarize 742 pages in a few lines to find out everything about the latest developments in technology or use of AI. Don’t thank us!
Here is a challenge: could you summarize your ebook in a few sentences?
It’s a big challenge! My process is often to update my books. This year, I added about a hundred pages compared to the previous version, which takes stock of technological developments and uses around AI.
To really summarize, recent developments have mainly concerned technologies and processors, but we can also see notable progress in project management, especially in the preparation of training data. They help speed up machine learning projects.
What changes in AI have you seen since your last publication in 2019?
I have observed several types of developments :
We can also mention some interesting new additions to the ebook :
You have identified two major limitations to AI, what are they? How to resolve these biases?
I noticed that an amalgam was made in terms of AI biases: many people think that artificial intelligence biases come from algorithms when in reality, these biases have their main roots in the training data themselves, which quite simply reflect the biases of our societies. The rules of the algorithms are all quite neutral, as a rule. In contrast, algorithms can be used to debit training databases by increasing the proportion of under-represented groups. For example, if we have a database that reproduces the biases of society (not enough women, not enough minorities, various socio-demographic biases, etc.), the algorithm can increase the proportion of categories deemed too weak by concern for fairness.
The challenge for companies that use classification and prediction tools is to detect company biases as early as possible to resolve data biases using rebalancing algorithms, or by changing the sample. of the training base. It’s a question of social science.
This is also why it is essential to focus this type of thinking on generative AI. The latter are found in visual or textual content, and are capable of generating content themselves (scripts, films, poems, texts, etc.). They don’t really invent anything since they always create information which is a reconstruction made from existing data, mainly of human origin. They are a mixture of existing data, no more, no less.
And how is it a problem?
This is a problem insofar as AI will automatically reproduce societal biases, which will be perpetually reproduced. Every day, AI systems are based on partial data or extrapolations, and therefore perform poorly. This is the stupidity of AI and it can have disastrous consequences on people’s lives.
If, for example, we consider a system trained to offer higher education courses depending on the existing one. A pupil X living in Neuilly would be more likely to succeed than a pupil Y living in the 93 department, regardless of school results, only because the AI considers that the place of residence is decisive in the school success. This is the famous confusion between correlation and causation.
A more concrete example that took place at the end of 2018 at Amazon concerns gender bias. Amazon had implemented a Deep Learning system to analyze candidates’ CVs, which selected profiles with as many words as possible corresponding to the job description while taking into account socio-demographic characteristics. This system had been put in place to recruit tech profiles, and women made up only 5% of the training database, which was automatically reflected in reality. Indeed, the algorithm tended to disadvantage women who applied for these types of positions despite having the same skills as men. There was therefore a bias resulting from the training data. This can be integrated and corrected by Data Scientists taking into account the probabilistic mechanisms put in place by Machine Learning systems.
The solution for this kind of model is to overweight the under-represented category, or increase the percentage share of that category in the training base. It would be about doing positive discrimination.
Of all the forms of AI you mentioned (evolutionary algorithms, knowledge representation, affective AI, transfer learning), which is the most complex form, and therefore the most recent? How does it work in practice?
I’m still struggling to fully understand how the newer language processing algorithms, transformers, work. These are deep learning neural network systems capable of implementing what is called compositionality, that is, of processing text with a very good understanding of its context. It is a revolutionary solution for translating, classifying or generating text from an existing one. Even if these systems remain in a probabilistic approach to the text, the progress is significant. However, these applications are still artisanal in the sense that they assemble several interconnected bricks. It is still difficult to explain the results. So it’s quite difficult to grasp especially as it evolves very quickly. Indeed, each year, there are more than fifty new neural networks – of the Deep Learning type – which appear.
However, these systems still have serious limitations. For example, the German-born online deep learning translation service produces beautiful, grammatically correct translations from French to English. But the resulting English is not idiomatic. You can see from afar that it is French translated into English and not an English-native phrasing.
Basic Machine Learning working on structured data does not have this component of mystery: it is more coded, more segmented, it is based on better known and older mathematical models, which makes it easier to understand and explain the results.
How will AI evolve over the next few years?
In my opinion, two waves of AI are brewing: one aims to correct the flaws of current AIs, the other will be based on functionally improved AIs.
This first wave of corrective AI is already underway. We will find there the notion of frugal AI, because we know that AI consumes a lot of energy, especially in objects. Much research is underway to reduce the size of training games and ensure that less data is used for the same result. Other work relies on the quantization mechanism to reduce computational time when we use trained AI. This mechanism is based on the manipulation of whole numbers rather than floating point numbers. Finally, improved processors will also optimize power consumption.
The second element of the corrected AI relates to data biases that remain to be resolved. The third element concerns the protection of privacy. A technique promoted by Google in 2017 has spread widely in 2019 and 2020, called Federated Learning, and involves having an AI react to situations related to personal data to train a central AI, but without that personal data is sent to Google. A very widespread concrete example is provided by Google with its function of pre-filling emails for users.
The second wave will be a wave of improved AI, as opposed to corrective AI. In this context, many startups and developers are working on the mythical AI known as AGI (artificial general intelligence). It is an artificial intelligence which would reason generally like humans, and would therefore be able to solve any problem in the same way, if not better. This is a myth because I’m not sure you can reproduce this technically. Moreover, the experts working on the subject do not really define the uses that could be made of it.
As I became interested in the history of computing, I was fascinated by how researchers worked on the creation of the first graphics computers and the mouse we use today. These researchers conceptualized things in the sixties, and it wasn’t until twenty years later that the material saw the light of day. It took eleven years of work between the first graphics computer created by Xerox in 1973 (the Alto) and the first Apple Macintoshes. Creative cycles are very long, but researchers and innovators have shown phenomenal creativity. This is why I think the same is true for artificial intelligence: innovators will find clever solutions that will revolutionize the uses of this technology.
What will the AI of 2030 look like? Should we consider new legislation? New forms of AI?
Yes, there will be legislation. Otherwise, we don’t know! How can we predict what people will be doing in 10 years?
One thing is certain, there will always be more data. But it is very difficult to predict how the algorithms will evolve. Elon Musk said “We were expecting flying cars, and we got Twitter”. There is always a gap between expectations and reality. It is very likely that many innovations will be in this scenario and rather unexpected and not necessarily that sophisticated from a technological point of view.
I would like to cite the example of GPT3, which is a language processing deep learning engine created by the American company OpenAI, initially launched by Elon Musk and largely funded by Microsoft more recently. This GPT3 is mainly driven by human intelligence since it contains all the data from Wikipedia, the Library of Congress, and a lot of data from websites. There are approximately 175 billion parameters in the neural network. GPT3 allows you to generate text and code. It’s truly amazing technology but it’s not yet embodied in the form of practical innovation!
At the same time, we have some totally dumb video conferencing software that doesn’t even detect open mics or active cameras when connecting to an online meeting. It is above all a question of the people behind the technologies. Some are creative while some go for the easy way. The various concerns that we may encounter are quite simply due to a lack of anticipation on the part of some developers.
The paradox is that AI is evolving in very sophisticated ways to answer ever more complex problems, but some basic problems still remain unresolved. This shows the gulf that there is between science and everyday use. I would like the two to come closer.
I also hope that the world of tomorrow will be a world where we can see people! Technology obviously allows us to stay in touch despite the context, but sometimes too much digital kills digital.
Olivier Ezratty advises companies on the adoption of deep techs in their innovation strategies, particularly around artificial intelligence and quantum computing. Author, speaker and trainer, he is notably the author of the ebook “The uses of artificial intelligence” (editions in 2016, 2017, 2018, 2019 and 2021) as well as “Understanding quantum computing” (editions in 2018, 2019 and 2020).
He is a trainer on artificial intelligence and quantum computing at Cap Gemini Institut, member of the Scientific Council of ARCEP as well as speaker and referent expert at IHEDN (class of 2019/2020).
Olivier Ezratty started in 1985 as a software engineer and R&D manager in editorial IT at Sogitec, a subsidiary of the Dassault group, then cut his teeth in marketing at Microsoft France to become Marketing and Communication Director and Relations Director. Developers (1990-2005). He is a Centrale Paris engineer (CentraleSupelec since 2015)