Artificial intelligence (AI) has quickened its progress in 2021.
A new administration is in place in the US and the talk is about a major push for Green Technology and the need to stimulate next generation infrastructure including AI and 5G to generate economic recovery with David Knight forecasting that 5G has the potential - the potential - to drive GDP growth of 40% or more by 2030. The Biden administration has stated that it will boost spending in emerging technologies that includes AI and 5G to $300Bn over a four year period.
On the other side of the Atlantic Ocean, the EU have announced a Green Deal and also need to consider the European AI policy to develop next generation companies that will drive economic growth and employment. It may well be that the EU and US (alongside Canada and other allies) will seek ways to work together on issues such as 5G policy and infrastructure development. The UK will be hosting COP 26 and has also made noises about AI and 5G development.
The world needs to find a way to successfully end the Covid-19 pandemic and in the post pandemic world move into a phase of economic growth with job creation. An opportunity exists for a new era of highly skilled jobs with sustainable economic development built around next generation technologies.
AI and 5G: GDP and jobs growth potential plus scope to reduce GHG emissions (source for numbers PWC / Microsoft, Accenture)
The image above sets out the scope for large reductions in emissions of GHGs whilst allowing for economic growth.
GDP and jobs growth will be very high on the post pandemic agendas of governments around the world. At the same time those economies that truly proposer and grow rapidly in this decade will be those who adopt Industry 4.0 technology and in turn will lead to a shift away from the era of heavy fossil fuel consumption towards a digital world that may be powered by renewable energy and with transportation that is either heavily electric or over time, hydrogen based.
2021 will mark the continued acceleration of Digital Transformation across the economy.
Source for image above Lean BI
The examples of how Machine to Machine Communication at the Edge enabled by AI could work maybe demonstrated by the following image as an example:
In the image above the Machine to Machine communication allows for broadcast across the network that a person has been detected stepping onto the road so that even the car that does not have line of sight of the person is aware of their presence
It is important to note that AI alongside 5G networks will be at the heart of this transition to the world of Industry 4.0.
5G will play an important role as 5G networks are not only substantially faster than 4G networks, but they also enable significant reductions in latency in turn allowing for near real-time analytics and responses, and also enable far greater capacity for connection thereby facilitating massive machine to machine communication for IoT devices on the Edge of the network (closer to where the data is created on the device).
The image below sets out the speed advantage of 5G networks relative to 4G.
Source for image above Thales Group
However, as noted above 5G has many more advantages over 4G than speed alone as shown in the image below:
Source for image above Thales Group
The growth in Edge Computing will reduce the amount of data being sent backwards and forwards between a remote cloud server and thereby make the system more efficient.
Source for image above Thales Group
The economic benefits of 5G are set out below:
• $13.2 Trillion dollars of global economic output
• 22.3 Million new jobs created
• $2.1 Trillion dollars in GDP growth
Towards AI at the Edge (AIIoT)
To date AI has been most pervasive and effective in the areas of Social Media and Ecommerce giants whose large digital data sets give them an advantage and where edge cases don't matter so much in terms of their consequences. No fatalities, injuries or material damages arise from an incorrect recommendation for a video, a post, or an item of clothing, other than a bad user experience.
However, when we seek to scale AI into the real world, edge cases and interpretability matter. Issues such as causality and explainability become key in areas such as autonomous vehicles and robots and also in healthcare.
Equally data privacy and security also really matter. On the one hand as noted above, data is the fuel for Machine Learning models. However, on the other hand in areas such as healthcare much of that data is often siloed and decentralised plus also protected by strict privacy rules in the likes of the US (HIPAA) and Europe (GDPR). It is also an issue in areas such as Finance and Insurance where data privacy and regulation are of significant importance to the operations of financial services firms.
This is an area where Federated Learning with Differential Privacy could play a big role in scaling Machine Learning across areas such as healthcare and financial services.
Source for image above NVIDIA What is Federated Learning?
It is also an area where the US and Europe could work together to enable collaborative learning and help scale Machine Learning that also provides for Data Security and Privacy for end users (patients). The Healthcare sector around the world is at breakpoint due to the strains of the Covid-19 pandemic and augmenting our healthcare workers with AI to reduce the strain upon them whilst ensuring that patient data security is maintained will be key to transforming our Healthcare systems to reduce the strain on them and deliver better outcomes for the patient.
Source for Image above TensorFlow Federated
For more on Federated Learning see: Federated Learning an Introduction.
In relation to AI, we will need to move away from the giant models and techniques that were predominant in the last decade towards neural compression (pruning) that in turn will enable models to operate more efficiently on the Edge and help preserve battery life of devices and also reduce carbon footprint with reduced energy consumption.
Furthermore, we won't only require Deep Learning models that may inference on the Edge, but also models that may continue to learn on the Edge, on the fly, from smaller data sets and respond dynamically to their environments. This will be key to enabling effective autonomous systems such as autonomous vehicles (cars, drones) and also robots.
Solving for these challenges will be key to enabling AI to scale beyond Social Media and Ecommerce across the sectors of the economy.
It is no surprise that the most powerful AI companies today and last few years tend to be from the Ecommerce and social media sector.
- Alphabet / Google with CBINSIGHTS observing Alphabet In AI: How Google Went From A Search Engine To An $800B Global AI Powerhouse
- Facebook have had a strategy for powerful AI capability dating back for many years;
- Wired noted how Deep Learning came to power Alexa, Amazon Web Services, and nearly every other division of the company;
- Ali Baba was reported as using AI to power record sales for Singles Day;
Furthermore, the images below from Valuewalk show how ByteDance (owner of TikTok) is the world's most valuable Unicorn and an AI company.
Source for image above Valuewalk, Tipalti, The Most Valuable Unicorn in the World 2020
Venture Capitalist and Angel Investors should also try to understand that in order to scale AI startup ventures access to usable data and meeting the requirements of their customer in terms of usability (which may include some or all of transparency, causality, explainability, model size, ethics) are key for many sectors.
The number of connected devices and volume of data is forecast to grow dramatically as Digital Technology continues to expand its reach for example the image below shows a forecast from Statista for 75 Billion internet connected devices by 2025, an average of over 9 per person on the planet!
In fact IDC forecast that " The global datasphere will grow from 45 zettabytes in 2019 to 175 by 2025. Nearly 30% of the world's data will need real-time processing. ... Many of these interactions are because of the billions of IoT devices connected across the globe, which are expected to create over 90 ZB of data in 2025."
Illustration of the AI IoT across the Edge
Source for infographic images below: Iman Ghosh VisualCapitalist.com
In the past decade key Machine Learning tools such as XG Boost, Light Gradient Boosting Machines and Cat Boost emerged (approximately 2015 to 2017) and these tools will continue to be popular with Data Scientists for powerful insights with structured data using supervised learning. No doubt we will see continued enhancements in Machine Learning tools over the next few years.
In relation to areas such as Natural Language Processing (NLP), Computer Vision and Drug Discovery efforts Deep Learning will continue to be the effective tool. However, it is submitted that increasingly the techniques will move towards the following:
Transformers (including in Computer Vision);
Neuro Symbolic AI (hybrid AI that combines Deep Learning with symbolic Logic);
Neuroevoutionary (hybrid approaches that combined Deep Learning with Evolutionary algorithm approaches);
Some or all of the above combined with Deep Reinforcement Learning.
This will lead to an era of Broad AI as AI starts to move beyond narrow AI (performing just one task) and starts working with multitasking but not at the level where AI can match the human brain (AGI).
My own work is focused on the above hybrid approaches for Broad AI as we seek to find ways to scale AI across the economy beyond Social Media and Ecommerce the above will be key to enabling true Digital Transformation with AI across traditional sectors of the economy and enabling our moving into the era of Industry 4.0.
Source for Image above David Cox, IBM Watson
MIT IBM Watson Lab define Broad AI and the types of AI as follows:
"Narrow AI is the ability to perform specific tasks at a super-human rate within various categories, from chess, Jeopardy!, and Go, to voice assistance, debate, language translation, and image classification. "
"Broad AI is next. We’re just entering this frontier, but when it’s fully realized, it will feature AI systems that use and integrate multimodal data streams, learn more efficiently and flexibly, and traverse multiple tasks and domains. Broad AI will have powerful implications for business and society."
"Finally, General AI is essentially what science fiction has long imagined: AI systems capable of complex reasoning and full autonomy. Some scientists estimate that General AI could be possible sometime around 2050 – which is really little more than guesswork. Others say it will never be possible. For now, we’re focused on leading the next generation of Broad AI technologies for the betterment of business and society."
I would add Artificial Super Intelligence (or Super AI) to the list above as this is a type of AI that often gains much attention in Hollywood movies and television series.
In Summary
Whether one views 2021 as the first year of a decade or not, 2021 will mark a year for reset across the economy and hopefully one whereby we start to move beyond the Covid pandemic to a post pandemic world.
California will remain as a leading area for AI development with the presence of Stanford, UC Berkley, Caltech, UCLA, and University of San Diego. However, other centres for AI will continue to grow around the US and the world, for example Boston, Austin, Toronto, London, Edinburgh, Oxford, Cambridge, Tel Aviv, Dubai, Abu Dhabi, Singapore, Berlin, Paris, Barcelona, Madrid, Lisbon, Sao Paulo, Tallinn, Bucharest, Kyiv / Kharkiv, Moscow and of course across China (many other examples of cities could be cited too). AI will become a pervasive technology that is increasingly in the devices (including within our mobile phones) that we interact with everyday and not just when we enter our social media accounts or go online to shop.
It will also mark a reset for AI to be increasingly on the Edge and across the "real-world" sectors of the economy with the emergence of Broad AI to take over from Narrow AI as we move across the decade.
Smaller models will be more desirable / more useful
GPT-3 is an exciting development in AI and shows the potential for Transformer models, however, in the future small will be beautiful and crucial. The human brain does not require the amount of server capacity of GPT-3 and uses far less energy. For AI to scale across the edge we'll need powerful models that are energy efficient and optimised to work on small devices. For example Mao et al. set out LadaBERT: lightweight adaptation of the BERT ( a large Transformer language model) through hybrid model compression.
The authors note "...a major blocking issue of applying BERT to online services is that it is memory-intensive and leads to unsatisfactory latency of user requests, raising the necessity of model compression. Existing solutions leverage the knowledge distillation framework to learn a smaller model that imitates the behaviours of BERT."
"However, the training procedure of knowledge distillation is expensive itself as it requires sufficient training data to imitate the teacher model."
"In this paper, we address this issue by proposing a hybrid solution named LadaBERT (Lightweight adaptation of BERT through hybrid model compression), which combines the advantages of different model compression methods, including weight pruning, matrix factorization and knowledge distillation. LadaBERT achieves state-of-the-art accuracy on various public datasets while the training overheads can be reduced by an order of magnitude."
Reducing training overheads and avoiding unsatisfactory latency of user requests will also be a key objective of Deep Learning development and evolution over the course of 2021 and beyond.
My Vision of Connectionism: Connecting one human to another (we're all human beings), connecting AI with AI, and AI with humans all at the level of the mind.
When I adopted the @DeepLearn007 handle on Twitter many years ago, I was inspired by the notion of connectionism, and the image that I selected for the account illustrates how 2 human beings could connect at the level of the brain and how the exchange of information, in effect ideas, drives innovation and the development of humanity. In the virtual world much of that occurs at the level of data and the analytical insights that we gain from that data through application of AI (Machine Learning and Deep Learning) to generate responses.
I remain a connectionist, albeit an open minded one. I believe that Deep Neural Networks will remain very important and the cornerstone of AI development but just like Deep Reinforcement Learning combined Reinforcement Learning with Deep Learning to very powerful effect with the likes of AlphaGo, AlphaZero, and MuZero resulting, so too developing hybrid AI that combines Deep Learning with Symbolic and Evolutionary approaches will lead to exciting new product developments and enable Deep Learning to scale beyond Social Media and Ecommerce sectors where the likes of medics and financial services staff want causal inference and explainability for trust in the AI decision making. For example, Microsoft Research state that "understanding causality is widely seen as a key deficiency of current AI methods, and a necessary precursor for building more human-like machine intelligence."
Furthermore, in order for autonomous vehicles to truly take off we'll need the model explainability for situations where things have gone wrong in order to understand what happened and how we may reduce the probability of the same outcome in the future.
The next generation of AI will be in the direction towards the era Broad AI and the adventure will be here in 2021 as we move towards the Edge, towards a better world as we move beyond the scars and challenges of 2020. The journey may require scaled up 5G networks around the world to really transform the broader economy and that may only really start to happen at the end of the year and beyond but the direction of the pathway is clear.
The exciting potential for healthcare, smart industry, smart cities, smart living, education, and every other sector of the economy will mean that a new of businesses will emerge that we cannot even imagine today.
Perhaps a good point to conclude is with the forecast from Ovum and Intel for the impact of 5G for the media sector (of course AI will play a big role alongside 5G in developing new hyper personalised services and products and have a symbiotic relationship together).
