Technology Megatrends in the Age of Hyperscale Innovation

Science and Technology Breakthroughs in 2022

Rafic Makki, Head Technologist and Fellow

We are living in the age of hyperscale innovation! From generative AI, intelligent edge sensors and the next frontier in computing to the metaverse and the personalization of everything, technology megatrends are greatly influencing how we live, work, socialize and communicate. In this paper, we will focus on two global megatrends addressing some of humanity’s most significant challenges: Electrification and Digital Biology. For each megatrend we highlight the top innovations of the year and provide some visibility on possible achievements over the next few years. Additionally, we will look at some of the key innovations and notable achievements of 2022.

Let’s start with the innovation of the year.

Innovation of the Year: A New Era for Generative AI

With the release of DALL-E and ChatGPT in 2022, the field of generative AI entered a new era. Generative AI has the potential to be the biggest enabler and accelerator of creativity and productivity since the Internet. It can now write essays, code, generate music and draw with credible human-like (or better) results. It is being used in various fields, including media, medicine and entertainment. Its value in terms of improving productivity and growing the global economy over the next decade is immense. Its value as a platform for innovative killer apps is explosive.

How did we get here? Generative AI has been around for decades. For example, in the 1970s David Cope developed a program to generate original music. In 2014, a new deep learning architecture called generative adversarial networks (GANs) enabled the creation of realistic images such as faces. In 2017, a landmark paper was published in the field of natural language processing [1]. It introduced a new neural network model called transformer, which is very effective at language understanding. Transformers use “attention” to focus on specific parts of the input when processing it, rather than considering the entire input at once. This allows the model to capture dependencies and relationships more easily in the input data. Another key feature of transformers is their ability to parallelize, making them much faster and more efficient.

Because of their efficiency and ability to process input sequences of any length, transformers have enabled the use of large language models. The size of a language model refers to the number of parameters it has, which is a measure of the amount of information the model has stored. A large language model has a higher capacity and is able to store more information than a smaller model. Consequently, it can perform better on a wide range of tasks.

Transformer models have contributed immensely to the field of generative AI. Many generative AI models, such as the Generative Pre-training Transformer (GPT), are based on transformer architectures and use attention mechanisms to generate high-quality outputs. This Includes both DALL-E and ChatGPT. DALL-E is capable of generating images from text descriptions and ChatGPT is capable of generating text in a coherent and fluent conversational style text. Images appearing in this article were generated by DALL-E.

Challenges: There are a variety of challenges to overcome including the possibility of biased outputs, explainability (how AI decisions are made), legal and ethical issues, and the business model for monetizing ChatGPT. Large language models require enormous compute resources for learning. For example, GPT-3 is reported to have 175 billion parameters and GPT-4 which is expected to be released late 2023 may have as many as one trillion parameters! ChatGPT models were trained on Microsoft’s supercomputer AI which has 285,000 CPU cores, 10,000 GPUs, and provides a connectivity of 400 gigabits per second per GPU server. With the release of ChatGPT Professional, it will be interesting to see how the business model will evolve.

Opportunities: APIs that provide access to large pretrained models and/or build custom models for specific cases across a multitude of application areas are among the biggest opportunities created by generative AI. Think of Google maps and all the APIs that plug into it, but then multiply it by several orders of magnitude! There is already an explosion of startups innovating on that front. Welcome to the age of hyperscale innovation!

Generative AI will be a key enabler of the world’s technology megatrends including Electrification and Digital Biology. We look at these two critically important megatrends next.

Megatrend I. Electrification

The next 10 years may very well be remembered as the decade of transition to electric vehicles (EVs) at scale and with it, the dawn of the electrification economy. Here’s why: Climate change data published in 2022 shows the planet warming at a rate not experienced in over 10,000 years. Greenland is losing nearly 250 billion metric tons of ice each year. To combat climate change, more than 20 countries have enacted electrification mandates with varying target dates for banning the sale of gasoline cars and more than 120 have made net-zero emission pledges.

EVs are expected to be the biggest component of the electrification boom. Both the EU and the state of California announced a ban on new internal combustion engine (ICE) car sales by 2035. Vehicle manufacturers such as GM will end the production of ICE cars and focus on EVs by 2035. This provides a window of 12 years to prepare the needed infrastructure to achieve such a transformation. Realizing such a transformation will require a massive effort across a multitude of technology areas, including:

Better, cheaper and safer battery technology, higher yield exploration and environmentally safe mining for battery metals, battery metal recycling; massive charging infrastructure; much more efficient hybrid and distributed power grid systems; and higher efficiency power conversion electronics.

According to a recent electrification study by the National Renewable Energy Laboratory (NREL), a transformational change for the US (a possible scenario involving a major transition to EVs and household heat pumps), would increase electric consumption by 67% through 2050 with a CAGR of 1.6% [2]. Additional demand implies additional energy supplies will be needed. The total carbon footprint of the EV sector will decrease only if the additional demand is supplied through renewables. It is worth noting that EVs are more energy efficient than ICE vehicles, meaning they can travel further on the same amount of energy. EV efficiency notwithstanding, meeting the electrification targets will be a massive undertaking. Let’s look at the progress made in 2022 along some of the more critical dimensions of electrification.

The hunt for new deposits: Today there are roughly 10 million EVs globally. Projections show as many as 400 million EVs by 2035 and 780 million by 2040 [3]. Billions of batteries will have to be manufactured, but where will the raw materials come from? According to Benchmark Minerals, we will need to find 384 new mines over the next 10 years. Even with major recycling operations in place, we would still need 336 new mines. Unfortunately, the cost of finding new deposits is skyrocketing and it is becoming more and more difficult to find them (99% of explorations fail to become mines).

Kobold Metals, a US startup employing ML to significantly improve the exploration process, announced last year that it had begun processing drilling data and optimizing exploration for copper and cobalt in Mingomba, Zambia. According to Kobold, Mingomba’s ore has an average grade of 3.64% copper which is 6 times better than Chile’s, the global leading producer of copper. Kobold uses machine learning (ML) to sift through a mammoth database representing over 100 years of information gathering about the Earth’s crust [4]. The equivalent of over 30 million pages of data, including hand-written field reports from as far back as 100 years ago, together with newer geologic reports, imagery and soil samples cleaned and aggregated into a single ML-ready data base. Kobold’s advanced ML algorithms learn from this data to provide decision-making intelligence that guides the exploration process, making it more efficient and less costly to find deposits deep underground.

The race for the ultimate battery: Li-ion batteries are expected to continue to dominate the EV market throughout this decade, and for good reason. The combination of light weight, charge density, cycling life (charging/discharging), discharge rate, power efficiency, reliability and cost is simply unmatched, so far. Although some newer technologies may excel in certain areas compared to Li-ion batteries, none has shown superiority in overall performance. It is important to note that the safety of Li-ion batteries is still a significant concern due to the risk of thermal runaway and fires.

Though the road ahead remains uncertain, various potential candidates for replacing Li-ion passed important milestones. One such candidate is in the area of solid-state batteries (SSB). SSBs address the safety issue by replacing the liquid electrolyte portion of the battery with a solid electrolyte. In October, ProLogium of Taiwan, announced the premiere of its fast charging 100% silicon oxide anode SSB having energy densities in the range of 295–330 W-h/kg. For comparison, the Tesla S battery is 272–296 W-h/kg. If the cycle life holds up and the cost becomes competitive, the Prologium SSB could be a major step forward in next generation battery technology.

Rechargeable batteries are also needed for grid storage. They provide significant benefit for both on-grid and off-grid applications. The benefits include grid stabilization and peak load management, load shifting, tiered energy costs and integration into the next evolution of the grid, distributed grid and microgrids.

In October, researchers unveiled a new battery for grid storage with energy density of 1010–1111 W-h/kg, amounting to three to four times the energy capacity of Li-ion. The room temperature Na/S battery [5] uses a type of molten salt that can be cheaply obtained from salt water. To attain the higher energy densities, the researchers needed to improve the reactivity of the salt. They were able to achieve much higher reactivity by employing carbon electrodes and a simple pyrolysis process. By operating at room temperature, the battery avoids the key safety problem of Li-ion batteries (thermal runaway) that can produce fires. It remains to be seen if this new technology can be commercialized at scale.

The car that harvests energy: Solar energy is used effectively to provide energy for our homes, so why not for our cars? Solar energy needs space, enough space to install lots of photovoltaic cells to provide the needed energy. We have that space on top of our homes, but not much space on a car? Well enough space as it turns out, provided you have high efficiency photovoltaics, combined with a lightweight aerodynamic EV. You can then use the sun to charge the car battery and extend the car’s range. This is exactly what the startup Lightyear One has done. Solar panels extend across the top of the car, including the top of the trunk and hood. In 2022, Lightyear’s version 2.0 became the most aerodynamic production car in the world with a drag coefficient of 0.175 and demonstrated one of the most efficient powertrains, measured at 97%. The car has a range of 450 miles. An engineering beauty!

The power grid needs to evolve: The aging power grid systems in the US are struggling to cope with demand, let alone events related to extreme weather conditions. Add the additional demand expected from electrification over the next decade, and the urgency becomes clear.

A US DoE study found that power outages have more than doubled between 2015 and 2020. In California, an investigation in 2020 found that aging power equipment sparked a major fire that killed dozens. There is no lack of policies to fix this. In fact, there were over 192 policy and incentive actions on modernizing the grid in the second quarter of 2022 alone [6]. Though the US infrastructure bill earmarked $65 billion to modernize the grid, it will likely cost much more. The aging US grid includes 7,300 power plants with 160,000 miles of high voltage power lines, with portions that date back to the mid-20th century. Fixing it will be a mammoth undertaking, and a great opportunity area for innovation.

Areas of innovation that are needed include:

Platforms for integrating distributed/renewable energy sources, including microgrids; bidirectional flow of electricity (sending electricity back to the grid); efficient ultra-low loss long term energy storage; platforms for intelligent dynamic management of demand, including the use of EV charging stations; second-life battery technology (redeploying used-up EV batteries for grid storage); novel multilayered security to protect the power grid from cyber threats; the Active Efficiency Consortium areas of focus, including performance-based utility models, smart sensors with real-time data analytics at the very edge of the network; and higher efficiency renewable energy capture and conversion.

WeaveGrid, a startup in San Francisco is integrating renewable energy on the grid with the goal of improving the EV driving experience. In August of last year, WeaveGrid launched the largest EV smart charging program in the US. Its platform leverages vehicle telematics to automate charging schedules based on customer preferences, utility rates, and grid needs. As its platform collects data, it continuously learns and improves. On the edge analytics side, Autogrid of Menlo Park announced the deployment of its utility-scale virtual power plant (VPP) in both California and Canada last year across several applications including realtime edge analytics for optimized demand response. Autogrid’s Energy Internet Platform utilizes AI and machine learning to predict and optimize distributed energy resources (DER) — its platform can control thousands of DERs.

The Future of Electrification: The timeline for electrification on a global scale remains uncertain. There has been no shortage of declarations and pledges for the transition to electrification. COP 27’s Acceleration to Zero (A2Z) coalition has a target of 100% of new cars and vans to be electric in leading markets by 2035 and the whole world by 2040. Though innovation on the battery front is beginning to accelerate, the A2Z targets are not likely to be achieved unless we have transformative change in the architecture, efficiency and energy sourcing of the power grid. There will certainly be steady progress on the grid, but it remains to be seen if the necessary investments for a major transformation will be made.

Megatrend II. Digital Biology

Digital biology, a marriage of technology and biology, has been one of the key drivers of innovation across various verticals of the economy. Applications of digital biology are truly massive, including DNA memory, electronic noses, design automation for complex biological systems, bio-inspired computing architectures such as cognitive computing (inspired by the human brain) and cytomorphic computing (inspired by the human cell), machine learning for de-novo drug discovery, wearable health monitors, exoskeletons, new materials, and brain-computer interfaces (BCIs). Digital biology is also helping scientists drug previously undruggable disease targets. In fact, it is a major weapon in the arsenal for curing incurable diseases. Let’s look at some of the notable innovations in digital biology in 2022.

Brain-Computer Interfaces Kill Brain Tumors: BCIs and their future benefits in healthcare were covered extensively in our 2021 edition which focused on the human brain [7]. In 2022, Stanford University scientists developed a wireless implant [8] that enables the continuous and precise targeting of brain tumors without open skull surgery. The wireless device is implanted between the skin and the skull. It provides a significant advantage over the traditional bulky and unwieldy headgear by emitting infrared light that can activate nanoparticles which increase temperature and kill cancer cells without damaging surrounding tissue. The power and wavelength can be tuned to the size of the tumor.

AI-derived Antibiotics: Drug resistant bacteria are a very serious growing global health threat. According to Nature [9], infections caused by bacteria that resist antibiotics are a leading cause of death, with more deaths than HIV/AIDS or Malaria. In 2022, researchers at the Chinese Academy of Sciences used NLP type machine learning algorithms to identify over 2300 potential antimicrobial peptide sequences, over half of which are completely novel with no similarity to known antimicrobials. These novel peptide sequences promise to significantly overcome bacterial resistance and pave the way to new lines of antibiotics.

The First Under-$100 Genome: The cost of sequencing the human genome dropped from $100,000,000 in early 2000s to around $1000 in 2015. That’s nearly an order of magnitude faster rate than Moore’s law. The cost of sequencing is driven by consumables including the sequencing reagents and the flow-cell. Last May Ultima Genomics announced a new technology [10] that reduces the cost to under $100. Ultima uses a dispense system to deliver reagents that are spread quickly and uniformly on a spinning silicon wafer (a technique popular in semiconductor manufacturing). Machine learning is deployed to convert raw signals from the wafer into sequence reads.

Brain on a Dish Plays Pong: In December of 2021, scientists at Cortical Labs published an article in Medium about growing neurons on a petri dish that develop networks, learn, and connect. They didn’t know exactly what they had and even proclaimed “we don’t know what we’re making because nothing like this has ever existed before”. But within a few months they had developed a system dubbed DishBrain, with microelectrodes to stimulate the brain cells in the dish [11]. They then taught the brain on the dish to play a game of Pong! The cells learned in 5 minutes, much faster than it takes an AI engine to learn the game. Perhaps the future of computing will be an integrated chip having a combination of human brain cells and electronic circuits working together in harmony. DishBrain is step 1.

The Future of Digital Biology: Imagine a world in which BCIs and robotic exoskeletons help the disabled walk and communicate. Imagine a world in which human brain cells and electronics are integrated on a single chip that runs generative AI software to invent new technology and solve the world’s toughest challenges. Imagine a world in which the world’s data is stored in synthetic DNA, becoming the digital storage medium of data centers and the cloud. All this is possible and some of it likely over the next decade. That’s the future of digital biology!

Other Notable Achievements in 2022

Amazing Feats in Space: On Sept 26 NASA’s Double Asteroid Redirection Test (DART) spacecraft designed to defend our planet from crashing asteroids, intentionally slammed into the asteroid moonlet Dimorphos to alter its course. DART altered Dimorphs’ course by 33 angular minutes (1 minutes is 1/60 degrees) and displaced over two million pounds of rock into space. In another first, the magnificent $10 bn Webb telescope captured amazing images of the oldest galaxies known, 350 million years after the big bang. The light from the four galaxies observed took 13.4 billion years to reach Webb.

Bioscience ploughs forward: In July, Deepmind, in partnership with the European Bioinformatics Institute announced that AlphaFold revealed the structure of the protein universe. The structure of more than 200 million proteins from 1 million species were deciphered, thereby significantly increasing our understanding of biology and paving the road for the next big breakthroughs in medicine.

Fusion research gains momentum: In a story that made international headlines in December, scientists at Lawrance Livermore National Laboratory pointed 192 laser beams at a tiny diamond sphere housing hydrogen fuel. The fuel ignited causing atoms to fuse, releasing more energy than the lasers put into the sphere, but not more energy overall taking into account the energy needed to supply the lasers (roughly 100 times more energy that put out). This is still an important milestone, but far from achieving practical fusion to power the grid.

In August, a major news story emerged from two studies, one published in Cell [12] and one in Nature [13]. Researchers were able to create and grow mouse embryos from stem cells without sperm or eggs for 8.5 days, longer than ever before. The embryos developed a brain and a beating heart as well as foundations for other organs. The two studies used similar techniques. Applying this research to humans will be difficult because the stage of organ formation happens a month after fertilization. There are ethical and legal concerns to be sure, but the possibility that life is being created without sperm or eggs is fascinating.

Key Takeaways

• The transition to electrification at scale will require a massive transformation, especially in the architecture, energy sourcing and distribution of the power grid. Substantial private and government investment will be needed.
• Digital biology is revolutionizing the way we understand and manipulate living systems, paving the way for unprecedented advancements in medicine and biotechnology.
• Generative AI is the enabler of enablers — it will change our world.

The next five years will be amazing!

References

1. Vaswani et.al,” Attention is all you need,” https://arxiv.org/abs/1706.03762, Dec, 2017.

2. T. Mai et.al., “Electrification Futures Study: Scenarios of Electric Technology Adoption and Power Consumption for the United States,” https://www.nrel.gov/docs/fy18osti/71500.pdf, 2018.

3. Zero Emission Vehicles Factbook, https://assets.bbhub.io/professional/sites/24/2022-COP27-ZEV-Transition_Factbook.pdf, Nov, 2022.

4. “These Algorithms Are Hunting for an EV Battery Mother Lode,” https://www.wired.com/story/these-mining-algorithms-are-hunting-for-an-ev-battery-mother-lode, Dec, 2022.

5. Zhang, “Atomically Dispersed Dual-Site Cathode with a Record High Sulfur Mass Loading for High-Performance Room-Temperature Sodium–Sulfur Batteries,” https://onlinelibrary.wiley.com/doi/epdf/10.1002/adma.202206828, Oct, 2022.

6. “The 50 States of Grid Modernization,” NC Clean Energy Report, Table 1, page 6, 2022.

7. R. Makki, “The Global Grand Challenge of Our Time,” https://www.linkedin.com/pulse/global-grand-challenge-our-time-rafic-makki.

8. H. Arami, et. al. “Remotely controlled near-infrared-triggered photothermal treatment of brain tumours in freely behaving mice using gold nano star ,” https://www.nature.com/articles/s41565-022-01189-y, Aug, 2022.

9. T. Thompson, “The staggering death toll of drug-resistant bacteria,” https://www.nature.com/articles/d41586-022-00228-x, Jan, 2022.

10. G. Almogy et. al. “Cost-efficient whole genome-sequencing using novel mostly natural sequencing-by-synthesis chemistry and open fluidics platform,” bioRxiv preprint doi: https://doi.org/10.1101/2022.05.29.493900, Aug, 2022.

11. BJ Kagan et. al., “In vitro neurons learn and exhibit sentience when embodied in a simulated game world,” https://www.biorxiv.org/content/10.1101/2021.12.02.471005v2.full, Oct, 2022

12. S. Tarazi et. al., “Post-gastrulation synthetic embryos generated ex utero from mouse naive ESCs,” Post-gastrulation synthetic embryos generated ex utero from mouse naive ESCs, https://www.cell.com/action/showPdf?pii=S0092-8674%2822%2900981-3, Aug, 2022.

13. G. Amadi, “Embryo model completes gastrulation to neurulation and organogenesis,” https://www.nature.com/articles/s41586-022-05246-3, Aug, 2022.