An annual placing of bets for tech in 2024.

Predictions for Tech in 2024

Every January I try to predict some tech trends I anticipate for the year ahead. I also go back to check how my predictions for the previous year weighed up in the end. Let’s see how accurate last year’s predictions were and take a stab at what lies ahead for 2024. Previous winners going back to 2021 and carrying on into 2023 included the rise of RISC-V which has continues to flourish, as well as a cloud exodus as some organizations have moved back to on-premise environments to save costs in 2023. Let’s see how predictions for 2023 landed.

Predictions for Tech in 2023

Recap 2023

Once again some of last year’s predictions were premature. Some far exceeded what even I thought. Going back further to 2022, the COVID chip shortage seems to have settled a bit, but 2023 raised tension between China and the western world with an escalation of tech embargoes and import bans. The real star of 2023 was a further surge in AI communities and a widespread exchange of LLM models. A full itemized list from last year’s predictions is found here:

1. The Metaverse will implode, taking Zuckerberg with it.
   It could be argued the Metaverse did implode, though not enough to dislodge Facebook leadership to any extent. If anyone else has heard of a CEO losing $46B and surviving, I’d love to hear about it (challenge — not including Twitter). The tech involved here has made impressive gains, with Oculus 3 introducing many advancements, and other headsets hitting the market with new innovation. I was approached by recruiters for a company that helps train soldiers virtually without spending $200,000 on Javelin missiles which is a clear value proposition, so I think the Metaverse still has practical applications but not the average consumer. That excludes the adult entertainment industry which apparently is producing more content than ever before.
2. Crypto exchanges will continue to collapse.
   The FTX trial ended like most thought it would, with SBF behind bars. A few other exchanges have had their share of controversy, but recent gains in cypto values have regulators largely looking the other way. If crypto implodes again I think we will see the regulatory microscope return to this space.
3. Semiconductor shortages ease🤞.
   This was largely true. TSMC continues to lead the way with ever shrinking microchip fabrication. Geopolotical tensions with Taiwan continue to put this at risk for other reasons but in general, chip availability was much improved. Chip makers continued to see their stocks surge, particularly king of the year Nvidia. Hedge fund manager Michael Burry famously shorted chip stocks in 2023 and took big losses this year, but I think this may be another of his bearish predictions that was just premature and we’ll see how things play out in 2024.
4. Project Caviar and AV1 will complete the free media vision.
   This prediction is really the unsung hero of 2023. Not many people talk about Opus or AV1 but they are quietly saving the Internet. While most people looking for their next phone, tablet, PC, or TV are looking for things like performance or screen quality, many devices have introduced native support for new efficient media codecs that dramatically free up internet bandwidth around the world. Video streaming takes up an estimated 65% of the world’s internet bandwidth. AV1 can easily cut that by half or more. 2023 was the first year I’ve had no problem finding the next device I buy with native AV1 decoding, which is usually hidden in fine print details. You can also switch your YouTube settings to prefer AV1. Any household full of kids or flatmates streaming should enable this to save significant bandwidth and avoid causing that unfortunate Zoom connection drop during an important call. I will double down with this prediction for 2024 below.
5. Tech infrastructure budgets will be squeezed.
   Tech budget cuts definitely took hold in 2023 in anticipation of a recession that may or may not have landed, depending on where you are. Most recession reports I find don’t mention what country or economic region they are discussing. I expected budget cuts to hit products and services but I didn’t anticipate how extreme the layoffs would be. AI is bound to replace a lot of information jobs in 2024 but pre-recession fear layoffs were stronger than I imagined. After COVID hiring binges a talent hangover has caused many blind spreadsheet layoffs, and a lot of good talent is currently on the bench looking for work. It seemed to be literally night and day as people in tech like myself saw waves of recruiter contacts replaced with silence and ghosting. Any organization in search of talent has full advantage as rates have dropped and it’s a hirer’s market. Mergers and acquisitions have also dropped, with the exception being the massive acquisition of VMware by Broadcom.
6. OpenAI will become a political battleground.
   This was spot on, culminating in the unceremonious removal of Sam Altman as CEO, followed by an employee protest and realization that Microsoft could hire Sam Altman and his team would follow. This would have left an empty shell of OpenAI and only a board with nobody and no value left behind. I had the privilege of briefly meeting Sam at YCombinator once and everybody in the Valley knows that he is perfectly capable of doing anything he puts his mind and his connections to and you just don’t mess with him. The board shakeup that followed was comical. The major surprise in this space for 2023 was Facebook’s leak/introduction of Llama open source LLM. This tech and framework gives anyone the ability to train and use models to become their own ChatGPT, much the way OpenAI was originally intended. See more on this in 2024 predictions.

The tech job market favors employers going into 2024.

All in all I would say 2023 predictions were fairly strong though most of it was pretty obvious going into 2023. Nothing was a long shot. Let’s see if I can pull a better hail Mary for 2024.

What about 2024?

Now let me try to guess what happens in the year ahead. These opinions are personally my own. What do I expect or look forward to in tech for 2024? Some of these are predictions and some may just be wishful thinking.

1. The LLM models ecosystem will be the star of 2024.
   Digital AI is really nothing new. Google’s Tensorflow is an amazing marvel of open source AI introduced 8 years ago in 2016 and allowed people to train and use neural networks similar to the ones used by OpenAI. The reason it didn’t catch on is the same reason a lot of Google’s technology doesn’t initially catch on: ease of use. A lot of Google’s innovations are designed by the smartest minds in the world but not designed to be used by those less than smart. Google introduced Cgroups to the Linux kernel and the concept of post-chroot containers but left it up to the user to figure out image management. It wasn’t until Docker simplified image management and sharing that containers took off for the rest of the world. Google introduced Tensorflow but left it up to the user to figure out how to train and use their own models. Now communities like HuggingFace and Facebook/Meta’s introduction of Llama are filling this gap by providing an area to build and exchange specialized AI models just like Docker Hub allowed the free sharing and publication of container images. This space is going to take off in a huge way and the open source nature of it fixes the proprietary license switch made by OpenAI. While OpenAI continues to climb in revenue with a goal of eventual profit there are huge headwinds in the open source community and I predict their revenue will peak around 2024 in the face of growing open source competition. The reason for that is anyone today can already run a local LLM rivaling ChatGPT for free using the open source Llama on top of community models from the HuggingFace community. The open source distribution that OpenAI was supposed to be is being replaced by actual open source models from the community and from startups like Mistral. Startups are often releasing simplified teaser models for free and then hopefully offering larger commercial versions of those models for purchase to enterprise customers. This business model presents great short-term opportunities in software if development costs are managed, because the competition is evolving rapidly. AI startups not continuously innovating or without a 1-2 year path to profit will find their AI products obsolete by the time they expect to be cashflow positive. The only business that can’t lose in this space is hardware acceleration and GPUs, and companies like Nvidia will continue to thrive in 2024.
2. Nvidia’s Competition Will Make Gains.
   There is no doubt that Nvidia makes the best GPUs and the best related software in almost every category. Video, 3D, and AI inference crowns all go to the current Nvidia lineup for Laptops, Workstations, and datacenter applications. Still most people aren’t aware that the king of market share still goes to Intel because of the widespread built-in GPUs across most of its consumer CPUs. Intel has recently had major troubles in its CPU space and its fabrication technology which hasn’t been able to keep up with TSMC’s smaller 7–2nm processes. They also lost Apple as a customer to Arm. But most people don’t appreciate the GPU innovations Intel and AMD have made in recent years. Also Intel’s troubled fabrication which hasn’t caught up with TSMC’s 5–2nm isn’t used for their GPUs. Intel ARC GPUs may not be the best in category but they are priced at a great value and they are actually produced using superior TSMC fabrication. Intel’s solid OpenCL support can be used for AI and LLM inference with impressive results at around 30–50% the cost of an Nvidia solution even if it doesn’t perform quite as well. Meanwhile Apple got rid of OpenCL support which I think was a mistake and requires a special build of Llama is required to use Apple GPUs for AI inference. I’ve personally tested Llama inference on Apple M2 with and without Metal GPU acceleration and the fact is that the M2 CPU and main RAM continues to be the star, with negligible gains from the Metal GPU. The low latency architecture of Apple’s built-in RAM-CPU combination makes for an amazing AI inference experience, with the limitation of RAM you can’t upgrade for larger models.
3. VRAM Fabricators Will Come Out on Top.
   In other words, the future of AI won’t just be won by GPU manufacturers but also CPU and especially VRAM manufacturers. Even Nvidia GPUs require RAM from a small list of VRAM manufacturers like Samsung, Micron, Elpida and Hynix. Today’s industrial GPU was designed years ago for the crypto mining craze and focused on compute but didn’t require large amounts of RAM. Tomorrow’s industrial GPU requires massive amounts of RAM for AI use cases and will help the bottom line of VRAM manufacturers. Take two similar Nvidia chips like the AD102 split into two use cases: an AD102 for a desktop use GPU with 16GB VRAM in a RTX 4080 Super retails for about $1,200 at listing. A similar chip in an RTX 5000 workstation GPU with 32GB of production-grade ECC VRAM ideal for AI and inference retails for about $5,000. Server and datacenter GPUs with premium HBM2/3 RAM go even higher and are hard to find right now. The common commodity required by all discrete GPUs and all vendors is expensive VRAM and demand couldn’t be higher in 2024.
4. Microsoft Will Win the Most Cloud Share.
   This prediction doesn’t come from opinion. This is objective fact. In fact if left to my cloud of choice I would use Google for its superb technology. But even last week I tried to test a newer Google instance type for LLM inference performance and hit quota limitations. Contacting sales to get an increase resulted in a sales autoresponse offering to help me. I responded asking to be introduced to a Sales Engineer. An AI response came back telling me generically about Google’s cloud offerings. By then I’d lost interest in benchmarking a GCP instance and was unable to write a blog comparing cloud inference. This is a big fat fail by Alphabet. Good technology is useless without a responsive sales team or at least a smooth escalation path. I can already test the same workloads without issue in AWS but when it comes to support AWS is very busy from already being the dominant provider in cloud offerings. When you contact Microsoft’s Azure team asking for help they will often have an entire sales team show up at your door tomorrow eager to win your business. This is fairly consistent whether contacting them from a large tier one bank or a regional small business. This is how Microsoft wins: they want it the most. AWS is not going away by any means but when you’re at the top, the only way is down. I could do an entirely separate post on this in 2024.
5. Continued Cloud Exodus to On-prem Kubernetes.
   Loss-leading deals from cloud hyperscalers have become their profitable plays after price adjustments. I earlier predicted there would be a rise in cloud exits as the push to cloud caused massive drops in hardware prices. I have finally seen a rising number of inquries about cloud savings and on-premise Kubernetes environments. Ian Miell of Container Solutions and I recently had a conversation and it seems more widespread than I thought. In fact I’ve finally seen upticks in pre-owned hardware prices which I monitor for signs of hardware trends. A proper price index adjusted for inflation would be a valuable metric, but for now I just monitor refurbisher pricing and even eBay sale prices. Google’s own infrastructure is largely run via bare metal containers for simplicity and performance, and the rest of the world is catching up. In the enterprise this usually means a solution like Alphabet’s Google Anthos, IBM’s Red Hat OpenShift, Broadcom’s VMware Tanzu, or SUSE’s Rancher. It will be interesting to see how Broadcom’s VMware acquisition affects Tanzu in 2024 but I expect to see gains from all three vendors.
6. AV1 Continues to Dominate Media.
   As a huge AV1 video codec advocate I have seen it become more widespread and start to cut wasted internet bandwidth on media streaming. Native decoding support saw massive gains in new devices in 2023. It’s time to go further with more hardware encoding. Google’s Tensor 3 chip was released in their new Pixel 8 series phones with the world’s first mobile AV1 encoder. Unfortunately Google still has not enabled this impressive feature with software or within the camera app. Next generation video recording solutions are already obsolete if they don’t have AV1 support. This includes phones, action cameras like GoPro, security cameras, DVRs, and more. The rest of these devices need to embrace encoding or face obsolesence. The fact that a set-top cable box or DVR can record 10 hours of AV1 video in the same storage space as 2 hours of H264 video means its time to make the switch. SSD and storage prices are expected to rise in 2024 and lets face it, the bulk of this need is taken up by media, video, and audio. Google could sell 100x the Tensor 3 chips if they used them to build a line of security camera and action cameras or else licensed them out to third parties. Not doing this is a massive missed opportunity at Alphabet.
7. Phone Numbers Will Cease to Exist.
   My longest long shot wish list prediction for 2024 is the death of telephone numbers. I highly doubt this will actually happen in 2024 or maybe even before 2030 but it is high time to end the obsolete and awful ancient concept of phone numbers. Phone numbers were first used in 1879 to avoid the need to ask an operator to direct each call manually. Now they’ve become a mandatory nuissance with most telco contracts. Just like internet security has gone from IP-based trust to identity based trust, the phone number is an easily enumerated identifier that is prone to scam attacks and SPAM dialing in a world without authentication. The fact that a phone number is frequently a required field when signing up for online services is silly — users can easily make up or use any phone number so long an auth token isn’t verified. Phone numbers are fixed length with optional prefix codes or Huffman codes. This means I can pick any random US number (say +1–614–624–2537) and dial that right now. Having mashed the keyboard for a random number, I can publicly look up its US area code and find Columbus, Ohio. If the call connects to someone unmuted, I make their phone ring day or night no matter where they are or what they’re doing. So I can call Average Joe right now from anywhere in the worldwhile he or she is probably on the toilet or doing something delicate like performing surgery on a horse. See how ridiculous the concept of a phone number is? The only number I care about on my phone is an IP address, and I expect it to change every time I move between connections. The rest is provided by apps which have modern concepts of identity and access control. I can easily set a status letting people know if I’m available or not. Classic voice telephony is already dead, and thanks to technologies like Starlink, VoiP and modern communication apps with voice or video work almost anywhere in the world. Please just kill the public unauthenticated phone number already. Those who object or claim the elderly won’t be able to use modern VoiP technology should speak to someone who asked an operator to connect them before phone numbers existed. Or maybe the first victim of an autodialer scam. It’s time to start by requiring telcos to make phone numbers optional with any internet contract. One way or another, phone numbers should and will die an overdue death.
8. AMMENDED 29-FEB-2024: One of the big three will try to acquire another big cloud provider.
   While I’m making hail mary predictions I’d like to make one more long shot bet. Given the American SEC has been asleep at the wheel of monopolies lately. I think one of the big three cloud providers will try to acquire one of the other big cloud providers within at least a few years. Apple has loads of cash on hand but has no cloud play. How wild would it be to see Apple acquire another big cloud player? This is especially likely given Apple just canceled their electric car project. Apple could easily jump into the public cloud space by acquiring a smaller player like Digital Ocean. Though that may appear petty in the grand scheme of cloud, I think it would be the smartest thing Apple did all decade. This includes their overpriced Apple Vision Pro headset which is already flopping down the hype slope as all overpriced VR headsets do. If I were Apple I would buy Digital Ocean and enter the cloud space, focusing Apple’s powerful Arm architecture on AI use cases.

Please let’s just end phone numbers.

Conclusion

It’s always fun to place bets on the year ahead. Given the recap of my 2023 low-hanging-fruit predictions, I wanted to go bigger this year with some things I would love to see happen. This year I invite anybody has thoughts or predictions to drop them here as timestamped comments for the future. Meanwhile if anyone is curious about any of the topics above or would like a consultation for any of them please contact me as I’m happy to discuss. TeraSky provides consulting and services for many of the products mentioned here.

GeForce RTX 4080 SUPER vs RTX 5000 Ada Generation

Telephone number — Wikipedia