NTT to launch its Large Language Model “tsuzumi” in March 2024

NTT will launch commercial services based on “tsuzumi,” a lightweight yet world-class Japanese language processing Large Language Model (LLM) developed by NTT, in March 2024 in order to help solve issues posed by the proliferation of LLMs such as increased power consumption and costs. Ahead of the launch, trials have already begun in October 2023 with partners including Kyoto University Hospital in the medical field and Tokio Marine & Nichido Fire Insurance Co., Ltd. in the contact center field.

Recently, LLMs like ChatGPT have drawn a lot of attention. While these models exhibit high linguistic processing capabilities by incorporating vast knowledge within the model, the energy required for learning is said to be equivalent to one nuclear power plant for one hour (in the case of GPT-3), and operating them requires massive GPU clusters. Tuning for specializing in various industries and inference costs are enormous, posing issues for sustainability and the economic burden on companies preparing learning environments.

Features of tsuzumi

Lightweight LLM

• NTT has developed lightweight versions of “tsuzumi” with 600 million and 7 billion parameters.
• Approximately 300 times lighter than OpenAI’s GPT-3 with 175 billion parameters (600 million version), and 25 times lighter (7 billion version).
• The 7 billion version can be inferred at high speed on 1 GPU, and the 600 million version on a CPU, enabling lower tuning and inference costs.
• When converted to GPU cloud usage fees, learning costs can be reduced to approximately 1/300 (600 million version) and 1/25 (7 billion version), and inference costs to approximately 1/70 (600 million version) and 1/20 (7 billion version).

Especially skilled in Japanese, but supports English & Japanese

• “tsuzumi” supports both Japanese and English. Leveraging insights gained from years of research, it demonstrates high performance especially in Japanese language processing.
• Confirmed to surpass GPT-3.5 and top domestic LLMs in Rakuda, a benchmark for generative AI.
• Also achieves world-class performance in English. Multi-lingual support is planned going forward.

Flexible tuning — base model plus adapter

• Using adapters that can efficiently learn knowledge, tuning for correspondence with language expressions and knowledge unique to specific industries is possible through minimal additional learning.

Multi-modal — language plus understanding of visual, audio, user situation

• Planned support for multimodal incorporating understanding of graphical displays, nuances in voice, facial expressions, etc. not expressed in language, enabling collaboration with people in the real world.

tsuzumi’s Positioning

First, tsuzumi will focus on specialized domains leveraging the ability to flexibly and securely learn data specific to each industry.

Rather than one massive LLM containing all knowledge, tsuzumi aims for a world where the collective wisdom of small specialized LLMs with expertise and individuality solves real-world social issues in coordination with diverse AIs.

To serve as an integration base for countless LLMs, a safe, low-latency environment on par with local environments is necessary. For tsuzumi learning, NTT constructed an environment utilizing the IOWN All-Photonics Network spanning data centers hundreds of kilometers apart, enabling secure LLM learning with minimal performance degradation by connecting GPUs and storage between data centers.

Future Developments

After launching commercial services, NTT will continue to enhance tuning functions and implement multimodal capabilities. NTT will also advance development for applications in cybersecurity and autonomously coordinating AIs like AI constellations.

Through these efforts, NTT will further accelerate initiatives for creating new value and elevating customer experiences.

This article is part of our Tokyo FinTech Publication, please follow us to read more from our writers, like hundreds of readers do every day. Please also register for our short weekly digest, the “Japan FinTech Observer”, on Medium or on LinkedIn.

Should you live in Tokyo, or just pass through, please also join our Tokyo FinTech Meetup. In any case, our YouTube channel and LinkedIn page are there for you as well.