Abu Dhabi researchers create first-of-its-kind telecom LLM
Khalifa University's 6G Centre and TII collaborate on building TelecomGPT
#UAE #LLMs - A joint research project from the 6G Centre of Khalifa University (KU) and Abu Dhabi's Technology Innovation Institute (TII) has developed a first-of-its-kind TelecomGPT large language model (LLM). The model is the first comprehensive model built and trained on a specialised data set for the telecom domain, which could provide telecom operators, telecom vendors, standardisation bodies and telecom researchers with a valuable LLM for day-to-day usage. According to the researchers, the fine-tuned TelecomGPT excels in the Telecom Math Modeling benchmark, outperforming some of the world's most powerful LLMs including GPT-4, Llama-3 and Mistral. The TelecomGPT model is currently available to the telecom industry via a proprietary licence.
SO WHAT? - Large language models have the potential to revolutionise 6G communications networks, bringing capabilities beyond conventional deep learning models, which are already widely used in telecom networks. LLMs could provide suitable tools for applications such as Radio Resource Management (RRM) in Open Radio Access Network (O-RAN), whilst facilitating more sophisticated and adaptive communication protocols, thereby enhancing the potential efficiency and resilience of telecom networks. However, few telecom-specific LLMs have been developed and the sector also lacks widely-accepted evaluation benchmarks for the telecom domain. The new research from TII and KU offers a first-of-its-kind telecom-domain specific LLM, proposes telecom LLM benchmarks and recommends a pipeline to allow future researchers to adapt any general purpose LLMs, to telecom-specific models.
Here are some key points about the TelecomGPT research:
Joint research from the 6G Centre of Khalifa University (KU) and Abu Dhabi's Technology Innovation Institute (TII) has produced TelecomGPT, a fine-tuned large language model developed specifically for the telecom sector.
According to the research team, TelecomGPT outperforms state-of-the-art LLMs including GPT-4, Llama-3 and Mistral in the Telecom Math Modeling benchmark.
The KU/TII project also created a one of its kind, telecom domain-specific data set. Researchers built a telecom-specific pre-train dataset, instruction dataset, and preference dataset, in order to perform continual pre-training, instruct tuning and alignment tuning respectively.
The researchers also propose a pipeline for building telecom-specific LLMs and introduces benchmarks for evaluating their capabilities. Other researchers can use the team’s framework to build their own telecom domain-specific models.
Due to the lack of widely accepted LLM evaluation benchmarks in the telecom domain, the research extends existing evaluation benchmarks and proposes three new ones: Telecom Math Modeling, Telecom Open QnA and Telecom Code Tasks, in order to provide a holistic evaluation of a model’s capabilities.
The new proposed benchmarks evaluate LLM’s math modeling, open-ended question answering, code generation, infilling, summarisation and analysis in the telecom domain.
The new TelecomGPT model could prove to be invaluable to telecom operators, telecom vendors, standardisation bodies, and researchers, such as those working in institutions around the world to define 6G standards.
The TelecomGPT model is currently available to be licenced on request via Technology Innovation Institute.
ZOOM OUT - The development of the TelecomGPT large language model by Technology Innovation Institute and Khalifa University's 6G Centre could impact the work of telecom operators, vendors and policymakers worldwide, in particular as the industry works to define 6G standards. The UAE is playing a key role in the development of, and global collaboration in defining 6G standards. Sixth generation networks promise to use untapped radio frequencies enabling high-speed, low-latency communication about 100 times faster than 5G. The UAE aims to become one of the first nations in the world to implement 6G in 2030, making the shape, function and timely delivery of 6G - crucial to UAE plans. TelecomGPT could support efforts, both at home and globally, as stakeholders work towards a fully-defined 6G standard.
🎧 Listen to the podcast of Middle East AI News LIVE on 16 May 2024, when Middle East AI News talked to Prof. Merouane Debbah, founding director of the 6G Centre at Khalifa University in Abu Dhabi about the UAE’s role in defining the 6G standard.
LINKS
Read more telecom stories:
Ooredoo to buy thousands of Nvidia GPUs (Middle East AI News)
Nokia to open MEA Open Innovation lab (Middle East AI News)