MBZUAI plugs the privacy gap at the heart of enterprise AI
DP-Fusion delivers token-level provable privacy without sacrificing output quality
#UAE #LLMs — Abu Dhabi-based Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) has presented DP-Fusion, a new method for protecting sensitive data during live AI inference, at ICLR 2026 (International Conference on Learning Representations) in Rio de Janeiro. The research addresses one of the most persistent vulnerabilities in deployed AI systems: the risk that large language models inadvertently leak sensitive information from their context. Information such as personal data, medical records or confidential documents can sometimes leak when AI generates responses. DP-Fusion provides mathematically provable privacy guarantees at the token level to solve the problem, while maintaining high output quality, achieving six times lower perplexity than competing privacy methods.
SO WHAT? — Training-time privacy in AI has received significant attention from global researchers, but what happens when a live model processes sensitive data during inferencing has not. Inferencing is not only where the real-world risk concentrates, but the demand for inferencing is witnessing explosive growth, taking Generative AI into more risk-averse operations. GenAI models are now being deployed more extensively in regulated environments, such as hospitals deploying AI on patient records, financial institutions running AI on client data, or governments using AI on classified documents. MBZUAI’s DP-Fusion was developed in response to that gap. The fact that it provides mathematically provable guarantees (rather than probable ones) makes it different from existing approaches.
KEY POINTS:
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) has presented DP-Fusion at ICLR 2026 in Rio de Janeiro, one of the most competitive venues in AI research. The paper introduces a token-level differentially private inference method for large language models that provably limits how much sensitive information can leak through AI-generated outputs.
The method works in four steps:
labelling sensitive tokens in the input;
running the model without those tokens to establish a baseline;
running the model with the sensitive tokens; and
blending the two output distributions so the final result stays within a mathematically bounded distance of the baseline (hiding sensitive information without destroying output quality).
DP-Fusion achieves six times lower perplexity than competing differentially private inference methods, meaning its outputs are significantly more coherent and useful while maintaining stronger privacy guarantees. The research suggests this is a substantially better privacy-utility trade-off than any previously published approach.
The method provides provable privacy bounds, meaning the advantage any attacker gains from trying to infer sensitive information is mathematically bounded. This is a stronger guarantee than scrubbing or paraphrasing methods, which offer no formal proof of privacy.
DP-Fusion also mitigates prompt injection and jailbreak attacks, by treating tokens retrieved from untrustworthy external sources as sensitive. This effectively gives the method a dual function as both a privacy tool and a security defence against adversarial manipulation of AI systems.
The research targets high-stakes deployment environments such as hospitals using AI to match patient symptoms to historical medical records, financial services firms processing client data through AI systems, and any organisation where personally identifiable information flows through a live language model.
The privacy-utility balance is controlled by a single parameter, where setting it to zero hides sensitive tokens entirely and higher values progressively trade privacy for improved output quality. The option gives operators practical control over the level of protection applied to different use cases.
The DP-Fusion research team includes: Rushil Thareja, Preslav Nakov, Praneeth Vepakomma, and Nils Lukas.
ZOOM OUT — The AI privacy challenge that DP-Fusion addresses is growing faster than most organisations realise. According to McKinsey, AI inference will account for more than 40 percent of total data centre demand and growing at 35 percent annually. The global AI inference market, currently valued at over $100 billion, is projected to reach $250 to $350 billion by 2030. Inference already accounts for upwards of two-thirds of all AI workloads today and represents 80 to 90 percent of the total lifetime cost of an AI system. Every one of those inference calls is a moment when a live model touches real data and, whenever it does so, there’s also a live operational risk.
[Written and edited with the assistance of AI]
LINKS
DP-Fusion research paper (arXiv)
DP-Fusion live demo (website)
DP-Fusion library (GitHub)
Pypi package (Pypi)
Example collab notebook (Google Colab)
Read more about recent MBZUAI research:
MBZUAI builds AI to save researchers drowning in papers (Middle East AI News)
UAE lab breaks the speed barrier in AI video generation (Middle East AI News)
MBZUAI advances medical AI, cut training costs (Middle East AI News)
MBZUAI officially launches K2 Think V2 with mobile apps (Middle East AI News)
MBZUAI, Inception launch enhanced Nanda Hindi LLM (Middle East AI News)