A Distributed Generative AI Approach for Heterogeneous Multi-Domain Environments under Data Sharing constraints

Federated Learning has gained increasing attention for its ability to enable multiple nodes to collaboratively train machine learning models without sharing their raw data. At the same time, Generative AI—particularly Generative Adversarial Networks (GANs)—have achieved remarkable success across a wide range of domains, such as healthcare, security, and Image Generation. However, training generative models typically requires large datasets and significant computational resources, which are often unavailable in real-world settings. Acquiring such resources can be costly and inefficient, especially when many underutilized devices—such as IoT devices and edge devices—with varying capabilities remain idle. Moreover, obtaining large datasets is challenging due to privacy concerns and copyright restrictions, as most devices are unwilling to share their data. To address these challenges, we propose a novel approach for decentralized GAN training that enables the utilization of distributed data and underutilized, low-capability devices while not sharing data in its raw form. Our approach is designed to tackle key challenges in decentralized environments, combining KLD-weighted Clustered Federated Learning to address the issues of data heterogeneity and multi-domain datasets, with Heterogeneous U-Shaped split learning to tackle the challenge of device heterogeneity under strict data sharing constraints—ensuring that no labels or raw data, whether real or synthetic, are ever shared between nodes. Experimental results shows that our approach demonstrates consistent and significant improvements across key performance metrics, where it achieves an average 10% boost in classification metrics (up to 60% in multi-domain non-IID settings), 1.1×—3× higher image generation scores for the MNIST family datasets, and 2×—70× lower FID scores for higher resolution datasets, in much lower latency compared to several benchmarks.

Training modern generative AI models typically requires centralized access to large datasets and powerful compute infrastructure, which is often infeasible in real-world environments where data is siloed due to privacy, security, or legal constraints. Meanwhile, vast numbers of edge and IoT devices remain underutilized, each with limited and highly variable computational capabilities. Existing decentralized paradigms—such as federated learning and split learning—address parts of this problem but fail to jointly handle device heterogeneity, non-IID data, multi-domain environments, and strict data-sharing constraints. As illustrated in Figure 2, current learning paradigms differ in how they trade off privacy, computation, and communication, while prior distributed GAN approaches address only subsets of these challenges. These limitations motivate a unified framework that enables collaborative generative model training across heterogeneous, privacy-sensitive, and multi-domain environments without sharing raw data or labels.

We propose HuSCF-GAN, a distributed generative learning framework that combines heterogeneous U-shaped split learning with clustered federated learning. As shown in Figure 3, each client’s generator and discriminator are split between the client and a shared server, enabling devices with different computational and communication capabilities to participate efficiently while keeping raw data and labels local. The server first determines optimal, client-specific split points using a genetic algorithm to minimize overall training latency. Training then proceeds via heterogeneous U-shaped split learning, illustrated in Figure 4, where only intermediate activations and gradients are exchanged. Periodically, the server clusters clients based on discriminator activations and performs clustered federated aggregation, weighting updates using both dataset size and divergence scores. Together, these components allow HuSCF-GAN to robustly handle device heterogeneity, non-IID data, and multi-domain settings under strict data-sharing constraints.

HuSCF-GAN is evaluated across a wide range of realistic scenarios, including IID and non-IID data distributions, single-domain and multi-domain environments, and highly heterogeneous edge-device configurations. The diversity of datasets used in the evaluation is illustrated in Figure 5, spanning handwritten digits, fashion images, medical imaging, natural images, and audio data. Across all settings, HuSCF-GAN consistently outperforms state-of-the-art distributed GAN baselines. It achieves an average ~10% improvement in downstream classification performance, with gains reaching up to 50–60% in challenging multi-domain non-IID scenarios. In terms of generative quality, HuSCF-GAN delivers 1.1×–3× higher image generation scores on MNIST-family datasets and 2×–70× lower FID scores on higher-resolution datasets. Crucially, HuSCF-GAN also improves efficiency, achieving significantly lower training latency than competing distributed GAN frameworks—reducing per-iteration latency by at least 5× and up to 58× in heterogeneous and multi-domain settings—while maintaining stable convergence. These results demonstrate that HuSCF-GAN delivers both performance gains and practical scalability without sacrificing efficiency.