### Apple’s MLX Framework Integrates CUDA Backend: A Major Breakthrough for Machine Learning
Apple’s MLX machine learning framework, originally designed for Apple Silicon, is now integrating a CUDA backend, signaling a significant enhancement in its functionalities. This initiative is led by developer [@zcbenz on GitHub](https://github.com/ml-explore/mlx/pull/1983), who started experimenting with CUDA support a few months back. The project has been divided into smaller modules, with several already incorporated into the main branch of Apple’s MLX.
Although the backend is still under development, it presently supports and has tested multiple fundamental operations, such as matrix multiplication, softmax, reduction, sorting, and indexing.
#### What is CUDA?
CUDA, or Compute Unified Device Architecture, is NVIDIA’s proprietary computing framework designed to exploit the capabilities of its GPUs for high-performance parallel computing tasks. It shares a comparable purpose with Metal, Apple’s graphics API, but is specifically optimized for NVIDIA hardware. CUDA has emerged as the benchmark for running machine learning tasks on NVIDIA GPUs, extensively used throughout the ML landscape, from academic research to commercial uses. Major frameworks like PyTorch and TensorFlow rely on CUDA for effective GPU acceleration.
#### Why is Apple’s MLX Incorporating CUDA?
Initially optimized for Apple Silicon and Metal, the inclusion of a CUDA backend in MLX represents a strategic pivot. This upgrade enables researchers and developers to prototype CUDA-based models locally on a Mac using MLX, making it easier to deploy on expansive NVIDIA GPU clusters that continue to lead machine learning training tasks.
Nonetheless, there are constraints to this new functionality. Not all MLX operators are currently available, and support for AMD GPUs is expected in the future. Despite these limitations, the capacity to shift MLX code to NVIDIA GPUs without extensive modifications offers substantial benefits for quicker testing, experimentation, and research, which is essential for AI developers.
For those keen on exploring this new feature, additional information can be found on [GitHub](https://github.com/ml-explore/mlx/pull/1983).
