In a study published in Newton, researchers from the Institute of Physics (IOP) of the Chinese Academy of Sciences developed a hardware-assisted probabilistic-gradient-of-activation-function-accelerated backpropagation (p-GAF-BP) algorithm that significantly reduces computational overhead and energy consumption during artificial intelligence (AI) training.

A fundamental component of AI systems is BP algorithm which enables deep neural networks to learn through iterative error correction. BP training requires extensive multiplication and gradient calculations, many of which are wasted on neurons operating in saturation regions where gradients approach zero and contribute little to network optimization. Existing AI accelerators mainly improve computational parallelism but do not fundamentally reduce the number of operations.

In this study, researchers utilized the intrinsic stochastic switching behavior of spin-orbit torque magnetic tunnel junctions (SOT-MTJs), a class of spintronic devices originally developed for magnetoresistive random-access memory technologies, and found that the probabilistic switching characteristics of SOT-MTJs closely match the mathematical form of activation function derivatives such as the sigmoid function.

Using twice-repeated stochastic sampling of SOT-MTJ states, researchers transformed complex gradient derivation operations into simple probabilistic sampling processes. This approach dynamically identifies effective gradients while bypassing unnecessary calculations associated with saturated neurons, thereby reducing the number of multiplication and addition operations without modifying the neural network structure.

The p-GAF-BP algorithm maintained model accuracy while improving energy efficiency. In handwritten digit recognition tasks based on the MNIST dataset, it achieved recognition accuracies above 97%, comparable to those of other BP algorithms, and it reduced the number of multiplication and addition operations during gradient computation by nearly one order of magnitude, and the overall energy consumption by approximately 79%. The algorithm also showed robust performance in deeper neural networks such as ResNet18 on the CIFAR10 dataset.

This study demonstrates how spintronic hardware can be integrated with neural network training algorithms to support the development of more energy-efficient and sustainable AI systems. "It establishes a new paradigm for hardware-algorithm co-designed energy-efficient AI training, which has strong potential for future low-power edge intelligence systems and large-scale AI model training," said Prof. HAN Xiufeng from IOP, one first author of the study.