Arrhythmia detection using electrocardiography (ECG) is a critical step in saving lives. With the widespread adoption of wearable devices providing single-lead ECG recordings, a major challenge has emerged: how can we achieve high diagnostic accuracy comparable to sophisticated medical devices? The answer lies in an innovative methodology: Multi-Teacher Decomposed Feature Distillation. Most state-of-the-art models currently rely on deep neural networks, which demand substantial computational resources. In contrast, portable devices require “lightweight” models. The conventional approach for knowledge transfer is Knowledge Distillation, where a large model (teacher) trains a smaller model (student). However, for the complex signals in ECG, a single teacher may not be sufficient to convey all the intricate details. Proposed Solution: Decomposed Feature Distillation This study introduces the idea of breaking down the ECG signal into separate components or features rather than processing it as a single block. The approach involves: Multi-Teacher Approach: Instead of relying on a single expert, multiple teacher models are employed, each specializing in extracting a specific pattern from the data (e.g., high-frequency components, temporal patterns, or structural variations in the P-QRS-T wave). Feature Decomposition: Extracted features are analyzed and decomposed into their fundamental elements. This allows the student model to understand why a heartbeat is classified as arrhythmic, not just what the classification is. Distillation Process: The small model is trained to learn these decomposed features from multiple teachers, enhancing its ability to detect anomalies even in noisy environments. High Accuracy in Single-Lead ECG: Overcomes the information limitation of a single channel compared to traditional 12-lead ECGs by extracting deeper features. Real-Time Inference: Thanks to the compact student model, these algorithms can run directly on smartwatches or home monitoring devices without delay. Higher Medical Reliability: Feature decomposition increases model interpretability, which is crucial in medical applications for gaining clinicians’ trust. Integrating multi-teacher knowledge distillation with feature decomposition represents a significant advancement in biomedical signal processing. This is not merely an attempt to shrink models—it is a strategy for smarter artificial intelligence capable of extracting maximum insight from minimal data. By adopting these techniques, we move closer to a future where accurate cardiac diagnosis is accessible to everyone, anytime, with minimal technical requirements. Al-Mustaqbal University is the first one university in Iraq.