Transformer Model TransAct Beats CNN-LSTM on Two of Three HAR Benchmarks
Researchers have introduced TransAct, a Transformer-based framework designed to improve human activity recognition in wearable devices.
The system utilizes optimized input projection and multi-headed self-attention mechanisms to process time-series data, demonstrating a performance advantage over traditional CNN-LSTM models on two of three tested benchmarks, according to findings published in Nature.
To evaluate the framework, developers compared TransAct against a hybrid CNN-LSTM architecture across the UCI HAR, PAMAP2, and OPPORTUNITY datasets. On the PAMAP2 benchmark, which uses raw inertial signals, TransAct reached 86.13% accuracy, compared with 82.75% for the CNN-LSTM baseline. Results on the UCI HAR dataset showed a higher held-out test accuracy for TransAct, though the statistical significance of this margin varied depending on the testing method used.
The OPPORTUNITY benchmark, which contains severe class imbalances, revealed mixed results. While overall accuracy was statistically indistinguishable between the two models, TransAct achieved a significantly higher macro-F1 score, a metric considered more informative for imbalanced datasets.
Despite these accuracy gains, the study said that TransAct is consistently slower than the CNN-LSTM baseline. Testing on a T4 GPU showed that both mean epoch time and per-sample inference latency were approximately two to three times higher for TransAct. An ablation study on the UCI HAR dataset found no statistically detectable contribution to accuracy from individual architectural components, including layer normalization and positional encoding.
Nature serves as a prestigious international venue for peer-reviewed research across the natural sciences and interdisciplinary fields. Recognized as one of the most influential scientific journals globally, it maintained an impact factor of 50.5 according to the 2022 Journal Citation Reports, underscoring its significant citation influence within the scientific literature. The journal maintains a highly selective publication process, accepting fewer than 8% of submitted papers.
The scope of the journal encompasses dense, technical research articles alongside commentary, analysis, and news regarding scientific policy and developments. While primary research papers are the journal’s focus, it also features spotlight articles on technological breakthroughs, which are distinct from research articles and do not receive the same recognition.
The researchers said that while their results indicate a reproducible advantage over recurrent baselines on two of the three benchmarks, this performance improvement is not universal and arrives with a consistent computational cost.
Related reading
- FT, NYT Tie Alice Huang’s Chip Network to Gaming Tycoons and PLA Suppliers
- Raghul Anandakrishnan Taken Into ICE Custody After Texas Conviction
- Researchers Report TransAct Matches CNN-LSTM Accuracy at Higher Computational Cost (time.news)
- RBA's Fifth Hike Adds About A$94 a Month on A$600,000; Three Voted to Hold (daybreakwire.com)