Title: A Robust Deep Learning Model for Fall Action Detection using Healthcare Wearable Sensors
Authors: Abdulwahab Alazeb1,*, Iqra Aijaz Abro 2,*, Naif Al Mudawi 1, Haifa F. Alhasson 3, Naif S. Alshammari4, Mohammed Alshehri1, and Fatimah Alhayan 5
Datasets Description: 
A. UP-fall dataset
The UP-fall detection dataset provides comprehensive data about identifying eleven fall-related activities [17] based on three trial executions from each subject. Healthy young people without disabilities participated in the dataset collection by performing eleven fall-related activities through wearable sensors and vision equipment with ambient sensors. The UP-fall detection dataset incorporates postures from six activities of daily living (ADL) which are standing, sitting, walking, lying down, jumping, and picking up objects alongside five fall-related activities that include falling forward with hands and knees and backward, sideways, and sitting on an empty chair actions. The fall activities span ten seconds yet the ADLs need sixty seconds of completion except for the jumping and object pickup which run for thirty seconds and ten seconds, respectively.
B. UR-fall dataset
The UR fall detection dataset was created from data obtained through a combination of two USB-connected Kinect cameras and an IMU device which operated at the waist via Bluetooth connectivity. The ADL events were recorded using only Camera 0 as well as an accelerometer. The sensor data acquisition involved PS Move hardware and the x-IMU system. The study was conducted with five volunteers who performed 70 sequences among 30 fall-induced sequences and 40 sequences of ADL activities within an office setting. Each participant executed multiple forward, and backward along with lateral falls onto the 2 cm thick carpet space with intended motions at least three times. The x-IMU sensor was operated by the researchers near the pelvic region. Participants performed the ADLs by standing, sitting squatted and bending their bodies after which they picked up objects and rested on the settee. Falls detected by the system demonstrated perfect accuracy but the sensor misidentified quick sitting activities because it confused them with true falls without additional sensor data. The provided dataset comprises falls that happen when someone is standing along with falls that take place during the sitting phase. Each dataset entry includes original accelerometer measurements that accompany depth images from two Kinect cameras as well as RGB image time sequences. A threshold-based fall detection technique served as one of the methods described by the researchers.


Abstract: The proposed technique begins with Butterworth's sixth-order filtering of the data followed by segmentation through Hamming window application. The identification of essential patterns is achieved through the utilization of feature extraction methods which include State Space Correlation Entropy (SSCE) coefficients together with Mel Frequency Cepstral Coefficients (MFCC), Linear Predictive Cepstral Coefficients (LPCC), Parseval’s energy and Auto-Regressive (AR) coefficients. The features are selected through PSO optimization then LSTM networks execute the classification process. The method received experimental assessment using three public datasets named UP-Fall and UR-Fall. The proposed method achieves a substantial improvement in classification results compared to traditional approaches while demonstrating enhanced accuracy outcomes in experiments. This methodology demonstrates exceptional performance for fall action detection systems because it combines advanced preprocessing techniques with robust feature extraction methods and LSTM networks optimized through PSO.

Key Contributions:
Data processing with Butterworth sixth-order filters and subsequent segmentation through Hamming windows enhances the extraction reliability of signals especially when sensor conditions vary.
The proposed framework uses a combination of SSCE coefficients together with MFCC, LPCC, Parseval’s energy, and AR coefficients as features for precise locomotion transition detection.
Feature selection based on the PSO system achieves optimization of discriminative features that are used for classification purposes.
LSTM networks form the basis for classification because they maintain superior capabilities to recognize sequential patterns related to complex actions and fall events.
The proposed system underwent thorough evaluation on the UP-Fall and UR-Fall datasets to prove its superior accuracy and reliability compared to standard techniques.
The implemented approach successfully handles sensor inconsistency problems alongside variation issues from both speed changes and movement styles and performs precise matching of equivalent action categories.

Usage:

Data Availability:

The UP fall detection and UR-fall detection dataset used in this study are available at:
UP-fall detection dataset:https://www.kaggle.com/datasets/pragyachandak/upfalldataset
UR-fall detection dataset:https://fenix.ur.edu.pl/mkepski/ds/uf.html


Code Repository:

The code for implementation is available in the supplemental files provided with the submission.
The supplemental files contain code for preprocessing, segmentation, and feature extraction from time-series data. The preprocessing step applies a sixth-order Butterworth filter to smooth the data. After preprocessing, the data is segmented using a Hamming window. Finally, features such as Linear Predictive Cepstral Coefficients (LPCC), Parseval's Energy, State Space Correlation Entropy (SSCE) coefficients, and Mel Frequency Cepstral Coefficients (MFCC) are extracted.

Code Information
The code for this fall action detection system follows a series of essential steps to process sensor data, extract relevant features, and classify fall events using deep learning models. Below are the key components and steps involved in the implementation:

1. Preprocessing
Butterworth Sixth-Order Filter: The raw data is first filtered using a Butterworth filter of the sixth order. This filter helps reduce noise and smooth the sensor signals, making them more reliable for subsequent analysis.

Hamming Window Segmentation: After filtering, the data is divided into smaller segments using the Hamming window technique. This segmentation process ensures that the data is manageable and suitable for feature extraction.

2. Feature Extraction
Linear Predictive Cepstral Coefficients (LPCC): LPCCs are used to extract temporal features that represent the spectral properties of the time-series data. They are effective for capturing the characteristics of the movement patterns in fall events.

Parseval’s Energy: This feature captures the energy content of the signal and helps in distinguishing different types of motion based on energy variation.

State Space Correlation Entropy (SSCE) Coefficients: SSCE coefficients provide information about the temporal correlations within the data, which helps in detecting transitions in movement and postures.

Mel Frequency Cepstral Coefficients (MFCC): MFCCs are widely used in speech and sound processing, and they are also effective in motion classification as they capture the frequency content of the sensor data.

3. Feature Selection
Particle Swarm Optimization (PSO): PSO is used for feature selection, optimizing the discriminative features that are most relevant for classification. This reduces the feature space while retaining the critical information for accurate fall detection.

4. Classification
Long Short-Term Memory (LSTM) Networks: The extracted features are used to train an LSTM network. LSTMs are well-suited for sequential data and help in identifying patterns in the temporal sequences of movement, which are essential for classifying different fall actions.

Steps:
Preprocessing: Butterworth sixth-order filtering for noise reduction.
Segmentation: Hamming windowing technique for dividing data into segments.
Feature Extraction:
LPCC (Linear Predictive Cepstral Coefficients)
Parseval's Energy
SSCE (State Space Correlation Entropy) Coefficients
MFCC (Mel Frequency Cepstral Coefficients)

Requirements:
To run the code, the following software and libraries are required:

Operating System: Windows 10 (x64)

Hardware: Intel Core i7-10510U CPU, 8GB RAM

Python Libraries:

PyTorch: For building and training the LSTM model.

OpenCV: For handling image processing and depth images.

NumPy: For numerical operations and handling large datasets.

SciPy: For signal processing tasks like filtering.

Additional libraries may be required for specific tasks. 

Methodology:

The human fall detection system uses inertial sensor data which requires pre-processing for data quality improvements. A Butterworth filter of the 6th order provides noise reduction and a Hamming window with a length of 10 seconds enables signal segmenting. The subsequent phase applies SSCE coefficients and MFCC and LPCC features along with Parseval’s energy and AR coefficients. The feature detection methods include SSCE for signal complexity assessment and MFCC for spectral compactness as well as LPCC for temporal pattern identification and Parseval’s energy measurement of intensity plus AR coefficients for temporal relationships. The PSO optimization technique enhances the features by cutting down computational complexity through the selection of the most relevant features. An LSTM network functions by detecting temporal patterns to perform data classification operations. The process follows four distinct phases which include preprocessing and feature extraction together with optimization before the classification step. Each stage is explained in detail throughout this text.

Code Repository:
The implementation is available in the supplementary files. It includes all the necessary code for:

Preprocessing: Applying Butterworth filtering and segmentation.

Feature Extraction: Extracting LPCC, Parseval’s Energy, SSCE, and MFCC from the segmented data.

Feature Selection: Using PSO to select the most relevant features.

Classification: Training and evaluating the LSTM network on the processed data.

Citations:
1. L. Martínez-Villaseñor, H. Ponce, J. Brieva, E. Moya-Albor, J. Núñez-Martínez, and C. Peñafort-Asturiano, “UP-Fall Detection Dataset: A Multimodal Approach,” Sensors, vol. 19, p. 1988, 2019, doi: 10.3390/s19091988.
2.S. Hafeez, S. S. Alotaibi, A. Alazeb, N. A. Mudawi, and W. Kim, “Multi-sensor-based action monitoring and recognition via hybrid descriptors and logistic regression,” IEEE Access, vol. 11, pp. 48145–48157, 2023.