Efficient Drowsiness detection system for drivers using image processing techniques

List Of Abbreviations

IR = Infrared Ray
V2V = Vehicle to Vehicle
M2M = Machine to Machine
EAR = Eye Aspect Ratio

Introduction

In today’s date different things are being developed with advance technology where things are getting advance to make our life more comfortable and secured. With the term of security, different devices have been developed to provide smart security features in a very smart and efficient way. If we talk about driver’s safety in car, different advance initiatives have been taken which can provide extreme security to the drivers [1]. The main reason behind using advance security features to avoid unexpected situation. Vehicles are getting smart so that it ca interact with its drivers in different situations [2]. With the help of IoT (internet of Things), a term V2V (vehicle to vehicle technology) helps to make an inter communication between cars through internet. As a result, if any car needs help, it can interact to the nearby vehicle in seeking of help. Apart from this, if the vehicle faces any issues, it can inform the same to the drivers so that drivers can solve it quickly. Different automatic technologies come with security features which are making vehicle more reliable in purpose of safety. Researchers and developers are working on the improvement of vehicle’s safety to reduce the accidental risk. Driver fatigue is one of the most frequent and surprising situations that might have major repercussions. Generally, during long drive, drivers get tired and may feel sleepy while driving and this can cause dangerous results like collisions. With the purpose of preventing this situation, different system has been developed to alert drivers to remove the drowsiness [3]. The systems are getting developed in such a way so that it can detect the drowsiness status of driver and alert the driver if he feels drowsy. Different embedded systems are being used where IR (infrared ray) sensors are being used even in commercially for drowsiness detection [4]. This type of system works well even in low light condition where infrared ray is being used to detect the blink of eyes. IR sensor sends the IR rays and the ray is reflected back to the receiver from eyes if the eye is closed. With this method, it is easy to identify that eye is close or not [5]. But the driver has to install the whole system in front of his eye otherwise the module will not be able to detect the eye perfectly [6]. And this becomes a big disadvantage for this kind of system as to install this type of module in front of eyes during driving may be risky and disturbing. In this paper, a system has been proposed which is able to detect the drowsiness status of a driver using live face detection and generate output accordingly to alert the driver so that kind of unexpected incidents can be overcome on time. The system is based on image processing techniques using computer vision where no extra devices are not needed to install in front of eyes. An integrated system having a camera can detect the face of the driver and is able to make the detection of drowsiness status of the driver.

Literature Review

Eddie E. Galarza et al. [1] In order to obtain the required outputs, image detection and prediction techniques are utilized to classify various images. These techniques allow images to be predicted and classified easily and efficiently. Drowsiness detection system helps to identify the drowsiness status of a particular person’s face by using image predictor. In this work, authors tried to implement different image predictor for real-time drowsiness detection and showed their performances accordingly.

Anil Kumar Biswal et al. [7] Image detection and prediction algorithms, get the popularity in Image processing where, using computer vision, different types of image-based applications are possible to be implemented. This paper proposed a detailed vision to build a face detection system which is able to detect faces and also able to identify correctly.

Mahek Jain et al. [2] Image processing become a common technique where it has been widely used in different applications. In this work, authors tried to describe the development of computer visionbased image processing algorithm and its application. The selection of a good image predictor that may be efficient enough to predict the particular image-object is the notable work in this paper.

 V B Navya Kiran et al. [8] The use of an image classification system that can categorize images based on their attributes is demonstrated in this research. Following a thorough introduction to several machine learning concepts and methodologies, such as SVM, Logistic Regression, and Random Forest methods, a web application based on the machine learning model was developed in order to classify the image object.

Femilda Josephin et al. [9] Open CV is recognized as one of the most used and popular image processing tools. One of the main reasons behind using Open CV is its easy and wide range of applicability with flexible nature. Following model construction and training, various machine learning-based image recognition systems have been created to detect or identify faces with this goal; nevertheless, their performance and accuracy are debatable. Therefore, using the DLib library, the authors of this study attempted to create a face detection system that is reliable and capable of high accuracy.

Sukrit Mehta et al.[10] Image processing is used in drowsiness detection where a good number different pre built libraries are being used to detect the drowsiness status. There are different predicting algorithms available through which one can get to know that any particular person is drowsy or not. The technology can use predictive algorithms to determine whether a person is sleepy. The authors attempted to use a machine learning algorithm that could anticipate a person's level of tiredness in order to create such a system with the same goal.

Jacek DybaÃÂ??a et al. [3] Nowadays, a good number of images processing based algorithms are being used in purpose of the development of different real-time problems. Image processing algorithms are being used in image detection, classifications, prediction etc. The selection of algorithm is based on the nature and working purpose of the application. So, it is necessary check the required output before finalizing the image processing algorithm. Haar casade is one of the mostly used image processing algorithm which is predict output by analyzing different features of the image.

Elena Magán et al. [11] This study offers an extremely effective method for identifying and forecasting images using machine learning classifiers, enabling the authors' system to precisely forecast a person's level of drowsiness. We can prevent any unforeseen disaster from occurring if we take preventative measures beforehand. The system has been designed with the same goal in mind, making it possible to receive the sleepiness status as an impending alert.

Rafael Barea et al. [4] In this work, the author showed different image detection techniques and prediction classifier where he selected one suitable classifier for his proposed system. The author mostly focused on the accuracy level of the classifier used in his model where with achieving good accuracy, the risk factor can be reduced. So, a reliable system attached on a car can be helpful to prevent the accidental risk.

S. Jansi Rani et al. [12] According to the authors, the work's findings demonstrated a close relationship between machine learning algorithms and picture applications, demonstrating that a good image classification system may be implemented using an effective machine learning method. A good analysis of data through deep studies done by the authors claimed that their proposed system can more effectively classify the images with less time and high accuracy.

Mohsen Poursadeghiyan et al. [5] In this paper, authors proposed an embedded system for drowsiness detection which is based on IR (infrared ray) sensor that can detect the blink of eyes. A module has been developed including a IR sensor where there is an emitter and a receiver. Emitter emits IR ray and receive the reflected IR ray from eyes. This module works perfectly in even low light condition.

Prof.Ankita V. Karale et al. [13] In this paper, authors developed a drowsiness detection system integrated with the Internet of Things where if the system detects drowsiness of the driver it will generate alarm as well as it will send a message to a particular number to alert other person as well to inform the situation.

Yaman Albadawi et al. [6] This paper is based on vehicle to vehicle (V2V) technology where a drowsiness system has been discussed which can send the drowsiness status to the nearby vehicle so that the nearby vehicle driver can get to know about the other driver status. This helps to reduce the chances of face to face collision and the nearby vehicle can also may assist by making him or her careful.

Table 1: Comparison on different proposed techniques

Methodology

 Implementation and design

A drowsiness detection system helps to alert the drivers while he/ she gets drowsy so that accidental risk can be reduced and driver will not lose the control over the car. An integrated system is required to perform the whole operation where a camera detects the frontal face of the drivers. Drowsiness detection systems are designed to detect the drowsiness status of a driver to prevent any unexpected incidents during driving. The system module is responsible to do this task efficiently where the module needs to be attached in front of the driver seat. The system can capture the driver’s facial image. An image detection algorithm helps to detect the face of the driver. A predictor is required to classify the image to take the decision. Figure 1 is sowing the detection process to identify the drowsiness status of drivers.

Figure 1: Drowsiness detection system environment for drivers

Face detection is one of the most basic part of computer vision. Face detection is considered as the foundation of other studies like identifying specific features of detecting faces.There are different pre-trained models like Haar cascades, dlib frontal face detector, MTCNN, and a Caffe model using OpenCV’s DNN module. The most used frontal face detection algorithms are Haar cascades and dlib frontal face detector. Both work well and widely being used in different face detecting applications. Haar cascade works slow but accuracy is high with compare to dlib. So, those applications where accuracy matters rather than running performance, Haar cascade would be suitable for that. But dlib works fast compared to Haar cascade so that those application where detection for each moment is necessary, dlib would be ideal. In our system, it needs to detect face continuously without any running interruption; Dlib was found more suitable so that frontal face can be detected in realtime with smooth experience. If we use Haar cascade, there may be delay to detect face where it can cause unsatisfactory results in terms of overall system performances.

In this system, “shape_predictor_68_face_landmarks.dat” file has been used which is basically a 68 landmark based shape predictor. In the Figure 2, it is shown that a face shape has been represented with 68 points where we can use those points according to requirement. In this work, the eye part has been focused mainly which has been marked with a rectangle in the Figure 2. So, the points37,38,39,40,41,42 represents the left eye and 43,44,45,46,47,48 represents the right eye. We can consider these points with P1, P2, P3, P4, P5, P6 and by finding the Euclidian distance between those points, we can measure the eye aspect ratio (EAR) with the mentioned formula.

Figure 2: Eye shape landmarks for EAR measurement

So, eye aspect ratio will define the status of the open and close of the eyes. In the proposed work, a particular threshold value of eye aspect ratio has been set which is 0.25. If the measured EAR value exceeds this threshold value, it will show active status, means eyes are open. If the value is in between 0.21 to 0.25, it will be considered as drowsy and less than 0.21 will show the status as sleeping.

Working functionalities with flow diagram

The system works with a defined sequence of steps where every step has its own task mentioned in the Figure 3. At the very first, the system needs to be started. After being started, it will load the classifier and start to read the video stream to detect the face with capturing frame. If the frame is captured successfully, it will apply the classifier to that frame otherwise it will continue the video streaming for capturing frame. After applying the classifier, it can get to know that eye is closed or not through measuring the eye aspect ratio. Then it will check the EAR value with the threshold value and give the output message accordingly and again start to capture new frame. The process will be continuing until it is stopped manually to end the process.

Figure 3: Flow chart of the proposed drowsiness detection system

Results

In the Figure 4, we can see that in the left window Active status is showing when my eyes are fully opened where EAR value must be greater than 0.25 and that is why it's showing “active!” status. In the right window the capturing face identified by a red square with 68 face land marks are visible.

Figure 4: Output result for active status

Here, in the Figure 5, eyes are partially closed that means EAR value must be lies between 0.21 to 0.25 and that’s why its showing “drowsy!” status.

Figure 5: Output result for drowsy status

In the Figure 6, the eyes are fully closed that EAR must be less than 0.21 so that its showing “sleeping!” status.

Figure 6: Output result for sleeping status

Conclusion

Safety is a big concern in terms of driving and huge development has been done for the safety of the drivers. Vehicles are being modified in such a way that maximum safety of the drives can be ensured by implementing or installing different types of modules designed with smart features. In this work, a similar kind of device has been discussed which can play a big role on the safety of the drivers which is able to detect the drowsiness of the drivers. The device can perform in a very systematic manner where a camera is responsible to capture the frontal image of the driver face. A detection algorithm has been used to detect the faces and predictor algorithm is used to achieve predicted outcome. The major task behind the work is to measure the aspect ratio of the eyes through the predictor landmark and according the selecting the particular threshold value and conditions, the system is able to detect the open and close status of the eyes. With this system, a safe and reliable environment can be provided for the drivers where a proper alert can save lives. The Drowsiness Detection System for drivers utilizing image processing techniques holds future potential for application in image processing-based monitoring of drowsiness during drug-induced sleep studies.

Future Enhancement

Some drivers cover their face while yawning or they have different signs of sleepiness like eye closure or falling head; in this case, future work may consist of combining the detection of different fatigue signs. When the driver wears glasses the system may not detect eyes which are the most noteworthy disadvantage of these systems. This issue has not yet been settled in near future it can be sorted out.

Author Contributions

GC contributed to the study conception and design, collected the data, performed the analysis and interpretation of results; UB and JD drafted the manuscript. All authors have reviewed the results and approved the final version of the manuscript.

Acknowledgement

The Department of Machine learning and Artificial intelligence and The Pharmacology Department of the Lovely Professional University in Punjab, India, is acknowledged by the author for its assistance.

Ethics Approval Statement

This research paper adheres to ethical research standards. All sources are properly cited to acknowledge the original authors’ contributions. No data fabrication, falsification, or plagiarism has been involved in the preparation of this manuscript. The authors have ensured that the work is original and does not infringe on any existing copyrights or intellectual property rights.

References

1. Galarza, E. E., Egas, F. D., Silva, F. M., Velasco, P. M., & Galarza, E. D. (2018). Real time driver drowsiness detection based on driver’s face image behavior using a system of human computer interaction implemented in a smartphone. In Proceedings of the International Conference on Information Technology & Systems (ICITS 2018) (pp. 563-572). Springer International Publishing.
2. Jain, M., Bhagerathi, B., & Sowmyarani, C. N. (2021). Real-time driver drowsiness detection using computer vision. International Journal of Engineering and Advanced Technology, 11(1), 109-113.
3. DybaÅ?a, J., Czapski, P., Skalski, P. (2017). Driver Drowsiness Detection Systems. Researchgate.
4. E. Rogado, J. L. Garcia, R. Barea, L. M. Bergasa and E. Lopez, "Driver fatigue detection system," 2008 IEEE International Conference on Robotics and Biomimetics, Bangkok, Thailand, 2009, pp. 1105-1110.
5. Poursadeghiyan, M., Mazloumi, A., Saraji, G. N., Baneshi, M. M., Khammar, A., & Ebrahimi, M. H. (2018). Using image processing in the proposed drowsiness detection system design. Iranian journal of public health, 47(9), 1371.
6. Albadawi, Y., Takruri, M., & Awad, M. (2022). A review of recent developments in driver drowsiness detection systems. Sensors, 22(5), 2069.
7. Biswal, A. K., Singh, D., Pattanayak, B. K., Samanta, D., & Yang, M. H. (2021). IoT�based smart alert system for drowsy driver detection. Wireless communications and mobile computing, 2021(1), 6627217.
8. Navya Kiran, V. B., Raksha, R., Rahman, A., Varsha, K. N., & Nagamani, N. P. (2020). Driver drowsiness detection. International Journal of Engineering Research & Technology (IJERT).
9. Josephin, J. F., Lakshmi, C., & James, S. J. (2020, December). A review on the measures and techniques adapted for the detection of driver drowsiness. In IOP conference series: materials science and engineering (Vol. 993, No. 1, p. 012101). IOP Publishing.
10. Mehta, S., Dadhich, S., Gumber, S., & Jadhav Bhatt, A. (2019, February). Real-time driver drowsiness detection system using eye aspect ratio and eye closure ratio. In Proceedings of international conference on sustainable computing in science, technology and management (SUSCOM), Amity University Rajasthan, Jaipur-India.
11. Magán, E., Sesmero, M. P., Alonso-Weber, J. M., & Sanchis, A. (2022). Driver drowsiness detection by applying deep learning techniques to sequences of images. Applied Sciences, 12(3), 1145.
12. Jansi, R., Anand, R., & Chandrasekhar, A. R. (2020). Drowsiness Detection System using Machine Learning. IJIRT, 7(1).
13. Bhogavkar, A., Badarkhe, S., Samage, P., & Bhavankar, S. (2023). Driver drowsiness detection system. Journal of Emerging Technologies and Innovative Research