Home About the Journal Latest Work Current Issue Archive Special Issues Editorial Board
<< Previous Next >>

2022, 4(3): 210-222

Published Date:2022-6-20 DOI: 10.1016/j.vrih.2022.01.007

Privacy-preserving deep learning techniques for wearable sensor-based big data applications

Abstract

Background
Wearable technologies have the potential to become a valuable influence on human daily life where they may enable observing the world in new ways, including, for example, using augmented reality (AR) applications. Wearable technology uses electronic devices that may be carried as accessories, clothes, or even embedded in the user's body. Although the potential benefits of smart wearables are numerous, their extensive and continual usage creates several privacy concerns and tricky information security challenges.
Methods
In this paper, we present a comprehensive survey of recent privacy-preserving big data analytics applications based on wearable sensors. We highlight the fundamental features of security and privacy for wearable device applications. Then, we examine the utilization of deep learning algorithms with cryptography and determine their usability for wearable sensors. We also present a case study on privacy-preserving machine learning techniques. Herein, we theoretically and empirically evaluate the privacy-preserving deep learning framework's performance. We explain the implementation details of a case study of a secure prediction service using the convolutional neural network (CNN) model and the Cheon-Kim-Kim-Song (CHKS) homomorphic encryption algorithm. Finally, we explore the obstacles and gaps in the deployment of practical real-world applications.
Results
Following a comprehensive overview, we identify the most important obstacles that must be overcome and discuss some interesting future research directions.

Keyword

Wearable technology ; Augmented reality ; Privacy-preserving ; Deep learning ; Big data ; Secure prediction service

Cite this article

Rafik HAMZA, Minh-Son DAO. Privacy-preserving deep learning techniques for wearable sensor-based big data applications. Virtual Reality & Intelligent Hardware, 2022, 4(3): 210-222 DOI:10.1016/j.vrih.2022.01.007

References

1. Onday O. Japan's society 5.0: Going beyond industry 4.0. Business and Economics Journal, 2019, 10(2):1–6 DOI:10.4172/2151-6219.1000389

2. Hamza R, Zettsu K. Investigation on privacy-preserving techniques for personal data. ICDAR'21: Proceedings of the 2021 Workshop on Intelligent Cross-Data Analysis and Retrieval. 2021, 62–66 DOI:10.1145/3463944.3469267

3. Gahi Y, Guennoun M, Mouftah H T. Big Data Analytics: security and privacy challenges. In: 2016 IEEE Symposium on Computers and Communication. Messina, Italy, IEEE, 2016, 952–957 DOI:10.1109/iscc.2016.7543859

4. Hamza R, Hassan A, Huang T, Ke L S, Yan H Y. An efficient cryptosystem for video surveillance in the Internet of Things environment. Complexity, 2019, 1625678 DOI:10.1155/2019/1625678

5. Jia B, Zhang X S, Liu J W, Zhang Y, Huang K, Liang Y Q. Blockchain-enabled federated learning data protection aggregation scheme with differential privacy and homomorphic encryption in IIoT. IEEE Transactions on Industrial Informatics, 5960, PP(99): 1 DOI:10.1109/tii.2021.3085960

6. Patil A S, Hamza R, Hassan A, Jiang N, Yan H Y, Li J. Efficient privacy-preserving authentication protocol using PUFs with blockchain smart contracts. Computers & Security, 2020, 97: 101958 DOI:10.1016/j.cose.2020.101958

7. Rafique A, van Landuyt D, Heydari Beni E, Lagaisse B, Joosen W. CryptDICE: Distributed data protection system for secure cloud data storage and computation. Information Systems, 2021, 96: 101671 DOI:10.1016/j.is.2020.101671

8. Dahl M, Mancuso J, Dupis Y, Decoste B, Giraud M, Livingstone I, Patriquin J, Uhma G. Private machine learning in tensorflow using secure computation. 2018

9. Wang M H, Zhu T Q, Zhang T, Zhang J, Yu S, Zhou W L. Security and privacy in 6G networks: new areas and new challenges. Digital Communications and Networks, 2020, 6(3): 281–291 DOI:10.1016/j.dcan.2020.07.003

10. Lebeck K, Ruth K, Kohno T, Roesner F. Towards security and privacy for multi-user augmented reality: foundations with end users. In: 2018 IEEE Symposium on Security and Privacy. San Francisco, CA, USA, IEEE, 2018, 392–408 DOI:10.1109/sp.2018.00051

11. Chen D W, Xie L J, Kim B, Wang L, Hong C S, Wang L C, Han Z. Federated learning based mobile edge computing for augmented reality applications. In: 2020 International Conference on Computing, Networking and Communications (ICNC). Big Island, HI, USA, IEEE, 2020,767–773 DOI:10.1109/icnc47757.2020.9049708

12. Syamimi A, Gong Y W, Liew R. VR industrial applications―A Singapore perspective. Virtual Reality & Intelligent Hardware, 2020, 2(5): 409–420 DOI:10.1016/j.vrih.2020.06.001

13. Zheng L Y, Liu X, An Z W, Li S F, Zhang R J. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection. Virtual Reality & Intelligent Hardware, 2020, 2(1): 12–27 DOI:10.1016/j.vrih.2019.12.002

14. González F C J, Villegas O O V, Ramírez D E T, Sánchez V G C, Domínguez H O. Smart multi-level tool for remote patient monitoring based on a wireless sensor network and mobile augmented reality. Sensors (Basel, Switzerland), 2014, 14(9): 17212–17234 DOI:10.3390/s140917212

15. Li Y, Zheng L, Wang X W. Flexible and wearable healthcare sensors for visual reality health-monitoring. Virtual Reality & Intelligent Hardware, 2019, 1(4): 411–427 DOI:10.1016/j.vrih.2019.08.001

16. Yin J H, Chng C B, Wong P M, Ho N, Chua M, Chui C K. VR and AR in human performance research―An NUS experience. Virtual Reality & Intelligent Hardware, 2020, 2(5): 381–393 DOI:10.1016/j.vrih.2020.07.009

17. Kim H, Kwon Y T, Lim H R, Kim J H, Kim Y S, Yeo W H. Recent advances in wearable sensors and integrated functional devices for virtual and augmented reality applications. Advanced Functional Materials, 2021, 31(39): 2005692 DOI:10.1002/adfm.202005692

18. Gulhane A, Vyas A, Mitra R, Oruche R, Hoefer G, Valluripally S, Calyam P, Hoque K A. Security, privacy and safety risk assessment for virtual reality learning environment applications. In: 2019 16th IEEE Annual Consumer Communications & Networking Conference. Las Vegas, NV, USA, IEEE, 2019,1–9 DOI:10.1109/ccnc.2019.8651847

19. Fun T S, Samsudin A. A survey of homomorphic encryption for outsourced big data computation. KSII Transactions on Internet and Information Systems, 2016, 10(8): 3826–3851 DOI:10.3837/tiis.2016.08.022

20. Gao W C, Yu W, Liang F, Hatcher W G, Lu C. Privacy-preserving auction for big data trading using homomorphic encryption. IEEE Transactions on Network Science and Engineering, 2020, 7(2): 776–791 DOI:10.1109/tnse.2018.2846736

21. Wang D, Guo B, Shen Y, Cheng S J, Lin Y H. A faster fully homomorphic encryption scheme in big data. In: 2017 IEEE 2nd International Conference on Big Data Analysis. Beijing, China, IEEE, 2017, 345–349 DOI:10.1109/icbda.2017.8078836

22. Aono Y, Hayashi T, Phong L T, Wang L H. Privacy-preserving logistic regression with distributed data sources via homomorphic encryption. IEICE Transactions on Information and Systems, 2016, E99.D(8): 2079–2089 DOI:10.1587/transinf.2015inp0020

23. Esperanca P, Aslett L, Holmes C. Encrypted accelerated least squares regression. Artificial Intelligence and Statistics. 2017

24. Yonetani R, Boddeti V N, Kitani K M, Sato Y. Privacy-preserving visual learning using doubly permuted homomorphic encryption. In: 2017 IEEE International Conference on Computer Vision. Venice, Italy, IEEE, 2017, 2059–2069 DOI:10.1109/iccv.2017.225

25. Fang H K, Qian Q. Privacy preserving machine learning with homomorphic encryption and federated learning. Future Internet, 2021, 13(4): 94 DOI:10.3390/fi13040094

26. Halevi S. Homomorphic encryption. In Tutorials on the Foundations of Cryptography. Springer, Cham, 2017, 219–276 DOI:10.1007/978-3-319-57048-8_5

27. Yagoub M A, Laouid A, Kazar O, Bounceur A, Euler R, AlShaikh M. An adaptive and efficient fully homomorphic encryption technique. ICFNDS'18: Proceedings of the 2nd International Conference on Future Networks and Distributed Systems. 2018, 1–6 DOI:10.1145/3231053.3231088

28. Yan X Y, Wu Q L, Sun Y M. A homomorphic encryption and privacy protection method based on blockchain and edge computing. Wireless Communications and Mobile Computing, 2020, 8832341 DOI:10.1155/2020/8832341

29. Iezzi M. Practical privacy-preserving data science with homomorphic encryption: an overview. In: 2020 IEEE International Conference on Big Data (Big Data). Atlanta, GA, USA, IEEE, 2020, 3979–3988 DOI:10.1109/bigdata50022.2020.9377989

30. Pramanik M I, Lau R Y K, Hossain M S, Rahoman M M, Debnath S K, Rashed M G, Uddin M Z. Privacy preserving big data analytics: a critical analysis of state-of-the-art. WIREs Data Mining and Knowledge Discovery, 2021, 11(1): e1387 DOI:10.1002/widm.1387

31. Tran H Y, Hu J K. Privacy-preserving big data analytics a comprehensive survey. Journal of Parallel and Distributed Computing, 2019, 134: 207–218 DOI:10.1016/j.jpdc.2019.08.007

32. Vijaya K A, Sujith M S, Sai K T, Rajesh G, Yashwanth D J S. Secure Multiparty computation enabled E-Healthcare system with Homomorphic encryption. IOP Conference Series: Materials Science and Engineering, 2020, 981(2): 022079 DOI:10.1088/1757-899x/981/2/022079

33. Li D, Liao X F, Xiang T, Wu J H, Le J Q. Privacy-preserving self-serviced medical diagnosis scheme based on secure multi-party computation. Computers & Security, 2020, 90: 101701 DOI:10.1016/j.cose.2019.101701

34. Hesamifard E, Takabi H, Ghasemi M. Deep neural networks classification over encrypted data. CODASPY'19: Proceedings of the Ninth ACM Conference on Data and Application Security and Privacy. 2019, 97–108 DOI:10.1145/3292006.3300044

35. Podschwadt R, Takabi D. Classification of encrypted word embeddings using recurrent neural networks. 2020

36. Graepel T, Lauter K, Naehrig M. ML confidential: Machine learning on encrypted data. In International Conference on Information Security and Cryptology. Springer, Berlin, Heidelberg. 2012, 1–21 DOI:10.1007/978-3-642-37682-5_1

37. Li P, Li J, Huang Z G, Li T, Gao C Z, Yiu S M, Chen K. Multi-key privacy-preserving deep learning in cloud computing. Future Generation Computer Systems, 2017, 74: 76–85 DOI:10.1016/j.future.2017.02.006

38. Bost R, Popa R A, Tu S, Goldwasser S. Machine learning classification over encrypted data. In: Proceedings 2015 Network and Distributed System Security Symposium. San Diego, CA, Reston, VA: Internet Society, 2015 DOI:10.14722/ndss.2015.23241

39. Takabi D, Podschwadt R, Druce J, Wu C, Procopio K. Privacy preserving neural network inference on encrypted data with GPUs. 2019

40. Badawi A A, Hoang L, Mun C F, Laine K, Aung K M M. PrivFT: private and fast text classification with homomorphic encryption. IEEE Access, 2020, 8: 226544–226556 DOI:10.1109/access.2020.3045465

41. Jung W, Kim S, Ahn J H, Cheon J H, Lee Y. Over 100x faster bootstrapping in fully homomorphic encryption through memory-centric optimization with GPUs. IACR Transactions on Cryptographic Hardware and Embedded Systems, 2021, 114–148 DOI:10.46586/tches.v2021.i4.114-148

42. Hesamifard E, Takabi H, Ghasemi M. Cryptodl: Deep neural networks over encrypted data. 2017

43. Zhao Z P, Bao Z T, Zhang Z X, Cummins N, Sun S H, Wang H S, Tao J H, Schuller B W. Self-attention transfer networks for speech emotion recognition. Virtual Reality & Intelligent Hardware, 2021, 3(1): 43–54 DOI:10.1016/j.vrih.2020.12.002

44. Li M C, An L, Yu T, Wang Y G, Chen F, Liu Y B. Neural hand reconstruction using a single RGB image. Virtual Reality & Intelligent Hardware, 2020, 2(3): 276–289 DOI:10.1016/j.vrih.2020.05.001

45. Mishra P, Lehmkuhl R, Srinivasan A, Zheng W, Popa R A. Delphi: A cryptographic inference service for neural networks. In 29th Security Symposium Security. 2020, 2505–2522

46. Sarmah S S. An efficient IoT-based patient monitoring and heart disease prediction system using deep learning modified neural network. IEEE Access, 2020, 8: 135784–135797 DOI:10.1109/access.2020.3007561

47. Ge C P, Yin C C, Liu Z, Fang L M, Zhu J C, Ling H D. A privacy preserve big data analysis system for wearable wireless sensor network. Computers & Security, 2020, 96: 101887 DOI:10.1016/j.cose.2020.101887

48. Xu R H, Joshi J B D, Li C. CryptoNN: training neural networks over encrypted data. In: 2019 IEEE 39th International Conference on Distributed Computing Systems. Dallas, TX, USA, IEEE, 2019, 1199–1209 DOI:10.1109/icdcs.2019.00121

49. Abdalla M, Bourse F, De Caro A, Pointcheval D. Simple Functional Encryption Schemes for Inner Products. Berlin, Heidelberg, Springer Berlin Heidelberg, 2015, 733–751 DOI:10.1007/978-3-662-46447-2_33

50. Gilad-Bachrach R, Dowlin N, Laine K, Lauter K, Naehrig M, Wernsing J. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, Edited by Maria Florina B, Kilian Q W. PMLR 2016, 201–210

51. Bos J W, Lauter K E, Loftus J, Naehrig M. Improved security for a ring-based fully homomorphic encryption scheme. IACR Cryptology EPrint Archive, 2013, 75

52. Halevi S, Shoup V. Algorithms in HElib. Berlin, Heidelberg, Springer Berlin Heidelberg, 2014, 554–571 DOI:10.1007/978-3-662-44371-2_31

53. El Saj R, Sedgh Gooya E, Alfalou A, Khalil M. Privacy-preserving deep neural network methods: computational and perceptual methods—an overview. Electronics, 2021, 10(11): 1367 DOI:10.3390/electronics10111367

54. Alkhelaiwi M, Boulila W, Ahmad J, Koubaa A, Driss M. An efficient approach based on privacy-preserving deep learning for satellite image classification. Remote Sensing, 2021, 13(11): 2221 DOI:10.3390/rs13112221

55. Zhang Y S, Xiao X L, Yang L X, Xiang Y, Zhong S. Secure and efficient outsourcing of PCA-based face recognition. IEEE Transactions on Information Forensics and Security, 2020, 15: 1683–1695 DOI:10.1109/tifs.2019.2947872

Related

1. Zike YAN, Hongbin ZHA, Flow-based SLAM: From geometry computation to learning Virtual Reality & Intelligent Hardware 2019, 1(5): 435-460

2. Hayat ULLAH, Sitara AFZAL, Imran Ullah KHAN, Perceptual quality assessment of panoramic stitched contents for immersive applications: a prospective survey Virtual Reality & Intelligent Hardware 2022, 4(3): 223-246

3. Lianyu ZHENG, Xinyu LIU, Zewu AN, Shufei LI, Renjie ZHANG, A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection Virtual Reality & Intelligent Hardware 2020, 2(1): 12-27

4. Zhiyuan ZHANG, Yuchao DAI, Jiadai SUN, Deep learning based point cloud registration: an overview Virtual Reality & Intelligent Hardware 2020, 2(3): 222-246

5. Muhammad IRFAN, Muhammad MUNSIF, Deepdive: a learning-based approach for virtual camera in immersive contents Virtual Reality & Intelligent Hardware 2022, 4(3): 247-262

6. Yuanyuan SHI, Yunan LI, Xiaolong FU, Kaibin MIAO, Qiguang MIAO, Review of dynamic gesture recognition Virtual Reality & Intelligent Hardware 2021, 3(3): 183-206

7. Xiaojiao SONG, Jianjun ZHU, Jingfan FAN, Danni AI, Jian YANG, Topological distance-constrained feature descriptor learning model for vessel matching in coronary angiographies Virtual Reality & Intelligent Hardware 2021, 3(4): 287-301

8. Wei LYU, Zhong ZHOU, Lang CHEN, Yi ZHOU, A survey on image and video stitching Virtual Reality & Intelligent Hardware 2019, 1(1): 55-83

9. Mengting XIAO, Zhiquan FENG, Xiaohui YANG, Tao XU, Qingbei GUO, Multimodal interaction design and application in augmented reality for chemical experiment Virtual Reality & Intelligent Hardware 2020, 2(4): 291-304

10. Mingxuan CHEN, Ping ZHANG, Zebo WU, Xiaodan CHEN, A multichannel human-swarm robot interaction system in augmented reality Virtual Reality & Intelligent Hardware 2020, 2(6): 518-533

11. Yonghang TAI, Junsheng SHI, Junjun PAN, Aimin HAO, Victor CHANG, Augmented reality-based visual-haptic modeling for thoracoscopic surgery training systems Virtual Reality & Intelligent Hardware 2021, 3(4): 274-286

12. Yukang YAN, Xin YI, Chun YU, Yuanchun SHI, Gesture-based target acquisition in virtual and augmented reality Virtual Reality & Intelligent Hardware 2019, 1(3): 276-289

13. Yuan GAO, Le XIE, A review on the application of augmented reality in craniomaxillofacial surgery Virtual Reality & Intelligent Hardware 2019, 1(1): 113-120

14. Jinyu LI, Bangbang YANG, Danpeng CHEN, Nan WANG, Guofeng ZHANG, Hujun BAO, Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality Virtual Reality & Intelligent Hardware 2019, 1(4): 386-410

15. Xiaomei ZHAO, Fulin TANG, Yihong WU, Real-time human segmentation by BowtieNet and a SLAM-based human AR system Virtual Reality & Intelligent Hardware 2019, 1(5): 511-524

16. Chan QIU, Shien ZHOU, Zhenyu LIU, Qi GAO, Jianrong TAN, Digital assembly technology based on augmented reality and digital twins: a review Virtual Reality & Intelligent Hardware 2019, 1(6): 597-610

17. Wang LI, Junfeng WANG, Sichen JIAO, Meng WANG, Shiqi LI, Research on the visual elements of augmented reality assembly processes Virtual Reality & Intelligent Hardware 2019, 1(6): 622-634

18. Pengfei HAN, Gang ZHAO, A review of edge-based 3D tracking of rigid objects Virtual Reality & Intelligent Hardware 2019, 1(6): 580-596

19. Shenze WANG, Kaikai DU, Ningfang SONG, Dongfeng ZHAO, Di FENG, Zhengqian TU, Study on the adaptability of augmented reality smartglasses for astigmatism based on holographic waveguide grating Virtual Reality & Intelligent Hardware 2020, 2(1): 79-85

20. Lingfei ZHU, Qi CAO, Yiyu CAI, Development of augmented reality serious games with a vibrotactile feedback jacket Virtual Reality & Intelligent Hardware 2020, 2(5): 454-470

21. Xiang ZHOU, Liyu TANG, Ding LIN, Wei HAN, Virtual & augmented reality for biological microscope in experiment education Virtual Reality & Intelligent Hardware 2020, 2(4): 316-329

22. Jie REN, Chun YU, Yueting WENG, Chengchi ZHOU, Yuanchun SHI, Design and evaluation of window management operations in AR headset+smartphone interface Virtual Reality & Intelligent Hardware 2022, 4(2): 115-131

23. Yun-Han LEE, Tao ZHAN, Shin-Tson WU, Prospects and challenges in augmented reality displays Virtual Reality & Intelligent Hardware 2019, 1(1): 10-20

24. Mohib ULLAH, Sareer Ul AMIN, Muhammad MUNSIF, Muhammad Mudassar YAMIN, Utkurbek SAFAEV, Habib KHAN, Salman KHAN, Habib ULLAH, Serious games in science education: a systematic literature review Virtual Reality & Intelligent Hardware 2022, 4(3): 189-209

25. Ahmed L ALYOUSIFY, Ramadhan J MSTAFA, AR-assisted children book for smart teaching and learning of Turkish alphabets Virtual Reality & Intelligent Hardware 2022, 4(3): 263-277