EyeShield: Real-Time Protection of Mobile Device Screen Information from Shoulder Surfing

ABSTRACT

People use mobile devices ubiquitously for computing, communication, storage, web browsing, and more. As a result, the information accessed and stored within mobile devices, such as financial and health information, text messages, and emails, can often be sensitive. Despite this, people frequently use their mobile devices in public areas, becoming susceptible to a simple yet effective attack -- shoulder surfing. Shoulder surfing occurs when a person near a mobile user peeks at the user's mobile device, potentially acquiring passcodes, PINs, browsing behavior, or other personal information. We propose, EyeShield, a solution to prevent shoulder surfers from accessing/stealing sensitive on-screen information. EyeShield is designed to protect all types of on-screen information in real time, without any serious impediment to users' interactions with their mobile devices. EyeShield generates images that appear readable at close distances, but appear blurry or pixelated at farther distances and wider angles. It is capable of protecting on- screen information from shoulder surfers, operating in real time, and being minimally intrusive to the intended users. EyeShield protects images and text from shoulder surfers by reducing recognition rates to 24.24% and 15.91%. Our implementations of EyeShield achieved high frame rates for 1440 x 3088 screen resolutions (24 FPS for Android and 43 FPS for iOS). EyeShield also incurs acceptable memory usage, CPU utilization, and energy overhead. Finally, our MTurk and in-person user studies indicate that EyeShield protects on-screen information without a large usability cost for privacy-conscious users. This is joint work with Brian Tang and was presented at USENIX Security’23. 

BIOGRAPHY

KANG G. SHIN is the Kevin & Nancy O'Connor Professor of Computer Science in the Department of Electrical Engineering and Computer Science, The University of Michigan, Ann Arbor. His current research focuses on QoS-sensitive computing and networking as well as on embedded real-time and cyber-physical systems.

He has supervised the completion of 91 PhDs, and authored/coauthored close to 1,000 technical articles, a textbook and about 60 patents or invention disclosures, and received numerous awards, including 2023 IEEE TCCPS Technical Achievement Award, 2023 SIGMOBILE Test- of-Time Award, 2019 Caspar Bowden Award for Outstanding Research in Privacy Enhancing Technologies, and the Best Paper Awards from 2023 VehicleSec, 2011 ACM International Conference on Mobile Computing and Networking (MobiCom’11), the 2011 IEEE International Conference on Autonomic Computing, 2010 and 2000 USENIX Annual Technical Conferences, as well as the 2003 IEEE Communications Society William R. Bennett Prize Paper Award and the 1987 Outstanding IEEE Transactions of Automatic Control Paper Award. He has also received several institutional awards, including the Research Excellence Award in 1989, Outstanding Achievement Award in 1999, Distinguished Faculty Achievement Award in 2001, and Stephen Attwood Award in 2004 from The University of Michigan (the highest honor bestowed to Michigan Engineering faculty); a Distinguished Alumni Award of the College of Engineering, Seoul National University in 2002; 2003 IEEE RTC Technical Achievement Award; and 2006 Ho-Am Prize in Engineering (the highest honor bestowed to Korean-origin engineers).

He has chaired Michigan Computer Science and Engineering Division for 3 years starting 1991, and also several major conferences, including 2009 ACM MobiCom, and 2005 ACM/USENIX MobiSys. He was a co-founder of a couple of startups, licensed some of his technologies to industry, and served as an Executive Advisor for Samsung Research.