Biometrics-based Recognition System on Wearable Devices Weitao Xu

Biometrics-based Recognition System on Wearable Devices Weitao Xu

Biometrics-based Recognition System on Wearable Devices Weitao Xu University of Queensland Supervisor: Prof. Neil Bergmann Dr. Wen Hu Outline

Introduction Project 1-Face recognition Project 2-Key generation Project 3-Gait recognition Conclusion 2 of 51 Introduction What is biometrics? The users behavioral or physiological characteristics to determine or verify an identity.

Fingerprint Handwritten signature Face 3 of 51 Introduction Wearable devices Smart watch

Smart glass Smart wristband 4 of 51 Introduction Sensors Accelerometer Gyroscope Compass Camera Micphone

GPS Bluetooth Applications Navigation Localization

Health monitoring Tracking Remote control 5 of 51 Research projects Biometrics 1 2

Wearable device 3 face recognition on smartglass Key generation system for wearable devices Gait recognition system using energy harvester

6 of 51 Research project 1 Sensor-assisted face recognition system on smartglass 7 of 51 Background Face recognition Security

Friends Tagging Face Unlock Smart glasses: a third eye 8 of 51 Main Challenges Short battery life Challenges

Limited processing capability Depend on cloud architecture Possible Solution Wireless transmission is expensive Our goal Robust and efficient in-situ face recognition system on smart

glass 9 of 51 System Architecture Step 1: capture multi-view face images Step 2: sampling optimization MASO Step 3: face recognition Target MVSRC

User ID User MASO: Max Accuracy Sampling Optimization MVSRC: Multi-view Sparse Representation Classification 10 of 51 MVSRC MVSRC exploits the sparsity of face images from different

views and apply a weighted model to improve accuracy. 11 of 51 Max Accuracy Sampling Optimization Problem: applying MVSRC directly on a face image sequence is expensive Observation: adjacent frames are similar Solution: downsample a face image sequence What is a good down sampling strategy? 12 of 51

Max Accuracy Sampling Optimization We propose an optimization strategy based on machine learning. We relate the face recognition accuracy with angles of face images We use IMU sensors(accelerometer, gyroscope, magnetometer) to estimate the angles of the face images We use support vector regression (SVR) to downsample face images. Recognition process 13 of 51

Obtain Angle Information Target Final view angle final First view angle start User start .

1 User 2 k final 14 of 51 Evaluation Datasets

Honda/UCSD dataset: 20 subjects 59 image sequences Frames of each video clip vary from 12 to 645 Private dataset: 10 subjects (2 females and 8 males) 9 different combinations of expressions and environments 10.8K face images 15 of 51 Evaluation of MVSRC

Comparison with competing multi-view face recognition methods Private dataset Honda dataset MVSRC is 5%-10% more accurate than other methods 16 of 51 User study

Implementation: Vuzix M100 Smart glass 5 users, 10 subjects Different environments Comparison with OpenCV face recognition methods 7%-15% Statistic Our system Eigenfaces Computation

time 516-582 ms 247-331 ms Energy consumption 506-535 mJ 316-410 mJ Expected

battery life 0.28 hr 0.37 hr Memory Usage 55-64 MB 38-44 MB Same level

17 of 51 Demo 18 of 51 Research project 2 Key generation system for wearable devices 19 of 51 Background

Wearable devices smartphone, smart glass, smart watch etc. Wearable devices pairing Data exchange 20 of 51 Traditional Methods

Explicit Input Labor-intensive Small form factor 21 of 51 Traditional Methods Key Exchange Public key infrastructure Cannot distinguish legitimate (D-H protocol) devices and attackers

22 of 51 Traditional Methods Static key Vulnerable to attackers Key revocation 23 of 51 System Overview Walkie-Talkie: On-body Authentication

Automatic Secure Pairing Spontaneous Key Generation Devices on the same body measure the same gait signals 24 of 51 Model User model: only the devices on the same body can be paired together Adversarial model: Active imposter Passive imposter

25 of 51 Design details Signal Processing Alice REQ Key generation Secure Communication

RSP Bob Source Separation REQ Signal Alignment RSP

Data Collection Quantization Signal Processing Reconciliation Key Generation Privacy

Amplification Data Sharing 26 of 51 Signal processing- signal alignment Space alignment -Coordinate system of different system are not the same -We transform the acceleration signal to body reference system 27 of 51

Key Generation Gravity direction Acc Quantization Sideway direction Reconciliation Walking direction Privacy Amplification

Key Generation 28 of 51 Key Generation-quantization Segmentation Calculate boundaries Quantization K1,K2,K3

KAlice=[K1,K2,K3] 29 of 51 Key Generation-reconciliation Reconciliation: Error-correcting phase KAlice KBob KAlice=1101001 KBob=1001101 Steps: Exchange key indexes

LAlice=[1,3,5,6,7,9,10] LBob=[1,2,5,6,8,9,10] Reserve common indexes L=[1,5,6,9,10] Reserve key generated at same index KAlice=10101 KAlice=1101001 KBob=10101 KBob=1001101 30 of 51 Key Generation-privacy amplification Privacy amplification: Correlation exits as walking is repetitive

Reconciliation reveal information to attackers Steps: Segmentation K1Alice(30bits) K2Alice .. KAlice=11010011011001011. Perform XOR KAlice=[(K1Alice XOR K2Alice) ,(K3Alice XOR K4Alice),.]

31 of 51 Evaluation Dataset 20 subjects (14 males, 6 females) 32 of 51 Evaluation Impact of reconciliation

Reconciliation reduces bit rate, however it can improve bit agreement rate up to 100%. 33 of 51 Evaluation Security analysis Passive imposter Active imposter Mutual Information among different devices

An active imposter can only achieve approximately 50% bit agreement rate (=0.8)) 34 of 51 System Implementation Implementation Moto E2 smartphone Bluetooth Low Energy (BLE) mode AES encryption/decryption Computation Time (ms)

Energy Consumption (mJ) Signal processing 108.3 71.2 Key generation 208.1 12.7

AES encryption 0.2 0.1 AES decryption 0.2 0.1 Total

316.8 85.6 Account to 0.002 of the battery supply only 35 of 51 Demo 36 of 51

Research project 3 Gait recognition system for wearable devices 37 of 51 Background-gait recognition Gait recognition refers to identify an individual by his/her unique walking pattern. Mission: Impossible 38 of 51

Background-energy harvester Kinetic energy harvester (KEH): - generate power from kinetic motions like walking, running 39 of 51 Background-energy harvester AMPY-world's first wearable motion-charger SolePower-self-sustaining smart boot 40 of 51

Motivations Accelerometer-based gait recognition - continuous sampling consumes a lot of energy Our goal: - use generated voltage as source signal to realize gait recognition 41 of 51

System overview Traditional accelerometer-based gait recognition system: Proposed KEH-based gait recognition system: Accelerometer Save energy 42 of 51 Prototype design Piezoelectric energy harvester (PEH) Electromagnetic energy harvester (EEH)

43 of 51 Data collection Dataset 20 subjects (14 males, 6 females) Indoor, Outdoor 44 of 51 Feasibility study 45 of 51 Evaluation results

Comparison with accelerometer-based system KEH-Gait can achieve comparable recognition accuracy when we use more than 5 gait cycles. 46 of 51 Energy consumption analysis Our system can reduce energy consumption by 75.8)8)%. 47 of 51 Demo 48 of 51 Conclusion We implement three biometrics-based recognition system on wearable devices We conduct extensive evaluations to evaluate their performance and security.

We implement the systems on commercial devices to demonstrate the feasibility. With the ubiquity of wearable devices, more security and privacy issues will arise. Future work will focus on addressing these challenges on emerging wearable devices. 49 of 51

References 1. Weitao Xu, Yiran Shen, Neil Bergmann,Wen Hu. Sensor-assisted Face Recognition System on Smart Glass via Multi-view Sparse Representation Classification IPSN 2016 (CORE A*) 2. Weitao Xu, Girish Revadigar, Chengwen Luo, Neil Bergmann,Wen Hu.Walkie-Talkie: Motion-Assisted Automatic Key Generation for Secure On-Body Device Communication IPSN 2016 Best paper runner-up award (CORE A*) 3. Weitao Xu, Guohao Lan, Qi Lin, Sara Khalifa, Neil Bergmann, Mahbub Hassan, Wen Hu. KEH-Gait: Towards a Mobile Healthcare User Authentication System by Kinetic Energy Harvesting conditionally accepted by NDSS' 2017 (CORE A) 50 of 51

Thanks for your attention! Questions and suggestions? 51 of 51

Recently Viewed Presentations

  • Dissociative and Somatic Symptom Disorders

    Dissociative and Somatic Symptom Disorders

    Illness Anxiety disorder. Formerly called Hypocondriasis. Illness Anxiety Disorder is a somatic symptom disorder characterized by the misinterpretation of normal bodily functions as signs of serious illness. People with this disorder fear or mistakenly believe that normal bodily reactions represent...
  • All-terrain Vehicle (Atv) Safety

    All-terrain Vehicle (Atv) Safety

    PREVENTION WHAT IS IT? June 2008 What is Prevention? Prevention is the act of impeding, or preventing, something from happening such as disease or a social condition.
  • Chapter 15 Life of the Mesozoic Era Mesozoic

    Chapter 15 Life of the Mesozoic Era Mesozoic

    and the fact that many cephalopods were nektonic . make them excellent guide fossils. Mollusks. The Ammonoidea, cephalopods with wrinkled sutures, constitute three groups: the goniatites, ceratites, and ammonites. Ammonites, while present during the entire Mesozoic,
  • BIO 342 HUMAN PHYSIOLOGY - Wofford College

    BIO 342 HUMAN PHYSIOLOGY - Wofford College

    Tropic hormones control the function Trophic hormones promote survival and growth of targets Releasing Hormone Release-inhibiting Hormone Hypothalamo-hypophyseal portal system A A S S P P P P P P P P P P P S 1 Tropic hormones control...


    Although evidence for non-pharmacological interventions was not reviewed, harms of such approaches are minimal ... Parenteral availability of haloperidol may still warrant short-term use in emergencies. ... excretion, interactions with other medications longer time to reach steady state levels ...
  • Kinematics in 1-D - Mrs. Alinger's Science Pages

    Kinematics in 1-D - Mrs. Alinger's Science Pages

    Kinematics in 1-D. Learning Target. I can differentiate between position, distance, displacement, speed, and velocity. What is motion? The mathematical description of motion is called kinematics. The explanation of motion in terms of its causes is called .
  • Judaism - MR. DORAN'S WEBSITE - Home

    Judaism - MR. DORAN'S WEBSITE - Home

    Unquestionably monotheistic—this is why Judaism, despite its size, is so important to world religions ... This begins the notion of prophecy which is central to all Abrahamic religions. ... illustrated values of Judaism, and importance to the Jewish faith—think: if...
  • Atoms, Molecules and Ions - Berry College

    Atoms, Molecules and Ions - Berry College

    * VI. Proton, Neutron and Electron Amu = 1/12 the mass of a carbon atom containing 6 protons and 6 neutrons. This makes the proton ~1amu. The proton and electron have electrical charge * A. Elements What makes elements unique...