Using Fingerprint Sensor as a Touchpad on Mobile Devices

Nishant Arora 16/May/2017

I have been away for a while, many of you know I started with my Masters at UofT last fall and since then I have not had time to update my blog. Right now I started with my internship and my daily routine is falling into place. I now have some time available to write about things I did in the last eight months.

This is one of the most interesting projects I did in the Spring Term at UofT. The inspiration for this project was the 9GAG android app. Everyone who knows me, knows that I am an avid 9gag reader and I often browse through memes before dozing off in my bed. I can't be the only one to have dropped the phone on my face while using the phone in the bed. Those who are unaware about this problem, this is what it looks like (this is the actual project proposal):

9GAG solved this problem by allowing navigation through the posts using the volume rocker buttons. This was a gamechanger for me, I could now browse 9GAG peacefully in bed (unfortunately the latest release of the app has ditched this feature). The problem of the phone falling turns out to be because of two main reasons, first being our tendency to use the phone unimanually (single-handedly) and the second being the muscle fatigue caused in the hand due to unimanual use of the phone. Since we primarily use our thumbs to interact with the front facing touchscreen, the hand grip gives way for the phone to fall.

So, what can be an alternative to this problem? I suggest we start interacting with the back of the device, where fortunately our index finger is already comfortably placed. Instead of building a new hardware to study this problem, I set out to convert the rear fingerprint sensor on my LG G5 into a touchpad. This enables us to perform various gestures like tapping and swiping which in turn activate actions like scrolling, zooming and clicking in the phone UI.

I developed two dummy android apps to perform the study:

This simply loads a webpage and allows scrolling via gestures on the fingerprint scanner. Swiping scrolls down, Tapping scrolls up.
Ideally it's a clone of instagram, but loads data locally (it's WIP). Each swipe loads the next picture and each tap likes the picture.

The aim was to develop a MVC, perform a study on users and quantify how this new input modality helped users. The results were interesting, read the full study here: Alternative Mobile Input Methods via Onboard Hardware a Usability Study

Here is my presentation for this study during the ECE1780: Advanced Mobile User Interfaces class:

I am thankful to my ECE1780 professor, Prof. Parham Aarabi. He was really supportive about pursuing this wild idea of using the fingerprint sensor as a touchpad.

Dr. David Holman my professor for another course I studied (Human Computer Interaction) helped me understand the design of a usability study and proof-read my drafts. Thank You!

Overall this was fun, I loved it. Feel free to use the code I posted on GitHub and PRs are welcome.

Cheers :)