Muhammad Sarfraz - Advancements in Computer Vision Applications in Intelligent Systems and Multimedia Technologies

• Sanjida Nasreen Tumpa
  https://orcid.org/0000-0003-3024-4420
  University of Calgary, Canada
• K. N. Pavan Kumar
  https://orcid.org/0000-0002-7893-0588
  University of Calgary, Canada
• Madeena Sultana
  University of Calgary, Canada
• Gee-Sern Jison Hsu
  https://orcid.org/0000-0003-2631-0448
  National Taiwan University of Science and Technology, Taiwan
• Orly Yadid-Pecht
  University of Calgary, Canada
• Svetlana Yanushkevich
  https://orcid.org/0000-0003-4794-9849
  University of Calgary, Canada
• Marina L. Gavrilova
  University of Calgary, Canada

ABSTRACT

Smart societies of the future will increasingly rely on harvesting rich information generated by day-to-day activities and interactions of its inhabitants. Among the multitude of such interactions, web-based social networking activities became an integral part of everyday human communication. Flickr, Facebook, Twitter, and LinkedIn are currently used by millions of users worldwide as a source of information, which is growing exponentially over time. In addition to idiosyncratic personal characteristics, web-based social data include person-to-person communication, online activity patterns, and temporal information, among others. However, analysis of social interaction-based data has been studied from the perspective of person identification only recently. In this chapter, the authors elaborate on the concept of using interaction-based features from online social networking platforms as a part of social behavioral biometrics research domain. They place this research in the context of smart societies and discuss novel social biometric features and their potential use in various applications.

INTRODUCTION

The biometric authentication system can be referred to as an identification technique based on the physical, and behavioral attributes of an individual (Jain et al., 2008). Biometric systems use pattern recognition processes to identify a person by matching the feature vectors obtained from their physiological or behavioral characteristics (Jain et al., 2006).

Traditionally, two types of biometrics: physiological and behavioral, have been considered (Jain et al., 2008). Physiological biometrics relies on physical attributes of individuals, such as face, palm, fingerprint, iris, ear, etc. Behavioral biometrics are based on human activities dictated by persons behaviour, such as handwriting, gait, signature and voice, which also afford discriminability among individuals. Behavioral biometrics are more volatile to changes an individual may undergo through their life but have the advantages of being non-intrusive and cost effective over physiological biometrics (Yampolskiy & Govindaraju, 2008). It is gaining an increasing demand for person identification and verification purposes to reduce security threats, especially in the cyber worlds. Another application where behavioral biometrics show significant potential is in combination with smart devices. One of the obvious benefits is combining behavioral characteristics with the mobile devices, in order for example to keep track of an individuals health status (i.e. heart rate, body temperature), or to remind to change the posture during an exercise based on the gait. Huge benefits can be also obtained in using such information during an emergency response, while providing service for those seeking admission to the homeless centre, or in providing an assistance to an elderly person if a fall is detected by sensors in a smart building.

In a highly interconnected society, people spend a lot of time interacting with each other on social media or in online virtual worlds. Users leave their virtual footprints expressed as shared thoughts, writing style, interaction patterns, in online communities, virtual worlds and as recorded by smart devices. It is possible to observe repetitive patterns in a user's social interactions via online since they are driven by human behaviours and habits. A persons social connections can be obtained by analysing whom the user most frequently and consistently communicates over time.

In this chapter, we first classify the existing behavioral biometrics into two main categories: machine-independent human behaviour and machine-dependent human behaviour. Machine independent behavioral biometrics are well established in the community, with most of the early works falling under this category. For example, person identification from his/her unique signature, gait or walking style, or from a piece of drawing are being studied over last decade (Monwar & Gavrilova, 2013; Bazazian & Gavrilova, 2015; Al-Zubi et al., 2003). The latter category is emerging compared to the former and more demanding field of research due to increasing number of cybersecurity threats. Due to the wide range of applications of computing devices and software, users can interact with computers and each other in many ways.

We classify machine dependent human behaviours further into the following three sub-categories based on the type of communication with the machine: interaction-based, skill-based, and intelligence-based. First subcategory is based on human interaction with input device that does not consider any knowledge or intelligence of the user. Some examples are keystroke dynamics, mouse dynamics, touchscreen interaction, etc. (Bakelman et al., 2013; Jorgensen & Yu, 2011; Bo et al., 2013; Frank et al., 2012). Second subcategory takes into account the style or preference of a user during interaction with the computer. Coding style of a programmer, browsing style or handshaking style would fall under this category (Spafford & Weeber, 1993; Olejnik & Castelluccia, 2013; Guo et al., 2013). Finally, the third subdivision includes human intelligence, knowledge, and skills into account during interaction with software such as game playing strategy, car driving skill, hobby or habits, for instance (Yampolskiy & Govindaraju, 2010; Igarashi et al.,2004; Jiang et al., 2013).

Human activities that involve machine or sensor interaction are not limited to programming, computing, gaming or typing. Now, in the era of social networking and internet-of-things, human identity as well as everyday activities have been naturally extended into web-based and sensor-based dimensions (Eastwood et al., 2015; Dattner & Yadid-Pecht, 2011). Our first hypothesis is that, similarly to a physical world, behavioral patterns and habits are present in the daily activities of virtual world users. Patterns can be found in different online Social Networking Service (SNS) activities such as: tweets, status updates, likes, URLs, photo and video tagging, uploads, comments, communications and so on. Our second hypothesis is that a person can be identified based on his activities and information which accumulates through online social networking platforms. Therefore, in this chapter we are introducing the fourth subcategory of machine dependent behavioral biometrics: web-based social biometrics . We also investigate the feasibility of using users web-based social networking activities as novel behavioral biometrics to identify a person in virtual domain.

In this chapter, we introduce web-based social and sensor data as behavioral biometric features . The concept of how these features can be used for person identification will also be analyzed. Some existing works based on relevant web-based social data are reviewed in the background study. Some state-of-the-art web-based biometric features are analyzed thereafter to understand the significance of the novel biometric. The key challenges and applications are also identified to discover the potential directions for the future research. This chapter is an updated version of the book chapter Online User Interaction Traits in Web-based Social Biometrics, that appeared in Computer Vision, Image Processing, Intelligent Systems Multimedia Technologies book in 2014 (Sultana et al., 2014c).