A facial recognition system is a technology capable of identifying or verifying a person from a digital image or a video frame from a videosource. There are multiple methods in which facial recognition systems work, but in general, they work by comparing selected facial features from given image with faces within a database. It is also described as a Biometric Artificial Intelligence based application that can uniquely identify a person by analysing patterns based on the person's facial textures and shape.[1]
While initially a form of computer application, it has seen wider uses in recent times on mobile platforms and in other forms of technology, such as robotics. It is typically used as access control in security systems and can be compared to other biometrics such as fingerprint or eye iris recognition systems.[2] Although the accuracy of facial recognition system as a biometric technology is lower than iris recognition and fingerprint recognition, it is widely adopted due to its contactless and non-invasive process.[3] Recently, it has also become popular as a commercial identification and marketing tool.[4] Other applications include advanced human-computer interaction, video surveillance, automatic indexing of images, and video database, among others.[5]
History of facial recognition technology
During 1964 and 1965, Bledsoe, along with Helen Chan and Charles Bisson, worked on using the computer to recognize human faces (Bledsoe 1966a, 1966b; Bledsoe and Chan 1965). He was proud of this work, but because the funding was provided by an unnamed intelligence agency that did not allow much publicity, little of the work was published.[6]Based on the available references, it was revealed that the Bledsoe's initial approach involved the manual marketing of various landmarks on the face such as the eye centers, mouth, etc., and these were mathematically rotated by computer to compensate for pose variation.[6] The distances between landmarks were also automatically computed and compared between images to determine identity.[6]
Given a large database of images (in effect, a book of mug shots) and a photograph, the problem was to select from the database a small set of records such that one of the image records matched the photograph. The success of the method could be measured in terms of the ratio of the answer list to the number of records in the database. Bledsoe (1966a) described the following difficulties:
| “ | This recognition problem is made difficult by the great variability in head rotation and tilt, lighting intensity and angle, facial expression, aging, etc. Some other attempts at face recognition by machine have allowed for little or no variability in these quantities. Yet the method of correlation (or pattern matching) of unprocessed optical data, which is often used by some researchers, is certain to fail in cases where the variability is great. In particular, the correlation is very low between two pictures of the same person with two different head rotations. | ” |
| — Woody Bledsoe, 1966 | ||
This project was labeled man-machine because the human extracted the coordinates of a set of features from the photographs, which were then used by the computer for recognition. Using a graphics tablet (GRAFACON or RAND TABLET), the operator would extract the coordinates of features such as the center of pupils, the inside corner of eyes, the outside corner of eyes, point of widows peak, and so on. From these coordinates, a list of 20 distances, such as the width of mouth and width of eyes, pupil to pupil, were computed. These operators could process about 40 pictures an hour. When building the database, the name of the person in the photograph was associated with the list of computed distances and stored in the computer. In the recognition phase, the set of distances was compared with the corresponding distance for each photograph, yielding a distance between the photograph and the database record. The closest records are returned.
Because it is unlikely that any two pictures would match in head rotation, lean, tilt, and scale (distance from the camera), each set of distances is normalized to represent the face in a frontal orientation. To accomplish this normalization, the program first tries to determine the tilt, the lean, and the rotation. Then, using these angles, the computer undoes the effect of these transformations on the computed distances. To compute these angles, the computer must know the three-dimensional geometry of the head. Because the actual heads were unavailable, Bledsoe (1964) used a standard head derived from measurements on seven heads.
After Bledsoe left PRI in 1966, this work was continued at the Stanford Research Institute, primarily by Peter Hart. In experiments performed on a database of over 2000 photographs, the computer consistently outperformed humans when presented with the same recognition tasks (Bledsoe 1968). Peter Hart (1996) enthusiastically recalled the project with the exclamation, "It really worked!"
By about 1997, the system developed by Christoph von der Malsburg and graduate students of the University of Bochum in Germany and the University of Southern California in the United States outperformed most systems with those of Massachusetts Institute of Technology and the University of Maryland rated next. The Bochum system was developed through funding by the United States Army Research Laboratory. The software was sold as ZN-Face and used by customers such as Deutsche Bank and operators of airports and other busy locations. The software was "robust enough to make identifications from less-than-perfect face views. It can also often see through such impediments to identification as mustaches, beards, changed hairstyles and glasses—even sunglasses".[7]
In 2006, the performance of the latest face recognition algorithms was evaluated in the Face Recognition Grand Challenge (FRGC). High-resolution face images, 3-D face scans, and iris images were used in the tests. The results indicated that the new algorithms are 10 times more accurate than the face recognition algorithms of 2002 and 100 times more accurate than those of 1995. Some of the algorithms were able to outperform human participants in recognizing faces and could uniquely identify identical twins.[8][9]
U.S. Government-sponsored evaluations and challenge problems[10] have helped spur over two orders-of-magnitude in face-recognition system performance. Since 1993, the error rate of automatic face-recognition systems has decreased by a factor of 272. The reduction applies to systems that match people with face images captured in studio or mugshot environments. In Moore's law terms, the error rate decreased by one-half every two years.[11]
Low-resolution images of faces can be enhanced using face hallucination.
Techniques for face acquisition
Essentially, the process of face recognition is performed in two steps. The first involves feature extraction and selection and, the second is the classification of objects.[5] Later developments introduced varying technologies to the procedure. Some of the most notable include the following techniques:
Traditional
Some face recognition algorithms identify facial features by extracting landmarks, or features, from an image of the subject's face. For example, an algorithm may analyze the relative position, size, and/or shape of the eyes, nose, cheekbones, and jaw.[12] These features are then used to search for other images with matching features.[13]
Other algorithms normalize a gallery of face images and then compress the face data, only saving the data in the image that is useful for face recognition. A probe image is then compared with the face data.[14] One of the earliest successful systems[15] is based on template matching techniques[16] applied to a set of salient facial features, providing a sort of compressed face representation.
Recognition algorithms can be divided into two main approaches, geometric, which looks at distinguishing features, or photometric, which is a statistical approach that distills an image into values and compares the values with templates to eliminate variances. Some classify these algorithms into two broad categories: holistic and feature-based models. The former attempts to recognize the face in its entirety while the feature-based subdivide into components such as according to features and analyze each as well as its spatial location with respect to other features.[17]
Popular recognition algorithms include principal component analysis using eigenfaces, linear discriminant analysis, elastic bunch graph matching using the Fisherface algorithm, the hidden Markov model, the multilinear subspace learning using tensor representation, and the neuronal motivated dynamic link matching.
3-Dimensional recognition
Three-dimensional face recognition technique uses 3D sensors to capture information about the shape of a face. This information is then used to identify distinctive features on the surface of a face, such as the contour of the eye sockets, nose, and chin.[8]
One advantage of 3D face recognition is that it is not affected by changes in lighting like other techniques. It can also identify a face from a range of viewing angles, including a profile view.[13][8] Three-dimensional data points from a face vastly improve the precision of face recognition. 3D research is enhanced by the development of sophisticated sensors that do a better job of capturing 3D face imagery. The sensors work by projecting structured light onto the face. Up to a dozen or more of these image sensors can be placed on the same CMOS chip—each sensor captures a different part of the spectrum....[11]
Even a perfect 3D matching technique could be sensitive to expressions. For that goal a group at the Technion applied tools from metric geometry to treat expressions as isometries[18]
A new method is to introduce a way to capture a 3D picture by using three tracking cameras that point at different angles; one camera will be pointing at the front of the subject, second one to the side, and third one at an angle. All these cameras will work together so it can track a subject's face in real time and be able to face detect and recognize.[19]
Skin texture analysis
Another emerging trend uses the visual details of the skin, as captured in standard digital or scanned images. This technique, called Skin Texture Analysis, turns the unique lines, patterns, and spots apparent in a person's skin into a mathematical space.[13]
Surface Texture Analysis works much the same way facial recognition does. A picture is taken of a patch of skin, called a skinprint. That patch is then broken up into smaller blocks. Using algorithms to turn the patch into a mathematical, measurable space, the system will then distinguish any lines, pores and the actual skin texture. It can identify the contrast between identical pairs, which are not yet possible using facial recognition software alone.[20]
Tests have shown that with the addition of skin texture analysis, performance in recognizing faces can increase 20 to 25 percent.[13][8]
Facial recognition combining different techniques
As every method has its advantages and disadvantages, technology companies have amalgamated the traditional, 3D recognition and Skin Textual Analysis, to create recognition systems that have higher rates of success.
Combined techniques have an advantage over other systems. It is relatively insensitive to changes in expression, including blinking, frowning or smiling and has the ability to compensate for mustache or beard growth and the appearance of eyeglasses. The system is also uniform with respect to race and gender.[20]
Thermal cameras
A different form of taking input data for face recognition is by using thermal cameras, by this procedure the cameras will only detect the shape of the head and it will ignore the subject accessories such as glasses, hats, or makeup.[21] Unlike conventional cameras, thermal cameras can capture facial imagery even in low-light and nighttime conditions without using a flash and exposing the position of the camera.[22] However, a problem with using thermal pictures for face recognition is that the databases for face recognition is limited. Diego Socolinsky and Andrea Selinger (2004) research the use of thermal face recognition in real life and operation sceneries, and at the same time build a new database of thermal face images. The research uses low-sensitive, low-resolution ferroelectric electrics sensors that are capable of acquiring longwave thermal infrared (LWIR). The results show that a fusion of LWIR and regular visual cameras has greater results in outdoor probes. Indoor results show that visual has a 97.05% accuracy, while LWIR has 93.93%, and the Fusion has 98.40%, however on the outdoor proves visual has 67.06%, LWIR 83.03%, and fusion has 89.02%. The study used 240 subjects over a period of 10 weeks to create a new database. The data was collected on sunny, rainy, and cloudy days.[21]
In 2018, researchers from the U.S. Army Research Laboratory (ARL) developed a technique that would allow them to match facial imagery obtained using a thermal camera with those in databases that were captured using a conventional camera. This approach utilized artificial intelligence and machine learning to allow researchers to visibly compare conventional and thermal facial imagery.[23] Known as a cross-spectrum synthesis method due to how it bridges facial recognition from two different imaging modalities, this method synthesize a single image by analyzing multiple facial regions and details.[24] It consists of a non-linear regression model that maps a specific thermal image into a corresponding visible facial image and an optimization issue that projects the latent projection back into the image space.[22]
ARL scientists have noted that the approach works by combining global information (i.e. features across the entire face) with local information (i.e. features regarding the eyes, nose, and mouth). In addition to enhancing the discriminability of the synthesized image, the facial recognition system can be used to transform a thermal face signature into a refined visible image of a face.[25] According to performance tests conducted at ARL, researchers found that the multi-region cross-spectrum synthesis model demonstrated a performance improvement of about 30% over baseline methods and about 5% over state-of-the-art methods. It has also been tested for landmark detection for thermal images.[24]
Application
Mobile platforms
Social media
Social media platforms have adopted facial recognition capabilities to diversify their functionalities in order to attract a wider user base amidst stiff competition from different applications.
Founded in 2013, Looksery went on to raise money for its face modification app on Kickstarter. After successful crowdfunding, Looksery launched in October 2014. The application allows video chat with others through a special filter for faces that modifies the look of users. While there is image augmenting applications such as FaceTune and Perfect365, they are limited to static images, whereas Looksery allowed augmented reality to live videos. In late 2015, SnapChat purchased Looksery, which would then become its landmark lenses function.[26]
SnapChat's animated lenses, which used facial recognition technology, revolutionized and redefined the selfie, by allowing users to add filters to change the way they look. The selection of filters changes every day, some examples include one that makes users look like an old and wrinkled version of themselves, one that airbrushes their skin, and one that places a virtual flower crown on top of their head. The dog filter is the most popular filter that helped propel the continual success of SnapChat, with popular celebrities such as Gigi Hadid, Kim Kardashian and the likes regularly posting videos of themselves with the dog filter.
DeepFace is a deep learning facial recognition system created by a research group at Facebook. It identifies human faces in digital images. It employs a nine-layer neural net with over 120 million connection weights, and was trained on four million images uploaded by Facebook users.[27][28] The system is said to be 97% accurate, compared to 85% for the FBI's Next Generation Identification system.[29] One of the creators of the software, Yaniv Taigman, came to Facebook via their acquisition of Face.com.
ID Verification Solutions
Emerging use of Facial recognition is in use of ID verification services. Many companies are working in the market now to provide these services to banks, ICOs, and other e-businesses.[30]
Face ID
Apple introduced Face ID on the flagship iPhone X as a biometric authentication successor to the Touch ID, a fingerprint based system. Face ID has a facial recognition sensor that consists of two parts: a "Romeo" module that projects more than 30,000 infrared dots onto the user's face, and a "Juliet" module that reads the pattern.[31] The pattern is sent to a local "Secure Enclave" in the device's central processing unit (CPU) to confirm a match with the phone owner's face.[32] The facial pattern is not accessible by Apple. The system will not work with eyes closed, in an effort to prevent unauthorized access.[32]
The technology learns from changes in a user's appearance, and therefore works with hats, scarves, glasses, and many sunglasses, beard and makeup.[33]
It also works in the dark. This is done by using a "Flood Illuminator", which is a dedicated infrared flash that throws out invisible infrared light onto the user's face to properly read the 30,000 facial points.[34]
Deployment in security services
Policing
The Australian Border Force and New Zealand Customs Service have set up an automated border processing system called SmartGate that uses face recognition, which compares the face of the traveller with the data in the e-passport microchip.[35][36] All Canadian international airports use facial recognition as part of the Primary Inspection Kiosk program that compares a traveler face to their photo stored on the ePassport. This program first came to Vancouver International Airport in early 2017 and was rolled up to all remaining international airports in 2018-2019.[37] The Tocumen International Airport in Panama operates an airport-wide surveillance system using hundreds of live face recognition cameras to identify wanted individuals passing through the airport.[38]
Police forces in the United Kingdom have been trialling live facial recognition technology at public events since 2015. However, a recent report and investigation by Big Brother Watchfound that these systems were up to 98% inaccurate.[39]
National security
The U.S. Department of State operates one of the largest face recognition systems in the world with a database of 117 million American adults, with photos typically drawn from driver's license photos.[40] Although it is still far from completion, it is being put to use in certain cities to give clues as to who was in the photo. The FBI uses the photos as an investigative tool, not for positive identification.[41] As of 2016, facial recognition was being used to identify people in photos taken by police in San Diego and Los Angeles (not on real-time video, and only against booking photos)[42] and use was planned in West Virginia and Dallas.[43]
In recent years Maryland has used face recognition by comparing people's faces to their driver's license photos. The system drew controversy when it was used in Baltimore to arrest unruly protesters after the death of Freddie Gray in police custody.[44] Many other states are using or developing a similar system however some states have laws prohibiting its use.
The FBI has also instituted its Next Generation Identification program to include face recognition, as well as more traditional biometrics like fingerprints and iris scans, which can pull from both criminal and civil databases.[45]
In 2017, Time & Attendance company ClockedIn released facial recognition as a form of attendance tracking for businesses and organizations looking to have a more automated system of keeping track of hours worked as well as for security and health and safety control.
In May 2017, a man was arrested using an automatic facial recognition (AFR) system mounted on a van operated by the South Wales Police. Ars Technica reported that "this appears to be the first time [AFR] has led to an arrest".[46]
As of late 2017, China has deployed facial recognition technology in Xinjiang. Reporters visiting the region found surveillance cameras installed every hundred meters or so in several cities, as well as facial recognition checkpoints at areas like gas stations, shopping centers, and mosque entrances.[47][48]
Additional uses
In addition to being used for security systems, authorities have found a number of other applications for face recognition systems. While earlier post-9/11 deployments were well-publicized trials, more recent deployments are rarely written about due to their covert nature.
At Super Bowl XXXV in January 2001, police in Tampa Bay, Florida used Viisage face recognition software to search for potential criminals and terrorists in attendance at the event. 19 people with minor criminal records were potentially identified.[49][50]
In the 2000 Mexican presidential election, the Mexican government employed face recognition software to prevent voter fraud. Some individuals had been registering to vote under several different names, in an attempt to place multiple votes. By comparing new face images to those already in the voter database, authorities were able to reduce duplicate registrations.[51] Similar technologies are being used in the United States to prevent people from obtaining fake identification cards and driver's licenses.[52][53]
Face recognition has been leveraged as a form of biometric authentication for various computing platforms and devices;[13] Android 4.0 "Ice Cream Sandwich" added facial recognition using a smartphone's front camera as a means of unlocking devices,[54][55] while Microsoft introduced face recognition login to its Xbox 360 video game console through its Kinect accessory,[56] as well as Windows 10 via its "Windows Hello" platform (which requires an infrared-illuminated camera).[57] Apple's iPhone X smartphone introduced facial recognition to the product line with its "Face ID" platform, which uses an infrared illumination system.[58]
Face recognition systems have also been used by photo management software to identify the subjects of photographs, enabling features such as searching images by person, as well as suggesting photos to be shared with a specific contact if their presence were detected in a photo.[59][60]
Facial recognition is used as added security in certain websites, phone applications, and payment methods.
The United States' popular music and country music celebrity Taylor Swift surreptitiously employed facial recognition technology at a concert in 2018. The camera was embedded in a kiosk near a ticket booth and scanned concert-goers as they entered the facility for known stalkers.[61]
Advantages and disadvantages
Compared to other biometric systems
One key advantage of a facial recognition system that it is able to person mass identification as it does not require the cooperation of the test subject to work. Properly designed systems installed in airports, multiplexes, and other public places can identify individuals among the crowd, without passers-by even being aware of the system.[62]
However, as compared to other biometric techniques, face recognition may not be most reliable and efficient. Quality measures are very important in facial recognition systems as large degrees of variations are possible in face images. Factors such as illumination, expression, pose and noise during face capture can affect the performance of facial recognition systems.[62] Among all biometric systems, facial recognition has the highest false acceptance and rejection rates,[62] thus questions have been raised on the effectiveness of face recognition software in cases of railway and airport security.[citation needed]
Weaknesses
Ralph Gross, a researcher at the Carnegie Mellon Robotics Institute in 2008, describes one obstacle related to the viewing angle of the face: "Face recognition has been getting pretty good at full frontal faces and 20 degrees off, but as soon as you go towards profile, there've been problems."[8] Besides the pose variations, low-resolution face images are also very hard to recognize. This is one of the main obstacles of face recognition in surveillance systems.[63]
Face recognition is less effective if facial expressions vary. A big smile can render the system less effective. For instance: Canada, in 2009, allowed only neutral facial expressions in passport photos.[64]
There is also inconstancy in the datasets used by researchers. Researchers may use anywhere from several subjects to scores of subjects and a few hundred images to thousands of images. It is important for researchers to make available the datasets they used to each other, or have at least a standard dataset.[65]
Data privacy is the main concern when it comes to storing biometrics data in companies. Data stores about face or biometrics can be accessed by the third party if not stored properly or hacked. In the Techworld, Parris adds (2017), “Hackers will already be looking to replicate people's faces to trick facial recognition systems, but the technology has proved harder to hack than fingerprint or voice recognition technology in the past.”
Ineffectiveness[edit]
Critics of the technology complain that the London Borough of Newham scheme has, as of 2004, never recognized a single criminal, despite several criminals in the system's database living in the Borough and the system has been running for several years. "Not once, as far as the police know, has Newham's automatic face recognition system spotted a live target."[50][66] This information seems to conflict with claims that the system was credited with a 34% reduction in crime (hence why it was rolled out to Birmingham also).[67]However it can be explained by the notion that when the public is regularly told that they are under constant video surveillance with advanced face recognition technology, this fear alone can reduce the crime rate, whether the face recognition system technically works or does not. This has been the basis for several other face recognition based security systems, where the technology itself does not work particularly well but the user's perception of the technology does.
An experiment in 2002 by the local police department in Tampa, Florida, had similarly disappointing results.[50]
A system at Boston's Logan Airport was shut down in 2003 after failing to make any matches during a two-year test period.[68]
In 2014, Facebook stated that in a standardized two-option facial recognition test, its online system scored 97.25% accuracy, compared to the human benchmark of 97.5%.[69]
In 2018, a report by the civil liberties and rights campaigning organisation Big Brother Watch revealed that two UK police forces, South Wales Police and the Metropolitan Police, were using live facial recognition at public events and in public spaces, but with an accuracy rate as low as 2%.[39] Their report also warned of significant potential human rightsviolations.[39] It received widespread press coverage in the UK.[70]
Systems are often advertised as having accuracy near 100%; this is misleading as the studies often use much smaller sample sizes than would be necessary for large scale applications. Because facial recognition is not completely accurate, it creates a list of potential matches. A human operator must then look through these potential matches and studies show the operators pick the correct match out of the list only about half the time. This causes the issue of targeting the wrong suspect.[41][71]
Controversies
Privacy violations
Civil rights right organizations and privacy campaigners such as the Electronic Frontier Foundation,[72] Big Brother Watch[39] and the ACLU[73] express concern that privacy is being compromised by the use of surveillance technologies. Some fear that it could lead to a “total surveillance society,” with the government and other authorities having the ability to know the whereabouts and activities of all citizens around the clock. This knowledge has been, is being, and could continue to be deployed to prevent the lawful exercise of rights of citizens to criticize those in office, specific government policies or corporate practices. Many centralized power structures with such surveillance capabilities have abused their privileged access to maintain control of the political and economic apparatus, and to curtail populist reforms.[74]
Face recognition can be used not just to identify an individual, but also to unearth other personal data associated with an individual – such as other photos featuring the individual, blog posts, social networking profiles, Internet behavior, travel patterns, etc. – all through facial features alone.[75] Concerns have been raised over who would have access to the knowledge of one's whereabouts and people with them at any given time.[76] Moreover, individuals have limited ability to avoid or thwart face recognition tracking unless they hide their faces. This fundamentally changes the dynamic of day-to-day privacy by enabling any marketer, government agency, or random stranger to secretly collect the identities and associated personal information of any individual captured by the face recognition system.[75] Consumers may not understand or be aware of what their data is being used for, which denies them the ability to consent to how their personal information gets shared.[76]
Face recognition was used in Russia to harass women allegedly involved in online pornography.[77] In Russia there is an app 'FindFace' which can identify faces with about 70% accuracy using the social media app called VK. This app would not be possible in other countries which do not use VK as their social media platform photos are not stored the same way as with VK.[78]
In July 2012, a hearing was held before the Subcommittee on Privacy, Technology and the Law of the Committee on the Judiciary, United States Senate, to address issues surrounding what face recognition technology means for privacy and civil liberties.[79]
In 2014, the National Telecommunications and Information Association (NTIA) began a multi-stakeholder process to engage privacy advocates and industry representatives to establish guidelines regarding the use of face recognition technology by private companies.[80] In June 2015, privacy advocates left the bargaining table over what they felt was an impasse based on the industry representatives being unwilling to agree to consent requirements for the collection of face recognition data.[81] The NTIA and industry representatives continued without the privacy representatives, and draft rules are expected to be presented in the spring of 2016.[82]
In July 2015, the United States Government Accountability Office conducted a Report to the Ranking Member, Subcommittee on Privacy, Technology and the Law, Committee on the Judiciary, U.S. Senate. The report discussed facial recognition technology's commercial uses, privacy issues, and the applicable federal law. It states that previously, issues concerning facial recognition technology were discussed and represent the need for updated federal privacy laws that continually match the degree and impact of advanced technologies. Also, some industry, government, and private organizations are in the process of developing, or have developed, "voluntary privacy guidelines". These guidelines vary between the groups, but overall aim to gain consent and inform citizens of the intended use of facial recognition technology. This helps counteract the privacy issues that arise when citizens are unaware of where their personal, privacy data gets put to use as the report indicates as a prevalent issue.[76]
The largest concern with the development of biometric technology, and more specifically facial recognition has to do with privacy. The rise in facial recognition technologies has led people to be concerned that large companies, such as Google or Apple, or even Government agencies will be using it for mass surveillance of the public. Regardless of whether or not they have committed a crime, in general people do not wish to have their every action watched or track. People tend to believe that, since we live in a free society, we should be able to go out in public without the fear of being identified and surveilled. People worry that with the rising prevalence of facial recognition, they will begin to lose their anonymity.
Facebook DeepFace
Social media web sites such as Facebook have very large numbers of photographs of people, annotated with names. This represents a database which may be abused by governments for face recognition purposes.[83] Facebook's DeepFace has become the subject of several class action lawsuits under the Biometric Information Privacy Act, with claims alleging that Facebook is collecting and storing face recognition data of its users without obtaining informed consent, in direct violation of the Biometric Information Privacy Act.[84] The most recent case was dismissed in January 2016 because the court lacked jurisdiction.[85] Therefore, it is still unclear if the Biometric Information Privacy Act will be effective in protecting biometric data privacy rights.
In December 2017, Facebook rolled out a new feature that notifies a user when someone uploads a photo that includes what Facebook thinks is their face, even if they are not tagged. Facebook has attempted to frame the new functionality in a positive light, amidst prior backlashes.[86] Facebook's head of privacy, Rob Sherman, addressed this new feature as one that gives people more control over their photos online. “We’ve thought about this as a really empowering feature,” he says. “There may be photos that exist that you don’t know about.” [87]
Imperfect technology in law enforcement
All over the world, law enforcement agencies have begun using facial recognition software to aid in the identifying of criminals. For example, the Chinese police force were able to identify twenty-five wanted suspects using facial recognition equipment at the Qingdao International Beer Festival, one of which had been on the run for 10 years.[88] The equipment works by recording a 15-second video clip and taking multiple snapshots of the subject. That data is compared and analyzed with images from the police department's database and within 20 minutes, the subject can be identified with a 98.1% accuracy.[89] In the UK, the police's use of facial recognition technology has been found to be up to 98% inaccurate.[39]
Facial recognition technology has been proven to work less accurately on people of color.[90] One study by Joy Buolamwini (MIT Media Lab) and Timnit Gebru (Microsoft Research) found that the error rate for gender recognition for women of color within three commercial facial recognition systems ranged from 23.8% to 36%, whereas for lighter-skinned men it was between 0.0 and 1.6%. Overall accuracy rates for identifying men (91.9%) were higher than for women (79.4%), and none of the systems accommodated a non-binary understanding of gender.[91]
Experts fear that the new technology may actually be hurting the communities the police claims they are trying to protect.[92] It is considered an imperfect biometric, and in a study conducted by Georgetown University researcher Clare Garvie, she concluded that "there’s no consensus in the scientific community that it provides a positive identification of somebody.”[93]
It is believed that with such large margins of error in this technology, both legal advocates and facial recognition software companies say that the technology should only supply a portion of the case – no evidence that can lead to an arrest of an individual.[93]
The lack of regulations holding facial recognition technology companies to requirements of racially biased testing can be a significant flaw in the adoption of use in law enforcement. CyberExtruder, a company that markets itself to law enforcement said that they had not performed testing or research on bias in their software. CyberExtruder did note that some skin colors are more difficult for the software to recognize with current limitations of the technology. “Just as individuals with very dark skin are hard to identify with high significance via facial recognition, individuals with very pale skin are the same,” said Blake Senftner, a senior software engineer at CyberExtruder.[93]
Facial recognition technology market worth a staggering $4.6bn in 2019 - and set to grow by another 25% over next 9 years.[94]
In May 2019, the San Francisco Board of Supervisors voted to prohibit police and other government agencies from using facial recognition technology, making San Francisco the first U.S. city to ban this practice.[95]
Emotion detection
Facial recognition systems have been used for emotion recognition[96][97] In 2016 Facebook acquired emotion detection startup FacioMetrics.[98][99]
Anti-facial recognition systems[edit]
In January 2013 Japanese researchers from the National Institute of Informatics created 'privacy visor' glasses that use nearly infrared light to make the face underneath it unrecognizable to face recognition software.[100] The latest version uses a titanium frame, light-reflective material and a mask which uses angles and patterns to disrupt facial recognition technology through both absorbing and bouncing back light sources.[101][102][103][104] In December 2016 a form of anti-CCTV and facial recognition sunglasses called 'reflectacles' were invented by a custom-spectacle-craftsman based in Chicago named Scott Urban.[105] They reflect infrared and, optionally, visible light which makes the users face a white blur to cameras.[106]
Another method to protect from facial recognition systems are specific haircuts and make-up patterns that prevent the used algorithms to detect a face, known as computer vision dazzle.[107]
References[edit]
- ^ "What is Facial Recognition? - Definition from Techopedia". Techopedia.com. Retrieved 2018-08-27.
- ^ "Face Recognition Applications". Animetrics. Retrieved 2008-06-04.
- ^ Zhang, Jian, Yan, Ke, He, Zhen-Yu, and Xu, Yong (2014). "A Collaborative Linear Discriminative Representation Classification Method for Face Recognition. In 2014 International Conference on Artificial Intelligence and Software Engineering (AISE2014). Lancaster, PA: DEStech Publications, Inc. p.21 ISBN 9781605951508
- ^ "Facial Recognition: Who's Tracking You in Public?". Consumer Reports. Retrieved 2016-04-05.
- ^ a b Bramer, Max (2006). Artificial Intelligence in Theory and Practice: IFIP 19th World Computer Congress, TC 12: IFIP AI 2006 Stream, August 21-24, 2006, Santiago, Chile. Berlin: Springer Science+Business Media. p. 395. ISBN 9780387346540.
- ^ a b c de Leeuw, Karl; Bergstra, Jan (2007). The History of Information Security: A Comprehensive Handbook. Amsterdam: Elsevier. pp. 264–265. ISBN 9780444516084.
- ^ "Mugspot Can Find A Face In The Crowd -- Face-Recognition Software Prepares To Go To Work In The Streets". ScienceDaily. 12 November 1997. Retrieved 2007-11-06.
- ^ a b c d e Williams, Mark. "Better Face-Recognition Software". Retrieved 2008-06-02.
- ^ R. Kimmel and G. Sapiro (30 April 2003). "The Mathematics of Face Recognition". SIAM News. Retrieved 2003-04-30.
- ^ "Face Homepage". nist.gov.
- ^ a b Crawford, Mark. "Facial recognition progress report". SPIE Newsroom. Retrieved 2011-10-06.
- ^ "Airport Facial Recognition Passenger Flow Management". hrsid.com.
- ^ a b c d e Bonsor, K. "How Facial Recognition Systems Work". Retrieved 2008-06-02.
- ^ Smith, Kelly. "Face Recognition" (PDF). Retrieved 2008-06-04.
- ^ R. Brunelli and T. Poggio, "Face Recognition: Features versus Templates", IEEE Trans. on PAMI, 1993, (15)10:1042-1052
- ^ R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice, Wiley, ISBN 978-0-470-51706-2, 2009 ([1] TM book)
- ^ Zhang, David; Jain, Anil (2006). Advances in Biometrics: International Conference, ICB 2006, Hong Kong, China, January 5-7, 2006, Proceedings. Berlin: Springer Science+Business Media. p. 183. ISBN 9783540311119.
- ^ Kimmel, Ron. "Three-dimensional face recognition" (PDF). Retrieved 2005-01-01.
- ^ Duhn, S. von; Ko, M. J.; Yin, L.; Hung, T.; Wei, X. (1 September 2007). "Three-View Surveillance Video Based Face Modeling for Recogniton". Three-View Surveillance Video Based Face Modeling for Recognition. pp. 1–6. doi:10.1109/BCC.2007.4430529. ISBN 978-1-4244-1548-9 – via IEEE Xplore.
- ^ a b "How Facial Recognition Systems Work". HowStuffWorks. 2001-09-04. Retrieved 2018-04-09.
- ^ a b Socolinsky, Diego A.; Selinger, Andrea (1 January 2004). "Thermal Face Recognition in an Operational Scenario". IEEE Computer Society. pp. 1012–1019 – via ACM Digital Library.
- ^ a b "Army Builds Face Recognition Technology that Works in Low-Light Conditions". AZoRobotics. April 18, 2018. Retrieved August 17, 2018.
- ^ "Army develops face recognition technology that works in the dark". Army Research Laboratory. April 16, 2018. Retrieved August 17, 2018.
- ^ a b Riggan, Benjamin; Short, Nathaniel; Hu, Shuowen (March 2018). "Thermal to Visible Synthesis of Face Images using Multiple Regions". ResearchGate.
- ^ Cole, Sally (June 2018). "U.S. Army's AI facial recognition works in the dark". Military Embedded Systems. p. 8.
- ^ Shontell, Alyson (2015-09-15). "Snapchat buys Looksery, a 2-year-old startup that lets you Photoshop your face while you video chat". Business Insider Singapore. Retrieved 2018-04-09.
- ^ Simonite, Tom. "Facebook Creates Software That Matches Faces Almost as Well as You Do". MIT Technology Review. Retrieved 2018-04-09.
- ^ "Facebook's DeepFace shows serious facial recognition skills". Retrieved 2018-04-09.
- ^ "Why Facebook is beating the FBI at facial recognition". The Verge. Retrieved 2018-04-09.
- ^ "A glimpse at bank branches of the future: video walls, booth-sized locations and 24/7 access". USA TODAY. Retrieved 2018-08-13.
- ^ Kubota, Yoko (2017-09-27). "Apple iPhone X Production Woe Sparked by Juliet and Her Romeo". Wall Street Journal. ISSN 0099-9660. Retrieved 2018-04-10.
- ^ a b "The five biggest questions about Apple's new facial recognition system". The Verge. Retrieved 2018-04-10.
- ^ "Apple's Face ID Feature Works With Most Sunglasses, Can Be Quickly Disabled to Thwart Thieves". Retrieved 2018-04-10.
- ^ Heisler, Yoni (2017-11-03). "Infrared video shows off the iPhone X's new Face ID feature in action". BGR. Retrieved 2018-04-10.
- ^ "Smartgates". Australian Border Force. Retrieved 11 March 2019.
- ^ "Our history". New Zealand Customs Service. Retrieved 11 March 2019.
- ^ "Facial recognition technology is coming to Canadian airports this spring". CBC News. Retrieved 2017-03-03.
- ^ Vogel, Ben. "Panama puts names to more faces". IHS Jane's Airport Review. Archived from the original on 12 October 2014. Retrieved 2014-10-07.
Under the USD11 million contract, a cluster of sixty computers, a fibre optic network, and 150 surveillance cameras were installed in the terminal and at about 30 gates.
- ^ a b c d e "Face Off: The lawless growth of facial recognition in UK policing" (PDF). Big Brother Watch.
- ^ FORTUNE. "Here's How Many Adult Faces Are Scanned From Facial Recognition Databases".
- ^ a b "The trouble with facial recognition technology (in the real world)".
- ^ "Real-Time Facial Recognition Is Available, But Will U.S. Police Buy It?". NPR.org.
- ^ "Police Facial Recognition Databases Log About Half Of Americans". NPR.org.
- ^ Knezevich, Kevin Rector, Alison. "Maryland's use of facial recognition software questioned by researchers, civil liberties advocates".
- ^ "Next Generation Identification". FBI. Retrieved 2016-04-05.
- ^ Anthony, Sebastian (6 June 2017). "UK police arrest man via automatic face-recognition tech". Ars Technica.
- ^ "China's massive investment in artificial intelligence has an insidious downside". Science | AAAS. 7 February 2018. Retrieved 23 February 2018.
- ^ "China bets on facial recognition in big drive for total surveillance". Washington Post. 2018. Retrieved 23 February 2018.
- ^ Greene, Lisa (15 February 2001). "Face scans match few suspects" (SHTML). St. Petersburg Times. Archived from the original on 30 November 2014. Retrieved 2011-06-30.
By using Viisage software, police matched 19 people's faces to photos of people arrested in the past for minor pickpocketing, fraud and other charges. They weren't charged with any game-day misdeeds. THIS IS A FARCE
- ^ a b c Krause, Mike (14 January 2002). "Is face recognition just high-tech snake oil?". Enter Stage Right. ISSN 1488-1756. Archived from the original on 24 January 2002. Retrieved 2011-06-30.
- ^ "Mexican Government Adopts FaceIt Face Recognition Technology to Eliminate Duplicate Voter Registrations in Upcoming Presidential Election". Business Wire. 11 May 2000. Retrieved 2008-06-02.
- ^ House, David. "Facial recognition at DMV". Oregon Department of Transportation. Archived from the original on 5 February 2007. Retrieved 2007-09-17.
Oregon DMV is going to start using “facial recognition” software, a new tool in the prevention of fraud, required by a new state law. The law is designed to prevent someone from obtaining a driver license or ID card under a false name.
- ^ Schultz, Zac. "Facial Recognition Technology Helps DMV Prevent Identity Theft". WMTV News, Gray Television. Retrieved 2007-09-17.
Madison: ...The Department of Motor Vehicles is using... facial recognition technology [to prevent ID theft]
- ^ Heater, Brian. "Don't rely on Face Unlock to keep your phone secure". TechCrunch. Retrieved 2017-11-02.
- ^ "Galaxy S8 face recognition already defeated with a simple picture". Ars Technica. Retrieved 2017-11-02.
- ^ "How Facial Recognition Works in Xbox Kinect". Wired. Retrieved 2017-11-02.
- ^ "Windows 10 says "Hello" to logging in with your face and the end of passwords". Ars Technica. Retrieved 17 March 2015.
- ^ Kubota, Yoko (September 27, 2017). "Apple iPhone X Production Woe Sparked by Juliet and Her Romeo". The Wall Street Journal. Archived from the original on September 28, 2017. Retrieved September 27, 2017.
- ^ "Windows 10's Photos app is getting smarter image search just like Google Photos". The Verge. Retrieved 2017-11-02.
- ^ Perez, Sarah. "Google Photos upgraded with new sharing features, photo books, and Google Lens". TechCrunch. Retrieved 2017-11-02.
- ^ Giaritelli, Anna (December 13, 2018). "Taylor Swift used airport-style facial recognition on concertgoers". www.washingtonexaminer.com. Retrieved December 13, 2018.
- ^ a b c "Top Five Biometrics: Face, Fingerprint, Iris, Palm and Voice". Bayometric. 2017-01-23. Retrieved 2018-04-10.
- ^ Haghighat, M.; Abdel-Mottaleb, M. (2017). "Low Resolution Face Recognition in Surveillance Systems Using Discriminant Correlation Analysis". 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017): 912–917. doi:10.1109/FG.2017.130. ISBN 978-1-5090-4023-0.
- ^ "Passport Canada - Photos". passportcanada.gc.ca. Archived from the original on 1 March 2009.
- ^ Albiol, A., Albiol, A., Oliver, J., Mossi, J.M.(2012). Who is who at different cameras: people re-identification using depth cameras. Computer Vision, IET. Vol 6(5), 378-387.
- ^ Meek, James (13 June 2002). "Robo cop". London: UK Guardian newspaper.
- ^ "Birmingham City Centre CCTV Installs Visionics' FaceIt". Business Wire. 2 June 2008.
- ^ Willing, Richard (2 September 2003). "Airport anti-terror systems flub tests; Face-recognition technology fails to flag 'suspects'" (Abstract). USA Today. Retrieved 2007-09-17.
- ^ Meyer, Robinson (2015). "How Worried Should We Be About Facial Recognition?". The Atlantic. Retrieved 2 March 2018.
- ^ Dodd, Vikram (2018-05-14). "UK police use of facial recognition technology a failure, says report". the Guardian. Retrieved 2018-05-29.
- ^ White, David; Dunn, James D.; Schmid, Alexandra C.; Kemp, Richard I. (14 October 2015). "Error Rates in Users of Automatic Face Recognition Software". PLOS ONE. 10(10): e0139827. doi:10.1371/journal.pone.0139827. PMC 4605725. PMID 26465631.
- ^ "EFF Sues FBI For Access to Facial-Recognition Records". Electronic Frontier Foundation.
- ^ "Q&A On Face-Recognition". American Civil Liberties Union.
- ^ "Civil Liberties & Facial Recognition Software". About.com, The New York Times Company. pp. pp. 2. Archived from the original on 1 March 2006. Retrieved 2007-09-17.
A few examples which have already arisen from surveillance video are: using license plates to blackmail gay married people, stalking women, tracking estranged spouses...
- ^ a b Harley Geiger (6 December 2011). "Facial Recognition and Privacy". Center for Democracy & Technology. Retrieved 2012-01-10.
- ^ a b c Cackley, Alicia Puente (July 2015). "FACIAL RECOGNITION TECHNOLOGY Commercial Uses, Privacy Issues, and Applicable Federal Law" (PDF).
- ^ "Facial Recognition is getting really accurate, and we have not prepared". 11 October 2016.
- ^ "This creepy facial recognition app is taking Russia by storm". 18 May 2016.
- ^ What Facial Recognition Technology Means for Privacy and Civil Liberties: Hearing before the Subcommittee on Privacy, Technology and the Law of the Committee on the Judiciary, United States Senate, One Hundred Twelfth Congress, Second Session, July 18, 2012
- ^ "Privacy Multistakeholder Process: Facial Recognition Technology". National Telecommunications and Information Association. Retrieved 5 April 2016.
- ^ McCabe, David. "Facial recognition talks break down as privacy advocates withdraw". TheHill. Retrieved 2016-04-05.
- ^ Weaver, Dustin. "Business eyes facial recognition guidelines". TheHill. Retrieved 2016-04-05.
- ^ Martin Koste (28 October 2013). "A Look Into Facebook's Potential To Recognize Anybody's Face". NPR. Archived from the original on 1 November 2013. Retrieved 2013-12-25.
- ^ "Facebook Keeps Getting Sued Over Face-Recognition Software, And Privacy Groups Say We Should Be Paying More Attention". International Business Times. Retrieved 2016-04-05.
- ^ Herra, Dana. "Judge tosses Illinois privacy law class action vs Facebook over photo tagging; California cases still pending". cookcountyrecord.com. Retrieved 2016-04-05.
- ^ "Singel-Minded: Anatomy of a Backlash, or How Facebook Got an 'F' for Facial Recognition". WIRED. Retrieved 2018-04-10.
- ^ "Facebook Can Now Find Your Face, Even When It's Not Tagged". WIRED. Retrieved 2018-04-10.
- ^ Beijing, Agence France-Presse in (2017-09-01). "From ale to jail: facial recognition catches criminals at China beer festival". the Guardian. Retrieved 2018-03-08.
- ^ "Police use facial recognition technology to detect wanted criminals during beer festival in Chinese city of Qingdao | OpenGovAsia". www.opengovasia.com. Retrieved 2018-03-08.
- ^ "Photo Algorithms ID White Men Fine—Black Women, Not So Much". WIRED. Retrieved 2018-04-10.
- ^ Joy Buolamwini; Timnit Gebru (2018). "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification". Proceedings of Machine Learning Research, vol. 81. pp. 1–15. Retrieved 8 March 2018.
- ^ Buranyi, Stephen (2017-08-08). "Rise of the racist robots – how AI is learning all our worst impulses". the Guardian. Retrieved 2018-04-10.
- ^ a b c Brel, Ali (2017-12-04). "How white engineers built racist code – and why it's dangerous for black people". the Guardian. Retrieved 2018-04-10.
- ^ "Facial Recognition Technology Market Report 2019-2029". Visiongain. Retrieved 2019-03-12.
- ^ Paul, Kari (May 14, 2019). "San Francisco is first US city to ban police use of facial recognition tech". The Guardian. Retrieved May 15, 2019.
- ^ "Emotion detector: Facial expression recognition to improve learning, gaming". Science Daily. Retrieved 4 January 2017.
- ^ "Facial Recognition Market - Global Forecast to 2021". Digital Journal. Retrieved 4 January 2017.
- ^ Constine, Josh. "Like by smiling? Facebook acquires emotion detection startup FacioMetrics". TechCrunch. Retrieved 4 January 2017.
- ^ "Facebook acquires FacioMetrics to add 'fun effects' to photos and videos". VentureBeat. Retrieved 4 January 2017.
- ^ "These Goofy-Looking Glasses Could Make You Invisible to Facial Recognition Technology". Slate. 18 January 2013. Retrieved 22 January 2013.
- ^ Hongo, Jun. "Eyeglasses with Face Un-Recognition Function to Debut in Japan". Wall Street Journal. Retrieved 9 February 2017.
- ^ Osborne, Charlie. "Privacy visor which blocks facial recognition software set for public release". ZDNet. Retrieved 9 February 2017.
- ^ Stone, Maddie. "These Glasses Block Facial Recognition Technology". Gizmodo. Retrieved 9 February 2017.
- ^ "How Japan's Privacy Visor fools face-recognition cameras". PCWorld. Retrieved 9 February 2017.
- ^ Magnetic. "Be Seen and Unseen! Reflectacles are the Sunglasses of the Future". Magnetic Magazine.
- ^ "Reflectacles - Reflective Eyewear and Sunglasses".
- ^ Harvey, Adam. "CV Dazzle: Camouflage from Face Detection". cvdazzle.com. Retrieved 2017-09-15.
- Tucker, Jennifer (23 November 2014). "How faicial recognition technology came to be". Boston Globe. Retrieved 24 February 2019.
What is Facial Recognition?
How computers use face scan technology to identify users
by
Computers use face scanning software to identify a person by mapping their facial features and comparing that information with a database of faces. Facial recognition technology is quickly becoming the most secure and reliable tool for user authentication.
How Does Face Scan Technology Work?
Facial recognition systems use a number of measurements and technologies to scan faces including thermal imaging, 3D face mapping, cataloging unique features (also called landmarks), analyzing geometric proportions of facial features, mapping distance between key facial features, and skin surface texture analysis. Facial recognition technology falls under the category of biometrics, or the measurement of biological data. Other examples of biometric technologies include fingerprint scanning and eye/iris scanning systems.
Uses for Facial Recognition Technology
Facial recognition software has a variety of applications, but it's most commonly used for security and law enforcement purposes. Airports use facial recognition software to search for individuals suspected of crimes and also to compare passport photos with in-person faces to confirm identities.
Law enforcement agents use facial recognition software to identify and apprehend people who commit crimes. Several states use facial recognition software to prevent people from getting fake identification cards or driver’s licenses. Some foreign governments have even used facial recognition technology to crack down on voter fraud
Limitations of Facial Recognition
While face recognition programs can use a variety of measurements and types of scans to detect and identify faces, there are limitations:
- Poor resolution images and poor lighting can reduce the accuracy of face-scanning results.
- Different angles and facial expressions, even a simple smile, can pose challenges for face matching systems.
- Facial recognition loses accuracy when the person is wearing items like glasses, hats, scarves, or hairstyles that cover part of the face. Makeup and facial hair can also pose issues for face detection programs.
- Facial scans don’t necessarily connect with a profile, meaning that a scan of a person’s face may not be useful if there are no photos of them in an accessible database. Without a match, the identity of the person behind the face scan can remain a mystery.
Concerns over privacy or security can also pose limitations for how facial recognition systems are used. For example, scanning or collecting facial recognition data without a person’s knowledge and consent violates the Biometric Information Privacy Act of 2008.
Facial recognition technologies can be exploited for nefarious purposes. For example, facial recognition data that positively matches online photos or social media accounts could allow identity thieves to gather enough information to steal a person’s identity.
Facial Recognition Use in Smart Devices and Apps
Facial recognition is a growing part of our everyday lives through devices and applications. For example, the Facebook facial recognition system, DeepFace, can identify human faces in digital pictures with up to a 97 percent accuracy rate.
Apple has also added a facial recognition feature called Face ID to the iPhone X. Face ID is expected to replace Apple’s fingerprint scanning feature, Touch ID. It stores a mathematical representation of your facial scan in a secure location on the device itself; the information doesn’t get copied or stored on Apple’s servers.
Face ID uses depth perception and infrared sensors to ensure that the camera is scanning your actual face and not a photo or 3D model. The system also requires your eyes to be open, which prevents another person from unlocking your phone while you are asleep or unconscious.
Apple has cautioned that identical siblings (twins, triplets) would be able to unlock each other’s phones. Even without an identical sibling, Apple has estimated that there is approximately a one-in-a-million chance that the face of a complete stranger will have the same mathematical representation of their facial scan as you do. Children under 13 are not good candidates to use this technology because their faces are still growing and changing shape.
Android and laptop users can download several apps that add facial recognition capabilities to their device, but none of them are as reliable as Face ID. Options include True Key by McAfee and Face Lock Screen. As the technology improves, expect to see more devices with built-in facial recognition features.
