How Does a Computer Recognise a Human Face?

A face is not a password

A phone can unlock when its owner looks at it. An airport gate may compare a traveller with a passport image. A camera connected to a database may search for a person in a crowd. All three can be called facial recognition, but they ask different questions and carry very different consequences.

The common technical core has stages. First a system detects a face in a picture and locates it well enough to work with despite angle, light and expression. It then extracts features into a compact numerical representation, often called a template or embedding. Finally it compares that representation with a reference and produces a similarity score. The UK Information Commissioner's Office describes biometric capture, feature extraction and template creation as distinct steps.¹

The numbers are not a tiny photograph. They are a mathematical summary designed to make similar images of the same person close enough for comparison. A decision still needs a threshold: how similar is similar enough? Set it loosely and more impostors may be accepted. Set it tightly and more genuine users may be rejected.

One-to-one and one-to-many

A phone unlock is normally verification: does this face match the reference enrolled by this device's owner? It is a one-to-one comparison. A search for someone in a gallery is identification: which, if any, of many stored faces might match? That is a one-to-many task. The ICO distinguishes these uses because they answer different identity questions.¹

The difference is practical, not semantic. If a phone rejects its owner, they may try again or use another method. If a search wrongly nominates a person from a large database, that result may draw scrutiny from officials. The score should be treated as a lead requiring context and checks, not as a verdict. The larger and less controlled the search, the more attention must be paid to false matches and the way the system is used.

Face detection is also not the same as recognition. A camera can draw a box around a face without identifying anyone. Software that estimates a smile or counts people is doing a different task again. Combining all these tools under one label obscures the relevant privacy and performance questions.

Why does it fail?

A face in a studio portrait is easier to compare than one under a shadow, partly hidden by glasses or turned away from the camera. Ageing, expressions and camera angle can change the image. Training data and the design of the algorithm matter too. A system tested on one kind of photograph cannot be assumed to perform equally well on poor video stills.

There are two central errors. A false negative misses a real match; a false positive associates two different people. Both matter, but their costs differ with the use. NIST's facial recognition evaluations show that performance varies widely among algorithms and that demographic differences can appear in both error types.² Image quality can create some differences, while false-positive differences may persist even with high-quality images. It is inaccurate to describe every system as equally biased or every system as neutral.

The relevant test is therefore specific: which algorithm, which population, which cameras, which reference images, which threshold and which human decision after the result? Laboratory accuracy is useful evidence, but a deployment needs evaluation under its own conditions. A headline number stripped of those details can be more misleading than informative.

What does a template reveal?

A stored facial template may look abstract, but it is still linked to a person and can be used to recognise them. Unlike a password, a face cannot simply be changed if a database is misused. Systems must decide where templates are stored, how long they remain, who can search them and how access is audited. The ICO treats biometric recognition as processing special-category biometric data under UK data protection law.¹ Specific legal requirements depend on the application and jurisdiction.

Consent is also contextual. A person can choose to enrol their face on a private device. People walking through a public space may not have a meaningful opportunity to opt out of remote identification. Even a highly accurate system can create an environment in which movements are tracked or assembled into a record. Privacy is not merely the probability of an incorrect match; it also concerns what a correct match allows an organisation to do. The ICO warns that public-space recognition can identify people without their meaningful awareness or ability to opt out.³

Where is it useful?

Identity checks can be convenient when they reduce the need for a card or typed password. In controlled settings, with a clear purpose and another route for people who cannot or do not wish to use face recognition, the trade-off may be acceptable. It can also assist in locating missing people or managing secure access, subject to safeguards.

The risks rise when the database is broad, the stakes are high, or an automated result is treated as conclusive. Operators should be able to explain the purpose, evidence for performance, error-handling process and limits on storage. Human review is meaningful only if reviewers can challenge the system rather than merely confirm its suggestion.

A computer does not know a face in the way a friend does. It compares numbers produced from images and reports how closely they fit a stored reference. That can be useful, but the social decision sits outside the model: who gets enrolled, who gets searched, and what happens after a possible match.