The iPhone X replaces the Home button and Touch ID with gesture controls and Face ID, sophisticated facial recognition software that learns to recogni
The iPhone X replaces the Home button and Touch ID with gesture controls and Face ID, sophisticated facial recognition software that learns to recognize your face and only lets you use your device once it has confirmed it is you. It is a new technology, here’s what we know about it so far:
The hype machine
This is how Apple’s chief marketer, Phil Schiller put it: “With the iPhone X, your iPhone is locked until you look at it and it recognizes you. Nothing has ever been more simple, natural, and effortless. This is the future of how we’ll unlock our smartphones and protect our sensitive information.”
How it works
Short version: Face ID works by using the selfie camera in the iPhone X. Apple calls this its TrueDepth camera system – it’s not just a camera, it consists of ambient light and proximity sensors, an infrared and a 7MP cameras, a flood illuminator and a dot projector.
When you set Face ID up (see below) the technology gets to know your face by creating a 3D image using over 30,000 dots positions cast on your face. There are 80 generally recognized nodal points on a face, so Apple’s tech is sophisticated.
Once Face ID is set up on your iPhone X, the device will automatically recognize you when you hold it up to your face, and open the iPhone. The system will not work if you do not stare directly at it or if you close your eyes. The process takes minimal time.
What is the process?
When you look at your iPhone X, Face ID implements the following steps:
- The flood illuminator detects your face
- The infrared camera captures an image of your face
- The dot projector casts 30,000 infrared dots on your face to create a model of it
- The data from the infrared camera and dot projector is sent to the A11 chip for processing.
- The A11 chip compares this information with what it knows about you. The chip also carries a whole bunch of AI information to help accurate identification.
What AI information is used?
Apple has analysed over a billion images for data about faces. This data was used to train what Apple calls a “neural network”, information that is compared to the model the iPhone has created of you face.
Can you explain TrueDepth?
TrueDepth captures an image of your face and then builds on top of that image using the dot positions cast by the Dot Projector module. It captures more information about your face each time it is used, which means its ‘map’ of your face improves – and changes –with you. It builds a 3D depth-sensitive picture of your face, which makes it highly accurate.
Does this mean Apple has a map of my face?
The biometric data created by the Face ID system is never shared with Apple. It is stored in the Secure Enclave inside the A11 Bionic chip. This means the information is also not shared between devices, so you’ll have to train each system up as and when you need it. Apple does not receive a map of your face.
What is ‘User attention’?
As I mentioned, you need to stare directly at the Face ID camera in order to successfully unlock your iPhone X. That’s why it does not work if you close your eyes or look away. Apple calls this ‘user attention’.
What if I change my face?
The system is sufficiently smart that it will still work if your appearance changes. Get a haircut, put on some glasses or wear a hat and it will still recognize who you are.
Can I fool it with a photo?
Face ID is not like the facial recognition system Samsung uses in its Note 8, which can be fooled using a photo. Face ID is not so easily fooled because it captures a 3D map of your face, and uses machine intelligence to ensure accuracy during verification. That’s why you can’t fool Face ID with a photo.
Can I fool it with a mask like Arya Stark?
Apple claims its tests showed that you cannot fool Face ID by wearing a mask of a person’s face, no matter how sophisticated that mask happened to be. We don’t know if Arya Stark will agree – security experts are quite sceptical. “It is only a matter of time before enough external data will be combined to side-step this control,” warned Kevin Cody.
Does Face ID work in the dark?
Among other things, Apple uses an infrared camera and a flood illuminator in Face ID. The first technology reads the dot pattern while the flood illuminator enables the system to work, even in the dark.
Does Face ID work if I’m asleep?
The Face ID system requires eye contact to function (“user attention”). This means it will not work if you are unconscious or asleep.
Can multiple people share the device?
If you use Touch ID you can create a fingerprint for a friend or family member you trust who you are happy to let use your iPhone. You can’t do this with Face ID, which can only track one person. I guess you could give other people your passcode, but I wouldn’t recommend it.
How secure is the system?
Apple claims that while Touch ID had a 1:50,000 chance of being unlocked by another person’s fingerprint, Face ID has just a 1:1,000,000 chance of being so unlocked.
Apple warns that someone who shares a close genetic relationship with you will be more likely to be able to fool Face ID – bad news if you have an evil twin.
It is worth noting that researchers were able to use Facebook photos to create 3D virtual models of a person’s face which they then used to fool five different face recognition applications.
Face ID was not one of these, but security experts have been and remain quite concerned that face recognition systems if cracked may open the door to authoritarianism or identity theft.
What about law enforcement?
While criminals can still force people to hand over their passcode, use a Touch ID sensor, or stare at an iPhone, what’s the situation regarding law enforcement?
The U.S. State Department already has a facial recognition database of 117 million American citizens, so can the police force you to look at and thus open your iPhone without a warrant?
You can refuse to stare directly at the device, or keep your eyes closed when you do stare at it, but ACLU legal director, Matthew Segal says the legal situation isn’t yet clear.
Is there an emergency way to switch Face ID off?
Press the Sleep/Wake button five times in rapid succession you’ll enter the Emergency SOS screen, tap the Cancel button and Face ID will be disabled and you will need to manually enter your passcode in order to begin using your iPhone. This should let you disable Face ID discreetly in some situations and can be seen as Face ID’s ‘emergency brake’.
Must I use Face ID?
You don’t need to use Face ID. You can choose to require a passcode to open your iPhone, like we did in the olden days before Touch ID. I imagine most enterprise deployments will demand employees use alphanumeric passcodes, which may cause a problem if their security policy precludes use of face recognition on the iPhone.
What about passcodes?
Once you get your hands on an iPhone X you must create a passcode before you can enable Face ID. Apple is very clear that systems like Touch ID or Face ID are there for convenience – the passcode you use on your devices is always higher up the security food chain. Touch ID and Face ID may make your devices easier to use, but you still need to use a passcode to enable either system.
How do I set up Face ID?
When setting up Face ID you will be asked to move your head around in various positions to help your iPhone build its first database. A white screen with a Mac OS-inspired smiling face will appear when the task is done. You can use Face ID to unlock your iPhone, for Safari Autofill, at the iTunes and App Store and for making a payment via Apple Pay. You can disable some or all of those features if you like.
Where did Face ID come from?
Some of the technologies used in Face ID were developed by 3D scanning firm, PrimeSense, an Israeli company Apple purchased in 2013. It is likely that Face ID also uses tech purchased when Apple acquired Emotient in 2016, and RealFace in February 2017.
How else is Face ID being used?
Apple couldn’t resist making some use of the face scanning technology it has created. The company told us that Snapchat is already developing some fun filters that use these systems to create face masks that sync with your facial movement.
Apple has created Animoji’s which work in a similar way, only in this case the way you move your face is used to give life-like animation to cute Animojis.
What next for Face ID?
Apple hasn’t told us what it plans to do, but I imagine we will see the company introduce Face ID APIs so developers can use more of the information put together by the system in their own apps at some point in future.
In the long term, there are other potential deployments — banks in Macau are beginning to use face recognition at ATMs and Mexico has used a face recognition system to prevent voter fraud.
Will governments one day recognize Face ID as an approved imaging system suitable for use in passports? If this happens then you’ll be able to travel across borders carrying nothing but an iPhone.
What does Edward Snowden say?
Edward Snowden may be a divisive character, but we know your right to privacy is one of the things he cares about. In a Tweet, he praised the security and emergency brake Apple has built inside Face ID, but voiced concern that the technology may impact privacy.
Good: Design looks surprisingly robust, already has a panic disable.
Bad: Normalizes facial scanning, a tech certain to be abused.
— Edward Snowden (@Snowden) September 12, 2017
What do you think?
I’m ever so interested in what you think. Will you use Face ID? What do you like about it, and what do you dislike? I’ve put together a poll here.
Google+? If you use social media and happen to be a Google+ user, why not join AppleHolic’s Kool Aid Corner community and join the conversation as we pursue the spirit of the New Model Apple?
Got a story? Drop me a line via Twitter. I’d like it if you chose to follow me there so I can let you know when fresh items are published here first on Computerworld.