Metropolitan Digital

The Conversation

  • Written by Siwei Lyu, Professor of Computer Science; Director, Computer Vision and Machine Learning Lab, University at Albany, State University of New York

Deepfake videos are hard for untrained eyes to detect because they can be quite realistic. Whether used as personal weapons of revenge, to manipulate financial markets or to destabilize international relations, videos depicting people doing and saying things they never did or said are a fundamental threat to the longstanding idea that “seeing is believing.” Not anymore.

Most deepfakes are made by showing a computer algorithm many images of a person, and then having it use what it saw to generate new face images. At the same time, their voice is synthesized, so it both looks and sounds like the person has said something new.

One of the most famous deepfakes sounds a warning.

Some of my research group’s earlier work allowed us to detect deepfake videos that did not include a person’s normal amount of eye blinking[1] – but the latest generation of deepfakes has adapted, so our research has continued to advance.

Now, our research can identify the manipulation of a video by looking closely at the pixels of specific frames. Taking one step further, we also developed an active measure to protect individuals from becoming victims of deepfakes.

Finding flaws

In two recent[2] research papers[3], we described ways to detect deepfakes with flaws that can’t be fixed easily by the fakers.

When a deepfake video synthesis algorithm generates new facial expressions, the new images don’t always match the exact positioning of the person’s head, or the lighting conditions, or the distance to the camera. To make the fake faces blend into the surroundings, they have to be geometrically transformed – rotated, resized or otherwise distorted. This process leaves digital artifacts in the resulting image.

You may have noticed some artifacts from particularly severe transformations. These can make a photo look obviously doctored, like blurry borders and artificially smooth skin. More subtle transformations still leave evidence[4], and we have taught an algorithm to detect it, even when people can’t see the differences.

A real video of Mark Zuckerberg. An algorithm detects that this purported video of Mark Zuckerberg is a fake.

These artifacts can change if a deepfake video has a person who is not looking directly at the camera. Video that captures a real person shows their face moving in three dimensions, but deepfake algorithms are not yet able to fabricate faces in 3D. Instead, they generate a regular two-dimensional image of the face and then try to rotate, resize and distort that image to fit the direction the person is meant to be looking.

They don’t yet do this very well, which provides an opportunity for detection. We designed an algorithm that calculates which way the person’s nose is pointing[5] in an image. It also measures which way the head is pointing, calculated using the contour of the face. In a real video of an actual person’s head, those should all line up quite predictably. In deepfakes, though, they’re often misaligned.

Detecting deepfakes by looking closely reveals a way to protect against them When a computer puts Nicolas Cage’s face on Elon Musk’s head, it may not line up the face and the head correctly. Siwei Lyu, CC BY-ND[6]

Defending against deepfakes

The science of detecting deepfakes is, effectively, an arms race – fakers will get better at making their fictions, and so our research always has to try to keep up, and even get a bit ahead.

If there were a way to influence the algorithms that create deepfakes to be worse at their task, it would make our method better at detecting the fakes. My group has recently found a way to do just that.

Detecting deepfakes by looking closely reveals a way to protect against them At left, a face is easily detected in an image before our processing. In the middle, we’ve added perturbations that cause an algorithm to detect other faces, but not the real one. At right are the changes we added to the image, enhanced 30 times to be visible. Siwei Lyu, CC BY-ND[7]

Image libraries of faces are assembled by algorithms that process thousands of online photos and videos and use machine learning to detect and extract faces. A computer might look at a class photo and detect the faces of all the students and the teacher, and add just those faces to the library. When the resulting library has lots of high-quality face images, the resulting deepfake is more likely to succeed at deceiving its audience.

We have found a way to add specially designed noise[8] to digital photographs or videos that are not visible to human eyes but can fool the face detection algorithms. It can conceal the pixel patterns that face detectors use to locate a face[9], and creates decoys that suggest there is a face where there is not one, like in a piece of the background or a square of a person’s clothing.

Subtle alterations to images can throw face detection algorithms way off.

With fewer real faces and more nonfaces polluting the training data, a deepfake algorithm will be worse at generating a fake face. That not only slows down the process of making a deepfake, but also makes the resulting deepfake more flawed and easier to detect.

As we develop this algorithm, we hope to be able to apply it to any images that someone is uploading to social media or another online site. During the upload process, perhaps, they might be asked, “Do you want to protect the faces in this video or image against being used in deepfakes?” If the user chooses yes, then the algorithm could add the digital noise, letting people online see the faces but effectively hiding them from algorithms that might seek to impersonate them.

Authors: Siwei Lyu, Professor of Computer Science; Director, Computer Vision and Machine Learning Lab, University at Albany, State University of New York

Read more

Metropolitan republishes selected articles from The Conversation USA with permission

Visit The Conversation to see more

Entertainment News

Phish’s 2018 Fall Tour to Conclude with Four Performances at MGM Grand Garden Arena

LAS VEGAS (May 15, 2018) – Phish, the American rock band known worldwide for its dedicated fan base, recently announced a 14-date Fall tour which will conclude in Las Vegas with four performances at...

Blane Ferguson - avatar Blane Ferguson

Dave Damiani and The No Vacancy Orchestra are “Bending The Standard”

Tina Sinatra, Dave Damiani & Landau Murphy Jr. celebrate 100 years of Frank Sinatra in Los Angeles There have been stories about independent filmmakers, but how about the independent big band...

Tom Estey - avatar Tom Estey

Billboard Chart-Topping Saxophonist VANDELL ANDREW Returns With New Single

From the vantage point of 30, his age and the name of his infectious, sensually grooving new full length album, Vandell continues to be fueled by the impressive roar of accolades and achievements th...

Metropolitan Digital - avatar Metropolitan Digital

Metropolitan Business News

Office Cleaner Takes Ownership of the Neglected Dishwasher

In an office that a dishwasher is being used communally, it is pretty difficult to set rules on how a certain appliance needs to be taken cared of. A dishwasher is a responsibility of no one until you...

News Company - avatar News Company

Amazing tips to become a successful trader

Everyone is working very hard to secure their financial freedom. Most of the people find it hard to support their family even after having a 9-5 day job. For this very reason, people often look for ...

News Company - avatar News Company

Best Practice For Young Professionals Working Through HR Internships

The industry of human relations is vitally important to the health and prosperity of a business.   Whether they are operating in textile manufacturing, accounting, sports, IT development or hospit...

News Company - avatar News Company

4 Easy Steps To Gaining SEO Momentum For Your Business

SEO (search engine optimisation) does not have to be a tiresome and overbearing exercise that diverts attention away from the core functions of a business.   SEO Shark affirms this as a smart and ...

News Company - avatar News Company

How to Manage An SEO Project On Limited Funds

SEO operators don’t need thousands upon thousands of dollars to become successful.   What SEO practitioners needs more than ever is the skills and diligence to identify problems that are acting a...

News Company - avatar News Company

An Introduction To Coworking For Australian Business Owners

Advancing technology is bringing with it great advantages in communications and networking, and to survive in business you need to keep up. With the rate at which everything changes these days, that...

News Company - avatar News Company


New Baggage Regulations to Help Aussie Parents Travel with Infants

Travelling around the globe has never been easy, especially when infants tag along for the trip. One of the main issues that parents often have to deal with is the need to bring extra item...

News Company - avatar News Company

Maya Beach Opens to Tourists

Despite recent reports that Southern Thailand's famous Maya Beach will close for three months this year, in fact no decision to this effect has been made by Thai authorities. Phi Phi Nati...

Maevadi Rosenfeldt - avatar Maevadi Rosenfeldt


SKYN®, Australia’s best-selling condom*, today launches its very first SKYN® Places of Intimacy Guide.   Curated in partnership with GQ Magazine and Conde Nast, the Guide features 30 lux...

SKYN - avatar SKYN