Should You Believe All Conclusions From Machine Learning Models?

Photo by Charles Deluvio on Unsplash

Here’s a machine learning algorithm that presumably can tell whether or not a person might convict a crime based on their facial features alone. It has analyzed thousands and thousands of criminal and civilian faces to increase its accuracy of detecting potential criminals from harmless individuals.

The scientists who created the algorithm claimed that using computer vision and machine learning models, they can reveal subtle cues and patterns to determine what features of the human face are associated with criminality. They claim that their model can do so with nearly 90 percent accuracy, obviously devoid of biases and prejudices that come from being human. Has physiognomy become real science again? Should you believe this model?

Turns out, you shouldn’t. Although we can argue that we have no clue what the “black box” does to evaluate and determine the output, we can still look at the input data to decide the validity of such models. (It’s called a black box due to the complexity of machine learning models, so no one knows exactly how these ML models come to their conclusions)

By looking at the pictures that were used to train the model, we see two massive problems. The first is that the images of noncriminals were selected to cast the individuals in a positive light. Meanwhile, the images of the criminals were official ID photos.

Secondly, these photos are of people who were convicted. This means that even if the model picks up facial features that represent criminality, we won’t know whether these differences are associated with committing crimes or with being convicted.

Even more problematic is the output of the ML model. The algorithm finds that criminals have shorter distances between the inner corners of the eyes, smaller angles between the nose and the corners of the mouth, and higher curvature to the upper lip. What does this mean?

There’s a simple and quick trick for you to allow your face to fit these exact criteria — simply smile.

Yes, if you smile, the angle between your nose and the corners of your mouth gets smaller, and the curvature of your upper lip gets higher. So what this means is that the noncriminals in the pictures mostly smiled while the criminals were mostly frowning. The scientists haven’t created a criminality detector — they invented a smile detector.

This is a classic example of the potential errors an ML model has. Although we praise them as algorithms that discover truths in our world, the reality isn’t so simple. ML models have to be crafted very carefully. Not only should the scientists pick the best model for the right situation, but they also have to be very sure that the data is clean and mostly free from biases. This is because that’s what ML models do: they pick up patterns and correlations in data. If there are biases in the data, ML models will pick them up as patterns, and the output will become distorted. Although scientists try their best to eliminate data with biases, there always will be subtle biases they might not have found.

Although machine learning is continuously allowing us to rise to unfathomable heights, we still have to be careful of these situations and be aware that at the end of the day, these are correlations and not causations.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Simon Jeong

Simon Jeong

An optimist, pessimist, and just a boring indifferentist