Note: I wrote this essay for ART 271 Photography II at Allegheny College in spring 2025, as the written companion to Readable, my computer vision art project. The original included the analyzed photographs; their captions are kept in place here. Lightly formatted for the web, otherwise as submitted.
In an age where machines are looking back at you, artists have started exploring what technology is truly seeing and how model training has changed how it is interprets images. One such artist is Trevor Paglen. Trevor Paglen, born in 1974, is an American photographer exploring where artificial intelligence and art collide with a focus on computer vision (Aperture, 2019). Computer vision is an area of artificial intelligence that aims to teach computers to be able to interpret and understand digital images, videos, and other forms of visuals through various methods of artificial intelligence training (IBM, 2025). Paglen’s work with computer vision specifically focuses on how artificial intelligence is trained and how the technology can be used for surveillance (Aperture, 2019). Additionally, Paglen explores the unseen analysis of our images and how these can represent hidden forms of power but in the process he reveals these technologies by making their content visible using photography (Aperture, 2019; Paglen 2016). The work featured in this essay is a response to Paglen’s work with computer vision and a recognition of the bias it has.
ImageNet Roulette
One of the most controversial works Paglen has been ImageNet Roulette which was made in 2020 and co-authored with Kate Crawford. The work specifically tries to highlight problematic data sets and how training artificial intelligence using them often results in things going wrong (Paglen, 2020). The data set called ImageNet, made by Princeton and Stanford Universities in 2009, is often considered one of the most important training sets in history and was used to train the model in ImageNet Roulette (Paglen, 2020; Hern, 2019). Artificial intelligence would use computer vision to classify people in images into different categories, and this would result in very offensive outputs. A user could upload their own image to be classified and in some cases it would output offensive terms including racial slurs (Hern, 2019). The overall message of the work is pointing out how training sets are often problematic, however people were upset about the method in which ImageNet Roulette uses to demonstrate this fact.
Capture of ImageNet Roulette from https://paglen.studio/2020/04/29/imagenet-roulette/
Personally, I agree with how this work was a harmful way of demonstrating the classification of people. I think the interactive element of being able to upload your own image, or images of others, and getting called a slur is too far and an example of how this sort of work should not be interactive. There should be a filter preventing any classification like that because realistically computer vision and other artificial intelligence programs do have a filter and would not act like that under normal circumstances. The point of the work was to show that image classification can go wrong, and it definitely accomplishes that but at the same time harming others which cannot be justified. I think it is important to have consideration for the viewer.
Another thing to point out that is problematic about this work is when presenting this work on his personal website or elsewhere whenever it is a picture of the creators it seems to be words with positive connotations when recognizing the artists calling Paglen a “microeconomist” and Kate Crawford a “newsreader”.
Trevor Paglen and Kate Crawford from https://paglen.studio/2020/04/29/imagenet-roulette/
However, images of other people are often accompanied by descriptions of negative connotations. This is an interesting choice on how to present this to an audience. It likely was not his intention but it feels like it is only okay to use insulting wordage whenever it is not directly on the artist. What the artist chooses to present and what they do not say a lot about the piece and it is interesting to see that the only positive screen captures of this project are of himself and his co-author.
This work does show how computer vision often boxes things it thinks are a face and can categorize them almost instantly by comparing them to other photos within its dataset. The work also used a variety of photographs with often busy backgrounds making it more interesting when the computer vision is able to correctly find the faces within a photo even if it has 20 or more people.
Machine-Readable Hito and Holly
On the other hand, Machine Readable Hito and Holly, made in 2017-2018 accomplishes a similar feat of using computer vision to analyze people’s faces but instead focuses the efforts on the faces of two other artists. This project uses different points on the face to calculate different characteristics about their faces where the number represents the amount of certainty in that guess. The characteristics include things like gender, expression, wearing glasses, or if they have a beard. In this piece, the uncertainty of the analysis is particularly interesting especially when gender is considered. In some cases, there is less than 50% certainty in Hito’s gender begging the question of what 100% certainty looks like (Paglen, 2021) but it also brings to light how certain faces are more accurately represented than others. Many facial recognition algorithms struggle to interpret certain faces especially of people of color which is very problematic (Broussard, 2024). This is likely due to the training set for these systems being non-inclusive. Machine-Readable Hito and Holly represent two ways of image data understanding, visual and textual, but also presents a bigger issue with facial recognition in general.
Machine-Readable Hito, 2017 from https://paglen.studio/2020/04/09/machine-readable-hito-and-holly/
My Project
For my work I wanted to combine the inaccuracy of ImageNet Roulette and the portrait photography of Machine-Readable Hito and Holly using individuals in which I have a close relationship to including myself. I chose to use the DeepFace model made in 2015 which was created by researchers at Facebook to look at their users' photos and enhance the accuracy of facial recognition (Taigman, 2014). I wanted to use the same method Trevor Paglen used to use facial recognition on images but test it with both natural and more staged images to see if the computer can accurately recognize the expressions of myself and those closest to me.
This process was of particular interest because of the idea surrounding data lost or gained transitioning from a 3D to a 2D space and how this process can lead to misinterpretation especially through the lens of technology (Aguera y Arcas, 2022). It has been recorded before that artificial intelligence has a hard time understanding concepts such as glass shattering or plants wilting because it has some level of impact that is understood 3-dimensionally (Aguera y Arcas, 2022). Natural language processing still struggles with these concepts that are permanent in our 3-dimensional realm so how can we expect image processing to understand our expressions and complex emotions?
For this final project, I started by creating an analysis program that would recognize a face, box it, and display the predicted emotions on screen. This looks similar to ImageNet Roulette with the way the content is overlaid on top of the image, but the content is more like Machine-Readable Hito and Holly. Faces are boxed in red, and the words are black with a white border. The analysis displays a combination of possible emotions including angry, disgust, fear, happy, sad, surprise, and neutral as well as a percentage of confidence the computer has that it is that emotion.
Busy Photos
99% Fear with a Smile
I started with busier photos from my life that I would post online, similar to the style of ImageNet Roulette. I chose to analyze photos I had taken from the ARGO Formal, an event that has great personal significance to me. All of the photos have very positive emotions attached to them for both me and my partner. However, when processed through the facial analysis, the algorithm would frequently get my face incorrect either emotion, gender, or my age. However, when it came to my white male presenting partner it had no difficulty reading the expression, age, gender, or his race. This demonstrates the difficulty facial recognition algorithms have with certain demographics over others. When technology can consistently and accurately interpret one person over this can be evidence of bias within the training set to be able to recognize white masculine presenting people over others.
In the first photo I analyzed it described my partner near perfectly as a white man aged 26 (his real age is 25), and 99% chance of being happy. However, when it analyzed my face it inaccurately described me as a white man aged 32 with over a 99% certainty that I am feeling fear (I was very happy at that moment). Again, analyzing another photo of just my face taking a selfie resulted with an answer of a white man aged 23 with a 45% confidence in the answer being happy and 32% confidence in my expression being fearful again.
A Misgendered Selfie at 32% Fear
This is at the heart of what Trevor Paglen hopes to recognize with his work, the fact that facial recognition is not perfect and can lead to issues with having a bias against or for certain demographics. This is especially important when we realize that facial recognition is starting to be used for policing (Broussard, 2024). This is extremely dangerous because wrongful recognition would be more likely to happen with people of color since systems have a harder time recognizing them (Broussard, 2024).
Another interesting aspect of these busy photos was that the algorithm would incorrectly recognize a face in different aspects of the photo like the background or in the folds of a dress. This phenomenon is the result of the computer not interpreting the 3D world correctly because it does not have these concepts of dimensions trained into it. Instead, the training is based on points of the face and alignment (Taigman, 2014). A face is just connected points so it can misinterpret faces in other shapes that share a similar resemblance in the eye of a computer like the human phenomenon of pareidolia (Natsume, 2018).
Me, A Pig, and 5 Other “Faces”
What is fascinating about these glitches is that the algorithm still attempts to assign race, gender, age, and emotion to these points even if they have no human connection.
Machine-Readable Lloyd, Jelaine, and Mike
To experiment with Trevor Paglen’s process more I also created my own version of machine readable people but with my family. Machine-Readable Lloyd, Jelaine, and Mike takes away the busyness of the background but keeps the facial analysis. I also wanted to practice the style of simple portrait images with dramatic expressions. The hope was that by eliminating the background the faces would be easier to recognize. I staged the photos and prompted the models to make expressions matching the program including happy, sad, anger, disgust, fear, and neutral.
To my surprise the facial recognition did just as poorly with my parents, Jelaine and Mike, as it did with me. The analysis program struggled to identify Mike’s face and his age analysis was off by at least 20 years less than his real age. The analysis program did even worse with my mother misgendering her in 48 out of the 49 photos analyzed. Additionally, it tended to get her face and expression completely wrong. Lloyd on the other hand had amazing results where the analysis consistently got close to his age, and got his race, his gender, and expressions nearly always correct. The smiling photos were analyzed correctly more often in the portrait photos which may be a sign the facial recognition program was trained off of a lot of smiling portraits.
Machine-Readable Mike. Notice the issues reading the face in the beard and incorrect race.
Machine-Readable Jelaine. Notice that the image was misgendered and labeled as sad.
These photos act as a direct example of how bias in training can provide better results for some people over others. The program struggled to understand the faces of women as well as people with complex facial features such as wrinkles. This not only shows a bias towards gender but also ages as Lloyd the 25-year-old man had the best results. Jelaine was misgendered consistently, similar to how Hito was misgendered in Paglen’s work Machine-Readable Hito and Holly. This brings to light how unreliable these facial recognition programs really are. This experiment was also very interesting because this program was developed to analyze photos posted on Facebook however it struggled with photos that more closely resembled social media posts and did better with ones that were simple portraits.
Machine-Readable Lloyd. Notice the correct race, emotion, and gender. The age is nearly correct.
Reflections
This project made me realize that there are more issues with facial recognition than just the inaccuracies. Facial recognition involves the categorization of people and makes a lot of assumptions in the process which can be very dehumanizing. On the surface it can be entertaining to see how wrong the program can be but there are some not-so-funny real-life implications of these errors. Especially when we consider how facial recognition is being used in policing, security cameras, airports, and so much more, these glitches can have serious impacts on people’s lives. It is especially important to use these systems ethically and train inclusively so these problems can be avoided in the future. Trevor Paglen’s work brings to light the issues with facial recognition software, how it can be harmful, and makes outright incorrect assumptions about people. His work acts as a window into seeing how problematic relying on artificial intelligence can be and how unseen systems of power may use these issues to their advantage. Paglen makes the invisible facial analysis systems visible by adding text from the systems onto the photos revealing the unseen analysis of our everyday lives and how biased they may be.
This final for me was about understanding Paglen’s process of experimentation with technology and learning about how repeated trials can uncover bias within these systems. My work was different from Paglen’s because it stayed very personal with only pictures of my close family. The images were also curated by me, and I only analyzed the images I wanted to. In the future I think it would be interesting to continue exploring this contrast between analyzing busy photos versus simple portrait photos to see which perform best overall. I also would like to look into how race plays a role in analysis more since that part of the bias did not get touched on because all of my models were white. Overall, I enjoy the workflow of staging images and using artificial intelligence to analyze or interpret them because I am interested in seeing how technology interprets the world differently and similarly to humans.
Bibliography
- Aguera y Arcas, Blaise. “Do Large Language Models Understand Us?” Dædalus, vol. 151, no. 2, 2022, pp. 26-43. American Academy of Arts and Sciences, https://www.amacad.org/publication/do-large-language-models-understand-us.
- Aperture. “Trevor Paglen on Artificial Intelligence, UFOs, and Mind Control.” Aperture, 12 Mar. 2019, https://aperture.org/editorial/trevor-paglen-on-artificial-intelligence-ufos-and-mind-control/.
- Broussard, Meredith. More than a Glitch: Confronting Race, Gender, and Ability Bias in Tech. “Recognizing Bias in Facial Recognition,” The MIT Press, 2024, pp. 29-44.
- Hern, Alex. “ImageNet Roulette: Facial Recognition Website Backlash over Racist Results.” The Guardian, 17 Sept. 2019, https://www.theguardian.com/technology/2019/sep/17/imagenet-roulette-asian-racist-slur-selfie.
- IBM. “What Is Computer Vision?” IBM, https://www.ibm.com/think/topics/computer-vision. Accessed 5 May 2025.
- Paglen, Trevor. “ImageNet Roulette.” Trevor Paglen Studio, 29 Apr. 2020, https://paglen.studio/2020/04/29/imagenet-roulette/.
- Paglen, Trevor. “Invisible Images (Your Pictures Are Looking at You).” The New Inquiry, 8 Dec. 2016, https://thenewinquiry.com/invisible-images-your-pictures-are-looking-at-you/.
- Paglen, Trevor. “Machine-Readable Hito and Holly.” Trevor Paglen Studio, 9 Apr. 2020, https://paglen.studio/2020/04/09/machine-readable-hito-and-holly/.
- Paglen, Trevor. “On the Hilarity of Scoring Gender Likelihood in Machine Readable Hito.” HOLO, 2021, https://www.holo.mg/stream/trevor-paglen-on-the-hilarity-of-scoring-gender-likelihood-in-machine-readable-hito/. Accessed 6 May 2025.
- Natsume, Ryota, et al. "Understanding Fake Faces." ECCV Workshops, 2018, https://openaccess.thecvf.com/content_ECCVW_2018/papers/11131/Natsume_Understanding_Fake_Faces_ECCVW_2018_paper.pdf. CVF Open Access.
- Taigman, Yaniv, et al. “DeepFace: Closing the Gap to Human-Level Performance in Face Verification.” Facebook AI Research, 2014, https://research.facebook.com/publications/deepface-closing-the-gap-to-human-level-performance-in-face-verification/.