The Recognition Machine(2018—)

A collaboration with photographer Antje Van Wichelen, sound artist Rokia Bamba, and the Troubled Archives collective. The Recognition Machine takes the form of a photobooth that matches visitors faces with those in the re-photographed portraits based on archival photographs from colonial archives.

The project draws a parallel between colonial and contemporary surveillance technology and the relation to race. In a colonial context, photography was used in combination with pseudo-science to establish false truths about race that fit a racist agenda. Today, artifical intelligence techniques are used in similarly questionable ways that contemporary critics like Ruha Bejamin have described as a “New Jim Code” and Safiya Umoja Noble have termed “technological redlining”.

A “ready made” model is used that has been trained on a data set of images pulled from the Internet labelled with emotions. Emotions make the subjective aspect of these labels apparent: who determines what appears to be a “happy” or “sad” face? In the data set several images of Barak Obama have been labelled “angry”; these judgements reflect the biases and prejudices of not only those who placed the images online, but the manner in which these images and labels were (semi) automatically gathered using web search engines and automation using code. Rather than reinforce the claims of these data models, the project wants to trigger discussions about how and why these algorithms work they way they do, and call their use into question.

Visitors to the photobooth are encouraged to place a copy of their photostrip along with others, the impromptu collection echoing the way modern data sets are gathered, often without the (full) understanding of participants

The photostrip produced by the photobooth shows three pictures, the visitor, the match from the archive, and an image from the data set used to train the algorithmic emotion classifier. Rather than simply giving an “absolute” response (the predominant matching emotional label), the model is used in a relational way. Given an image, the algorithm outputs a set of 7 “weighted probabilities” for each of the 7 possible emotional labels (for instance 33% happy, 27% sad, 8% disgusted, etc.). The archive is then searched to find the image which produced the most similar response. In this way the project creates an analogic relationship, you are (the visitor is) paired with an image that the model (mis)recognizes in a similar way. Combined with the subjectivity of the emotional labels, the idea is to attempt to construct a kind of empathetic link and solidarity between the visitor and the people whose faces are appear in the archive.

Photobooth with FER dataset images, photo strips on wall behind, standing visitors, Noisy Images, Cologne, 2018

The full FER2013 dataset (over 30,000 images labelled with one of 7 emotions) as displayed on the exterior of the photobooth

The project has been exhibited in Brussels, Cologne and Turin.

These are Situationist Times (2018—)

Collaboration with Ellef Prestsaeter and Torpedo Press (Oslo). These are Situationist Times is a digital interface to accompany a printed publication celebrating the 6 editions of the influential art magazine from the 1960s and speculating about a never realized 7th edition related to pinball. Prestsaeter and I conducted in-person interviews with creator Jacqueline de Jong at her home in Amsterdam. Using a custom-made “video wiki” editorial interface, we transcribed and edited the materials, adding cross references between the video and the magazine. The final interface presents an annotated interface to scans of each of the original Situationist Times magazines with cross linked audio/video commentary from de Jong. In addition to a online version, a tablet-based version (with one tablet dedicated to each magazine) has been used in a series of exhibitions: Oslo, Malmo, Silkeborg, Hamburg, and the Jacqueline de Jong retrospective at the Stedelijk Museum, Amsterdam in 2019. The interface was also adapted to a multi screen “cinematic” version as a “project room” installation at the New York Art Book Fair celebrating the launch of the book in the fall of 2019.

Reference points on the magazine scans indicate corresponding points in the video/transcription. Viewers can navigate via the video, transcrition, or the scans of the magazine.
Markers on the scans are links to commentary in the video.
The videos were transcribed and edited and the cross references made using a custom-designed “video wiki” editors’ interface.
Original touch screen and magazine installation, Torpedo Press, Oslo, 2018
Touch screen installation, Pinball Wizard, Stedelijk Museum 2019
White gloves were added to signal the preciousness of the original publications on display
Installation at the New York Art Book Fair, Brooklyn, 2019

The digital interface became a two screen continuously running video at the Art Book Fair
Situological applications: Digitizing the Situationist Times describes the process of creating the digital interface.

A network of one’s own #01: etherbox (2016)

Networks Of One’s Own is a para-nodal periodic publication that is itself collectively written in a network. Each of the episodes is thought of as the ‘release’ of a specific software stack, contextualised in its specific practice. The series aims to document a set of tools, experiences, ways of working that are diverse in terms of their temporality, granularity and persistence.

The title Networks of One’s Own refers to Virginia Woolf’s classic essay A Room of One’s Own which makes the case for a woman needing a space to herself to write. It was published in the historical context of Victorian England where women were expected to occupy themselves with domestic and marital duties rather than the autonomous practice of writing. Woolf’s claim for a Room Of Her Own is complicated by the fact that her autonomy depended on her ability to hire staff that she could outsource her household chores to. Networks Of Ones Own takes this text as an inspiration to rethink digital intimacy, dependencies and relations in networked practices. It wants to re-imagine how technical work and content work grow together, and radically questions the way tools and practices are shaping collaborative content and vice versa.

The idea of a publication series arose from conversations with Aymeric Mansoux during the development of the Experimental Publishing Master at the Willem De Kooning academy in Rotterdam in the fall of 2016. With the advent of inexpensive so-called “single board computers” such as the popular Raspberry Pi project, it has become commonplace to distribute “disk images” that can be loaded onto an SD card which then contain a fully working software stack. The idea of a dual-form publication (both textual and executable) suggested the possibility where unique software platforms could themselves be published in tandem with content (collectively) written using the platform itself.

Networks Of One’s Own is set up as a series of such dual-form publications that reflect upon and experiment with collaborative on-line practices in a time when commercial interests dominate digital working conditions. It appears at a moment when under the guise of data protection (GDPR), self-hosted, un-supervised network practices are under threat of being reigned in as potential liability. “Cloud services” offered via Internet access are typically provided by multinational corporations and it is easy to forget that the seamless experience historically originates in an inter-network linking together different autonomous and disconnected networks of varying scales: institutional, personal, educational and military.

The series Networks Of One’s Own is taken care of by related but independent collectives. For each of the episodes, we propose different experimental tools for situated writing and publishing. In this way, the series allows for showing multiple perspectives, to generate multivoiced forms of documentation. It makes the editorial process of each episode a test case for the same technologies that it reflects upon. Like the old adage of the PERL community (‘There is More Than One Way To Do It’), this project is about the heterogeneity of tools and of practices, ultimately foregrounding the inherent sociality of software.

The printed publication with SD card containing software to produce a local hotspot accessible etherpad setup
Through work as co-system administrator at Constant, we have developed a set of portable infrastructure to use in worksessions where the presence of a local infrastructure allowed us to rely on collaborative servers like etherpad without adding demands to a venue’s Internet connection.
At an early Constant organized event (Relearn 2015) a server bar provided local servers and network switches for participants to alter as needed.
Connecting to the etherbox to work on the presentation…
Together with An Mertens and Sarah Garcin, I presented the etherbox project at Situated Publishing: writing with and for Machines, at Transmediale 2017, Berlin

The Institute for Computational Vandalism (2014—)

Asger Jorn: Nothing is worse than stupidity combined with a never-failing memory.

Matthew Fuller: Although we can have doubts about the latter quality on occasion, this pair of assets is among the range of exciting features that computing offers. Computers are stupid in the sense that they do exactly as they are told. Their capacity for memory, like this function of stupefied perfect recall, is what makes them so effective for archiving, and indeed so disturbing as an agent of social control.

Computational vandalism means working with this and other qualities of computing; the capacity for repetition, speed, interpretation by combination, the layering of operations and so on. In this sense, computational vandalism works with the aesthetic, social, material and imaginal forces that are gathered as compositional terms within computing.

From Vandlist Iconophilia

Formed in 2014, The Institute for Computational Vandalism is a collective (Ellef Prestsæter, Nicolas Malevé, Michael Murtaugh) that explores critical uses of software in relation to archives. In particular, the group uses software as means of conducting “probes” in search of critical new forms and understandings of digital archives. The collective’s name refers to the Scandinavian Institute for Comparative Vandalism (or SISV in Danish), a group formed around Asger Jorn in the 1960’s. The idea of the ICV was born in the process of exploring the physical archive of the SISV’s 25,000-some photographs housed at the Museum Jorn in Silkeborg, Denmark. The groups work is in many ways the continuation of Active Archives initiated by Malevé with Constant in 2006 and a project which I started actively contributing to in 2008.

Exploring the file cabinet

A contour detection algorithm traces the forms detected in one of the archival photographs

A digital interface of photographs mirroring the organization of the physical filing cabinet
Symposium, Museum Jorn, Silkeborg, 2014

The contour tracing interface, on display at the Photographers Gallery, London, 2016
SIFT features

Algorithmic Arabesques: A Vandalist Coloring Book (2019—)

Collaboration with Ellef Prestsaeter and the archive of Guttorms Guttorsgaard.

The original image is photograph of a drawing and watercolor made by Norwegian artist Guttorm Guttormsgaard of a tree trunk.
Suspended above the drawing table is the original tree trunk from the Guttormsgaard archive.
The coloring book shows the results of a contour tracer and the “algorithmic speculations” of a model trained on “quick drawings”, interpreting the image from each of four rotations.
Visitors select a blank page from the table use crayons to draw their intepretation of the form.
The digital interface is a scanner and a single button: The drawing is scanned, and the QR code is used both to link and align the drawing with the original image in the archive, which is revealed on the central display.
The monitors shows the flow of images from the archive photo, to the contour tracer, and back through the scanner.

The website provides an interface to view all the interpretations of the archival images, as well as a means of uploading new scans/photographs of drawings.

Kurenniemi: in 2048 (2012)

KURATOR working with dOCUMENTA (13) commissioned Constant to develop a prototype online archive project in collaboration with the Central Art Archive of the Finnish National Gallery. The project relates to the work of Erkki Kurenniemi, pioneering electro acoustic musician and inventor of early synthesizers, who had obsessively documented his life.

Imagine a picture.
An horizontal picture of 2592 pixels wide and of 1944 pixels height.
Its print size is 36×27 inches.

The picture was taken on the 06th of November 2004 at 21h56:37. The document set contains 45732 pictures by Erkki Kurenniemi for the year 2004. Erkki took 223 pictures in 2004 between the hours of 9 and 10pm. Of the 45732 pictures present in the dataset, Erkki took 33712 at night.

In the folder where this file is located, there are 28 other pictures. They have been taken between 21h56:32 and the next day at 19h21:18. The folder Harrin bileet can be seen as a sequence of 21 hours 24 minutes 46 seconds of the life of a man of 63 years and 4 months at the date the picture was taken. It took 10 of a 400th second for the camera to take the picture.
The blink of an eye.

Collaboration with Nicolas Malevé, first in a series of works (to become the Institute for Computational Vandalism) looking at the ways software and algorithms can be used to rethink access to media archives.

Talking about the work at the opening, with sound artist Tarek Atoui. Kassel, 2012.
The digital archive was presented in a corner of the Kurenniemi installation at Documenta 13, Kassel
The installation contained Kurenniemi’s instruments and was the site of a performance of his music at the opening.
Here, the Hough detector algorithm is used to attempt to sort Kurenniemi’s digital diary pages to find drawings (by excluding text pages)
Documenta curator Joasia Krysa checks out the installation.
The data browser was developed as a way to provide a means of accessing hundreds of thousands of images that couldn’t (for privacy reasons) be conventionally shown. The interface provided a multi-frame view that instead showed the output of various algorithmic “interlocutors” like metadata or face detection, or other statistical techniques like a histogram.

A later installation in Aarhus, Denmark titled Data Radio explored the hundreds of hours of digitized casette recordings Kurenniemi made. The exhibition made use of a radio stream and (textual) chat room.