Surprise Machines

Surprise Machines is a visual investigation that will take the form of a digital installation at the Harvard Art Museums in Spring 2022. The project sets out to visualize and curate the entire universe of the museums’ collections, with the aim of opening up unexpected vistas on the more than 200,000 objects that make them up. To accomplish these surprise encounters, “black box” algorithms are curatorially employed to shape the visualizations, and a “choreographic interface” has been designed to connect the audience’s movement with several unique views of the objects.

From a technical point of view, Surprise Machines relies on the Harvard Art Museums’ API, which provides access to images and metadata of more than 200,000 objects. Making use of the PixPlot library from the Yale DHLab, the images from the API are arranged in the Cartesian plane using a UMAP algorithm. The interface uses machine vision to mediate the interactive gestures, and AI to interpret them in each still frame that’s reviewed in a Python script. Surprise Machines is a work-in-progress and will premiere as a digital, participatory installation next Spring.

Lins Derry, Douglas Duhaime, Jordan Kruguer, Dario Rodighiero, Jeffrey Schnapp and Christopher Pietsch - Surprise Machines from Information+ Conference on Vimeo.

Surprise Machines is a collaboration of Dario Rodighiero, Douglas Duhaime, Christopher Pietsch, Lins Derry, Jordan Kruguer, and Jeffrey Schnapp.

Curatorial A(i)gents presents a series of machine-learning-based experiments with museum collections and data developed by members and affiliates of metaLAB (at) Harvard, a creative research group working in the networked arts and humanities.