Abstract
Using computer vision and sound analysis to create 'pataphysical films, all in real-time.
’Pataphysics will be, above all, the science of the particular, despite the common opinion that the only science is that of the general. ‘Pataphysics will examine the laws governing exceptions, and will explain the universe supplementary to this one. —Alfred Jarry
The Evolution of nSpace, is an automated-art-system consisting of custom-made equipment which uses computer vision and sound analysis to generate images which are computationally arranged into film sequences in real-time. This apparatus and its custom-coded software, is calibrated to be used with either an amplified phonograph record, or the live performance, of sound-art by Maurizio and Roberto Opalio (aka My Cat Is An Alien) in order to generate a parallel live-video feed of a unique science-fiction film. The film, although completely generated frame-by-frame in real-time, is designed to proceed through three distinct parts or acts. Each act is narrated through subtitles generated with custom-coded scripts for synthetic language which stems from, and contributes to, the ongoing creation of the film.
Phonon Conversion
The first performance introduces the ’pataphysical invention of ‘phonon conversion,’ the principle underpinning the generation of ‘Tonalytic Trajectories’, converts phonons (quantum units of sound) into photons (quantum units of light). (3) The synthetically generated narrative describes how, before the invention of the Phonon Converter, there was no known way to directly convert phonons, or the lattice vibrations in solids into photons. Additionally, the synthetic narrator describes how computational sound analysis and Cross-Modal Encoding, permits a quantum superposition that allows data (generated as sound) to be transmitted via photonic spectra. In other words, with phonon conversion, lattice vibrations in solids can now be harnessed to produce a photon beam capable of propelling and steering a spacecraft, by converting sound to photon-encoded data suitable for transmission via a synchronised audio-to-optical interface. (4)
Tonalytic Trajectories
A Tonalytic Trajectory system controls the movement of spacecraft based on photon beams produced by focused tones or music. Intentional music compositions are crafted, or, in the case of skilled musicians improvised, to direct a spacecraft along desired orbital paths. Different tones, rhythms, and chords produce different wave shapes, to sculpt the trajectory. (5) For example, music produced in a major key can send the spacecraft outward on escape trajectories; dramatic diminuendos may be used to slowly spiral inward; and complex polyrhythms used to enable intricate course adjustments.
Experiencing nSpace
The viewer is invited to directly experience travelling through nSpace through these new inventions under the “controlled conditions designed for safety and comfort”. As the viewer begins to move through various iterations of nSpace they may experience feelings of disorientation and other somatic sensations. These are expected. You are now entering nSpace. Welcome.
Note on the title of this system
The title of this project is taken from a series of photographs and related works Donnachie and Simionato began making and publishing in the early 2000s. For these projects, small objects (representing platonic shapes such as cubes, cones and spheres) were photographed on existing printed publications in ways intended to problematise the viewer’s perspective, either collapsing the illusion of depth, or otherwise disrupting our perceptual resolution, of the original, underlying image. We even created a series of anaglyph images of these same objects for a small exhibition in Paris, inviting the viewer to see the (2-dimensional) images of the (3-dimensional) objects through red/blue glasses. (6)
1 ’Pataphysics, as established by Alfred Jarry, is an artistic practice that challenges the determinism of classical Newtonian physics.; Alfred Jarry, 1996. Exploits & Opinions of Doctor Faustroll, Pataphysician: A Neo-scientific Novel. Boston: Exact Change, 21; Christian Bök explains Jarry’s ’pataphysics prefers to explore the “surrational potentials of such physics… what is random and absurd”; “’Pataphysics is speculative, waiting for its chance to happen, as if by accident, in a theme park of scientific conception.”; Bök reminds us that “Science suggests that what is probable coincides with what is most provable[…] not because they can be proven but because they can be probed. There are “probeable” systems.”; Christian Bök, 2002. ’Pataphysics: The Poetics of an Imaginary Science Northwestern University Press, 29; Andrew Hugill and James Hendler in “’Pataphysics and Computing” also trace an entanglement between artistic research and quantum computing through the lens of ’pataphysics.
2 The frames are derived, collaged, manipulated and AI generated from various sources and archives and are envisioned as a ‘quantum cloud’ —reference to the principles of superposition of quantum mechanics, where a particle doesn’t have a definite position until it’s measured; Superposition in quantum mechanics refers to the ability of a quantum system to exist in multiple states at once. The principle of superposition, as described by Paul Dirac, suggests that a quantum system can be in a combination of states, each with its own probability. This means that a quantum system can be considered as being partially in several states at once, without these states interfering with each other; “The intermediate character of the state formed by superposition thus expresses itself through the probability of a particular result for an observation being intermediate between the corresponding probabilities for the original states, not through the result itself being intermediate between the corresponding results for the original states” P.A.M. Dirac. 1947. The Principles of Quantum Mechanics (2nd ed.). Clarendon Press, 12.
3 The Tonalytic Trajectory uses NASA’s open-source GMAT system for space trajectory optimisation, navigation, and mission analysis. See https://gmatcentral.org
4 Phonons represent the quantum basis of acoustic propagation through solids. On a quantum level, the vibrational motions caused by the music involve discrete quantized excitations of the vibrational states of the crystal lattice— and these quantum mechanical vibrational quanta are called phonons. Through careful tuning of the system (in a probabilistic combination of spatially separated vibration states), applying a phonon at a certain frequency stimulates the emission of a photon in a predictable way while preserving the superposition. Detecting the output photon ‘collapses’ the superposition (into either one vibrational state or the other). In the interim, there is a direct linkage between the phonon input and photon output at a quantum correlational level through the shared phononic superposition. In other words, phonon conversion effectively translates between macro-scale sound wave vibrations and light emission events tied to underlying molecular phonon and photon dynamics.
5 “Photonics Reshapes the Future of Computing” in Photonics Spectra Feb 2024. https://www.photonicsspectra-digital.com/photonicsspectra/february_2024?folio=36; See also Programmable photonic integrated circuits (PPICs) which process light waves for computation, sensing, and signaling, in https://phys.org/news/2024-02-key-photonic-components-supercomputing-technology.html
6 Similar to the subjective experience of superposed acoustic phenomena such as Shepard’s tone, superposed light waves may conjure unique visions in viewers. These waves may be perceived by the audience as images, although studies in Quantum dream state visualisation reveal that due to the subjective influence photon wavefunctions have on human consciousness, each participant may experience unique perceptions of images and environments (in other words, results may vary); Traditional visualisation shows photon waves move linear forward, while they wobble and pulse in planes perpendicular to their direction of motion. Visually the wave shape oscillates up and down sinusoidally and cycles in intensity over space and time.
7 “Evolution of nSpace.” in Archistorm, Paris, France, 2010, curated by Catherine Geel; in Activities in Space and Time, Warm Grey Gallery, Paris, 2010; in “Pink Laser Beam,” This is a Magazine: Compendium #6, 2009.