huyaqi

I found a website project: Clickclickclick.click (Moniker & VPRO Medialab, 2016), an interactive website with almost no visible content. When you arrive, a plain page shows a single green button. As you move your mouse, scroll, resize the window, or leave the tab, a disembodied voice narrates each action aloud (“The user has moved their cursor to the left,” “The user is hesitating”), and a growing log of “achievements” records what it has observed. The site never asks permission and never explains itself, yet it feels like it looking at you and maybe teasing you (and you are called subject somehow made you feel like you are the laboratory mouse).

What makes the piece work is not technical sophistication but framing. Nothing it detects is secret; it uses ordinary browser events. The magic comes from the voice, the timing, and the escalating sense of a presence on the other side of the screen. This is precisely the effect I want in my project, where a meditative “connect with me” intro leads into a mind-reading act powered by webcam eye tracking. Where Clickclickclick.click is cold and surveillant, I want a warmer, theatrical register closer to a mentalist’s stage act. Accoriding to Derren Brown describes “”in Tricks of the Mind (Brown, 2006).,audiences will attribute intention and insight to a system that simply reflects their own behaviour back at them.

Studying it, the skills I need become clear. Technically, I need front-end development (JavaScript, React, animation libraries such as Rive or GSAP), plus experience integrating browser-based gaze estimation. WebGazer.js (Papoutsaki et al., 2016) is the most accessible option, but its accuracy is limited, so I will need to design “tricks” around dwell time and coarse regions rather than precise coordinates, and build silent (alternative)fallbacks (mouse hesitation, timing) for when tracking fails. Visually, I need illustration and frame-by-frame animation skills for a hand-drawn UI, and sound design for the trance-like intro.

For research, the psychology of cold reading and the Barnum effect, so the “readings” land regardless of tracking accuracy; the ethics and UX of webcam access, including how to secure consent without breaking the illusion; and interactive narrative design, so the experience has a satisfying arc from invitation to reveal. Clickclickclick.click deliberately withholds its reveal; I want to decide whether mine should end with a moment of explanation, turning surveillance-anxiety into wonder rather than unease.

This week’s progress. This week I focused on scoping rather than building. I wrote out the user journey (question prompt, breathing sequence, gaze calibration disguised as ritual, the reading, and a reveal) and researched the tool stack, settling provisionally on React with WebGazer.js for gaze tracking, Rive for hand-drawn interactive animation, and browser-exposed device data (time, battery, timezone, referrer) for the “did my data leak?” moment. I have not written code yet. For next 2 weeks, I plan to build a bare prototype of the calibration-to-card-choice sequence to test how reliably WebGazer can distinguish which of four cards a user is looking at, and to start sketching the visual language for the intro screens.

References

Brown, D. (2006). Tricks of the mind. Channel 4 Books.

Moniker, & VPRO Medialab. (2016). Clickclickclick.click [Interactive website]. https://clickclickclick.click

Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J. (2016). WebGazer: Scalable webcam eye tracking using user interactions. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI 2016) (pp. 3839–3845). AAAI Press.

In “Machinic Rhetorics and the Influential Movements of Robots,” Miles C. Coleman (2018) introduces the concept of “machinic rhetoric” to describe how machines can act as rhetorical agents rather than simply functioning as neutral tools. One important idea in the article is the “hidden layer” between humans and machines. We can see what a machine produces, but we often cannot see all of the processes that produce its output. This hidden process can make a machine appear to have its own knowledge, character, or authority. Coleman uses examples from both historical and contemporary technologies to show how machines can become persuasive partly because they appear to act on their own. When the mechanisms behind a machine are difficult to see, users may attribute meaning and authority to the machine itself. In this sense, the machine is not only communicating information; its apparent autonomy and the way it presents information can also influence how people understand and trust it (Coleman, 2018).

I am interested in a similar “hidden layer” between the material machine and the meaning that people experience through it. I see an interesting parallel between Coleman’s machinic rhetoric and magic performance. In a magic show, the audience can see the magician performing an action, but they are not supposed to see the mechanism behind the trick. A magician friend told me that successful magic is not only about the physical trick. The lines, timing, attention, and story are also important because they guide the audience toward a particular interpretation of what is happening. The audience may begin to believe that something meaningful or impossible is happening, even though the mechanism is actually hidden. This makes me wonder how much of an experience comes from the material mechanism itself, and how much comes from performance, imagination, subjective belief, and rhetoric. This question is particularly relevant to my installation, where I want to use magic and ritual to obscure the mechanical and computational processes behind physiological data. The visitor may feel that the machine is discovering something about them, while the system is actually interpreting bodily signals through computational models. Coleman helps me think about this not simply as an interaction with technology, but as a process in which material mechanisms, forms of presentation, and epistemic authority are transformed into an experience of belief and trust. I want to explore this transformation further as I develop the project.

This week, I have been thinking more about how the ideas from this reading could become an actual experience rather than simply a theoretical concept in my project. I am especially interested in how the installation can create uncertainty about whether the machine is really “reading” something about the visitor or whether that feeling is being produced through performance and ritual. Next week, I plan to focus more on the form of the installation and determine how I want the interaction to be presented. I am currently considering using eye-tracking as part of the interaction, so I will investigate the skills, technical possibilities, and flexibility of available eye-tracking systems. I also want to learn more about how to use and design with this technology before deciding whether it should become part of the final installation.

Reference:

Coleman, M. C. (2018). Machinic rhetorics and the influential movements of robots. Review of Communication, 18(4), 336–351. https://doi.org/10.1080/15358593.2018.1517417

For my MA project, I am planning to design an interactive installation that explores the relationship between machines and human ritual. The installation will create a kind of “pseudo-spiritual” space in which a visitor is invited to sit down and interact with a mechanical display. The machine collect physiological signals such as heart rate, heart-rate variability, respiration, or skin conductance. These signals will become part of a ritualistic interaction involving guided breathing, a magic performance, and spiritual discourse. The key idea is to use magic to obscure the mechanical nature of the machine, and then use ritual to obscure the magic itself. The visitor may feel that the machine is discovering something about them, while the system is actually interpreting physiological data through computational models. I am interested in how scientific-looking data, algorithms, and machines can acquire a kind of authority similar to that traditionally associated with spiritual rituals. I’m also thinking about to develop the installation as a research project, using the collected data to explore physiological modeling, human-machine interaction, and the psychological effects of ritual and suggestion.

Two projects from the MA showcase booklet particularly stood out to me: Josh Kelly’s P.E.T. Sounds: Presence, Experience & Transcendence in Music and Ketterick Waddell’s What Do You Know From Funny?. Both projects use a personal or embodied experience as a starting point and transform it into a larger exploration of meaning, identity, and human connection.

Kelly uses podcasting, autoethnography, interviews, and philosophical reflection to investigate how music can create social bonds and give meaning to memory and experiences of transformation. Waddell similarly combines personal narrative with media, performance, and theory, but uses stand-up comedy to explore identity, oppression, emotional expression, and healing.

I find their combination of personal experience and broader theoretical questions really relevant to my idea. Their works make me think about the importance of choosing a form that is inseparable from the question being asked. Rather than simply presenting my research about technology and ritual, I want the installation itself to function as an experience of those ideas: the visitor should physically participate in a ritual, encounter the apparent authority of a machine, and only gradually become aware of the ambiguity between technology, interaction, and belief. This make me think of the final project a complete media experience with its own narrative, atmosphere, and critical dimension.