{"id":9962,"date":"2026-04-20T11:49:05","date_gmt":"2026-04-20T15:49:05","guid":{"rendered":"https:\/\/sites.temple.edu\/tudsc\/?p=9962"},"modified":"2026-04-20T11:49:05","modified_gmt":"2026-04-20T15:49:05","slug":"envisioning-metamorphosis-simulating-habitats-of-worms-with-point-clouds-and-unity-vfx-graph","status":"publish","type":"post","link":"https:\/\/sites.temple.edu\/tudsc\/2026\/04\/20\/envisioning-metamorphosis-simulating-habitats-of-worms-with-point-clouds-and-unity-vfx-graph\/","title":{"rendered":"Envisioning Metamorphosis: Simulating Habitats of Worms with Point Clouds and Unity VFX Graph\u00a0"},"content":{"rendered":"\n

By Jiayu Yang<\/p>\n\n\n\n\n\n\n\n

Introducation<\/h2>\n\n\n\n

Can you imagine becoming a worm in a beehive, in a jar, or on a disk in a laboratory? How would a worm perceive man-made structures? Kafka\u2019s The Metamorphosis<\/em> (1915) follows the transformation of the protagonist, Gregor, into a giant bug, offering a metaphor to look back at the alienating and dehumanizing aspects of modern human society. Today, when insects are increasingly integrated into human society, what could be seen if viewed through the senses of a parasite or a worm in a lab? More than postulating metaphors, my project at the LCDSS aims to model artificial habitats from a speculative worm\u2019s view in order to reveal the unstable captivity of nonhuman creatures.\u00a0\u00a0\u00a0

To recap, I am a documentary filmmaker and digital media artist working with wax worms (Galleria Mellonella), which were first found in nature as parasites of beehives, and are now proteins for pets, model organisms for immunology, and sources of plastic-degrading enzymes. In an era of species endangerment, wax worms are a unique case of thriving insects that are making habitats across the urban landscape. In my previous blog post<\/a>, I detailed the emergence of various artificial habitats for wax worms across apiculture, worm farms, and laboratories, and how I want to transport these habitats into VR environments through Photogrammetry and Unity. In this blog post, I will share details about my workflow and how the 3D models will be displayed and interacted with.\u00a0<\/p>\n\n\n\n

Becoming an Insect Through Visual Simulation<\/h2>\n\n\n\n
\"\"
Figure 1. Photo from In the Eyes of Animals <\/em>(2015)by Marshmellow Fest<\/figcaption><\/figure>\n\n\n\n

To simulate a worm\u2019s perception, which is haptic and non-visual, I find the format of point clouds to be of great potential. Point clouds are collections of three-dimensional information, with each dot representing the x, y, and z coordinates on the perceivable surface of a physical space or an object. Primarily generated for computer processing, point clouds are used to tell photogrammetry programs where a surface is located; their density indicates how much information is gathered and how confident the computer should be in reconstructing that surface. They are an intermediary translation of the physical world into the digital. <\/p>\n\n\n\n

Breaking solid space into contingent, porous clouds, point clouds reflect a nonhuman vision that alienates familiar physical spaces from the human eye. Today, point clouds have become a new aesthetic in visual art, with many visual artists working with them to simulate animal vision and represent the world in an alienated form. Examples that have inspired me to work with point clouds are Marshmellow Fest\u2019s In the Eyes of the Animals<\/a> (2015), Reality Design Lab\u2019s Echovision<\/a> (2024), and Refik Anadol\u2019s Echos of the Earth: the Living Archive<\/a> (2024).<\/p>\n\n\n\n

Methods <\/h2>\n\n\n\n

To transform a physical space into metamorphosing particles, there are a few steps I take: <\/p>\n\n\n\n

Step 1<\/strong>: Photogrammetry <\/p>\n\n\n\n

Step 2<\/strong>: Import to Unity VFX Graph <\/p>\n\n\n\n

In the following sections, I will walk through these steps in more detail.<\/p>\n\n\n\n

Photogrammetry<\/strong><\/h3>\n\n\n\n

Photogrammetry is a method of 3D scanning that interprets multiple photos of a physical space or an object and constructs 3D models based on the spatial information (x,y,z) in them. In my case, I used an iPhone camera, a Sony a7s2 camera, and an Insta360 camera to take the photos. Usually, I take around 70 to 150 photos from all angles, depending on the size and details of my object. With cameras that have higher resolution and higher shutter speed, such as the Sony full-frame mirrorless camera and the 360 camera, I often record videos first, then extract the frames from the recording as photos for photogrammetry. After all, videos are photos playing at 24 frames per second. <\/p>\n\n\n\n

After the photos were ready, I used Agisoft Metashape to interpret the spatial information. Metashape would first  \u201calign photos\u201d and then build a \u201ctie point\u201d (figure 2) out of them. At the stage of \u201ctie point,\u201d the 3D structure is sparse and lacks details. I then asked Metashape to build a \u201cpoint cloud\u201d by locating more points in the 3D space. With a point cloud, the model becomes more legible, while retaining a surreal look (figure 3). Then, simple edits such as \u201cselect\u201d and \u201cdelete\u201d can be done with the software. After some quick edits, I exported the point cloud as .ply files for creative use later. <\/p>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n<\/figure>\n\n\n\n

Figure 2. The tie points of the beehives.<\/p>\n\n\n\n

\n
\"\"<\/figure>\n\n\n\n
\"\"<\/figure>\n<\/figure>\n\n\n\n

Figure 3. Point clouds created by the tie points. 
<\/p>\n\n\n\n

Import to Unity VFX Graph<\/strong><\/h3>\n\n\n\n

Using Unity as the primary platform for viewing and animating the point cloud, I have learned to import point clouds as Unity assets with a point cloud importer\/renderer, since .ply is not natively supported in Unity. To import point clouds into Unity, I found the PCX package <\/a>shared by Keijiro Takahashi <\/a>(a renowned visual artist) on GitHub helpful. By installing this package in Unity, I was able to import the .ply file and render it as a position map and a color map. These maps are then used to tell VFX Graph how to position and color the particles.\u00a0<\/p>\n\n\n\n

Unity\u2019s VFX graph is an updated particle system, allowing faster, more efficient rendering of large numbers of particles. It uses a node-based structure to program the particles. To create a VFX graph, I first installed it via the Unity package manager. Then it became an asset I could create and add to my scene.\u00a0<\/p>\n\n\n\n

After creating a preset looping VFX graph, I went into the \u201cinitial particle\u201d block to add two more blocks: \u201cset position from map\u201d and \u201cset color from map.\u201d These blocks allow me to link the position and color maps to the VFX graph (figure 4). After that, I adjusted the spawn rate of the VFX graph to match the point count of my point cloud, so the particles would look exactly like the point clouds I generated.  <\/p>\n\n\n\n

\"\"
Figure 4. Using the position map and color map generated by the pcx package to initialize the VFX graph. 
<\/figcaption><\/figure>\n\n\n\n

Once translated into a VFX graph, the point clouds become clouds of particles that can be given lifetimes, sizes over time, velocity, or attraction. I can set them to flow with turbulence or slowly reconfigure into other shapes. The potential to manipulate and animate is infinite. 
<\/p>\n\n\n\n

A Work in Progress <\/strong><\/h2>\n\n\n\n

So far, I have built a scene of an infested artificial beehive in Unity. This is the first scene I built because these hives are where the worms first enter civilization as parasites. When set to morph with turbulence, the point clouds of the boxes of the hive become floating particles in this scene, as if they are bees or pollen (figure 5). <\/p>\n\n\n\n

\"\"
Figure 5. Point clouds transformed into flowing particles by the VFX graph 
<\/figcaption><\/figure>\n\n\n\n

I am building more scenes and working on the design of haptic interactions to simulate elusive visions or unattainable memories. My goal for the near future is to translate viewers\u2019 hand movements into forces that disrupt and reshape the point clouds of a habitat. This way, a viewer can see themselves dissipate the human narrative and reimagine these emerging organisms as something beyond the scientific captures. <\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

By Jiayu Yang<\/p>\n","protected":false},"author":37444,"featured_media":9976,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[447,297,2,352],"tags":[238,174,6,75,45],"class_list":["post-9962","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ecology","category-film-studies","category-grad-students","category-visual-art","tag-3d-modeling","tag-photogrammetry","tag-top-news","tag-unity","tag-virtual-reality"],"_links":{"self":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/9962","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/users\/37444"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/comments?post=9962"}],"version-history":[{"count":12,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/9962\/revisions"}],"predecessor-version":[{"id":10002,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/9962\/revisions\/10002"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media\/9976"}],"wp:attachment":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media?parent=9962"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/categories?post=9962"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/tags?post=9962"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}