The proposed project aims to create an innovative and interactive web art experience for users by building a text-based virtual character that responds to the user through the Face Mesh model. The primary objective of the project is to explore the relationship between viewer and machine learning model, rather than focus on its pros and cons. Through the metaphor of a romantic relationship, the project challenges the user to engage with the virtual character in a unique and exciting way.
The text-based virtual character will interact with the user through a series of popup messages generated by the chat GPT. Btw this text is also generated by chat GPT. Depending on the user's stay duration, different messages will appear, mimicking the dialog when an individual is trying to get someone's attention. However, the website does not provide a method to reply to the messages. The virtual character will also have a demanding characteristic, which will be implemented by gradually turning the text blurry based on the stay duration. Viewers can hover over the paragraph to read unfiltered text. The website also requires access to the camera and users to face the website; otherwise, it will show nothing.
The project will utilize ml5's pre-trained Facemesh model, which allows for facial landmark detection in the browser.
Description
Facemesh is a machine-learning model that allows for facial landmark detection in the browser. It can detect multiple faces at once and provides 486 3D facial landmarks that describe the geometry of each face. Facemesh works best when the faces in view take up a large percentage of the image or video frame and it may struggle with small/distant faces.
The ml5.js Facemesh model is ported from the TensorFlow.js Facemesh implementation.
In conclusion, this project aims to push the boundaries of interactive web art by creating a text-based virtual character that responds to the user through Face Mesh technology. By exploring the potential of machine learning as a tool for artistic expression, the project challenges the user to engage with the virtual character in a new and exciting way. While the virtual character will have a demanding characteristic, the project will provide a friendly and safe environment by screening conversations for sensitive content. By utilizing ml5's pre-trained Facemesh model, the project will demonstrate the potential of machine learning to enhance interactive web experiences.