This piece will consist of a "magic mirror," a device combining an electronic display with an interface that remains hidden behind a reflective surface. This mirror will capture the image of an individual standing before it. It will then, deploy a deep learning algorithm that will search the web for that individual's information through the use of an image recognition algorithm. The algorithm will then sift through the sources it finds, narrowing down to one person. Once the algorithm has pinpointed a single individual and has reached a certain degree of confidence, it will proceed to extract information from a preselected list of domains. After the information is compiled, it will begin to be displayed on the screen in a seemingly magical way—their name, age, professional affiliations, where do they live, family bonds, trips they made, achievements, who are their best friends and other relevant personal data retrieved will cascade around their image in the mirror. All the operational mechanics described here will remain hidden from the user's view. The individual will then be able to engage in a conversation with what appears to be themselves: a profile created in seconds, capable of seamlessly impersonating them. The project also aims to make a bridge with Lewis Carrol's piece Through the Looking-Glass, and What Alice Found There in which Alice is transported to a reflection world ruled by a queen with twisted logic and the risk of our world being controlled by machines with alien logic. Since existing models capable of performing such tasks are predominantly proprietary and not open to the public, the development of this image recognition model will be undertaken independently in the residency. Due to the potential risks of privacy breaches, this model will not be made public. The intent is to provide a profound and contemplative experience that prompts discourse on privacy, digital identity, and the pervasive reach of contemporary AI-driven search methodologies.
Sineglossa poses the challenge of amaze in order to inform about the risks of misinformation. This artwork is designed to demystify the state of the art of AI algorithms and reveal how significant is the impact that they are capable of making on our lives. It transforms complex ideas of AI-driven data gathering and digital identity creation into an interactive, tangible experience, making the elusive concept of digital misinformation more relatable. As a long-time advocate for internet privacy, I often encounter people who believe that having nothing to hide means there's no issue with their lives being scrutinized. This project's hands-on approach aims to provide a fresh perspective, highlighting the depth and significance of this issue in today's world. The project uses the allure of technology not just as a tool for creation but as a subject of critical examination itself. The "magic mirror" is not just a display of technical prowess; it is a medium through which the audience can reflect on their own digital footprint, the reliability of online information, and the role of AI in shaping their perceptions of reality and truth.
The High-Performance compute made available by the project will be instrumental in training my deep learning networks needed for this project. Besides that, the allocated budget for software will be used to acquire ethically sourced data. I also believe that collaborating with the expert group. It will be extremely important to help me guide into how to create an interface that's both useful and beautiful to users. Their expertise in ethics will also be key in ensuring that the experience is educational rather than intimidating. I also have several legal concerns that I would appreciate help to solve.
I work as a software engineer for many years, I have been involved with the open source movement for more than 15 years and have been an activist for security, privacy and ethical use of technology for all this time. My background includes technical training in electronics as I have studied electrical engineering in the past. In the past I have worked with artists to bring the technical aspects of their art alive. Relevant to this project I have experience both in building electronics and making use of generative models. I believe these skills are going to be central for the development of this project since I will have to build the magic mirror and the deep learning models myself. More about these contributions are found on my portfolio information.
Timeline:
April - Initially it's necessary to source datasets and learn about the latest scientific research on image search technology. The goal is to start the development of a preliminary deep learning model, laying the foundational groundwork for the project.
May - I will focus on starting the development of the electronic design of the mirror. This includes researching and buying all necessary components and installing the basic operational system. Concurrently, a plan will be developed for the user interface and interaction design that the individuals will have.
June to August - By July or early August, the deep learning algorithm development should reach a stable state, meaning it's then a good time to buy a dataset to train the final model. This stage is anticipated to be the most time-intensive, as it requires extensive research, development of unique code solutions, and iterative testing with various datasets to achieve a refined result tailored to the environment of the magic mirror.
September - The task for this month is to refine the output of the deep learning algorithm. It is necessary to come up with heuristics to accurately narrow down the search results to a single individual. Once that part is done, we move into scraping the identified list of websites and incorporating this data into the AI, enabling it to accurately mimic the identified person.
October - The objective is to train a simplified Large Language Model (LLM) that can convincingly impersonate the person in front of the mirror. Given that I have created similar models before I expect this process to take no longer than two weeks. The remainder of the time will be spent on improving the graphic interface of the mirror.
November - The final month will see the integration of the developed model with the mirror, culminating in the completion of the product. I expect to have several small problems here as the interaction of hardware and software is never an easy one. One month should be enough time to complete the integration.
Budget:
For budget I will need to purchase the mirror parts. That will cost around 500 euros. Then I will need a computer to run the model and perform the search. If I am able to develop a model small enough to fit in an embedded computer that will cost 50 euros. If not it might cost much more, 2000 euros. There's also the possibility of using an embedded or a cheaper computer and run the model in a cloud service which I estimate to cost in total 150 euros. The dataset is hard to estimate as I've said before there are no models available to the public, prices could go as high as 10000 dollars here but I don't estimate I will need that much data and that high quality data. I know some freely available databases and these might be enough. I intend to use the 8,000€ for data scientists and computer scientists’ skills to hire a few hours of a senior machine learning engineer to review my deep learning algorithm and offer insights on how to improve, as well as in the beginning of the project to offer me guidance on what paths I should pursue.
As a generative machine learning artist I've created a software that allows people to generate entire universes with simple inputs. The software contains a world building model that requires the input of things such as agents and environments. With these it is able to create universes where these agents are alive: they each contain their own personality and thoughts. They talk to each other, visit different locations, reflect about their lives and plans and decide on what to do. This project was presented in an art exposition in New York City, US July 2023 - Little Martians with Vanessa Rosa. You can find the technical review of this work here: https://techforgoodresearch.substack.com/p/yoasi-a-generative-llm-based-universe And a presentation I gave about it in a conference in Hamburg, Germany November 2023 here: https://www.youtube.com/live/0Fp_lQ1oimE?si=VHgNTGxilB5I4Rhn&t=22209
Another example of my generative artist work can be seen in this report written by the art and science collective RADAR: https://centaur.radardao.xyz/ . For this group I have developed a chatbot that concentrates the amalgam of the collective's personality called the RADAR Mind. It's essentially a chatbot fine tuned on all the data coming from the community that has a specific personality.
On the electronics side I've setup the entire electronics part of this art exposition Berlin, Germany December 2022 (https://www.behance.net/gallery/185276969/Taming-the-Horizon-objects-of-separation), for that I have developed my own DMX controller. My main contribution to this exposition was the piece you see portrayed in the second picture. It required the combination of software and motion capturing hardware and as people passed through it the squares would be rearranged depending on how people would interact with them.
Besides this project you can see all the different kinds software projects I've worked on my profile: https://github.com/marimeireles
Mariana is a software engineer, cooperative AI researcher and artist from Brazil. She is interested in creating art that it's pungent and will cause people to "wake up" and reflect. She wants these reflections to be uplifting and create a net positive effect in the world. She hopes her unique perspective coming from the technical side of AI and her passion for many arts allow her to bridge the gap between the deeply technical and the deeply touching.
Hi all, I’m Mariana. The idea behind "Through the Looking-Glass and What You Found There" is to draw attention to the advancements of AI systems. The concept involves taking a user's picture through a mirror, behind the scenes we will have an algorithm searching for them on the web, filtering out their information, and feeding it to a large language model capable of emulating them. The user can then engage in a conversation with this digital version of themselves through this mirror, asking personal questions to discovery the extent of how well this software is able to emulate them. The model will simulate specific language traits of the person, as well as provide broad details about their life.