News & Blog

Data-Driven Design of Abstract Art

Posted on December 14, 2017

Post by Matthew Higgs, Data scientist at The Data Lab


An Example of Negative-Time Evaluation in The Intention Economy

Motivation

Every prototype you build is an investment of time and/or money, and it’s only when you start getting feedback on a prototype that you can start to assess whether it was a good/bad investment. What if there was a way to obtain feedback from consumers on an idea before the prototype is built? What if you could use such feedback to guide your investments? In a number of cases the technology already exists, so how can we build such systems?

Fair use of images

This article contains a number of images for nonprofit educational purposes to communicate a trend in generative design. All the images have already been published publicly elsewhere and the source of each image is given. Message me if you are unhappy with my use of any of the images involved.

Tinder for Art

There is a company called Wydr who have been referred to as the “Tinder for art”. Artists can upload images of their art to the app, and users can indicate whether they “dislike” or “like” the work. To give you an idea of the user experience, I created some mock screenshots for such an app:

Mock screenshots of a “Tinder for Art” application.

BREAKING NEWS: What do you think of the images used? Would you “dislike”/“like” them? What if I told you the images were not created by humans, but by a machine?

Generative Adversarial Networks

The machine-generated images in the screenshots above come from a paper authored by researchers working at the intersection of artificial intelligence (AI) and art. Importantly, the researcher’s experiments showed humans could not tell the difference between machine- and human-generated art.

Generators and Discriminators

The researchers trained an algorithm, called a Generator, to generate images similar to existing art, but dissimilar to art easily classified into existing genres of artistic style. More precisely, the Generator (G) continually generates images and learns through feedback from another algorithm, called a Discriminator (D), where the feedback is negative if the image can’t be classified as art or can be classified into an existing genre:

Training the Generator (G) through feedback from the Discriminator (D).

The feedback-loop used to train the Generator can be built because there exists a collection of images (WikiArt) all labelled “Art” and individually labelled by genre. This information is encoded into the Discriminator to simulate an art critic, and over time the Generator learns which images receive a positive reward:

Example images with negative and positive rewards.

Negative-Time Evaluation

So why should you care about a Generative Adversarial Network? Well, consider a “Tinder for Art” app that occasionally switches from images of human-generated art to images generated by a machine. There is an opportunity to gain feedback from consumers on an artifact before the artifact is built (i.e. “Negative-Time Evaluation”). If we consider the feedback to be a good proxy for the consumer’s intent to buy, then we can use the collection of negative-time feedback to drive design:

An Example of Negative-time Evaluation in The Intention Economy. Machine-generated images are evaluated by dropping them into the normal user (U.) experience.

Plagiarism

Note the algorithm previously discussed is trained to generate images that are sufficiently different from those it has seen before. This encourages the algorithm to be novel, and opens up very interesting discussions about the legalities of selling such work.

Tinder for [insert]

Generative Adversarial Networks aren’t just built for art.

Tinder for Fashion

Up in Edinburgh we have our very own “Tinder for Fashion” named Mallzee. To give you an idea of the user experience, I created some mock screenshots for such an app:

Mock screenshots of a “Tinder for Fashion” application.


BREAKING NEWS: Only one of these images is real, the other two were generated by a Generative Adversarial Network built by a team at Vue.ai. Such a system could enable fashion designers to obtain feedback on early ideas and maybe even identify emerging future trends.

Tinder for Music

A harder one to communicate in mock screenshots, Spotify and many other music streaming services already use consumer feedback to tailor playlists, and the tools to build generative models for music are slowly emerging in projects like Magenta. Linking the two together, it might not be long before listener feedback is used to generate popular musical derivatives.

Tinder for Celebrities

One last one just for fun. The images in the mock screenshots below were generated by a Generative Adversarial Network trained on images of celebrities by a team at NVIDIA. These are not real people, they are what the Generator thinks celebrities look like. Anyone want to co-create the characters in an RPG? Designer babies in 2050?

Mock screenshots of a “Tinder for Celebrities” application.

Discussion

This article introduces how Generative Adversarial Networks might be combined with Tinder-style feedback to achieve negative-time evaluation for data-driven design. This has some interesting implications.

Consumer -> Creator

By incorporating consumer feedback early on in the design process, such a system blurs the line between creator and consumer. This might lead to an IKEA effect, and managing the ownership of artifacts generated by such systems will require a system in itself. Smart contracts might be a solution here (e.g. Mattereum). It’s also possible that the algorithm designers become the artists and “art” becomes about creating artefact-generating processes rather than single artefacts. Why pay for one image when you could pay for a system that continually generates images curated to your tastes?

Economic Feasibility

I’ve painted a very simple picture of a complex disruptive technology, and developing and maintaining such systems can be costly. People/hardware with the necessary skills/specs are in high demand, and any project planning on using AI technology should assess the economic feasibility. Such assessments require predictions about future demand and this might be might be difficult in creative domains. However, this is innovation, and taking risks is part of it.

Next Steps

The Data Lab is an innovation centre focused on helping Scotland realise the value in data science. If you are a Scotland- based organisation or individual and the above ideas are of interest to you, then feel free to reach out.

Thank you for reading, and feel free to follow me on Twitter: Matthew C. Higgs.

View by Categories


Mailing List


To recieve updates from The Data Lab please enter your details below.