Awesome Machine Learning Art

🤖 🎨 🎸 A curated list of awesome projects, works, people, articles, and resource for creating art (including music) with machine learning.

Contents

• People to Follow
• Projects
• Articles and Talks
• Learning Resources
• Libraries
• TODO

People to Follow

• Tero Parviainen - Software developer, music hacker, and writer. Building the design tools of the future at creative.ai.
• Gene Kogan - an artist and a programmer who initiated ml4a.
• 大トロ(hardmaru) - Research Scientist at Google Brain, Tokyo.
• Douglas Eck - The leader of Magenta, Google Brain.
• Adam Roberts - Music researcher in Magenta, Google Brain.
• Kyle McDonald - An artist working with code. He is a contributor to openFrameworks.
• Mario Klingemann - Artist, Neurographer, Coder, Data Collector, Archivist, Artist in residence @googleart.
• Memo Akten - Artist, researcher and philomath working with computation as medium, inspired by the intersections of science and spirituality.
• Robbie Barrat - Artist working with AI who is 19 years old and work in a research lab at stanford.
• Janelle Shane - Research Scientist in optics. Plays with neural networks.
• Daniel Shiffman - The greatest source of any topics on creative coding for beginners.
• Samim - Currently working for Google. Designer & Code Magician. Machine Learning, Flora-Fauna-Human-Computer-Interaction.
• Luba Elliott - Curator, researcher, organizer of several crative AI events.
• Nao Tokui - Runs a creative lab, Qosmo, in Tokyo. He is the creator of the "AI DJ" project.
• Sofia Crespo - An artists who is playing around botany, microscopy, and neural networks.
• Anna Ridler - An artists who specilizes in machine learning and drawing.
• Rebecca Fiebrink - The creator of The Wekinator (an interacitve machine learning tool).
• Sofia Crespo - An artist based in Berlin. Her works are around microscopy, memetics, botany, and neural nets.

Projects

Visual

• Learn to see - 👁️ An artificial neural network making predictions on live webcam input, trying to make sense of what it sees, in context of what it’s seen before. It can see only what it already knows, just like us.
• art-DCGAN - 🎨 Modified implementation of DCGAN focused on generative art.
• Fast Style Transfer - ⚡ Extremely easy example for fast real-time style transfer in the browser.
• Dirty Data - 😈 What happens when you use ‘dirty’ data? Does the network learn anything? If so, what does it learn? Is there anything interesting we can get out of it?
• Everyone Dance Now - 💃 transfer any person into a professional dancer immediately.
• Fall of the House of Usher - 🎥 12-minutes animation. Eash still is generated by a neural net (pix2pix) trained on the artist’s ink drawings.
• What I saw before the darkness - A neural network imagines a person. Then, one by one, neurons in the network are being switched off...
• Drawing Orientations
• neural-style-pt - A PyTorch style transfer implementation. Easy to install, runs on all operating systems, has extensive wiki guides, companion scripts, and other neural models.

Music

• Magenta - An open source research project exploring the role of machine learning as a tool in the creative process.
• The Infinite Drum Machine - 🥁 Thousands of everyday sounds, organized using machine learning.
• rapping-neural-network - 🎤 Rap song writing recurrent neural network trained on Kanye West's entire discography.
• Beat Blender - 🥁 Blend beats using machine learning to create music in a fun new way.
• Melody Mixer - 🎶 A fun way to explore music using machine learning.
• Performance RNN - 🎹 Real-time performance by a reccurent neural network (RNN) in the browser.
• Neural Beatbox - 🎤 RNN-based rhythm geration + audio classification = fun!
• AI DJ - 💽 A live performance featuring an Artificial Intelligence (AI) DJ playing alongside a human DJ. It won “Honorary Mentions” Award at Prix Ars Electronica 2018.
• Sornting - A game based on a musical machine learning algorithm which can interpolate different melodies. The player has to listen to the music to find out the right order, or "sort" the song.
• RUNN - A game based on a musical machine learning algorithm which can generate melodies. The player has to finish the side-scrolling game to listen to the full song.
• Jazz RNN - Listen to the jazz created by an algorithm.

Text

• Generated Recipes
• GPT-3 Creative Fiction - Creative writing by OpenAI’s GPT-3 model, demonstrating poetry, dialogue, puns, literary parodies, and storytelling.

Interactive

• The Wekinator - It allows anyone to use machine learning to build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more. It's free and open source.

Misc

• Machine Learning for Creativity and Design 2019
• Machine Learning for Creativity and Design, NeurIPS 2018 Workshop - 👨‍👩‍👧‍👦 It features 35 papers about mahcine learning art, incluing a wide range of different disciplines.
• Runway - It is a toolkit that adds artificial intelligence capabilities to design and creative platforms.
• Autonomous Trap 001 - the artist used ritual magic to trap self-driving cars.
• Fake New Generator - The model can generate almost meaningful text from any title.

Articles and Talks

• Machine learning for artists (a.k.a ml4a) (Gene Kogan) - This article compares the emerging of ML in art as the case of CV in early 2000s.
• Artists and Machine Intelligence - A program at Google that brings artists and engineers together to realize projects using Machine Intelligence.
• MusicVAE: Creating a palette for musical scores with machine learning
• Generating Abstract Patterns with TensorFlow
• BBC Sounds: The arts and artificial intelligence - A painting by a GAN model is sold for $432,500 USD in an auction (NOTE: the original code is written by Robbie Barrat, The Verge). The talk is joined by Mario Klingemann and Anna Ridler.
• The AI Art At Christie’s Is Not What You Think - Jason Bailey interview both Huge from Obvious and Robbie Barrat to investigate further into the controversial auction of Christie.
• How Generative Music Works: A Perspective - It's a website describing generative music interactively.

Learning Resources

Beginners

• TensorFlow.js - Intelligence and Learning (The Coding Train)
• Machine Learning with TensorFlow, ml5.js, and Spell (The Coding Train)
• Beginners Guide to Machine Learning in JavaScript (The Coding Train)

Medium

• Learning Machines - Taught by Patrick Hebron at NYU/ITP, Fall 2017.
• Machine Learning for Musicians and Artists (Rebecca Fiebrink)
• ml4a (Machine Learning for Artists)
• The Neural Aesthetic @ ITP-NYU, Fall 2018 - An amazing course by Gene Kogna. It's full of open mateirals about the machine learning art.

Adanced

• Neural Style Transfer: Creating Art with Deep Learning using tf.keras and eager execution
• Creative Applications of Deep Learning with TensorFlow (Parag Mital)
• cs231n - The notes accompany the Stanford Computer Science class CS231n (Convolutional Neural Networks for Visual Recognition).

Libraries

• tensorflow.js - ⚡ A JavaScript library for training and deploying ML models in the browser and on Node.js.
• ml5.js - 🤖 🖌 It aims to make machine learning approachable for a broad audience of artists, creative coders, and students.
• p5.js - 🎨 🎸 p5.js is a client-side JS platform that empowers artists, designers, students, and anyone to learn to code and express themselves creatively on the web.

TODO

• awesome-lint
• add profile picture of this repo
• add "For Non-Programmers" section

Contribute

Contributions welcome! Read the contribution guidelines first.

License

The content of this project itself is licensed under the Creative Commons Attribution 3.0 license.