Name: Brainstorm AI 2022: Salvador Dali Meets Wall-E
Uploaded: 2023-12-18
Duration: 10 min 17 s
Description: Mark Chen, Research Lead, OpenAI

Brainstorm AI 2022: Salvador Dali Meets Wall-E

Uncategorized

Brainstorm A.I. 2022 - Salvador Dali Meets Wall-E

Across our conversations. We've been discussing how A I is evolving and how it's being integrated into the fabric of all types of organizations. Now, we'd like to share a demo on A I's remarkable creative abilities brought to you by one of the world's leading A I research labs open A I imagine creating original digital images simply by describing with words what you would like to see in a bespoke demo. Let's meet Wally. A nod to both Wally, the 2008 animated movie about an autonomous robot and Salvador Dali, the surrealist painter. Please welcome to the brainstorm A I stage, Mark Chen research scientist at Open A I together with Dolly. Hi, everyone. Thank you for the kind introduction. And I'm really excited to be here at Fortune brainstorm A I and uh today I'm here to talk to you about the why, what and how of Dali two. And um I want to sincerely apologize if you're here to talk about chat G BT, but we'll leave that for another conference. So before I begin, I want to talk a little bit about open A I and our core mission. So our core mission is twofold. First, we want to build safe and beneficial artificial general intelligence. And by A G I, we mean systems that can match or outperform humans at most economically valuable tasks. It's equally important though to ensure that A G benefits all of humanity and not just a small slice. So earlier this year, we announced Dall E two and we released it in a platform called labs. And as Fortune pointed out, this represented a big step forward in text image generation. And the big breakthrough here was approaching photo realism by which I mean that at a quick glance, it's difficult to tell whether some of these images are computer generated or human generated. So at its core, dolly provides an interface where a user can provide a prompt which is a sentence, a couple of sentences of text and dolly can render a high resolution image that matches this prompt. So if I said something like fortune brainstorm A I conference in the style of Picasso, you might see an image like shown on the screens over here. So let's peer under the hood a little bit and talk about how do two was trained. So you can imagine Dali two at the beginning of training like a small child who has very little concept of what text or images are all about. And you start showing this child many flash cards one at a time. Each flash card has an image and a caption on it. And you can imagine that after seeing a million, hundreds of millions of these flash cards that the child starts to form some associations. Like when it sees the word panda, it might say, oh hey, in the image, there's a fuzzy animal that's black and white. And so it actually learns through seeing so many images and text pairs this own internal language which we're representing by this strip of colors for representing both vision and language. Now, in order to generate a new image, what the model does is it takes the prompt, let's say fall colors next to a lake and it turns it into its internal language or representation. And then it uses what's called a diffusion model to render it into a highly realistic image. Now, if you've been following our work since dolly one, this was using a technology more similar to GP D three where we're generating the image row by row and then column by column within the the each individual row. In contrast, what powers dolly two is a diffusion model and you can see a visualization of how these are working in the slide. So you can see that the image starts out as very grainy noisy textures. And over the course of generation it resolves into something that's crisp and sharp. So as I said at its core, do two has text to image capabilities which means that given a prompt, we can create an image that's true to its concept, attributes and styles. And just to illustrate this point, let's say that I have a prompt like the f the following say it's a theater with sunglasses and a suit, classic Hong Kong film, actor, action movie, close up with dramatic lighting. So even though the Otter wearing sunglasses and suits don't appear in our data set at all, our models able to generalize and basically compose all of these elements together to form a realistic image. And it's also not just able to combine these objects together. It also respects the mood lighting that you want and the overall ambience that you want. Another example here is if I ask for vibrant and smiling bowling balls rolling down a bowling alley in the style of digital art, it's creative enough to imagine that the finger holes could be eyes and the thumb hole could be a mouse and it really synthesizes it in the aesthetic of like this kind of neon bowling alley vibe. So text image isn't the only capability of dolly. There's also in painting in which you can make realistic edits to existing photographs. And to give you an example, say I start with this image of a yellow couch in a modern living room. And I don't really like this couch. I want to change it to something like a chic gray chair. So I can just mask out the middle part of the image and ask for a chic gray chair and I can get one in its place and notice that this respects all the lighting and shadows that are implied by the original image, the lights coming in from the right side and you can see that the shadows are generated appropriately. One other interesting thing is that you can change these edits in succession. So I don't have to stop here. I can ask for a modern fireplace on the wall. And so you can imagine this kind of very iterative editing procedure with images. Next, we have out painting which is powered by the same technology as our in painting. So what we can do here is we take an original image and draw around the borders of it to synthesize a larger arbitrary aspect ratio image. So let's take a famous painting, let's say the girl with a pearl earring and artist August Camp used our tool which you can kind of see visualized here and drew around this painting to visualize what could have been in the background. And you can see that plausibly this girl was in a very messy kitchen and living room. And finally, one thing we can do is variations, which means that we can take an input image and synthesize other images that look like this image. But with slightly different features. To give you an example, I can start with a dog dressed in formal attire. And if I run it through our variations engine, I can get these other images and you can notice that, you know, they're all dogs dressed in formal attire, but they have different ties uh are facing slightly different ways. And we've seen artists respond really well to this. They can visualize their work from different perspectives or take some art that they feel like isn't quite right and run it through the Variations engine to find out and and come up with new inspirations. So finally, I want to talk about a couple of practical applications of Dai. So when we launched Ali, one interesting class of users who came to us were urban planners and in particular, this one organization, better streets A I they were occupied with how do we show people what our streets could look like if we designed them around pedestrian experiences rather than vehicles? And you can see that here, they've taken some city streets and rendered pedestrian and walkable spaces. And this illustrates the power of showing people not just telling people next. When we were creating dolly, one tagline that we always thought about was we wanted to create an image generator for your imagination or image, search for your imagination. And Microsoft does this fairly literally. Um If you're not able to find the images that you want in, in being search, uh you're able to go and synthesize your own images. Of course, the potential for video games is also very clear. And in this case, we had a video game designer, create a character sprite sheet and also all the background and playable elements using dolly and actually create a playable video game out of it. One kind of niche but fairly heartwarming application is in generating graphics for children's storybooks. So children's storybooks, of course, they have this like very crayon drawn aesthetic. Um and it's these very vibrant colors and we've noticed that, you know, many people have been bringing their storybooks to life. Uh Here is one particular example where Mit Professor Phil Sola, he had a children's book that he wrote a while ago and, and made images for them. And finally, I want to showcase Ka which is an end to end uh fashion ideation and design platform. And this is an application where users can go and use dolly to synthesize different clothing that they feel like could suit them and get it made and shipped to them in the real world. So this illustrates one really strong point of using Dali. It allows you to personalize images so that they suit individual tastes. And this is also a big reason that I was so invested in creating Dali for myself. I'm not a very artistic person and I feel like it's so empowering to give myself the ability to create images that are personalized for me. And to conclude, we went through what dolly is all about how it works. Uh the different modes in which you can use it, you can use it for text image for in painting, for out painting, for variations. We went through some applications for how you can deploy in the real world. And I still think the most compelling applications for this technology are still to come. Thank you very much.