Name: Let the Games Begin
Uploaded: 2025-12-03
Duration: 22 min 16 s

AI has been core to the video game industry from the very beginning, as designers coded expansive worlds, responsive characters, and immersive experiences for millions of players. Now, how are today’s technologies—generative AI, agentic AI, virtual reality, and so on—changing how people make and play games? And can the design lessons of gaming, from its visual language to how users respond to incentives, be applied to other domains?

Let the Games Begin

Conferences

OK, so AI has always been core to the video game industry. How are generative and agentic AI and other digital technologies like virtual production changing the way people make and play games, and can design lessons of gaming be applied to other domains? Our next three speakers are well equipped to answer that. We have Haiyan Zheng as a general. and partner gaming AI at Xbox, who's heading up science efforts for consumer facing AI experiences such as gaming co-pilot. She was formerly chief of staff to the CEO of Microsoft Gaming and senior director of innovation at Microsoft Research Cambridge, where she served as technical adviser to the lab director. We have Lee Shueman who created and developed successful platforms, products, and devices at Nintendo, Microsoft, and Xbox. His work on rare LTD's Connect sports franchise earned him two BAFTA awards, including Best Family Games. Now, as chief product officer at Effecta Education Group, he leads the vision and development of Effeta's AI-powered education system. And finally we have Betta Liu, who is the AI lead for Justice Mobile, Netty's Games, one of China's largest PC and mobile game developers. She oversees large scale AI systems for the sort of Justice title, enabling players to create short form animations reaching billions of views across platforms like TikTok. She's also worked across product design and design strategy, exploring how AI design, and players themselves can develop new forms of expression. Fortune Asia editor and avid video gamer Nick Gordon will moderate this discussion and audience members, please also find headsets on your tables for simultaneous translation for one of our speakers. Thank you. Hello everyone, I'm very, very excited for this conversation. We, we have some content we're going to put up on screen, so let's, let's show that now. But, but you know, Lee, I want to start with you. You know, you're from, you know, Effect Education group, but there is a reason why you are on our stage today at this gaming panel. Could you talk a bit more kind of about, about your path through the gaming industry and then eventually at at Effecta? Yeah, yeah, thanks, Nicholas and hello everybody. So my name is Lee, Lee Shinneman. Um, I'm what you probably call a vintage gamer. I've been in the games industry since the mid 90s. You'll see some of my games playing up on the, on the screen behind. So, you know, a long, a long time caring about how to get players to love what they're playing and you know, and the importance of great design that brings all of that together. So 25 years through a whole range of You know, different devices, different consoles. I had the good fortune to work with some fantastic teams over the years, and for those that know the British, the British awards, I got lucky and won a few BAFTAs. In fact, good timing for this game that that's playing now. And you know, especially over the years I've got to work with some of the most talented game designers in the world, including people like Miya Motosan and Nintendo. And a whole range of like lessons learned about how to make playful experiences, make engaging, and now how to bring that, you know, into the world of education, which is, you know, why I'm on the stage today. I'm seeing Star Fox adventures. I have a soft spot in my heart for that video game which I played as a big player of this as a young. as a as a teenager, um, but Hayan, I want, I want to turn to you because you're, you're in, you're at Xbox, working gaming at Xbox, but you also kind of have a broader scope than that and we have a video we're going to show that that should kind of tell us a little bit of kind of like all the work that you're doing. OK. All right, I'm gonna try and replicate this here. We're off to a great start. And what this board does is I can connect into it through these wires, these um tiny coin cell motors. Oh my God. It makes me forget that I have a trevor. I one of them for a long time. So Haiyan, you know, how does this story kind of then connect to your broader experience in gaming, in design? How does this all come together? Right, thanks, Nicholas. Uh, so I wanted to bring this video today, uh, because, you know, this is a project I worked on several years ago before I came back into gaming, uh, that it's very rare for Us to be witness to these acute moments where technology can have this really profound effect to elevate somebody's day to day, where we can just see it happen that someone just a mundane thing like Emma drawing a straight line that she couldn't do because of her Parkinson's disease, that technology could elevate that moment and make that. This, you know, amazing transformational point for her. And as I moved into gaming as well, it also reminds me of the power of gaming and play in that, you know, at Xbox we talk about that our mission is really to bring the joy and community of gaming to every single person on the planet. And if you think about games, games are really this creative expression embodied through high technology that is really meant to elevate somebody's day to day, bring them to that moment of flow to these kind of new experiences, and that's how I think about technology, that to elevate people's experiences. So we have an opportunity now to kind of dive in deep into one game in particular, Sword of justice, Beta, that's the game you're working on, and we're going to kind of walk through some of the things that you're working on with this massively multiplayer online video game. Hello Aaron, my name is Beta Liu. I'm the game AR designer from Sword of Justice. So what is justice? It's a flagship open world MMO we launched at 2023. Uh, right now we have millions of players. So AI is not only a tech buzzword, but also their companions and also their creation tools in our game. In justice, you could customize your own companion from appearance to voice to their background settings. It will remember all of the interactions and involve over time. It's not only just a quest giver, but also a really true relationship. And also we provide all kinds of high freedom tools for the players with AI assisted, for example, the motion capture and the face sculpting and archi architecture building. So, uh, literally the players are building all around the world in our game. And also we help build a 3D TikTok in our game just like the uh the video you could just swipe and also uh shoot with their own avatars in just a second. This has, this has had millions, bills of players or the Chinese TikTok. So the design philosophy for us is very simple. The first one is agency. The world will react to every choice and the creation the players made. And the second one is interdependence, which means that we use AI to connect players to make sure that they feel they are part of the community. And right now we are working towards our um our end, which is a digital Western world. We are trying to shift from exploring maps to exploring societies. It's a world where the drama is unscripted and the possibilities are infinite. So Lee, I want, I want to turn back to you. You know, you've made this jump now from gaming to education. You know, what are some lessons that you think you've learned from working in game design that you're now bringing to this new industry? Yeah, well, let's let's first take a look at the pain of education, I guess, and I'm sure many of you in room of sat in, maybe not in Victorian time classrooms, but classrooms don't haven't really changed that much for many, many hundreds of years. And when you see us going into online lessons or Zoom meetings and everything else that we do, this kind of set up of sitting there all in a row and listening to a teacher and not really engaging with with the with the learning material and really focused on the learning outcome. So to the next slide please, Nicholas. So then. You know, at at EF sector we really care about making sure the student achieves the learning outcome, and I think, you know, many, many educational services around the world attempt it, but I think a big part of this is how to unlock the skill of the teacher. So what you're seeing on the screen here is the teacher is in front of a green screen. We've taken all the key lessons from live streaming and video gaming and brought it into this unique experience where the teacher is front and center inside the content. We run about 5 million classes, online classes like this every year, and you know that amazing set of information and data that we're getting now which will come to my next section in a minute. But I think what you can see from here on screen is, is, you know how we're taking lessons like immersion. making sure the teacher has agency and is able to teach how they want, how the student is able to learn how they want some of these visual tricks, and what you're seeing now on screen is the next iteration of that as we move into a world of agentic AI teaching. Now, you know, I should preface this to say we're not here to remove the human out of the loop. That's not the intention at all, but a big part of the world that we deploy our platform and solution in the global South, there's a huge shortage of teachers, maybe about 5 or 6 million shortage of English teachers, 8 million STEM teachers. So where we, you know, we're operating today maybe in Brazil in All across Latin America we care about giving those students that real opportunity to transform their lives and if that means then bringing AI into the class in a unique way and blending all of these different gaming elements, then you know this, this is, this, this really ultimately is that future. Um, you're seeing some different, different aspects on screen. This one now is just more about role playing for some different, different target segments, maybe not, maybe not in school, but we've deployed now to 4 million students across Latin America. We have new pilots rolling out across the world and again, like I said, the goal here is to now put agentech AI into the classroom, but there to support and sit alongside the teacher all the time. So high end, we take the slide down now, you know, I, I like going to turn to you. So what exactly are you doing specifically at Xbox in terms of kind of what you're doing with AI? Right, so, uh, you know, at Xbox we're really thinking about how to leverage AI to elevate the gaming experience, elevate that player experience, um, so one of the things we Launched recently is a gaming co-pilot. We launched this about 2 months ago on mobile and on PC, and the idea is that when you're playing a game, you can bring up a copilot and you can have a conversation while you're playing. You can, if you get stuck somewhere, you can ask questions and be like, hey, what's this boss? How do I, you know, what are some tips and trick tricks for getting past this boss? Um, you know, tell me about, I forgot what this mission was about. Tell me about the mission, and that might seem like super simple ideas, but it really, I think, um, brings to life gaming and what gaming is about because, you know, when, when a developer is making a game, they want the player to have that flow experience, to be um immersed in the game. As Lee said, to be really kind of deeply focused, but what happens is, you know, you get to a puzzle and you just can't solve it, and then you're like flipping to your browser, to your search engine, you're searching for stuff, you get on YouTube, you forget what you're doing, and then much later, maybe like a couple of weeks later, you come back to the game. So what we want to do is we want to make gaming even more fun, even more exciting through gaming co-pilot, and we're also thinking. About other AI features whether that's AI that sits on your device or AI that sits in the cloud that really highlights and makes special the gaming experiences you have with your friends. As someone who is constantly trying to look for the solution to a puzzle or a boss fight and not being able to find anything but a 3 hour long YouTube video. It can be tricky as a player, I can tell you, you know, but I want to go back to you. You know, given that you are working on designing and developing this video game, you know, beyond the AI conversation, what are some other lessons that you think can be learned about, you know, how to keep people engaged, how to make these games fun, how to keep them playing your game for a long period of time like MMOs are supposed to be? What are some lessons that that you've learned? Yeah, the first one would be, um, Will be the player agency and then the second one, I think it will be the social connection. Uh, for the player agency, we find out that uh the beauty and freedom is like the universal language across the scene. It's very essential to provide high freedom to for the players. Nowadays for younger players, the hardcore not only means combat, but also means the deep expression. The players will be willing to create and share what they created to Douyin, to Xiaohongshu, which will bring back a lot of traffic for the game. The second one is, um, social connection. We want to make the players will be needed in the game and um it will drive them back to the game in the long term. You know, Lee, I, I want to build on something that you sort of talked about as part again, lessons from the game industry to the rest of, to the rest of the industry, you know. Gamification, it is a buzzword. It's people talk about, oh, we have to gamify this. We have to give people scores. We have to turn this into a game, and that's how we engage people and I know from our, from our briefing chats that this is not a thing that you necessarily endorse. So, so, so what's the better way to think about about this idea of gamification and what should designers actually do? I mean, following on what what Betta just mentioned that that idea of player agency. is true in learning, you know, we, our platform is student driven, so by giving the student the ability to kind of drive that learning outcome as much as by themselves with that teacher there to facilitate, I think it's ultimately the like the core thing to lean in on. Um, you know, this idea of gamification has been around for many years. It tends to manifest itself as high scores, streaks, you know, all of these things that are that are meaningful for maybe certain social media products. But at the end of the day, like if you, if you're, if you're there to learn, you have to put in the time and then the effort. There's too much conversation. In around, you know, I can make learning faster. I can make learning easier, but really what you should be focused on is how to make learning more engaging, how to make learning learning more meaningful, and then from that point, take those elements to keep you in the learning experience, the same as keeping you in a gaming experience and finding ways to keep you motivated, motivated as a student. And you know, for everyone in here, we're all life lifelong learners, otherwise you know you guys wouldn't be here in this conference. So again, a lot of what we're talking about is how to maintain that ability to learn from, from the moment you start in school to the moment you're, you're, you know, it's. End of your days, but ultimately finding the ways to engage and bring all of these gaming elements that are really what gaming is about, which is like I said, that that giving the player control and agency and letting them take the lead and the fun that comes from that. Hi Anne, I want to go back to you and talk a bit more about kind of what Microsoft's doing with AI and gaming. I know there's this, there's the muse model, and I'm not going to remember everything that it can do, but there's, but, but you're experimenting with kind of these, these more expansive AI. AI possibilities, including for things like game preservation. I want you to talk a bit more about these other kind of more experimental things that you're trying to do. Um, well, earlier in the year in February, our research partners in Microsoft Research, they published a paper in Nature, and this is the first kind of video game AI model publication in nature. And what they did was they took a video game, and then they trained an AI model. On all the telemetry, all the footage of that video game, so that the AI model could basically simulate a playable version of the game without the original game engine. So earlier you heard Rockwell Group talk about unreal engine and talk about um how you know, putting visualization into the engine. Well, imagine in the future an AI model encompasses all. All of that and can simulate the world, um, whether that be a game world or a hotel interior, and it is all inside the model. And I think that the real capability of that is that now the AI can allow you to express to prompt for what you really want to do with that world, for what you want to do in that space. So as an example, this publication was called the Muse Model in nature. Um, as an example, the research. Team, they discovered that the AI model could do certain things like you could cut cut out a character and you could drop it into as you're playing a game, you could just cut and paste a character into your level and suddenly the AI model sees the character that you've just cut and pasted out of 2D pixels and starts animating it and starts making it come alive. So immediately it changes what you could do with video games. And I think this really can spark imagination about what is the future of play. And I think the challenge is always we don't quite, we we need that imagination. We need everybody in this room and creators across the world to think about what new things that you could do with gaming through the use of AI and I think that will really help us drive this new capability. And these are new models. This is not GPT, this is not Gemini, this is a new kinds of models that models that we call world models. Um, and as part of this, you know, we had a discussion about game preservation and um it's kind of a side topic, but game preservation is really interesting, you know, we engage a lot in backwards compatibility, making sure that all of the games of past generations can be played on Xbox on future devices. At the same time, I know that, you know, I love. And I love Dig Doug, um, and Cubbert, but younger, but that's because I'm more into the retro, the nostalgia of it, right? But if I put my 10 year old daughter in front of Cubbert, um, or Dig Dug, she's going to get bored very easily. And what I've discovered is that, you know, there are, uh, game preservation platforms out there, something like Antstream. I think is a fantastic platform and what they've discovered is in order to introduce games to a new audience, they've had to take the essence of the game and create new experiences. So they take a game and they allow you to jump directly to the key point in the game that's the most fun, and you can play and you can compete with your friends. So it's not exactly preservation of the original game, it's Introducing the essence of that experience to to a new generation, and I think there is now capability for AI for AI to learn, um, the backward, back catalog of games and to be able to allow new generations to. Experience these games in new ways. So these are all still speculative, um, but this is some of the research that we are driving forward in order to open up new capabilities for, for games that, you know, also beta is working on as well. Well, I think with that kind of very expansive view of what AI is bringing to the gaming industry, what the gaming industry brings more broadly to design questions and other industries, I want to thank Le Haiyan, Beta. Thank you so much for talking us through this huge industry today. Please give him a round of applause. Thanks.