Fortune Staff

Change the World

Impact Segment

Sector

Prior Year Rank

Future 50

Overall Score

Sea Company Profile, News, Rankings

legacy

Sea Company Profile, News, Rankings | Fortune

Chris Morris

The sand is so hot in Florida that nearly all sea turtles born there are now female

/2022/08/08/florida-sea-turtles-female-hot-sand/

Sean Kerins says his “foot is on the gas” when it comes to hiring tech talent.

Arrow Electronics' new CEO says supply chain is doing better, but still not back to normal

Leadership

Arrow Electronics' CEO: Supply chain better, but not normal

Sean thank you so much for joining us to talk about arrow electronics. You know, most people probably don't know about the critical role that your company plays if they're looking to buy a car, like an electric vehicle or a smartphone or some high tech home appliance? Tell our viewers exactly what does Well Susie, we're a global technology distributor and that means we help some of the biggest names in technology get their products to market. And that could be everything from semiconductors to data center computers, to cyber security appliances, to software and cloud solutions, You name it. And you know, we like to say an arrow that if a device takes an electronic charge, there's a good chance that we probably played a role in designing it, building it and certainly selling it well last year there was a big shortage of chips, semiconductor chips and we all know that there were long wait times if you were looking to buy any kind of electronic device. What about now is that chip supply crunch over? There's been progress for sure. Are all the issues fully resolved? No. And the way we know that is we see, you know, lead times for all the semiconductor technology that we helped sell are still twice their pre pandemic average. And that means that, you know, if you just bought a car a few weeks ago or if you're still waiting on something you ordered for home automation. I think we got a little ways to go before supply and demand fully normalizes and those lead times can become more predictable. What are you hearing from your other customers? Besides the consumers, you sell components to big companies like Microsoft or IBM Panasonic, what are they telling you? So they're seeing a couple of things. One in fact in places they're still working their way through inventory that built up over the past couple of years while they waited for that, that famous golden screw. And they're also telling us that they're now starting to re evaluate their own supply chain processes, that they get better visibility to semi semiconductor supply going forward. And despite all of that, your company just reported very strong sales. In fact you told investors they were the strongest financial results of any year in the history of the company. So can you keep up that pace or can you do better in 2023, given everything you've told us, I think there's enough uncertainty out there to uh to tell us that, you know, looking much beyond 90 days is going to be a bit of a challenge. Having said that we still believe in the big mega trends over the long term and you know, we want to come out of this year, even if it's more challenging in a position to really hit the ground running in 2024 beyond. So we're gonna stay invested in our our growth priorities as best we can. So what if the U. S. Economy goes into recession, what does that do for your plans? Well, you know, I think this is a time where the phrase uh cautiously optimistic is very appropriate obviously if a recession were to occur more deeply, you know, we're certainly well prepared. I think the big question is you know, are we in for a moderate decline over a reasonable period of time or are we in for something sharper and longer lasting? And there's still I think a number of variables to play out including you know, the question around you know, when does the chinese economy get back up on its feet? Well whatever happens with the economy and the business environment, I understand that you are laser focused on hiring, tell us why what is your thinking on that? Well look, we think beyond this year we think the you know the big technology megatrends are all lining up in our favor. You know, we talk about the electrification of everything, you know, E. V. In the automotive industry is a great example. We talk about, you know, the automation that we're now seeing embraced in the medical device and healthcare industry. We think that's a great future market for us as well. So I think the job of any Ceo and his or her team uh in a more challenging year is to make sure they emerge even stronger uh than when they came into the year. And so we know exactly where we think growth will come from long term and for us that means more sales capacity, that means more engineers. And that means more people to be able to help our big customers and suppliers manage their supply chains. And I really don't want to take the foot off the gas when it comes to those disciplines uh for our prospects down the road. Well, are you finding it easier for Arrow to find top talent given that there are so many talented tech people looking for jobs right now? I wouldn't call it easy. Um and I would say attrition is still higher than it's been historically across the industry. So we've developed a pretty good motion around, you know, constantly looking to replace people that may have left at the same time looking for new talent to fill uh new roles that we're investing in. I think our our HR team does a really good job sourcing from all kinds of places. And um you know, I'd say we're keeping up pretty, respectively, you've also made it a priority to hire and promote more women in leadership positions. You've said we need to do better. How's that going so far? You know, Suzy, I've always believed that, you know, we're going to be most effective when we look like the world around us and so we're making progress, you know, building a more diverse uh workforce, not just in the individual contributor ranks, but also at the top of the house. I'm really proud to say that we now have four women on our executive committee, but don't get me wrong, there's still a lot more for us to do. Well sean, let's talk a little bit about you, you became Ceo of arrow just last summer and this is the first time that you've been Ceo, what's the most important leadership lesson that you've learned so far as you're running a huge fortune 500 company? You know, I'd say two things. The first is learning is paramount as a ceo. You now encounter things that maybe you never did before in your career. So you've got to be willing to probe and learn, uh, each and every day. The second is you've also got to learn how to let go and rely increasingly on others to get the right things done for your company. And I would say I'm making progress with both, but it is a journey. Well you said many times that you learned a lot about team building and leadership from playing basketball at Syracuse University, you are the captain, you were the star player. And what would you say that college sports taught you about leadership? You know, I think sports is a great proxy for what you'll find in the professional and working world. Uh, you just don't happen to know it at the time, but you know, being part of a team means you're part of something bigger than just you. Uh, and there's a lot of reward in that. But great teamwork comes with lots of trust and lots of sacrifice and that's a lesson that stood with me over time. I think the other things you learned from competitive support, competitive athletics at a young age is this idea of overcoming adversity when things don't go well, this idea of persistence and work ethic because you know, you don't get good overnight, you get good over time. Um and that's true of companies and leadership as well. Uh you know, you've got to continue to work on the skill sets that will make you better. Well, your coach at Syracuse University was the legendary Jim Boeheim. What's the best advice you got from him about leadership when it comes to coach Boeheim? He was the one that taught me that as a leader. It's all about the examples you set and the deeds you do not the words you speak. And that's a lot about, it says a lot about how I think about leadership now as the Ceo of Arrow electronics. Well let's talk about that because in the corporate world, the job description for Ceo has changed so much over the past couple of years. The expectations um from customers from employees and from even the public has really increased. So knowing all of that, how would you describe the way that you want to leave Arrow electronics? You know, I think listening is more important than speaking, I think being accessible uh to your leaders and their employees in the right ways at the right times, uh, means a lot to them. I think being able to show them that you'll never ask the organization to do something that you're not willing to take on yourself. And being someone who, everyone understands, uh, will ensure the company always does. The right thing goes a long way to, you know, our character and our values going forward. And I will, you know, remain committed to those thoughts.

This intimate CFO Collaboration dinner in partnership with Workday and Deloitte will gather some of the Bay Area’s top finance leaders to examine the capital allocation future over the next five years, with a lens on how digital capabilities and multi-cloud platforms support new ways to model, measure and improve returns on invested capital. 

CFO Dinner: Generative AI and the Future of Finance

Michael Schrage, Research fellow, MIT Sloan School of Management 
Interviewer: Michal Lev-Ram, Editor-at-Large, FORTUNE


Generative AI and the Future of Finance

Good evening. Welcome to Fortunes CFO collaborative. Uh I'm Geoff Colvin, senior editor at Large at Fortune. People ask me what that title means. No one actually knows. Okay. It's my pleasure to welcome you here. It is, it's great. Uh it's my pleasure to welcome you here tonight with my colleagues um uh editor at Large and Cheryl Estrada, who must be right in the middle someplace there. Yes, there. Uh senior writer of the very popular CFO daily newsletter. Uh Fortune CFO collaborative has been building a community of finance chiefs to help you navigate the changing landscape. We've covered lots of ground in the past couple of years, from the economic shock waves of the pandemic to digital transformation talent challenges, E S G. And many other major issues facing CFOs today, we're delighted to bring you tonight's dinner session with our founding partner, Workday and our sponsor Deloitte. Our theme tonight is another critical one for C E O s across every sector and that's the future of capital investment. We will explore how the impact of digitization and transformation across companies, financial markets and industries globally is changing both the meaning and measurement of capital allocation. Over the next couple hours, we'll hear from experts and finance leaders that would be you about how best to support new ways to model measure and improve returns on invested capital. Lastly, a brief reminder, before we begin this opening interview tonight is on the record for fortune purposes, but when we dig into the dinner conversation later, that is totally off the record. Nothing will be reported so cut loose. All right. Uh let's get right to it. It's my pleasure to hand things over now to Fortune's my hologram together with Michael Schrag research fellow with the M. I. T. Sloan schools initiative on the digital economy. Please welcome both of them. Thank you so much. All right, well if if Jeff doesn't know what senior editor at large actually means, I certainly don't know what editor at large means. But I will do my best here tonight. Um Michael. We are thrilled to have you here. Those of you who have heard Michael speak or lecture or know him know that my job is gonna be very easy. I just have to wind him up and he goes um but I'm of the manage expectations down. All right. But that's just a little bit of context. You know, it is interesting. We're all sitting here in san Francisco this week. There's been a lot in the news on chat. Gpt generative ai very exciting stuff. And so I'm gonna just open with that Michael. I know you've got a lot of thoughts but how should or could the cfos in the room be thinking about this technology and how it impacts their their role, their function. Their companies. What are your thoughts? Well I want to be really careful on the M. I. T. And stanford and of course open ai the whole notion of not just chat. GPT three and chat. Gpt four and so called large language models. Very, very interesting. But the fundamental issue is that of what's called generative ai generative machine learning and the fundamental aspect there is depending upon how you pre train things. You can instead of optimize something, you can generate a variety of choices, a variety of options. So with dolly and stable diffusion you can generate all manner you can use it, give me an image that does such and such. So basically what we're really moving to is the notion of how do we want to collaborate with generative intelligence? Do we want to coach it? Do we want to tutor it? Let me say that one of the things I always you know, part of my research things, I always listen to what people don't say. I think chad GPT is very interesting for chatbots are very very interesting for how do you write a paper? I just saw something in my my feed today about how warden M B. A professor you know has had has published a paper that he didn't write but the bot wrote it for him which says something about both the quality of generative ai and warden tenure but that's it separate issue, apology, cheap shot. But the notion of what does this kind of collaboration, what are people not talking about? They're talking about collaboration with language, They're talking about collaboration with imagery. I think increasingly we're going to be seeing these kinds of generative ai generative machine learning being used for financial forecasts, scenario generation etcetera. I'm going to be doing a workshop with a group of chief with apologies here strategy officers for these kinds of generative technologies because you know the famous um what was it called? The shell scenarios they used to be, they had Pierre, Pierre the scenario group, what's the best case, Worst case? Most likely case and they would gain things because people would always pick the middle one. Okay. I think what we're going to increasingly see our finance people, people doing financial planning, use these kinds of generative tools to challenge the fundamental assumptions of their numbers. And the last thing I'm going to say on this is as important and as significant and as pervasive as I believe, generative ai ml is going to become the issues of interpret a ble explainable and transparent algorithms become even more important because the notion that we're going to become particularly finance people are going to become dependent upon black box issues. I don't see that at all. So I think it's going to be a very, very interesting thing how you get your quote unquote bright young things to bring you scenarios, Bring you projections, yes. Bring you spreadsheets and bring you forecasts that that force you require you to engage in a way you didn't expect that you don't feel it's a waste of your time in any way, shape or form. How's that for a long winded filibuster answer. Um you didn't even look at your notes. So okay, I know you really wanted to talk about this cutesy acronym you came up with. Okay, today Raro Kpi. So so so what I'm holding in my hand are not really my notes, they're actually the notes from an executive class that I ran a fortnight ago. And one of the, you know, my quote unquote leading indicator for a successful executive session is do people do the pre work? And so one of the sessions I teaches on future of KPI machine learning KPI and I asked the attendees, the participants to answer, you know, very briefly, because you can't get people in executive to spend more than 20 minutes on homework. Um what's the most what KPI provokes the most insight, what KPI provokes the biggest argument what KPI should you stop using right now? Because it's counterproductive and what key performance indicator does your organization need to adopt in the next 12 to 18 months to reinforce and and to to affirm the strategic direction that you're on and this as you might imagine, leads to very interesting feedback and very interesting breakout groups in this regard. And you know, just to put my markers on the table, I think one of the political organizational and please don't hiss at the use of this word epistemological challenges that cfos faces? Are you going to be partners and collaborators around KPI s throughout the organization? Or are you just going to own the financial KPI. And this is why I like the way Jeff introduced this. It's not just about capital as financial capital digitalization forgive me is a universal solvent, it dissolves everything you you get to deconstruct things into all manner of data streams. So I am curious about and what I want to hear in the questions in the comments and the follow up is to what extent our chief financial officer is going to be concerned about not just financial capital. Human capital intellectual capital social capital. Okay. Those are the kinds of issues. Are you going to be peers and partners? Are you going to be senior partners? Are you just going to go along with how your chr oh defines return on human capital investments. Are you just going to go along with the lawyers on I. P. And licensing stuff or is there going to be are there going to be shared KPI S that you play a direct role in that regard? And so my joke about are a risk adjusted return on KPI s. Is that every organization I've worked with advisory or an executive has KPI some organizations manage those KPI s portfolios. Some of those organizations manage those KPI s. Silos. Two separate companies that didn't know each other. They were b two B companies. We were having a discussion of C. L. V. What is C. L. V. And they had done some in my opinion serious descriptive statistics, serious descriptive analytic research on Cl B, what did they observe? They observed that there were certain reference clients that if those clients were used to close the deal, the deal got closed. And here was the debate. How do you change the C. L. V. Of the reference clients that close the deal? How much more valuable are they to you? How is that reflected in your CLD calculations and you know what their answer was? They hadn't a clue. They had no clue. I'm hoping to be invited back to at least one of those companies. But I will say this that was very very constructive conversation and I was grateful to be a part of it. I was also grateful to play a role in provoking that conversation. The CFOs and both of those organizations took that issue very seriously. And my last point is one of them said you know what there's probably a difference between cl v customer lifetime value and CLM customer lifetime monetization. Is that semantics or is that actually something really interesting? Again, I don't know. But I'm very interested to your reaction on that. So, Michael is a little bit of an instigator on top of his other credentials. Um Talk a little bit more about KPI key performance instigator and that's right. We'll talk a little bit more about KPI you we talked before about KPI is that not only data rich but that are are learning learning from each other. What does that look like? So if you were. So the research that I'm doing in collaboration with others at M. I. T. And you'll forgive me for saying this. I think this is pretty bloody obvious. And there is a multi billion dollar industry, multi $100 billion dollar industry based on this. And for the people who disagree with what I'm about to say, I would encourage you to speak up on this. So the view is KPI s are seen as metrics a way of measuring. But let's look at what KPI s also KPI s and and the beauty of this. I looked at the attendees here. You all this is a I'll be sycophantic and suck up for a moment. This is a pretty sophisticated crowd on this. Everybody in this room understands software agents. So the Duncan experiment, the thought experiment I'd like you to consider is what if the KPI it's a software agent clb data flow. How do reinforcement learning, how does the C. L. V. Optimize things? Oh wait we'll go back to what I said a few moments ago about portfolios. Let's take these as your KPI customer lifetime value employee experience. Ebitda or free cash flow sales growth. What what does it mean to optimize individually collectively that portfolio of KPI s. What kind of role should the CFO the ceo the chr oh the board play in how those KPI s learn to optimize themselves, learn gee I'm prepared to trade off this gee there seems to be a positive correlation indeed, causal inference between include improved employee experience and customer lifetime value and you you define Nps as one of your sub components in this regard. So again, what aspects of human capital, financial capital, social capital, intellectual capital are the P. C. A. You know, principal component analysis attributes that contribute to maximizing slash optimizing those KPI s. I'll go right back to the way I set this up. Is that going to be you something that CFOs are leaders in partners in or enablers in the good meaning of enablers? Not in the codependent meaning of that is the researchers point that what was that multi $100 billion business that's already doing that kind of software agent around KPI ad tech customer supplier, customer bidding for conversion rates etcetera? What do we want to optimize around this? So Martek, I would argue programmatic advertising, I would argue is a canary in the mine for learning KPI KPI that learn? I believe quite strongly because I do advisory work in this area, we will see those kinds of KPI S that learn in supply chain. What's the great obvious medium for doing this in doing this with digital twins. You test your hypotheses. Are you focusing on, optimizing or testing innovation hypotheses, that's what digitalization enables, that's the conversation we literally couldn't have had a decade ago. Um I want to ask you to drill down into one of the areas you talked about which is the talent side and again very topical and in the news, you know, we've seen many waves of, of layoffs on the tech sector side especially um talk a little bit about, I know you're, you're affiliated with the M. I. T. Sports analytics conference for years, lots of transformation and evolution there on how analytics is used to inform talent. So what, what can we learn from that? W A. R. Stands for wins above replacement. And I have a friend, I'll name drop Daryl Morey who's the general manager of the philadelphia 76 ers he was with with Houston Rockets. He was a co founder with Jessica gelman of the M. I. T. Sports analytics conference which began in the Sloan school classroom and now takes over the hinds arena in in in in boston pretty interesting. And by the way, you know, beauty of this is don't take my word for it is a wonderful chapter by Michael Lewis in the undoing project on Darrell's efforts at Houston, which is well worth reading. So what does win above replacement. Why did it emerge because you have all of these talented athletes that get paid literally millions or if you're really good tens of millions or if you're Lebron James hundreds of millions of dollars. So what is win above replacement when above replacement is a statistical analysis that gives it an indication of if if this player wasn't there and somebody else was. You know you guys can explain it to your your your your end of the table here that basically this is the added value shapely values. This is the added value that this player provides in this regard. I am increasingly seeing in talent oriented organizations performance analytics incorporating win above replacement sensibilities. If this software development manager, if this project manager, if this process manager is involved, how much more value do they add than our typical than our average? So I think you're going to increasingly see a mash up in integration between human capital management people analytics and contribution to economic value added to the firm. Is that under the rubric and under the aegis of the CFO or is that under the CHr oh I think the answer to those sorts of questions are up to the people sitting at this table. You like C. H. R. O. S. Right? You're not. I do you know in the in the spirit of full disclosure Cfos are not my natural my background is more innovation. Um I am not a CFOs are not my natural constituency here as my tax accountant would would would tell you. Um But whenever I work with organizations it's very very interesting that Cfos almost as much as corporate counsel are the ones who either veto or enable interesting new ways of analyzing how value is created, shared and understood throughout the enterprise I. E. To what extent do you want CFOs to become C. C. O. S. Or C. C. A. O. S. Chief capital allocation officers. The go to team. You don't want data. Yes, you need data analytics. But to what extent do the CFO does the CFO's office is in a position to analyze. Are the different parts of the organization using the right kind of KPI s to assure on a risk adjusted basis a good return on financial capital, human capital etcetera. Okay, well hopefully this has given you plenty of food for thought. Michael has lots of other acronyms if you would like. My meantime the acronym is the KPI for me but we'll let you eat your actual food now please enjoy your dinner. We're going to continue the conversation at the dinner table and again this portion will be off record. Um So thank you so much Michael for setting setting the stage here and I look forward to your off the record responses. Thank you very much.

For years, celebrities have held director seats and provided expertise within boardrooms.

Are celebrities valuable to boardrooms?

there are some growing trends in terms of celebrities that come into the boardroom, I'm here sitting on this board because I love the brand, I love the story, I love the people, I love the lives that you guys change. We see so many examples these days, you know, I always need to be a kodiak fan, I know how valuable it is to maintain a ritual focused on staying well. They want to have celebrities that their directors or even investors or even angels to their, it has been shown that it can generate some positive responses from external stakeholders, such as investors, given that they decided upon this particular celebrity and they decided to appoint them to the board. How does the stock market react when they publicly announced? I'm really excited, join Passmark because it really speaks to my core values and we basically find a positive reaction to that. It's even more favorable when the individual has what we call related business experience, it's not like just a flash in the pan where bang, we announced the appointment of celebrity to the board. The market likes that there's an uptick in stock price, we find that there's some sustained growth and sustained returns provided to the firms. In these modern corporations, boards are like the centerpiece of corporate governance mechanism, you know, these directors are appointed to act on behalf of shareholders and they served several roles right, monitoring the executives, they have to provide advice or resources through their networks um or expertise to the management and they are involving in important strategic decisions. So some of those celebrities can be, you know, business people um like Margaret Whitman formal ceo at HP who was serving as a director at Ebay. We know that Oprah Winfrey is also a director at weight watchers. Maybe you have a celebrity director that helps increase awareness of the company, You have a good finance person, you have a good accounting person, you have a good HR person, you have some with industry skills, that's a portfolio of skills, I suppose everyone having the same skills in papa, john's O'neill is appointed as director which had a significant impact and really, you know, when these celebrities are appointed um they can be a better ambassador almost um because um communicating strategic decisions, it's not always easy consumers or investors, they want to hear those strategic decisions, but you know, it would be better communicated through celebrities rather than through say, annual reports. You are able to do investor attention, invest their interest to the company, you're able to increase its visibility. That is an important contribution to the firm, but also at the same time, you know, appointing these celebrities is a little tricky in terms of effectiveness. The celebrity factor on the board may create a culture within the board that becomes ultimately dysfunctional. Start factor has a certain aura that may Intimidate other directors from really the full discharge of their fiduciary responsibilities. There is that attendance issue for example, J. M. Smucker noted that their celebrity director Nancy Lopez is a professional culture is the only director to attend less than 75% of the all board meetings in 2018 and similarly should kill O'Neill also attended less than 75% of board meetings at Papa John's in the year after joining the company. And so there are definitely more room for these celebrities to be part of corporate America. You're putting context without the other business people, other leaders that may open up opportunities in directions that you don't normally have or would be engaged with. And by bringing in those celebrity individuals as directors, it can signal that, you know, these firms are capable of bringing in those individuals and also there could be some positive spillover effect or halo effect coming from that.

Kevin Scott, Chief Technology Officer and Executive Vice President, Technology and Research, Microsoft 
Interviewer: Jeremy Kahn, FORTUNE

Brainstorm A.I. 2022 - A.I. And The Promise of Large Language Models

Conferences

A.I. And The Promise of Large Language Models


Brainstorm A.I. 2022 - A.I. And The Promise of Large Language Models


Welcome back everybody. Welcome to day two of Fortune brainstorm AI. We have got a great program ahead. We're going to explore the amazing advances made by large language models, and then we're going to get a firsthand account of the Ai focused digital transformation at a snack food giant. We're going to close out our day with a keynote conversation with Reid Hoffman at the end. So many of the people who have joined the conversations here at our gathering offer great examples of the kind of purposeful leadership that we want and need more of in the world. And that's exactly what led Fortune to create. Our newest membership community called Fortune connect. It's designed to help prepare those who aspire to become leaders of the future and we're really, really proud of this program. It's seen tremendous growth since its inception just 2/2 years ago. The goal is to create truly purpose driven leaders and to help them understand the dynamics of stakeholder capitalism. What it means, how it works when society is demanding so much more from companies such as yourself. The program is designed for mid career professionals who are rising fast on the executive track. I'm sure you have folks like that in your organizations and I would encourage you to consider putting them in this program. Um please check it out at fortune connect dot com when you can. Okay, before we start, I'd like to thank our partners for their work in bringing us all together here are presenting partner partner accenture and our partners C three ai and intel, we greatly appreciate your work in making all of these conversations come to life now let's get to it. They're called foundational models giant Ai systems trained on vast amount of text sometimes combined with images or even video. The results are systems that have achieved breakthroughs in a range of natural language tasks, from writing to summarization, to question answering as well as generating new visual content. Many see the models ushering in a revolution as these systems are deployed to accomplish knowledge work that was previously previously reserved for us. Lowly humans. But does big tech have a blind spot when it comes to the limitations of large language models? How do we assess these risks? We are really really fortunate to be able sit down with a veteran engineer at the top of the industry to share his insight into the promise of foundational models and his views on responsible ai. So please welcome kevin scott, Chief technology officer and executive Vice president of technology and research at Microsoft for a conversation with fortunes, jeremy Kahn. Hello everybody. Thanks for joining us this morning. Um Kevin, I want to start out by asking you why are these large language models and large foundation models? Such a big deal. Why do you think they're so important? Well, I think a bunch of people are getting clued into why they're a big deal. If you just look at what's happening on twitter with chat GPT over the past handful of days. Um so I think the thing that's really interesting about them is uh this very idea that they are foundations that they are platforms on which you can build a whole bunch of other things that they have just become powerful enough, where you're no longer in this mode that you have to do machine learning, where you have a narrow slice of data, your model, your training pipeline, and you're building something that is narrowly focused to a particular feature or product. Like we are now able to invest in building these these really big things and then thousands, tens of thousands, hundreds of thousands of people can use them to build all sorts of incredible things uh which is great because one of the things that it does, it really opens up the aperture of who can actually use Ai now. Uh so you uh will need new sorts of specialties in the future, but you don't need to have a PhD in computer science anymore to build an AI application, which I think is really, really exciting. Yeah, fantastic. Um I know that Microsoft uh you own GIT hub and with GIT hub you created something called co pilot, Can you tell us a little about what co pilot is, how it works and how you see that as a model for other products and features that you're hoping to roll out going forward. Yeah, so co pilot is the thing that we built in uh collaboration with open AI. so um we had this epiphany pretty early in the development of GPT three, that these large language models were actually able to write code. Um And so we uh invested additional you know energy and focus and built this product called Get Up Co pilot which is a coding assistance, so it allows a developer to express in natural language, a piece of code that they would like help writing and it admits it. Um and it it is you have been one of those things where the productivity gains are high and maybe more importantly the amount of love that developers who use it express for it and like how quickly it's become an essential part of their workflow has been just sort of surprising and wonderful to see that the you know, the thing that I think is interesting is that uh it is like get up co pilot for coding assistance is just 11 sort of co pilot that will exist in the world. Like I think in the limit you will have lots and lots of these things helping people with a pretty wide range of tasks, whether you are a video editor trying to pull together a piece of content or you're a journalist doing research or writing an article if you are a doctor trying to deal with your caseload, Like almost everything, I think that you can imagine where you're doing cognitive work, you will be able to see things like co pilot assisting with that work for everything, correct. Um How do you respond to customers who are really interested in this? Think this sounds you know like something that might help them in their business, but they're basically kind of afraid of using these models because they didn't train them themselves. They don't really understand the biases. I mean how do you sort of respond to customers who have those concerns? Yeah, like it's a thing that we hear all the time and I think the way that we get past this is we just sort of see how you you know, you build these dependencies like if you look inside of an organization, like any of you who run engineering teams, like this idea of taking a dependency on something that someone else built is just a hard thing to do in general. Um And yet we we do it all the time. Like people don't think about uh when they need to write a piece of software, do I have to write my own compiler or do I have to build my own operating system from scratch? Uh And so what what I think will happen. What we'll see over the coming years is that some part of the functionality of these big foundation models will start to look more like an operating system, a thing that, you know, it's just so complex where your organization probably won't be able to make the investment in building the most powerful possible thing um, and where like a small number of folks will be able to just because they are building a platform where the cost of building the platform gets amortized over a whole bunch of different things. Uh, and then we will have to figure out what the business model for this stuff looks like because you know, I think Bill Gates said once or at least it's been apocrypha li attributed to him that the true measure of success of a platform is when most of the economics and the value created on the platform, go to the creators, not to the person who built the platform, wow. Um, now I think we'd be remiss if we didn't ask you a little bit about this issue with co pilot, which is that as you know, has now been sued by a developer who says, well, my code was used in the training of this, um, but I'm not getting any compensation and I don't think, you know, is the proper use of the license of that. Is there anything you can say about that lawsuit? Well, obviously like, I can't say much about it. Um, but you know, the thing that I will say is like a bunch of this stuff will get figured out over the, over the coming years. Um, yeah, we they're, they're a bunch of different philosophical ways of thinking about how you build these models, uh, and I think we'll be okay as long as, you know, the world is building things that create value for lots and lots of people versus building things where you're sort of concentrating things into fewer and fewer hands, like that's where we will get into like real, real trouble. Right? Obviously, there are other concerns about safety around these models. You know, what are they gonna, will they produce biased content? How how have you at Microsoft? I thought about trying to mitigate those. Yeah, we we started many years ago before uh like transformers and deep neural networks were uh were even things thinking about how do you build a responsible AI practice? And it started as a thing in Microsoft research. We have now built a real responsible Ai discipline where we have an office of responsible ai that sits under brad smith and our legal team and we have a uh ethics committee that is composed of not just computer scientists, but social scientists and uh you know, folks who are experts in different disciplines that are very relevant to deciding, you know, what's safe, what's responsible, what's ethical when you are thinking about applications, these models, we have published a thing called the Responsible AI Standard, that's in its second version, it's like available for folks to have a look at. And like we have rigorous processes inside of the company, the try to evaluate uh what we're launching against that standard to make sure that at the very least we're being consistent with our own uh stated guidelines. And we're more than happy to have uh public debate. That's the reason why we published the standard about like where the lines ought to be and like we've drawn, you know, uh a bunch of very clear lines about like what a sensitive uses and like in in the cases of some of these sensitive uses, uh you know how you should deploy machine learning or AI systems, if at all. Right. Um Are there questions in the audience? I want to I want to let you all get a chance to to ask kevin anything that's on your mind. So please think of think of questions. Um there's one there go first. Please stand up and state where you're from. Hey, I'm Jill from capital G. Thank you so much. Um Obviously with co pilot, you guys had a really clear example of using the innovation from open AI sort of tightly coupled with the distribution of GIT hub. How do you think about other examples of that where open AI and Microsoft might work together. And I think you mentioned sort of other co pilot examples maybe, but a little bit more about how you expect that relationship to continue forward over the next several years. Yeah, I mean we we've got a really awesome partnership with open AI that's multifaceted. So like the biggest thing that we do together is we build ai computing infrastructure for training big models, so GPT three? GPT 35 which just got released chat. GPT like all of these things, like we're not only trained on Microsoft built infrastructure but uh the imprinting parts of what they're doing, so when you make an api call like that is going into like highly optimized infrastructure to support some like pretty technically complicated things that a lot of work went into. Um but we also collaborate with them on, you know, hey, here's an interesting application for these models. We have a Azure version of the Open AI A P I that runs inside of our cloud that is enterprise hardened and has all of the things that some customers both expected need from, from an enterprise version of something like this. Um And you know, we we even do a bunch of work together on safety and responsibility. Um So we've we have an awesome partnership with them. I just can't say enough good things about, about the people at open Ai they're just extraordinary folks. Other questions, there's a question down here in front, we can get a mic to this gentleman, please stand up and broad Meadows from the AI Foundation. I'm curious based on these new foundation models has the profiler skill sets that you hire for changed in the last couple years. Yeah. Um one of the big transitions that we've had and this has been going on for, you know, four or five years now, is that um what are machine learning engineers do uh in their job day to day has changed pretty dramatically. Um So yeah, I referenced it earlier, like it used to be the case that a machine learning applications team, inside of a product or feature team at the company, like had their own data, like had their own model, like they decided what the model architecture looked like, they would train it, they would uh like do the evaluation on the model that they trained, they would deploy it into some testing infrastructure and see how it performs and then they would sort of run their own like feedback cycle on top of it. Um More and more of what we're doing right now is like we have a very small number of foundation models that we use inside of the company that people use either directly like zero shot or one shot where there really is no fine tuning for a particular application or that you do a few patterns of like fine tuning to get it to perform in a particular way. Sometimes it's cost, sometimes it's uh yeah, you needed to have some performance characteristics in terms of quality that the base model doesn't have uh with, with no fine tuning. Um And so that like that's just a different mindset than the one that we had just a few years ago and like it's an interesting transition for folks in the organization to go through. Um and like oftentimes, it's a interesting thing and the teams who have little machine learning history, uh like can just go faster because they don't have a bunch of things, they have to go unlearned. Uh and they're just sort of open to a new approach rather than, you know, digging in and saying like, oh no, but my way is better. Um so it's a big transition happening. Um I think for folks who don't have their own specialized machine learning teams, like all of the foundation model stuff is almost nothing but goodness because it basically means that you have a more approachable set of expertise that you need to like get going. Um and that like, we, we still need that deep, deep expertise on like how to do all of the optimization, but like we can again, you know, focus that on foundations and then like that work gets, you know, advertised across the gazillion applications, kevin in 2020 you wrote a book called reprogramming the american dream where you kind of argued that, you know, Silicon Valley is this great place, It works really well, but we, we need to create this opportunity everywhere, not just in Silicon Valley. I'm curious if there's a, I have a role to play in that vision, I think it absolutely does. Um the the reason that I wrote that book is there, there's a vision of the future where um where these foundation models don't become platforms and where you see further concentration um into a smaller and smaller number of companies and you know, hands of experts that are mostly in like urban innovation centers in the United States and a couple of other places globally. And I think, you know the interesting thing if you concentrate on making these things available as platforms to lots of people and like you try to uh it's not even that they're free, but like if you get lots of people using it, you can drive the unit economics of what they cost to access even down. Um and it just creates a bunch of opportunity. Like if I look at the part of what I did in the book is I went back home to rural central Virginia where I grew up and I talked to a bunch of people who had entrepreneurial mindset and given these tools like without computer science degrees, like these folks will be able to see the opportunities that they give them and like they will absolutely be able to incorporate them into the things that they're doing and their businesses and to create new businesses that they hadn't imagined were possible before. And to me that feels really exciting, right? So I mean the idea is you could use machine learning in your business even though you have no background actually in it, you don't know how to code, you just instructed, you know, here's what I'd like to have this function do this app to and it is built for you. Yeah, I mean a lot of and and again you all are sort of sort of seeing it play out in real time. Like we we have uh we've been on this journey for many years now just because we we sort of had confidence that this is the way things were going for a while, but with chat gpt like what you're seeing is you can via prompt engineering with like expressing what you want to have accomplished a natural language, like you can get these models to do very very powerful things and so like that's just a much more approachable way of being able to harness the power of digital technology and like I've been talking about this for several years and like this is actually cribbed from Peter nor big, who gave a talk about it like seven or eight years ago where you know the thing that machine learning and ai does is it teaches, it transforms the task of Harnessing the power of your machine from programming to teaching. Uh and programming is an activity that's like hard to learn. Like you have to go to school, you have to practice like it, you know, believe Malcolm Gladwell or not like you probably need to put in your 10,000 hours to get really, really good at it. Whereas a toddler can teach someone how to do something like it is a thing that's inherent to us as humans. And so I just think it would be a more much more accessible way for people to get real power out of their technology than than this paradigm that we've had since Ada Lovelace, you know, 170 something years ago. And right now there's still quite a lot of art to kind of prompt engineering. And there's people saying, oh well we should start teaching kids how to prompt engineer. But I've heard other people say, no, I don't think we're gonna need to do that in five years. What's what's your view of? Oh I I think you are going to have to. I mean I was chatting with one of the teachers at my kid's school who saw chat GPT and they're like, oh my God, like now I'm not gonna be able to get my kids to write essays anymore because they can, you know, cheat, use chat Gpt to do the work for them. And I was like that is it's a pedagogical mistake to think that the essay like the artifact that you get at the end is the important thing about what it is you're trying to teach a student. Uh And so in a sense like nothing really is changing here other than you have this tool and like the student themselves like has to become the teacher to the model. So like the first thing they ask it is gonna be crap. Like it'll have factual mistakes that will the models. Like the hallucination rates like everybody should just sort of be tracking or like going down like you know chat gPT hallucinates way less than GPT three. And it's just getting better and better over time. But like still like the thing will make mistakes and so your job is like be the editor for this thing, be the teacher like coax it into getting you the output that you really need. Like that's the important thing. That's the special thing about us. Like the thing is just a tool so won't won't know much about history but we'll know a lot about prompt engineering. Like um No kevin thank you so much. We're just about out of time but thank you so much for joining us. This is a fantastic discussion.

Sea

Company Information

Change the World

Future 50