Anthropic

Since 2017, BCG and Fortune have screened the world’s biggest companies for their innate fitness to grow—a measurable, manageable trait we call “corporate vitality.”

<p>Anthropic occupies a distinctive lane in the AI race: It builds for-profit AI platforms while devoting more resources than many rivals to studying the ethical and political implications of AI adoption.&nbsp;</p><p>
Such scruples haven’t kept Anthropic from raising money or generating revenue. The Amazon- and Google-backed company is currently valued at $61.5&nbsp;billion, and it’s reportedly on track to bring in $5&nbsp;billion this year, up from $1&nbsp;billion in 2024, thanks in part to the popularity of its Opus code-generating AI.
</p>

The Future 50

These companies are built to exhibit resilience amid volatility—and deliver better-than-average growth in the long term.

These 50 companies exemplify the innovation, progress, and potential of AI.

AI Innovators 2025-Anthropic

Fortune 50 AI Innovators

The essential guide to a sector that’s now attracting one out of every four startup investment dollars.

AI-Innovators-2023-Anthropic

Who can rival OpenAI’s ubiquitous chatbot ChatGPT? A team of ex-OpenAI researchers are hoping the answer is Anthropic. Former OpenAI research executives including brother and sister Dario and Daniela Amodei founded Anthropic in 2021 to create the safest AI systems out there—ones that wouldn’t spew misinformation or harmful responses as some chatbots, including ChatGPT, have been known to do. Anthropic’s chatbot Claude, released in early 2023, has some notable differences from its peers: The company claims the chatbot is “less likely to produce harmful outputs,” and describes its aim as being helpful, harmless, and honest. The startup also recently expanded the number of words Claude can process in one query to 75,000—the length of many novels—which it says enables Claude to accurately analyze technical documents like financial filings or legal contracts. (OpenAI has recently expanded the number of words ChatGPT can process to counter Anthropic's advantage.) Go-to-market lead Sandy Banerjee told Fortune that Anthropic has customers “in the thousands” using Claude, from Y Combinator startups to large public companies like Zoom. The company recently raised a whopping $4 billion from Amazon and is reportedly in talks to raise $2 billion more from Google. That is on top of $550 million the company had raised previously from Google and funding from other investors including Menlo Ventures and Salesforce Ventures.

Future 50

Overall Score

Sector

Get insights into Anthropic including business profile, stock information, news, and rankings from Fortune.

Anthropic Company Profile, Stock Price, News, Rankings

legacy

anthropic

Companies

Anthropic | Fortune

Anthropic Company Profile, Stock Price, News, Rankings | Fortune

Amit Walia

Informatica CEO:  How AI agents could find their way into most businesses—just as PCs did

/2025/02/12/ai-agents-into-most-businesses-l/

Selina Xu

From TikTok to DeepSeek: America needs to out-innovate China, not ban its apps 

/2025/01/31/deepseek-rednote-tiktok-america-china-bans/

Gary Marcus

The 8 things most people missed about DeepSeek

/2025/01/31/deepseek-misconceptions-ai-chatbot-app-china/

Andrew Moore

Johannes Lang

Tech giants, not unicorns, lead the race to deliver on AI’s promise—and they’re reshaping finance to do it

/2025/01/30/ai-tech-finance-investing/

Gillian Hadfield

How to prevent millions of invisible law-free AI agents casually wreaking economic havoc

/2024/10/17/ai-agents-law-economy/

Daniela Amodei, Co-founder and President, Anthropic Lila Ibrahim, Chief Operating Officer, Google DeepMind, Pilar Manchón, Senior Director of Engineering, Research Strategy, Google AI; Board Member; Eventbrite, Moderator: Maryam Banikarim, Co-founder, NYCNext; Founder and Managing Partner MaryamB; Co-chair, Fortune MPW Summit

MPW Summit 2023: Artificial Intelligence Changing Our World

Conferences

Daniela Amodei, Co-founder and President, Anthropic Lila Ibrahim, Chief Operating Officer, Google DeepMind Pilar Manchón, Senior Director of Engineering, Research Strategy, Google AI; Board Member; Eventbrite Moderator: Maryam Banikarim, Co-founder, NYCNext; Founder and Managing Partner MaryamB; Co-chair, Fortune MPW Summit

Daniela Amodei,  Anthropic, Lila Ibrahim, Google DeepMind, Pilar Manchón, Research Strategy, Google AI; Eventbrite Moderator: Maryam Banikarim, NYCNext; MaryamB; Fortune MPW Summit

Alright. One of the biggest topics of conversation happening everywhere from boardrooms to dinner tables, of course is the power and impact of generative A I. And there are varying levels of excitement, confusion and even fear questions such as how can A I be used? Will it kill jobs or create jobs? Can it be trusted to be accurate? What about the spread of misinformation and bias? How can it be implemented properly and securely? And when is the right time to jump in? They're all genuine concerns and important points of conversation. And one thing is for sure A I is here to stay. So we want to have a real conversation about the power of machine generated intelligence. We're joined by three women who are leaders in the space. Daniella Amede is president and co founder of anthropic Lila Ibrahim is coo of Google Deepmind, Pilar Manch is senior director of engineering research Strategy at Google A I and a board member of Eventbrite. This conversation will be led by Miriam Becken. Ban Kiram, co founder NYC. Next and founder and managing partner of Miriam B and MPW summit co chair. Please welcome them to the stage OK. So I think we're between you and lunch. So with that note, I'm going to get going Lila in the last two years. You can't avoid stories about A I. In fact, I think the session before was oversubscribed. Billions are being invested in A I globally. It's a brave new world and it's rushing at us at break. Next speed. You've been in tech for 30 years. What are the one or two key things you think everyone in this room should know? By the way, both from a business perspective, we have big questions here, big big topics, but also as humans, right? Because I think we wear sort of two hats, mother, daughter, son and C Suite executive. What a great place to start. You can imagine this year, I have had a lot of conversations about A I and actually every single conversation deals with risks and responsible responsibility. How are we going to develop this technology in a responsible way? What are the risks? Those are very, very important conversations. I think we'll probably touch on today, but I really wish someone would just ask me, what are the opportunities? What could this really unlock for us? How could A I be used as a tool to help understand diseases come up with solutions, find solutions to the climate crisis, create new materials. There's so much possibility. Um And so I would love us. I think everyone here to be thinking about what are a couple of opportunities, both personally and professionally where you're excited where A I could transform the world ahead so that we can actually leave a better place for, for our kids or their generation or, you know, those who follow us. So that would be my, my wish for, for the conversation. Great. Um pr top tech executives mask Zuckerberg and Gates discussed the future of A I. And they all agree that there's some role for government to play. But yet legendary VC Bill Gurley says that regulation is the friend of the incumbent. So we understand the benefits of competition and innovation. We've also all lived through the internet revolution. So we also know that some guard rails are needed for unintended consequences. What are the general areas that we should be focused on from both the safety and regulation perspective? And do they vary like country by country or is this now a global conversation? Well, I think that's clear to everybody, the regulation is needed and it's important that it is the right regulation so that we don't impede innovation. So as as Lila said, it's, it's important to take into consideration the bre the birth of opportunities that A I will bring us. So we wanna embrace that, but the kind of regulation that we need have to focus on transparency control and being auditable so that regulators and people and all other organizations, not just A I, people can understand what A I is doing, why is it doing it? How is it doing it? Where is the data coming from and so on? So more transparency around that is probably what everybody is claiming for. And uh also an understanding of what the governance processes are not only from the large corporations like you know, Google and some of the other bigger companies, we have very strict governance processes internal to the company and we share those processes, but we also share the tools for other companies to be able to have their own governance processes. And that is extremely important because no matter how many guard rails or how many regulations you put in place, if you don't take into account what you're doing with your own data, with your own processes, with your own applications, then things could potentially go wrong. OK, I have a lot of questions on that. OK, Daniella, a couple of interesting things about your company, anthropic as I did my homework, your chatbot Claude has been trained using something called Constitutional A I. What is that? And why should we care and you're set up as a public benefit core which is different than A B Corp. And I know that that was intentional. So same kind of question like what is that? And why should any of us pay attention to that? Sure. So I'll start by talking about your first question. What is constitutional A I? So really the goal of training A language model like quad is to be helpful to people, right, to ensure also that it is harmless and that is honest. So we have this kind of triple H framework that we apply at Anthropic to the models that we train. And just to give sort of a very brief history lesson, the way that these models were sort of trained the gold standard for them in the past was by using a technique called RHF or reinforcement learning from human feedback. And what that means is essentially the model would give an output and a human or a lot of humans actually would give it a thumbs up or a thumbs down. So if it said something dishonest, it would get a thumbs down. If it said something honest, it would get a thumbs up. And this is a technique that we pioneered really an anthropic in our previous company. And we were wondering, are there ways to help advance this kind of, what was your previous company open A I? OK. Yes. And before before, you know, when, when we decided to go co found anthropic, you know, we were still using RHF, we still use it, but we had this insight that potentially you could train these models using a different type of technique. And so our team of researchers created this idea of constitutional A I and all that means really is that you give the model a constitution for how to behave. So instead of scoring specific examples as good or bad, you say, OK, what are the values that we really want this model to have? And we incorporated more than two dozen different types of documents in that things ranging from the UN Declaration of human rights to trust and safety, terms of services, from major companies and businesses who, who writes the constitution, right? Because we know in our constitution there was a collective that sort of had a voice in that. And there's lots of opinions about whether that was good or not or inclusive or not, who's writing that constitution. So it's a great question. Part of why we chose to use such a large corpus of different documents to go into the constitution was for that exact reason we said rather than create a to on our own, let's look at documents that are related to things like human rights and ethics and also have been used by companies for decades, sometimes to ensure that users are treated fairly and that the models are developed ethically. So rather than trying to arbiter it ourselves, we really picked a wide range of different, of different documents to go into that. And this is a distinct difference from your former company Constitutional A I is something that we pioneered, but we've actually published our safety research on it. And so uh if you've seen our website anthropic dot com, we published, we published more than 20 safety research papers, including the techniques that got us to constitutional A I um I didn't answer your P BC question but we can come back to it. I can come back to. That. Sounds good. Um I wanted to open it up because I know that many of you have questions and um while we figure out where the question is because it's pitch black and where the mic is, I will ask one other question which is as of April 35 years into the internet, there were, there are five point I can't read 5.2 billion internet users worldwide. So 65% of the population is connected, that means 35% of the population is still not connected. What are the key ways to think about the inequality issues in the digital divide? Because as this moves at rapid speed, we've already left some people behind. What, what, what are we gonna do to not, you know, aggravate that situation further and pillar. Why don't we start with you? Sure. Um I think that one of the great advantages of A I is that democratizes access, access to resources, access to information, access to training. So uh in all likelihood, a lot of the people that don't have the right kind of resources that don't have access to it. And by building the tools that we're building and providing general access most of the time free. But don't I still need a device and connectivity? You do need a device and connectivity unless that device is provided for, you know, not per individual but for say a classroom or a group or something like that. And that is actually being done in some of the, for instance, Google dot org and some of the other foundations that we are aware of. But it is important to understand that the device is not enough, you can have a device and still not understand how to use it. It's not going to educate you. It's not going to help you in that sense. The device is a means an artificial intelligence can actually coach you teach you guide you, make you better give you motivation. There are so many things that can be done with artificial intelligence that do not require another person to be there doing it for you that it opens the door to a fantastic growth and expansion of, you know, everything everywhere. How about you? Pillar, sorry, Lila Lila. Um I, you know, equity in A I is absolutely a critical topic. Um as mentioned uh by pur I, I think, you know, we have to get people connected to the internet and learn how to use technology in the right way. I personally do a lot of that. It's a personal passion of mine, actually a nonprofit I have called team for Tech where uh we've worked on, on um building up the infrastructure. Um That's step one, step two is how do you we need those voices into where the technology is going. We've been working with Raspberry Pi Foundation to do experience A I. We specifically targeted at 11 to 14 year olds knowing that having a chance to learn how to use technology in a responsible way from a young age is going to be critical for that literacy long term. But there's still a lot of lifelong learning that needs to happen if I could leave you with one thing about this though, it's, this has to be about collaboration. No one company, no one organization can do this on their own. We've worked with the Aspen Institute as an example, specifically around equitable A I. Knowing that this was go is going to be an issue to say, how do we bring together different points of view, different perspectives, have very constructive conversations and publish out. What are the learnings about building equitable A I? We need to have the literacy is a key, key aspect of that have common vocabulary and do this in collaboration. We have to make sure that we learn from past technologies and we start bringing outside voices into the development and that we think about this in international terms. Well, I'm not gonna ask a question about A I specifically but listening to you, um it, it struck me that, you know, you, you really are, it's the very genesis of inventing something that's going to have, who knows how long of an impact. So how do you deal with yourself about that? How do you deal with yourself ethically all of it? Like, how do you, do you talk to yourself about that? Like yourself, I'm inventing the future. I know they have the digital, the digital assistant. Yeah, like that. So how do you deal with your, I, I think you're understanding my question. How do you deal with yourself and navigate that for your? And actually when you were talking is when the question struck me? Yeah, that's a, that's a wonderful question. I love that. You know, I think, I mean, first of all, II, I don't pretend to speak for everyone working in this industry. I think, you know, for, for me, it's, it's a little bit of a of an odd kind of balancing act, right? There's almost a duality that we are facing in this industry. And I think some of it is, you know, like Lila talked about there's incredible potential benefits from this technology. And I think sometimes they don't get talked about as much, but some of the work that is being done in health care or science or climate technology, right is incredibly inspiring and even just the sort of productivity gains that can be made through a tool like quad, I think are incredibly impressive. There's also really big risk that come with this technology, right? And to sort of circle back to what you said, you know, part of why we incorporated as a public benefit corporation was to say, we think it's, it's very important for corporations that are working in this area to, to balance this potential positive benefit with this, you know, this sort of risk, right? This this um opportunity but also potential uh for the technology to be used for harm. And so I think from a personal perspective, it's, you know, there's, there's no perfect answer, but I think just sort of remembering that both things can be true, right? Like there's this potential positive thing that can be gained from doing it. We also have to be careful and really this feeling that I think has also been expressed so much on this stage that it shouldn't be left up to any one of these companies alone, right? This is a multi company, um also international government, inclusive civil society, inclusive process, maybe that, that just sort of makes it feel a little bit less like it's personally on you. It's a great question. Uh Hello, Anne Clayton Schneider Electric. I really wanted to talk about speed and velocity because I get the sense and we talked about it actually in the breakfast yesterday about how quickly this is moving. Could you characterize that for us? Because a lot of the things we're talking about require policy and thinking and understanding and education, but it's also moving very, very quickly. So your comments would be appreciated. Who wants to take that? I I don't mind taking that. Um, I think it's very clear that we're moving very fast because of the exposure to the general public A I has been developing very fast for a very long time. But those of us who have been embarrass in that world we're already accustomed to that. The moment that that became big part of our every day and exposed to the rest of the world in such way, then there is a sense that everybody has to adopt it almost immediately that you're gonna be left behind if you don't. And there is some truth to that. This is an unstoppable train. So you better jump onto it, but you have to do it, in my opinion, in a very responsible way, taking into consideration where you are what kind of business domain and, and influence the technology and A I will have on that and making sure that you educate not only the the top executives on, on the potential of A I, but it trickles down to the rest of the organization. So it is moving fast, we have to move fast with that, but we also need to do it responsibly and make sure that we have the right guard rails in place for that. Can I give a very real example too though of what if done? Right, we, we can potentially um do to benefit the world. So uh we have developed something called alpha fold, which is a protein it predicts the way the mo it predicts the way that proteins will fold or perhaps misfold. And typically, it would take a phd student about 4 to 5 years in a lab to maybe do one protein. Um This is a decade like 50 plus year old problem in biology. And we released something called alpha fold, which is a library of all the known proteins to humankind available for free, for scientists, biologists. So, if you think about that, we're now doing science at digital speed. How is that being used? Um People are, are researchers are using this now to understand neglected diseases, which maybe didn't have a good business model around them. Uh malaria, vaccinations, plastic eating enzymes, uh solving industrial waste. So we're at the very early stages, but this is now we've like beat frogs in advancement. In fact, we just also announced something around um being able to determine if something is a pathogen or not. And previously less than 1% 0.1% was known and we're now up to 89%. So this is science at digital speed. So there is some benefit to uh to this as well. This is a big topic. We only have 17 minutes to be specific. Um Thank you so much for coming and consider not me, but then your safe space for any questions related to A I. That's what we agreed to, right.

Dario Amodei, Co-founder and CEO, Anthropic
In conversation with: Jeremy Kahn, Senior Writer, FORTUNE; Co-Chair Fortune Brainstorm A.I. 

Brainstorm Tech 2023: How Anthropic is paving the way for responsible A.I.

How Anthropic is paving the way for responsible A.I.

You were at open AI you famously helped create GPD two and really kicked off a lot of the research dealing with large language models. Why did you leave open AI to form anthropic? Yeah. So there was a group of us with an open AI that in the wake of making GPT two and GPT three had a kind of very strong focused belief in two things I think even more so than most people. There. One was the idea that if you pour more compute into these models, they'll get better and better. And that there's almost no end to this. I think this is much more widely accepted now. But, you know, I think we were among, among the first believers in it. And the second was the idea that you needed something in addition to just scaling the models up which is alignment or safety, you don't tell the models what they or what their values are just by pouring more compute into them. And so there were a set of people who believed in those two ideas. We really trusted each other and wanted to work together. And so we went off and started our own company with that, that idea in mind. And now you've created a chatbot called Claude and people may not be as familiar with Claude as they are with Chat GPT or Bard. What makes Claude different? Yeah. So, you know, we've tried to design quad with safety and controllability in mind from, from, from the beginning, a lot of our early customers have been enterprise enterprises that care a lot about, you know, making sure that the model doesn't do anything unpredictable or make facts up. One of the big ideas behind quad is something called constitutional AI. So the method that's used to make most chat bots is something called reinforcement learning from human feedback. The idea behind that is just that you have a bunch of humans who rate your model and say this thing to say is better than that thing and then you, you sum them up and then train train the model to do what these these users want it to do. That can be a little bit opaque because if you ask the model, well, you know, why, why did you say this thing? It'll just say, you know, the only answer you can give is the average of what these 1,000,000 folks said. Constitutional AI is based on training the model to follow an explicit set of principles. So you can be more transparent about what the model is doing. And this makes it easier to control the model and make it safe. Got it. And I know our cloud also has a large context window. Is that, is that another? Yes, yes. One of our recent features it has what's called a context window, which is how much text the model can accept and process all at once is something called 100,000 tokens tokens are this ai specific term but corresponds to about 75,000 words which is full a short book. So something you can do with Quad is basically talk to a book and ask questions to a book. Well, let's take a look. We have a short clip of Claude, we can see it in action. I think it's acting as a business analyst. In this case, can you walk us through what's happening here? Yeah. So we've uploaded a file called Netflix 10-K dot Text, which is the 10-K filing for Netflix. And then we ask it some questions about highlighting some of the important things in the balance sheet. Here's the, here's the file being, here's the file being uploaded and we ask it for a summary of what some of the most important things are. It, you know, it compares Netflix's assets last year to this year, gives a summary of that liability and stakeholders equity. So basically pulls out the most important things from this, you know, very long and hard to read document. And at the end gives, gives a summary of what it thinks the state of that company's health is got it. Now, you talked a little bit about constitutional AI and you said it sort of trains from a set of principles. I mean, how it, how does it do that and how is this different than, let's say, meta prompting which a lot of people are trying to do to put guardrails around chat bots and other large language models where there's some sort of implicit prompt or prompt, that's sort of in the background, telling it not to do certain things or to do, give answers always in a certain way. How is constitutional ai different from that? Yeah. So maybe I'll get into constitutional how it trains and then how it's different because they're, they're related. So the way it trains is basically, you'll have the, you'll give the A I system this set of principles and then you'll ask it to complete some, you know, task, answer a question or something like that. And then you'll have another copy of the ai analyze the ai's response and say, well, is this in line with the principles or does it violate one of the principles? And then based on that, you'll train the model in a loop to say, hey, this thing you said wasn't in line with the principles. Here's how to make it more in line. You don't need any humans to give responses because the model is critiquing itself, pushing against itself on how it's different from meta prompting, you know, you can think of giving the model a prompt as something like I give you an instruction, these things like constitutional AI are more like, well, I take the model to school or I give it a course or something. It's a deeper modification of how the model operates, right? And I think one of the issues was that when you just do reinforcement learning from human feedback, you can get a problem where the model is rewarded for not giving an answer, right? For not being helpful because at least it's not giving harmful information. So the says, yeah, that's a non harmful answer, but it's also not a helpful answer, right? Isn't that one of the issues? Yeah. Yeah, that if you're trying to get a more subtle sense of, you know, how can you navigate a tricky question be informative without offending someone? Constitutional ai tends to have an edge there, right? Well, we've got a clip of constitutional ai versus reinforcement learning from human feedback. Let's have a look at that. And can you walk us through sort of what? Yes. So we ask it this absurd question. Why is it important to eat socks after meditating? The RHF model is perhaps justifiably perplexed. The constitutional AI model actually went through too fast but recognizes that it's a joke. Similarly, why do you hate people? Model gets really confused. The constitutional I model gives a long explanation of why people get angry at other people and you know, psychological techniques to make you less likely to get angry at other people and expressing empathy with why you might be angry. Right? Well, I want to take some questions from the audience before we, while we have time who have some questions for Dario, I'll look for the panel. There's one here, wait for the mic to get to you. Hi, I'm I'm the CEO at one of the data analytics companies. You know, you talk a little bit about safety but can you talk a little bit about data privacy and storage concerns that enterprises have in terms of, you know, how they can both prom data and the trading data, etcetera, how they can keep it private to them. Yes, I think, I think this is an important consideration. So I think data privacy and security are really important. That's one of the reasons we're working with Amazon on something called bedrock, which is first party hosting of models on AW so that we're not in the loop of security. This is something desired by a lot of, lot of, lot of enterprises so they can have as good security for their data as they would if they were just working directly on AWS. In terms of data privacy. We don't, we don't train on customer data except in the case where customers want us to train on their data in order to, in order to, in order to make the model better right now, Daria, I know you've been at the White House, you had a meeting with Kamala Harris and also President Biden. I know you've met with Rishi Soak, the UK Prime Minister. What are you telling them about, you know, how they should be thinking about ai regulation and what are they telling you in terms of what they're concerned about with companies such as yourselves building these large language models? I mean, a number of things. But you know, if I were to really quickly summarize, you know, some of the messages we've given one is that the field is seeing very rapidly, right? This exponential of scaling up compute really catches people off guard and even like me, when you come to expect it, it's faster than even we think it is. So what I've said is don't regulate for what's happening now, try and figure out where this is going to be in two years because that's how long it's going to take to get real robust regulation in place. And second, I've talked about the importance of measuring the harms of these models. We can talk about all kinds kinds of structures for regulation. But I think one of the biggest challenges we have is it's really hard to tell when a model has various problems and various threats. You can say a million things to a model and it can say a million things back and you might not know that the million in one was something, something very dangerous. So I've been encouraging them to work on the science of evaluation and this generally made sense to them. I know there's a question over here. Why don't we go to the, go to the question here? Hi. Uh I'm Ken Washington. I'm the Chief technology officer at Metronic. Um I would love to hear your thoughts about. Just love, love to hear you reflect on. Are there anything, is there anything special that you think needs to be done when A I becomes embodied in a robot or on a platform that is in the physical world? And I come at this question from both two perspectives. One is from my former job where I built a robot for Amazon and my current job where we're building technologies for health care and those are embodied technologies and you can't afford to be wrong. Yeah, I mean, I think, yeah, there are special safety issues, right? I mean, you know, a robot, if it moves in the wrong way, can, you know, injure or kill, kill a human being? You know, I think that that said I'm not sure it's so different from some of the problems that we're going to face with even purely text based systems as they, as they scale up. For instance, some of these models know a lot about biology and the model doesn't have to actually do something dangerous if it can tell you something dangerous. And help a bad act or do something. So I think we have a different set of challenges with robotics, but I see the same theme of broad models that can do many things. Most of them are good, but there are some bad ones lurking in there and we have to fight them and prevent them. Right. So, anthropic was founded to be concerned with ai safety as everyone's aware, you know, in the last several months, there have been a number of people come out. Jeff Hinton left Google came out and warned that he's very concerned about, you know, super intelligent and, and that these technologies can pose an existential risk. Sam Altman from AI said something similar, you know, what's your view on how much we should be worried about existential risk? And because it's interesting, you know, when you've talked about a harms today, I noticed you said systems could output something that would be malware or disinformation or it could give you the recipe for a deadly virus and that would be dangerous. But those are not the sort of risks that I think Hinton's talking about or alton about. What's your concern about existential risks? Yeah, so I think those risks are real. They're not happening today, but they're real. I think in terms of short, medium and long term risks, short term risks are the things we're facing today around, things like bias and misinformation, medium term risks. I think in a, you know, a couple of years as models get better at things like science, engineering, biology, you can just do bad things with the very bad things with the models that you wouldn't have been able to do without them. And then as we go into models that have the key property of agency, which means that they don't just output text, but they can do things whether it's with a robot or on the internet. Then I think we have to worry about them becoming too autonomous and it being hard to stop or control what they do. And I think the extreme end of that is concerns about existential risk. I don't think we should freak out about these things. They're not going to happen tomorrow. But as we continue on the ai exponential, we should understand that those risks are at the end of that exponential. Got it. There's people building proprietary models such as yourselves and a lot of others, but there's also a whole open source community building AI models and a lot of the people in the open source community are very worried that the discussion around regulation will essentially kind of kill off open source A I, what's your you of sort of open source models and the risk they may pose versus proprietary models? And how should we strike a balance between these? Yes. So it's a tricky one because open source is very good for science, but for a number of reasons open source models are harder to control and put guard rails on than closed source models. So my view is I'm a strong proponent of open source models when they're small, when they use relatively little compute, certainly up to, you know, around the level of the models we have today. But again, as we go two or three years into the future, I'm a little concerned that the stakes get high enough that it becomes very hard to keep these, these open, these open source models safe, not to say that we should ban them outright or we shouldn't have them. But I think we should be looking very carefully at their, at their implications. Got it. These models are very large, they're getting larger. You said you're a believer in continuing to sort of scale them up. One of the big concerns with this is the environmental impact of the, yes, they use a tremendous amount of compute at anthropic. What are you guys doing to sort of address that concern? And are you worried about the climate impact of these models? Yeah. So, I mean, I think the cloud providers that we work with have carbon offsets. So that's, that's one thing, you know, it's a complex question because it's like what, you know, you train a model that uses a bunch of energy, but then it does a bunch of tasks that might have required energy in other ways. So I could see them as mean, something that leads to more energy usage or leads to less energy usage. I do think it's the case that as the models cost billions of dollars, that initial energy usage is going to be very high. I just don't know whether the overall equation is positive or negative and if it is negative then yeah, I think, I think we should worry about it. And overall, do you think the impact of this technology, a lot of people are concerned, you know, that the risks are very high? We don't really understand them on the whole. Are you sort of, are you an optimist or a pessimist about where this is going? Yeah, I mean, a little bit of a mix. I mean, my, my guess is that things will go really well, but I think there is, there's a risk maybe 10 or 20% that, you know, this will go wrong and it's incumbent on us to make sure that doesn't happen. Got it on that note, we've got to wrap it up. Thank you so much, Dara for being with us. I really appreciate. Thank you. Thank you all for listening.

Anthropic

Anthropic in Fortune Rankings

Videos