FORTUNE — Ginni Rometty’s first customer conference as CEO of IBM (IBM) was an unusual affair, especially by Big Blue’s buttoned-up standards. The June confab took place in an airy loft in Manhattan’s hip Chelsea neighborhood. When the tiny elevator arrived to whisk a group of us to the meeting space, the doors opened and there was Rometty, flanked by a couple of visibly nervous assistants. “Really good to see you!” she said, clasping my hand warmly as her handlers checked their watches. The presentation was about to begin and Rometty still wasn’t wearing her microphone. “Isn’t this neat?” she asked.
The program started late. At 5-foot-11, with blond hair tucked behind a headband, Rometty, 55, has an almost regal bearing, but on this day she flubbed her entrance, bounding onto the stage before she could be introduced. She laughed it off. When an audience member’s ringing cellphone interrupted the events, she joked, “I hope that isn’t mine!”
You wouldn’t catch Lou Gerstner or Sam Palmisano trying to smooth over someone else’s faux pas. Rometty’s two predecessors also are unlikely to have hosted a sales meeting in a loft, and they definitely wouldn’t have described the proceedings as “neat.” But they surely would have approved of Rometty’s agenda that June day. She had assembled some familiar faces, the chief information officers who buy billions of dollars of software, tech services, and hardware from IBM (No. 19 on the Fortune 500), but she had also invited their chief marketing officers. (Thus the trendy venue.) Her ambitious — and yes, unusual — plan: Get the marketers to use IBM tools to sort their data for nuggets that will help them better reach customers and sell more stuff.
When Rometty (pronounced ro-MET-ty) became IBM’s ninth CEO — and its first woman chief executive — she took control of the 19th-largest company in the world by revenue (2011 sales surpassed $107 billion) and, at presstime, the fifth largest by valuation, with a market cap of $235 billion. Her influence on the world of technology and her company’s impact on the financial markets earn her the No. 1 spot in Fortune’s annual ranking of the Most Powerful Women in Business. She inherits a company with an enviable growth record for its enormous size. Over the past decade, the company has increased profits by an average 16% every year, returning 12% annually to shareholders.
She also needs to live up to almost ridiculously high expectations: IBM has said it will add $20 billion more in revenue growth in the next three years. To put that in perspective, that’s a business roughly the size of Nike (NKE), No. 136 on the Fortune 500.
Not that any of this is a surprise to Rometty, a 31-year veteran of IBM who is known to have thick binders of background material and data prepared for her in advance of meetings. Indeed, the most surprising thing about her June customer debut was how loose and improvisational it was. She’s not a stiff — “There’s nothing imperious or imperial about her,” notes Harvard Business School’s Rosabeth Moss Kanter — but Rometty rarely leaves anything to chance. For example, she declined to be interviewed in person for this article, and would answer questions only via e-mail.
Rometty was at Palmisano’s side for much of his decade-long tenure, and became a serious candidate to succeed him about four years ago. And she was personally involved in setting the high bar that she must now clear. She and other senior leaders helped him develop the five-year plan — dubbed “2015 Roadmap” — that has IBM targeting more than $125 billion in revenue that year.
For Rometty the challenge of meeting that goal is only partly about inventing new technologies to sell to her existing clients. Growth at IBM’s scale also means creating new markets, much the way it did with its Smarter Planet campaign, which sold nontechies such as mayors and police chiefs on the idea of using software to monitor and manage traffic, water systems, and sanitation trucks. Now Rometty is making a similar pitch to marketing executives, promising that technology will change the way they do their jobs. It won’t be an easy sell: Marketers are less apt than bureaucrats to be wowed by a charismatic CEO or statistics about petabytes. Many are accustomed to seeing computing as a tool to support their creative endeavors, not the starting point.
But if she can pull it off, Rometty could initially help change the way corporations communicate with their customers — and ultimately the way they use technology to build and sell their products. “This is a mindset shift, not a market shift,” Rometty explained to the CMOs at the Manhattan event. “It changes everything.” She was talking about her customers, but she could just as easily have been talking about IBM.
Rometty had always been a top performer, but she caught the eye of executives at headquarters in Armonk, N.Y., in 2002 when she managed the integration of IBM’s $3.5 billion purchase of PricewaterhouseCoopers’ IT consulting business. As general manager for IBM’s global services division — the unit that had been at the heart of Gerstner’s now legendary resuscitation of the company — Rometty pushed early for the acquisition and helped negotiate the deal. Overnight IBM became the world’s largest consulting business. And then Rometty had to figure out how to integrate 30,000 PwC consultants into her group of 150,000 IBMers.
It was a mishmash of cultures that could have gone horribly wrong, but Rometty managed the integration with a particular sensitivity to its impact on employees. IBM was buying talent, after all. The acquisition wouldn’t be successful unless Rometty persuaded the consultants, particularly the 1,000 or so incoming PwC partners, to stick around. She began planning for how the two cultures would fit together even before the dealmakers set financial terms. It was particularly challenging to navigate differing compensation packages. To bring salaries in line with IBM peers’, some of PwC’s top executive partners had to take as much as a 40% cut in cash compensation — and forgo perks like club memberships. To make up for the cash reductions, Rometty negotiated stock options that motivated the new employees to stay for at least four years. Ultimately, top performers could earn a higher payout.
She also reached out to all PwC employees personally. The morning the acquisition was announced, they arrived to the blinking red light of a voicemail notification on their phones. “Got to admit feelings were mixed,” wrote Tereza Nemessanyi, then a principal consultant at PwC, on her blog. “As a creative type, I was nervous of what I knew as a very rigid culture.” The following two-minute message welcomed her personally to IBM, assured her that IBM would retain the best elements of the PwC culture, and most important, got her excited. “Jeez, that woman leaves some seriously good voicemail,” wrote Nemessanyi, who tells me she was tempted to move to IBM but ended up staying with PwC’s parent.
This personal approach to leadership is the quality that resonates most with IBMers who have worked for Rometty. It’s one reason she has been able to keep talented, entrepreneurial employees like Manoj Saxena, a general manager in charge of commercializing the Watson technologies. A serial entrepreneur, Saxena arrived at IBM in 2006 after the company purchased Webify, his business-to-business software startup. “At first my venture capitalist friends were taking bets on how many quarters I’d stay,” he says. Rometty promised Saxena more resources and the opportunity to have a bigger impact at IBM, but she sold Saxena on the company a few years ago when he had a health issue. By that time Rometty was managing several hundred thousand people, but she regularly dropped him a personal e-mail to see how he was feeling. “She leads from both her head and her heart,” Saxena says.
Customers get head and heart too. Nick Donofrio, who worked closely with Rometty before his retirement from IBM in 2008, remembers helping her address the concerns of a large Midwestern client sometime around 2005. The client had installed some new IBM products that weren’t working well. Rometty called Donofrio, who was then executive vice president of innovation and technology, and told him they had to fly out immediately to see the client in person. The pair spent a day working closely with the client to get the project on track. But a day after they got back to New York, the client’s system wasn’t working again. Rometty insisted they fly out a second time to help the client fix the problem. “Most people wouldn’t go twice,” said Donofrio. “They’d send a junior person.” As it turned out, it wasn’t entirely IBM’s fault — the client hadn’t followed instructions. But Rometty said nothing. “Ginni’s not thinking, Did we do it wrong?” says Donofrio. “That’s not where her head is. She’s thinking about the client’s success.” (Perhaps tellingly, Rometty says she doesn’t recall this particular example.) The customer, says Donofrio, has become an even larger customer.
IBM is constantly restructuring its workforce, and the coming changes are sure to test Rometty’s leadership style. As IBM becomes more global, it will continue to bolster its ranks internationally, leaving the U.S. (an estimated 105,000 employees today) with a smaller number of workers — mostly researchers, executives who focus on sales and marketing, and talent coming from startup acquisitions. For many current employees — some of whom already feel taxed by the long hours and penny-pinching that IBM demands — the process will be wrenching: They’ll need to either reinvent themselves, or more likely, move on. And unlike when she was running global services, Rometty won’t be able to leave them all a voicemail.
The former Virginia Nicosia is the oldest of four children raised by a single mom outside Chicago. Her mother, with whom she remains very close, was among her strongest influences. By the time she arrived at Northwestern University in 1975, Rometty had matured into a striking, intelligent student who was popular among her peers and successful in the classroom. She pledged the elite sorority Kappa Kappa Gamma, and her pledge mom, Erin McInerney, remembers they connected in part because both came from modest backgrounds. “Unlike most of the other girls, we’d had summer jobs and were on scholarships and had loans,” she said. By senior year, Rometty was sorority president.
While many of her classmates studied the arts, Rometty was one of just a few of women to study computer science. In the late ’70s, Northwestern’s sole academic computer was so large that the university dedicated an entire building, the Vogelbach Computing Center, to housing it. Rometty and her fellow students learned to program it using punch cards. When Rometty needed help on an assignment, a KKG sister coaxed a fellow student named Craig Berman into tutoring her in exchange for dinner at the sorority house — probably the only place on campus where students were served meals on tablecloths. One Sunday morning Nicosia appeared in Berman’s fourth-floor dorm room, “a statuesque blond” who “kicked aside some dirty clothes stacked near the doorway.” She had a list of written questions and programming issues, and when he tried to answer the questions, she pushed him instead to explain his method for reaching the solution. “A lot of people look like they are listening, but she really listened,” he says.
Berman, who is now a vice president at Jeffries & Co. (JEF), describes his classmate as quiet, focused, always early, and organized. “If she fell off the sidewalk and into the lake, she would have come out wearing a scuba suit,” he says. And she had good advice as well. A professor once marked Berman’s homework incorrectly. In class he had an opportunity to show the professor up, proving he had the right answer. When he raised his hand, Rometty shoved it down, whispering, “You’ll win your argument in private later or lose it in public now.”
Rometty graduated in 1979 with a bachelor of science degree with high honors in computer science and electrical engineering. She had attended Northwestern on a scholarship from General Motors (GM), where she had interned between her junior and senior years, and she moved to Detroit to work for the automaker after graduation. There she met her husband, Mark Rometty, now an oil futures investor. The pair, who do not have children, enjoy scuba diving (as a matter of fact) and the occasional Broadway show together, as well as golfing near their Bonita Springs, Fla., home. (Rometty stays in a hotel-run condo in Westchester County when she is in New York.)
She credits her husband with being a great support to her. She describes a moment early in her career when her boss asked if she’d like to take a big promotion. She told her manager she didn’t feel ready, and asked for the night to think about it. At home she discussed it with her husband. “He just looked at me, and he said, ‘Do you think a man would have ever answered that question that way?’ ” Rometty took the promotion.
In 1981, Rometty took a job in IBM’s Detroit office as a systems engineer — a technical consultant of sorts — for banking customers. In the decades that followed, she moved up through a series of sales and management jobs, working with clients in all of IBM’s most important industries — banking, insurance, telecommunications, manufacturing, and health care. Before moving into senior management, Rometty spent the ’90s working in sales, a job for which her tech prowess and people skills made her uniquely qualified. Salespeople who met their annual sales quotas received commemorative pins and luxurious weekend getaways as a reward for earning membership in the prestigious 100% club. Rometty never missed a year.
Three weeks before Rometty was named CEO last fall, we sat together for an onstage conversation at Fortune’s annual Most Powerful Women Summit. By then speculation had begun that IBM would soon announce her promotion. I asked her what she’d learned from Palmisano. Rometty reflected. “What he always says is, ‘Nothing is inevitable.’ ” She went on to explain, “Whatever business you’re in — it doesn’t matter — it’s going to commoditize over time. It’s going to devalue. You’ve got to keep moving it to a higher value.”
To put it another way, Rometty learned that IBM must keep evolving. There is always a new shift coming in technology, and if she doesn’t help IBM become the first to discover and commercialize it, the company will lose its shirt.
It’s sage advice that came through experience. The company is still haunted by the near-death experience of the early ’90s, when it missed the technology shift to personal computers and came within a quarter of going bankrupt. Palmisano was the first CEO to step into the role after Gerstner turned the company around — and he responded by making bold research-based decisions about the future of the business, and enforcing maniacal discipline over how those decisions were carried out. Perhaps most important, and somewhat unusual for contemporary corporate culture, he nurtured a consistent team of senior executives — mostly IBM lifers — with very little turnover.
IBM likes its leaders to keep a low profile, preferring a ruthless focus on customers to coverage in the press. Earlier this year when the all-male Augusta National Golf Club, which has historically offered membership to CEOs of companies like IBM that sponsor the event, did not offer Rometty a membership, she kept mum about the affair, even as the snub ignited a media firestorm. Though she hasn’t explained herself, one gets the sense that her greatest contribution to feminism won’t be helped by speaking out on issues so much as making IBM successful. She has a company to run.
So in keeping with IBM’s traditional long-term approach to management, Rometty has been fulfilling the goals she helped her predecessor draft in IBM’s 2015 Roadmap. To hit the massive $20 billion goal, Rometty has spelled out four high-growth areas for the company to focus on. It will continue the Smarter Planet work, in which it infuses the traditional systems that make our towns and cities work with new forms of computing. Revenue from this initiative jumped 50% last year as IBM began to attach the word “smarter” to new market categories like cities and commerce. The company will also invest in helping companies adopt aspects of cloud computing, and it will supercharge its work in business analytics. Rometty will spend a good deal of her time personally nursing along deals in growth markets, which are expected to provide up to 30% of IBM’s revenue by 2015.
To that end, Rometty has been crisscrossing the globe since January. She set an early goal for herself of meeting 100 client CEOs in her first two months, and she quickly surpassed it. In April she traveled to Brazil to spend nearly a week hammering out the details of her first big CEO transaction, a deal with the energy business magnate Eike Batista, who calls her “transparent and straightforward.” IBM will take over the IT operations for Batista’s EBX Group in a contract that is estimated to be worth $1 billion over the next decade, and it will buy a 20% stake in an EBX subsidiary that provides tech services to industries like mining, energy, and shipbuilding.
As much as Rometty is acting on the strategy Palmisano laid out, she has put her own signature on the company as well. One big challenge with which IBM struggles is its reputation for being slow to execute projects, which are sometimes lumbered by layers of bureaucracy. So Rometty is attempting a fix. IBM’s most senior executives have long served on three teams — operating, technology, and strategy — to help guide the company. Rometty created a fourth leadership team, the Client Experience Team, which she chairs. There are no senior executives on the team. Rather, she appointed a series of client-facing executives to meet with her once a month. They bring in other business leaders — recently they hosted Ritz-Carlton chief sales and marketing officer Chris Gabaldon — to hear how other companies manage customer relationships. Rometty says it’s not about “clearing away roadblocks” so much as creating the “signature IBM relationship.” IBM hopes it will help the company’s reputation with customers as well.
As Rometty marches the company into IBM’s second century, technology is again shifting. Rometty says we are entering the “cognitive era” of computing. History has produced only three computing eras so far, she explains. The first encompassed machines that counted and tabulated — the calculators and punch-card machines that Thomas Watson first manufactured. The second era, which began in 1960, brought programmable computers to market. “Everything you know today — the iPad in front of you — is just programmable,” Rometty told the audience at that first customer conference. What comes next? “This era is machines that learn.”
The best example of this is IBM’s Watson supercomputer, which soundly defeated the two men holding the longest Jeopardy! winning streaks. The computer system understands natural language. It can generate hypotheses, recognizing that there are different statistical probabilities for each outcome. And it learns as it goes along, refining its responses. When Rometty talks about the cognitive era of computing, that is what she means.
Watson came from IBM’s labs, where the company invests an average of $6 billion annually. Now IBM is unleashing Watson on the business world. Longtime IBM customer Lori Beer drove to Yorktown Heights, N.Y., to watch that final match live, and that’s where she first met Rometty. Beer, an EVP who oversees tech purchases for health insurer WellPoint (WLP), thought maybe Watson could be useful. She visited IBM Research, and Rometty made several visits to WellPoint. “She actually spent the time to get to know us,” Beer says. “She really made sure she understood what the issues are.”
By September 2011 the companies had hammered out a deal for WellPoint to use Watson’s data crunching to help suggest treatment options and diagnoses to nurses and doctors. When presented with information about a new patient in the future, Watson will look for data on those with similar symptoms, as well as the treatments that have been most successful. It will provide a range of treatment options, going so far as to suggest how likely it is to be right about each selection. Less than a year after the contract was signed, the first pilots were rolled out in August with WellPoint nurses who manage complex patient cases and review treatment requests from medical providers. Meanwhile, Watson’s computers are being put through a medical school of sorts, absorbing medical records and other data so that by the end of 2013 they can be deployed in oncology practices to help doctors treat cancer patients.
Rometty, of course, believes that Watson has great promise beyond medicine. That’s why she had Beer speak at her first customer conference, the one that brought together technologists and marketers. The predictive nature of the technology could reinvent any company flooded with increasingly large amounts of data — nowadays basically every business in every industry. But first, Rometty will have to translate the idea to a host of new customers who never fancied themselves all that technically minded. And like the eight men who have run this iconic company before her, she’ll have to sell them on it.
This story is from the October 8, 2012 issue of Fortune.
Update: An earlier version of this story incorrectly said Rometty’s last name is pronounced RAH-metty. It is ro-MET-ty.