It may not qualify as a lightning-bolt eureka moment, but Jeffrey Immelt, chief executive of General Electric, recalls the June day in 2009 that got him thinking. He was speaking with GE scientists about new jet engines they were building, laden with sensors to generate a trove of data from every flight—but to what end?

That data could someday be as valuable as the machinery itself, if not more so. But GE couldn’t make use of it.

“We had to be more capable in software," Immelt said he decided. Maybe GE—a maker of power turbines, jet engines, locomotives and medical-imaging equipment—needed to think of its competitors as Amazon and IBM.

Back then, GE was returning to its heavy-industry roots and navigating the global financial crisis, shedding much of its bloated finance arm, GE Capital. That winnowing went on for years as billions of dollars in assets were sold, passing a milestone this summer when GE Capital was removed from the US government’s short list of financial institutions deemed “too big to fail".

But in 2011, GE also quietly opened a software centre in San Ramon, California, 38km east of San Francisco, across the bay.

Today, one of San Ramon’s most important projects is to build a computer operating system, but on an industrial scale—a Microsoft Windows or Google Android for factories and industrial equipment. The project is central to GE’s drive to become what Immelt says will be a “top 10 software company" by 2020.

Silicon Valley veterans are sceptical.

“GE is trying to do this the way a big company does, by throwing thousands of people and billions of dollars at it," said Thomas Siebel, a technology entrepreneur who is now chief executive of C3 IoT, a start-up that has done work for GE. “But they are not software people."

The San Ramon complex, home to GE Digital, now employs 1,400 people. The buildings are designed to suit the free-range working ways of software developers: open-plan floors, bench seating, whiteboards, couches for impromptu meetings, balconies overlooking the grounds and kitchen areas with snacks.

Many industries see digital threats, of course. Yet, the scope of the challenge is magnified at GE, a 124-year-old company and the US’s largest manufacturer, with more than 300,000 employees worldwide. Employees companywide have been making pilgrimages to San Ramon for technology briefings, but also to soak in the culture. Their marching orders are to try to adapt the digital wizardry and hurry-up habits of Silicon Valley to GE’s world of industrial manufacturing.

GE’s success or failure over the next decade, Immelt says, depends on this transformation. He calls it “probably the most important thing I have worked on in my career".

Apparently, there is no Plan B. “It’s this or bust," he said.

The march of digital technology—mainly inexpensive sensors, powerful computing and clever software—into the industrial world has been underway for years under the guise of “the Internet of things" or “the industrial Internet". It is the next battlefield as companies fight to develop the dominant software layer that connects the machines.

It promises to be a huge market for new products, improved service and efficiency gains in industries like energy, transportation and healthcare. By 2020, the industrial Internet market will reach $225 billion, GE executives predicted in a recent meeting with analysts.

So far, a major application has been predictive maintenance. Software analyses the data generated by a machine to identify early warning signals that it needs repair, before it breaks down.

The data volumes are exploding as machines new and old spawn sensors. By 2020, GE estimates that the data flowing off its machines in use will jump a hundredfold. That should enable far more detailed analysis, giving GE a chance to sell its customers not machines but “business outcomes", like fuel savings. Immelt sees this as a move up the industrial food chain.

New competition

Yet, all of this exposes GE to new competition beyond its traditional rivals like Rockwell Automation, Siemens and United Technologies. Tech giants, including Amazon, Cisco, Google, IBM and Microsoft, also have their eye on the industrial Internet market, as do a bevy of start-ups.

There is precedent for trouble in other industries, of course. Google and Facebook transformed media and advertising, Amazon redefined retailing, and Uber applied an entirely new business model to taxis, which hadn’t changed much in generations.

“The real danger is that the data and analysis becomes worth more than the installed equipment itself," said Karim Lakhani, a professor at the Harvard Business School. “GE has no choice but to try to do that itself."

Recently, GE has lured software engineers and data scientists from Amazon, Apple, Facebook and Google. Early on, though, it struggled simply to hire. Last year, it began running self-deprecating television ads, featuring young hires, aimed at closing the company’s image gap of industrial giant but digital midget. (Applications at GE Digital jumped eightfold, the company says.)

Until this year, Darren Haas hadn’t thought about GE, certainly not working there. To him, GE meant little more than kitchen appliances and light bulbs. “I had no idea," Haas said.

But he was intrigued after he met with Harel Kodesh, an expert in mobile and cloud computing who led teams at Microsoft and VMware, a maker of data-centre software. Less than two years ago, Kodesh joined GE, and he is now chief technology officer of GE Digital.

That someone of Kodesh’s calibre was a GE convert got Haas’ attention. Then, Haas started grasping the role that GE equipment plays across the economy—in transportation, in hospitals—“a whole world", he said. “I found that really, really compelling."

The other thing Haas, 41, found appealing was the big computing challenge that lies ahead for the company. In May, he joined GE from Apple, where he was a member of the founding team at Siri, the voice-based digital assistant start-up Apple acquired in 2010. When he left Apple, Haas was head of cloud engineering, managing the computing engine behind Siri, iTunes and iCloud.

At GE Digital, Haas has a similar title, head of platform cloud engineering, but in a different setting. He describes his job as applying modern software technology—machine learning, artificial intelligence and cloud computing—to the industrial arena. “I have got my work cut out for me," he said.

Haas is working on the centrepiece of GE’s software strategy, a product called Predix. Its evolution mirrors GE’s software ambitions.

Predix began as little more than a brand of software used by GE to service the gear it sells. One showcase use was in jet engines to do predictive maintenance, saving downtime. Around 2013, William Ruh, a former Cisco Systems executive brought in to put together the San Ramon software centre, started expanding Predix to other GE industrial businesses.

But that soon felt too small. The issue was “outside disrupters", Ruh said, citing the online lodging start-up Airbnb as an example of rivals that can appear seemingly out of nowhere and that “bring greater productivity—but don’t own the assets they sell". GE, Ruh notes, is the ultimate big-asset company.

The other threat was rising interest among tech companies to take their Internet connectivity expertise and try to apply it to industrial businesses. To get ahead of all this, GE reimagined Predix as a cloud-based operating system for industrial applications.

Kodesh leads that effort. He ticks off the looming competitors: Amazon Web Services, Google, Microsoft and others. “Those companies are going to encroach on the territory," Kodesh said. “Are we going to capitulate, or build something like Predix?"

The basic idea is that GE and outside software developers will write programs to run on Predix. This software might, for instance, monitor the health and fine-tune the operation of equipment like oil-field rigs and wind-farm turbines, improving performance, reducing wear and adapting to changing environmental conditions. It amounts to software delivering the equivalent of personalized medicine for machines.

GE has some built-in advantages. Its installed base is huge. For example, the company says more than a third of the world’s electricity is generated on GE equipment. It can make progress simply by winning over the aircraft makers, oil companies, hospitals and utilities that now depend on GE machinery.

GE is starting to attract a developer following. Tata Consultancy Services, for one, says it now has 500 programmers designing and developing Predix applications for customers in the electric-utility, aviation and healthcare industries. GE also promotes partnerships with Infosys, Wipro and Capgemini to help business write Predix software.

When he joined in 2011, Ruh had no illusions that making software a strength at a heavy-industry company would be easy. At the time, he told Immelt that would be “a 10-year journey", he said. “We are in the middle of that journey."

Part of that is an effort to change an engineering culture that stretches back generations. “If GE is truly going to be a digital-industrial company, we can’t be separate here," Ruh said of his software division. Digital “tools and habits" need to be embedded “in how people do their jobs", he said.

Digital metamorphosis

In its factory in Greenville, South Carolina, GE produces both giant power generators and evidence that this metamorphosis might work.

The three-storey building is crowded with immense cranes and milling, grinding and welding machines, overseen by manufacturing engineers and technicians. The finishing touches are being put on one of GE’s new gas turbines.

It looks like the business end of a rocketship lying on its side, a gleaming steel dynamo at rest. It weighs 430 tonnes. It fires up at nearly 1,593.333 degree Celsius, and it can generate enough electricity to supply more than 500,000 households.

The gas turbine was brought to market in half the typical five years. That kind of accelerated product development is a performance that GE hopes to replicate across its industrial businesses. And it is a story of changes in design and manufacturing practices made possible by digital technology.

John Lammas, 56, the vice-president for power generation engineering, started his working career 40 years ago, on the shop floor of a Rolls-Royce jet engine factory in Birmingham, England. He has been with GE for 31 years, moving up the ranks of the company’s jet engine and power turbine divisions.

“I’m an old mechanical guy," he said. But a couple of years ago, he issued an edict: no more paper drawings.

In the past, a model of a new part would be made and then converted to detailed blueprints running to 70 pages or more. These would then be physically sent to GE manufacturing engineers and outside suppliers to begin setting up the tooling, casting and cutting for the part.

This prototype-and-blueprint routine took up to eight weeks. Now, engineers use 3D computer models, skip the prototype step and instantly send the models electronically.

This goes a step beyond computer-aided design, which is commonplace. In Greenville, the designers are for the first time linked directly with manufacturers and suppliers in real time, in what GE calls a “digital thread". This means they can collaborate in ways that have changed the work process while making it more likely that problems or defects are spotted sooner.

Traditionally, one set of engineers designed a part, and only then passed it on to manufacturing. If a problem arose on the supplier side, the design was kicked back and the process started over. “Jobs are combining in this digital world," Lammas said.

Greenville’s own equipment has been a Predix guinea pig. The machinery and factory were retrofitted with data-generating sensors and the software. Matt Krause, the plant manager, said that last winter, when a snowstorm shut the factory for a day, the sensor network detected that the plant had consumed 450kg of argon, an inert gas used in coatings for parts. The leak was fixed, saving $350,000 a year.

“We can see things we never did before," Krause said. Overall, 60 of 200 steps in design and production have been automated or eliminated, reducing work time by 530,000 hours over three years, GE estimates.

Not all the start-up ideas that GE is trying to breed translate comfortably to heavy industry. Lean proponents urge companies to come up with “minimum viable products", particularly test versions of software programs. But no one wants a minimum viable jet engine or power generator. Yet in Greenville, engineers in the design stages are encouraged to move faster in smaller steps, conduct more experiments, and be willing to fail and try again. It amounts to a sea change in the engineering culture of heavy industry.

“As an engineer, not getting it right the first time, I find painful," said Bill Byrne, an engineering manager. “It’s uncomfortable. But it’s been incredibly liberating."

The old ways, said Lammas had merit. Each step and rule was logical on its own. But the emphasis on flawless execution and perfection fostered a fear of failure. “Overcoming that culture was probably the biggest challenge," he said. ©2016/THE NEW YORK TIMES