Kubrick’s fantasy machines to outwit traders

Kubrick’s fantasy machines to outwit traders
Comment E-mail Print Share
First Published: Fri, May 04 2007. 01 58 PM IST
Updated: Fri, May 04 2007. 01 58 PM IST
By Jason Kelly, Bloomberg
New York: Way up in a New York skyscraper, inside the headquarters of Lehman Brothers Holdings Inc., Michael Kearns is trying to teach a computer to do something other machines can’t: think like a Wall Street trader.
In his cubicle overlooking the trading floor, Kearns, 44, consults with Lehman Brothers traders as Ph.D.s tap away at secret software. The programmes they’re writing are designed to sift through billions of trades and spot subtle patterns in world markets.
Kearns, a computer scientist who has a doctorate from Harvard University, says the code is part of a dream he’s been chasing for more than two decades: to imbue computers with artificial intelligence, or AI.
His vision of Wall Street conjures up science fiction fantasies of HAL 9000, the sentient computer in “2001: A Space Odyssey.” Instead of mindlessly crunching numbers, AI-powered circuitry one day will mimic our brains and understand our emotions -- and outsmart human stock pickers, he says.
“This is going to change the world, and it’s going to change Wall Street,” says Kearns, who spent the 1990s researching AI at Murray Hill, New Jersey-based Bell Laboratories, birthplace of the laser and the transistor.
As finance Ph.D.s, mathematicians and other computer-loving disciples of quantitative analysis challenge traditional traders and money managers, Kearns and a small band of AI scientists have set out to build the ultimate money machine.
For decades, investment banks and hedge fund firms have employed quants and their computers to uncover relationships in markets and exploit them with rapid-fire trades.
Quants seek to strip human emotions such as fear and greed out of investing. Today, their brand of computer-guided trading has reached levels undreamed of a decade ago. A third of all US stock trades in 2006 were driven by automatic programmes, or algorithms, according to Boston-based consulting firm Aite Group LLC. By 2010, that figure will reach 50%, according to Aite.
AI proponents say their time is at hand. Vasant Dhar, a former Morgan Stanley quant who teaches at New York University’s Stern School of Business in Manhattan’s Greenwich Village, is trying to programme a computer to predict the ways in which unexpected events, such as the sudden death of an executive, might affect a company’s stock price.
Uptown, at Columbia University, computer science professor Kathleen McKeown says she imagines building an electronic Warren Buffett that would be able to answer just about any kind of investing question.
“We want to be able to ask a computer, ‘Tell me about the merger of corporation A and corporation B,’ or ‘Tell me about the impact on the markets of sending more troops to Iraq,”’ McKeown, 52, says.
Kubrick’s Dream
Some executives and scientists would rather not talk about AI. It recalls dashed hopes of artificially intelligent machines that would build cities in space and mind the kids at home. In “2001,” the novel written by Arthur C. Clarke and made into a movie directed by Stanley Kubrick in 1968, HAL, a computer that can think, talk and see, is invented in the distant future -- 1997.
Things didn’t turn out as ’60s cyberneticians predicted. Somewhere between sci-fi and sci-fact, the dream fell apart. People began joking that AI stood for “Almost Implemented.”
“The promise has always been more than the delivery,” says Brian Hamilton, chief executive officer of Raleigh, North Carolina-based software maker Sageworks Inc., which uses computer formulas to automatically read stock prices, company earnings and other data and spit out reports for investors.
Hamilton, 43, says today’s AI-style programmes can solve specific problems within a given set of parameters.
Chess vs Markets
Take chess. Deep Blue, a chess-playing supercomputer developed by International Business Machines Corp. (IBM), defeated world champion Garry Kasparov in 1997. The rules of chess never change, however. Players have one goal: to capture the opponent’s king. There are only so many moves a player can make, and Deep Blue could evaluate 200 million such positions a second.
Financial markets, on the other hand, can be influenced by just about anything, from skirmishes in the Middle East to hurricanes in the Gulf of Mexico. In computerspeak, chess is a closed system and the market is an open one.
“AI is very effective when there’s a specific solution,” Hamilton says. “The real challenge is where judgment is required, and that’s where AI has largely failed.”
AI researchers have made progress over the years. Peek inside your web browser or your car’s cruise control, and you’ll probably find AI at work. Meanwhile, computer chips keep getting more powerful. In February, Santa Clara, California-based Intel Corp. said it had devised a chip the size of a thumbnail that could perform a trillion calculations a second.
AI Believers
Ten years ago, such a computational feat would have required 10,000 processors.
To believers such as Dhar, Kearns and McKeown, all of this is only the beginning. One day, a subfield of AI known as machine learning, Kearns’s specialty, may give computers the ability to develop their own smarts and extract rules from massive data sets. Another branch, called natural language processing, or NLP, holds out the prospect of software that can understand human language, read up on companies, listen to executives and distill what it learns into trading programmes.
“There’s some nuggets of wisdom in the sea,” says Collective Intellect Chief Technology Officer Tim Wolters.
‘It’s Here’
The hope is that these systems will ape living neurons, think like people and, like traders, understand that some things are neither black nor white but rather in varying shades of gray.
Stock analyst Ralph Acampora, who caused a stir in 1999 by correctly predicting that the Dow Jones Industrial Average would top 10,000, says investment banks are racing to profit from advanced computing such as AI.
“It’s here, and it’s growing,” says Acampora, 65, chief technical analyst at Knight Capital Group Inc. in Jersey City, New Jersey. “Everybody’s trying to outdo everyone else.”
The computers have done well. A November 2005 study by Darien, Connecticut-based Casey, Quirk & Associates, an investment management consulting firm, says that from 2001 to ’05, big-cap US stock funds run by quants beat those run by nonquants.
Inside Lehman
Neither he nor Lehman executives would discuss how the firm uses computers to trade, saying the programmes are proprietary and that divulging information about them would cost the firm its edge in the markets.
At Lehman, Kearns is the big thinker on AI. He leaves most of the actual programming to a handful of Ph.D.s, most of whom he’s recruited at universities or computer conferences.
‘Catastrophic Risk’
Kearns says AI’s failure to live up to its sci-fi hype has created many doubters on Wall Street. He says people should be skeptical: Trading requires institutional knowledge that is difficult, if not impossible, to programme into a computer.
AI holds perils as well as promise for Wall Street, Kearns says. Right now, even sophisticated AI programmes lack common sense, he says.
“When something is going awry in the markets, people can quickly sense it and stop trading,” he says. “If you have completely automated something, it might not be able to do that, and that makes you subject to catastrophic risk.”
Turing Test
It wasn’t until 1951, however, that British mathematician Alan Turing proposed a test for a machine’s capability for thought. In a paper titled “Computing Machinery and Intelligence,” Turing, a computer pioneer who’d worked at Bletchley Park, Britain’s World War II code-breaking centre, suggested the following:
A human judge engages in a text-only conversation with two parties, one human and the other a machine. If the judge can’t reliably tell which is which, the machine passes and can be said to possess intelligence.
No computer has ever done that. Turing committed suicide in 1954. Two years later, computer scientist John McCarthy coined the phrase artificial intelligence to refer to the science of engineering thinking machines.
The Turing Test, as it’s now known, has fueled almost six decades of controversy. Some computer scientists and philosophers say human-like interaction is essential to human-like intelligence. Others say it’s not. The debate still shapes AI research and raises questions about whether traders’ knowledge, creativity, intuition and appetite for risk can ever be programmed into a computer.
Wall Street Smarts
During the 1960s and ’70s, AI research yielded few commercial applications. As Wall Street firms deployed computer-driven programme trading in the ’80s to automatically execute orders and allow arbitrage between stocks, options and futures, the AI world began to splinter. Researchers broke away into an array of camps, each focusing on specific applications rather than on building HAL-like machines.
Language Barrier
The programme worked backward, assessing possible trades and enabling trader-programmers to evaluate the impact of their actions. By working this way, the computer learns how to execute trades going forward.
Language represents one of the biggest gulfs between human and computer intelligence, Dhar says. Closing that divide would mean big money for Wall Street, he says.
Unlike computers, human traders and money managers can glimpse a CEO on television or glance at news reports and sense whether news is good or bad for a stock. In conversation, a person’s vocal tone or inflection can alter -- or even reverse -- the meaning of words.
Let’s say you ask a trader if he thinks US stocks are cheap and he responds, “Yeah, right.” Does he mean stocks are inexpensive or, sarcastically, just the opposite? What matters is not just what people say, but how they say it. Traders also have a feel for what other investors are thinking, so they can make educated guesses about how people will react.
‘Acid Test’
For Dhar, the markets are the ultimate AI lab. “Reality is the acid test,” says Dhar, a 1978 graduate of the Indian Institutes of Technology, or IIT , whose campuses are India’s best schools for engineering and computer science. He collected his doctorate in artificial intelligence from the University of Pittsburgh.
A professor of information systems at Stern, Dhar left the school to become a principal at Morgan Stanley from 1994 to ’97, where he founded the data-mining group and focused on automated trading and the profiling of asset management clients. He still builds computer models to help Wall Street firms predict markets and figure out clients’ needs. Since 2002, his models have correctly predicted the stock prices from month to month 61 percent of the time, he says.
‘Next Frontier’
Dhar says AI programmes typically start with a human hunch about the markets. Let’s say you think that rising volatility in stock prices may signal a coming “breakout,” Wall Street-speak for an abrupt rise or fall in prices. Dhar says he would select market indicators for volatility and stock prices, feed them into his AI algorithms and let them check whether that intuition is right. If it is, the programme would look for market patterns that hold up over time and base trades on them.
“That’s the next frontier,” he says.
What if?
McKeown is trying to upgrade her programme so it can answer broad “what-if” questions, such as, “What if there’s an earthquake in Indonesia?” Her hope is that one day, perhaps within a few years, the programme will be able to write a few paragraphs or pages of answers to such open-ended questions.
Dhar says computer scientists eventually will stitch together advances in machine learning and NLP and set the combined programmes loose on the markets.
A crucial step will be figuring out the types of data AI programmes should employ. The old programmer principle of GIGO -- garbage in, garbage out -- still applies. If you tell a computer to look for relationships between, say, solar flares and the Dow industrials and base trades on the patterns, the computer will do it. You might not make much money, however.
Courting Hedge Funds
“If I give an NLP algorithm ore, it might give me gold,” Dhar says. “If I give it garbage, it’ll give me back garbage.”
Collective Intellect, financed by Denver-based venture capital firm Appian Ventures Inc., is trying to sell hedge funds and investment banks on NLP technology.
Wolters says traders and money managers simply can’t stay on top of all the information flooding the markets these days.
Wall Street’s Future
Computers often did a better job of assessing risk than human loan officers, Thiemann, 50, says.
“And that is where Wall Street is going,” he says. Human traders will still provide insights into the markets, he says; more and more, however, those insights will be based on data rather than intuition.
Thiemann, who has a master’s degree in engineering from Stanford University and an MBA from Harvard Business School, knows algorithms. During the ’90s, he helped HNC Software Inc., now part of Fair Isaac, develop a tracking programme called Falcon to spot credit card fraud.
“If it’s a customer with a questionable transaction and you don’t mind losing them as a customer, you just deny it,” Thiemann says. “If it’s a great customer and a small transaction, you let it go through, but maybe follow up with a call a day or so later.”
Thiemann says he’s taking a similar approach with a trading system he’s building. He calls his programme Deep Green. The name recalls IBM’s Deep Blue -- and money.
DeepGreen evaluates market data, learns from it and scores trading strategies for stocks, options and other investments, he says. Thiemann declines to discuss his computerized hedge fund, beyond saying that he’s currently investing money for friends and family and that he plans to seek other investors this year.
“This is hard, like a moon launch is hard,” Thiemann says of the task ahead of him.
Searching for HAL
As AI invades Wall Street, even the quants will have to change with the times. The kind of conventional trading programmes that hunt out arbitrage opportunities between stocks, options and futures, for example, amount to brute-force computing. Such programmes, much like Deep Blue, merely crunch a lot of numbers quickly.
“They just have to be fast and comprehensive,” Thiemann says. AI systems, by contrast, are designed to adapt and learn as they go.
Dhar says he doubts thinking computers will displace human traders anytime soon. Instead, the machines and their creators will learn to work together.
“This doesn’t get rid of the rule of human creativity; it actually makes it more important,” he says. “You have to be in tune with the market and be able to say, ’I’m smelling something here that’s worth learning about.”’
At Collective Intellect, Vice President Darren Kelly, a former BMO Nesbitt Burns Inc. stock analyst, says tomorrow’s quants will rely on AI to spot patterns that no one has imagined in the free-flowing type of information that can be found in e-mails, on webpages and in voice recordings. After all, such unstructured information accounts for about 80% of all the info out there.
“The next generation of quant may be around unstructured analytics,” Kelly says.
After more than 50 years, the quest for human-level artificial intelligence has yet to yield its HAL 9000. Kearns says he’d settle for making AI pay off on Wall Street
“We’re building systems that can wade out in the human world and understand it,” Kearns says. Traders may never shoot the breeze with a computer at the bar after work. But the machines just might help them pay the bill.
Comment E-mail Print Share
First Published: Fri, May 04 2007. 01 58 PM IST
More Topics: Technology | Tech News |