The Future of E-mail
The Future of E-mail
une 12, 2006 (Computerworld) -- Your company scans incoming e-mail for viruses and outgoing messages for confidential information. Your spam filter snags most of the garbage, and it gets better as it learns the latest spamming and phishing spoofs. You're encrypting sensitive e-mail now, and you recently completed a project that keeps your messages safely archived in case federal regulators come knocking.
Indeed, with the right technology, the right policies and a little slice of your budget, you can pretty much manage the messaging madness. And new technology likely to emerge from the labs in the next year or two will help bring a little more civilization to the world of e-mail, ensuring its continued place among the most popular and important of all corporate applications.
However, e-mail's problems will accompany it into its second act, especially as users deploy a growing variety of mobile devices and discover new ways of communicating -- such as instant messaging, blogs, wikis and virtual reality spaces you've never even dreamed of. These will offer green pastures for hackers, spammers and phishers, and will require a whole new round of defensive tools, techniques and policies.
While today's efforts to improve e-mail are aimed mostly at curing its ills, research in vendor and university labs points to brave new uses for the humble e-mail message, from knowledge mining to workflow enhancement. Interviews with researchers, futurists and IT managers yielded the following conclusions about the future of e-mail.
1. New technologies, plus economic and political pressures, will eventually tame the malware.
Ray Tomlinson, a principal engineer at BBN Technologies in Cambridge, Mass., calls the struggle against spam, phishing and malware "pretty much a draw" at present. He has a good deal of perspective on these issues, having sent the world's first network e-mail message in 1971.
Tomlinson points with hope, but some exasperation, to alternate -- some would say competing -- proposals for stemming the tide of offensive, malicious and deceptive e-mail.
"It's not so much a hard technical problem; it's a hard business and political problem," Tomlinson says. "The players have vested interests in the various approaches, and they are fighting tooth and nail to get their approaches adopted. It's not the end users who are the bottleneck here."
Microsoft Corp. is pushing its Sender ID Framework, which verifies that a message was actually sent from a server authorized to send mail for the domain owner. John Scarrow, Microsoft's general manager of antispam and antiphishing strategy, says Sender ID has been adopted by 73% of Fortune 100 companies and is used for 31% of all e-mail messages.
An experimental system at HP Labs shows actual e-mail paths (the gray lines) overlaid on the lab's formal organizational structure (the black lines).
An experimental system at HP Labs shows actual e-mail paths (the gray lines) overlaid on the lab's formal organizational structure (the black lines).
"We are seeing the amount of spam now starting to plateau," he says. "It's a good indication the industry is starting to take a good bite out of the economics of the business."
More good news, Scarrow says, is that while IM and other modes of electronic communication also need to be protected, the technology for doing so is similar to that for e-mail.
Meanwhile, Yahoo Inc. and Cisco Systems Inc. last year submitted to the Internet Engineering Task Force a proposed standard called DomainKeys Identified Mail (DKIM), which, like Sender ID, is designed to guard against spoofing and phishing by authenticating an e-mail sender. DKIM verifies the domain of the sender and also cryptographically verifies the integrity of the message.
In addition to Sender ID, Microsoft has the SmartScreen filter, which uses statistical techniques to learn what's spam and what isn't, and the Phishing Filter add-in for the MSN Search Toolbar. But those tools are not enough, say the folks at Microsoft Research, where some 40 people work on new e-mail technology.
For example, researcher Joshua Goodman says the ultimate solution could be a four-pronged defense against spam called SmartProof. Here's how an experimental version of it works:
* First, a machine-learning filter, similar to SmartScreen, snags the obvious spam and quarantines it or throws it away. The filter passes on to the user's in-box any message that is from someone on the user's "whitelist."
* Messages suspected of being spam trigger replies to the senders, challenging them to prove they're not spammers.
* Senders may respond to the challenge by solving some kind of a puzzle -- one that's easy for a human but hard for an automatic spam generator.
* Alternately, senders can ensure the delivery of their messages by making credit card-based "micropayments." The payments may go to the recipient, the Internet service provider or a charity, or they can be refunded to the sender if the message turns out not to be spam.
"We thought if we could put all that together, we'd have a great long-term solution," Goodman says. "Obviously, it's a very ambitious plan, and I don't think we ever thought it would happen quickly."
Elsewhere at Microsoft, researchers are working on a prototype called MailScope that monitors e-mail routes and alerts users when significant delays are expected. If MailScope sees persistent delays between, say, Microsoft.com and Berkeley.edu, it warns users on those servers that delays are likely, much as a traffic report notifies drivers of congested routes.
In a related Microsoft project called SureMail, when a message is sent, a system posts a tamperproof notification to a table somewhere on the Internet. E-mail recipients periodically query the table and match notifications with messages received. If they find a notification for which there is no message, they know the message has been lost. Microsoft calls these "silent" losses because they so often go undetected. In controlled experiments over two months, using a variety of e-mail systems and carriers, Microsoft found that one in 140 e-mail messages disappeared without a trace. Delays averaged four minutes but lasted as long as 27 hours.
Despite the extensive research and development, some observers say technology can never completely cure e-mail's ills. Economic and regulatory tools will be needed as well, they say.
"Ultimately, I believe there will be a pay-per-message type of service that charges to ensure that e-mail is spam-free," says CIO Matthew Lynch at ShopKo Stores Inc. in Green Bay, Wis. E-mail carriers will charge companies a penny or two per message and will in exchange certify those messages as legitimate, he says. Lynch also predicts "stronger legislation around this topic."
A combination of technology, policy and market measures will keep e-mail among the top of all corporate applications, most users say. "E-mail will continue to be an integral form of communication," says Matthew Marks, head of integrated user services at Aetna Inc. "The capability to quickly and easily distribute a message with an attachment -- documents, links, objects, etc. -- to a large, dispersed audience with tracking and audit cannot be matched by IM, fax or snail mail."
2. E-mail -- just one of the many communications streams in the workplace -- will become part of a "puddle," or "activity thread."
Although e-mail seems unlikely to be supplanted by alternatives, the job of the IT manager is nevertheless complicated by the emergence of other options.
E-mail is in its "pimply adolescence," says futurist Paul Saffo at the Institute for the Future in Palo Alto, Calif. The problems of spam, phishing and e-mail-borne malware will be conquered, he predicts. In the meantime, he cautions, "you can't treat e-mail in isolation. All of our communications forms are melting away, and we are creating new things out of the puddle of old stuff."
Richard Golden, vice president for IT infrastructure at Circuit City Stores Inc. in Richmond, Va., says these threats will cause corporations to augment their technology defenses with strong policy defenses. He says it's relatively easy to protect e-mail systems with spam filters, virus scanners and the like because the systems are well defined, with discrete messages going from Point A to Point B through corporate IT assets.
"But things are converging into a world that is not as clearly definable as a corporate e-mail system," he says. "I think you'll see more policies about things like blogging, for instance. As the lines blur on the means for communications, it's going to require more focus on the information conveyed, regardless of the means used to convey it."
IBM Research is looking for ways to combine e-mail with other functions and integrate it seamlessly into users' daily activities. "It's not enough to help people manage their e-mail; it's important to help them manage their work," says Dan Gruen, a research scientist at the company's facility in Cambridge, Mass. That involves "connecting all the communications and information feeds around a topic or activity," he says.
For example, an IBM Research proto-type called Activity Explorer is a collaboration tool that pulls together e-mail messages, synchronous communication such as instant messages, screen images, files, folders and to-do lists. A project team can establish "activity threads" containing these feeds and can switch easily between asynchronous and real-time collaboration. An activity thread might include the messages, chats and files exchanged among members of a team that's writing a contract bid, for instance.
A more advanced experimental tool from IBM called Unified Activity Manager does all that and more, linking into other corporate applications such as workflow systems. It not only combines the elements of a current activity but also pulls in those elements from past similar activities. These notions of "activity-centric collaboration" will show up in the next release of Lotus Notes, dubbed Hannover, which is expected to ship next year, Gruen says.
Meanwhile, Microsoft Research has developed a way to combine e-mail, files, Web pages, calendar entries, to-do lists and other materials into one searchable archive. Called "Stuff I've Seen," the prototype uses MS Search to index a user's important content and then offers it through a unified interface with sorting, filtering, previews and thumbnail views.
3. New e-mail applications will emerge, including tools that mine message archives for corporate intelligence.
Even as e-mail yields turf to upstarts like IM, especially among younger users, new uses for e-mail are on the horizon. As companies and individuals begin to systematically archive messages, the e-mail becomes available for data mining, and researchers at a number of companies and universities are developing ways to make these archives more accessible.
For example, Hewlett-Packard Co. researcher Bernardo Huberman is devising ways to "harvest organizational knowledge" by mining the e-mail messages and PowerPoint presentations of employees. His techniques go way beyond the searching and categorization of messages that products do pretty well now. Huberman looks at the strengths of communication bonds among employees and patterns of communication that can reveal both hidden problems and opportunities.
"You can look at an organizational chart and make all sorts of inferences about how people work, but when you look at e-mail patterns, you see how they work in a different way," he says. "You discover leadership roles, such as who's the hub through which most of the e-mails go, that you wouldn't identify from the organizational chart."
The result of such pattern or network analysis might be to reorganize departments, projects or activities around those hubs, Huberman says.
HP Labs is now prototyping a tool called Knowledge Navigator that's based on those principles. It applies text mining, clustering algorithms and statistical analyses to employee e-mails and presentations stored on HP's servers. It could handle a query such as, "Who are the top five experts on topic x?" Huberman says, even when such expertise is not explicitly noted in org charts or personnel records.
Huberman says this kind of knowledge harvesting will be used by companies internally on their employees and externally on customers, resulting in the ability to generate messages and pitches aimed at both groups. "What we will see in the next few years is a very targeted way of placing information in the hands of relevant people," Huberman says. "Sure, it can be annoying, but it's better than getting spam on things you don't care about."
Despite the benefits, he acknowledges that mining messages raises ethical and potential legal issues. "In the next few years, we will see a blurring of the boundaries between what is considered private and public," Huberman says.
Mining employee e-mails is "something the company has an interest in, and we are starting to see that interest grow," says Carl Jones, director of collaboration services at The Boeing Co. in Chicago. He says the company has a knowledge management pilot project that, among other things, examines e-mail messages.
"If you have a business problem, you may be able to mine across the e-mail spectrum and find out, hey, there are people out in the field who are subject- matter experts that can help you," says Jones. But, he adds, "we'll have to be very careful about policies on privacy and so on."
Jon Kleinberg, a professor of computer science at Cornell University , says much can be learned from the networks created by people's activities on the Internet.
"How can you infer that someone is influential?" he says. "Is it the obvious things, like they send and receive the most messages, or is it more subtle things, like they operate at the periphery [of a group] but pull together groups that are otherwise weakly connected?"
Kleinberg says answers to such questions may have profound importance for companies that sell online and rely on word-of-mouth recommendations via customers' e-mail. He's looking into two competing theories as to why that kind of e-mail sometimes leads to snowballing sales and other times fizzles.
"Is it the attractiveness of the product, or is it something about the community of people who are into those kinds of products?" Kleinberg wonders.
He says e-mail pattern analysis could help a company answer questions such as, "Who are the key people to influence?" and "For which products is it worth it, and for which is it not?"
"Social network analysis is one of the great tools for productivity going forward, and very few people understand it," says Thornton A. May, a Computerworld columnist and dean at the IT Leadership Academy at Florida Community College at Jacksonville. "People tend to think of social network analysis as a list of people -- an address book. But it should tell you not just who knows who, but who knows what as well."
Users should see social network analysis as more than a way to find dates or customers. It can "solve problems, create teams or recombine organizations," May says.
IBM's Unified Activity Manager
IBM's Unified Activity Manager combines e-mails, files and schedules associated with a multiperson effort to respond to a request for proposals. This kind of capability will ship in IBM's next version of Notes, dubbed Hannover.
(Click image to see larger view)
September 12, 2006 at 02:15 AM in email | Permalink
| Top of page
| Blog Home
