Unearthing Recruiting Gold

By rooting through different databases, data mining connects the dots of candidate and employee data to speed up recruiting and improve sourcing.

Which employees are most likely to retire in the next five years and what skills do they have? What universities did your ideal job candidates attend? What cities have the best talent supply for the company’s new division?

If you can answer these questions, congratulations. You’re ahead of the curve. If not, read on about data mining, a powerful “business intelligence” technology making its way into the staffing world.


Making Mining Work


Data mining isn’t a “plug-and-play” solution; you can’t just fire up some software and get the results you want. Digging through data to find gold requires recruiting know-how.

“It takes somebody who understands both the technology’s capability and the recruiting process to get it done right,” says David Reed, Accenture HR Services’ vice president of resourcing services, North America.

Experts offer these guidelines to implementing data mining successfully: Define your objectives. You may not need elaborate data mining to achieve your recruiting goals. If you do need it, first define specifically what you want to know and why. Otherwise, it’s a wasted effort. “A fool with a tool is still a fool,” warns one analyst.

Don’t forget the human touch. They may speak different languages, but human resources and information-technology leaders must communicate and coordinate to buy and implement a successful data-mining solution. The results aren’t worth much without business leaders who interpret and act on them.

“Data mining is a guide; it’s not a tell-all crystal ball,” says Stephan Kudyba, a professor of management at the New Jersey Institute of Technology in Newark.

To outsource or not? Building a data-mining solution in-house requires technical expertise in the form of economists, statisticians or other savvy technical people who know what they’re doing. If you don’t have them on staff—and smaller firms typically don’t—hire an outside firm to perform a pilot project and judge its success before deciding whether to expand. Several vendors, including SAS, IBM, Oracle and Microsoft, sell data-mining solutions.

Avoid “garbage in, garbage out.” Successful data mining requires clean, properly structured data. In the case of staffing, this means up-to-date candidate information, so cull outdated and inactive resumes and applications and get rid of duplicative data.


Data mining is the process of extracting information from multiple databases to uncover and analyze trends, patterns and relationships. Using a combination of hardware, software and services, it turns raw data into useful knowledge, which can lead to strategic decisions that make a company more competitive.

A decade or so old, data mining is increasingly used by companies in manufacturing, finance and other industries to understand past events and to predict possible actions. Its best-known application is “customer relationship management,” the customer profiling that allows retailers to target certain products and advertising at certain buyers, with the aim of increasing sales and profits.

Data mining is just starting to catch on in recruiting, primarily at very large companies with lots of candidate information to manage. But even companies with as few as 3,000 employees can harness this technology to improve everyday sourcing as well as long-term workforce planning.

A Serious Investment

Plenty of HR software tools help recruiters handle the daily tasks of posting job ads, compiling resumes and tracking applicants. Standard systems work well to manage the Internet-generated glut of names, ages, addresses, skills and other candidate information—and that’s enough for many employers.

However, some employers need more. Large companies with scattered worksites and multiple databases may want to drill down into candidate data to discover patterns and trends that let recruiters better target their efforts. With data mining, employers can, for example, find internal job candidates, identify which universities or geographic areas yield the best candidates and discover which online sources produced the most hires in a given period.

“The ultimate goal is to make sure the recruitment process is not too lengthy or expensive and gets the best candidates,” says Aiman Zeid, a human capital management specialist at Cary, N.C.-based SAS, a provider of data-mining software and services.

Data-mining solutions are expensive but in the long run can make staffing more cost effective by giving recruiters top-grade information to work with, experts say. Data-mining technology solutions range in price from about $20,000 to $80,000, according to Stephan Kudyba, a professor of management at the New Jersey Institute of Technology in Newark and president of data-mining consulting firm Null Sigma Inc. in Wayne, N.J. Hiring a consultant to do data mining costs at least $10,000, depending on the scope and complexity of the project, Kudyba says. He adds: “The payoff in mining can often far exceed the costs, even very high costs.”

Data mining also can bring savings, says Alice Snell, vice president of iLogos Research, an independent staffing consulting division of Taleo Corp., a San Francisco-based HR software firm.

“Now you are getting to the short list [of candidates] more quickly,” Snell says. “Making the best match for the position has repercussions down the line,” including lower turnover and a 20 percent to 25 percent reduction in sourcing costs, she says. For example, the sourcing cost per candidate for a firm with a “candidate relationship database” is $6.20, compared with $7.70 for a firm that lacks such a database and whose sourcing budget is spent strictly on advertising, according to a 2002 iLogos report, Economics of Candidate Relationship Databases.

Consultants advise HR departments to consider their recruiting goals and budgets before taking the data-mining plunge. The solution is too complex and costly for the simplest recruiting functions, but it could be worth the investment for companies that count recruiting as a strategic component of future growth.

“We [in HR] need to be far more strategic and earn our place in executive management,” says Scott Schoenick, a senior knowledge consultant at Taleo. The right technology can make recruiting more data-driven and proactive, he says.

What Data Mining Does

Data mining’s best potential application in recruiting is for sourcing. Currently, many recruiters use software that allows key word searches querying their resume databases or job boards for candidates with certain skills and experience. The results are often too numerous or not quite right, making the process inefficient.

Data mining can yield more precise results. The solution recommended by some software executives and analysts is to create a database that stores job candidates’ self-rated “skills profiles.” Such a database also contains the normal candidate information—years of experience, past jobs, education and more—but the data is more detailed and better structured, so it provides better matches.

The detailed data “makes a huge difference in terms of effectiveness and accuracy,” says iLogos president Yves Lermusiaux.

Employers also can have current employees write profiles, creating an electronic inventory of internal skills that, combined with databases of candidate information, can become a valuable source of proprietary human capital information, Snell argues. “A candidate database is gold. Companies need to optimize their use of it,” she says.

One use is the ability to search internally for current employees who may not know of openings but could be candidates. Another is profiling the most successful or longest-serving employees to find their common characteristics—which can be sought in future hires.

“Data mining lets you build models of what makes these top achievers tick. You can use these models to recruit people in the future,” Kudyba says. Armed with this knowledge, employers can make good hires and prevent bad ones, reducing turnover, he adds.

Another application determines which of many sources are most useful for finding candidates with certain skills. Similarly, the best-performing new hires can be studied to see from which sources they came.

A company may find, for example, that its computer programmers with five years of Java experience were located through Dice.com, so the company knows to turn to that source when seeking programmers in the future. Along the same lines, a company may discover that these programmers graduated from the same few universities, knowledge that allows more targeted campus recruiting.

Evaluating source effectiveness is old hat, but “the new part of this is the real-time element,” says David Reed, Accenture HR Services’ vice president of resourcing services, North America. At Accenture, “we are able to track results of [advertising] campaigns and fine-tune strategies more quickly on the basis of this data,” he adds.

Forecasting Labor, Skills Needs

Higher-end HR software already includes “workforce analytics” tools that measure recruiters’ performance and signal a need to lower advertising costs, speed up time-to-hire or take some other action. Data mining goes further by enabling employers to predict and plan their workforce needs.

Most of the recruiting world hasn’t reached this level of technical sophistication yet. In the vanguard is SAS, which uses its own product to build predictive models that lead to targeted HR programs. “It’s a proactive form of workforce planning,” says Jeffrey B. Chambers, vice president of HR at the 10,000-person global company.

Who’s likely to jump ship? Who is eligible, and likely, to retire? SAS knows, thanks to data mining, so it can design programs to retain desirable employees. “We look at the critical employees with the critical skills first,” says Chambers. “Who can we not afford to lose? Who has significant business/institutional knowledge? Who is key to customer retention and development?

“Maintaining institutional expertise is critical to our company,” Chambers says. “We know 25 percent of our population will be eligible to retire in the next five years. We also know anecdotally they’ve been asking for a retiree health benefit.”

HR crunched the numbers and successfully made the case to top management to offer the benefit, he says. The health benefit was structured to favor longevity, so senior employees were encouraged to stay. “It also attracted people with domain expertise that’s needed to sell solutions to C-level executives,” says Chambers, “since no other employer is offering retiree health care to new hires. A true win-win!”

In the future, data mining may be used to forecast labor needs. Consider a hypothetical example: If HR knows that sales and turnover will each run about 10 percent more next year than this year, it can predict the number and type of employees who will be needed, whether they’re already within the company and where to find them if they’re not already on board.

Similarly, a skills gap analysis assesses an employer’s in-house talent supply based on skills, geography and other factors. The idea is to compare what a company has to what it will need down the road and to plan accordingly.

Finding Passive Candidates

Data-mining tools also can help recruiters spread their nets wider, locating candidates they otherwise would not have found or would have found only through outside search firms. For instance, one small company’s data-mining service has caught on as a way to identify passive candidates.

Eliyon Technologies Corp. of Cambridge, Mass., uses a text-mining tool that crawls the web to find names of business people and information about them in press releases, web sites, articles and other online sources. From this information Eliyon has compiled a database containing resume-like profiles of 20 million executives, managers and other professionals.

For $1,000 per month, recruiters can search Eliyon’s database by job title, company, university and other items. A search for a current president of a Fortune 500 biotechnology company yields four names, along with title, company, employment history, education, board membership and affiliations. It also shows where on the web Eliyon found the information.

“If you’re big enough to have an internal recruiting department, you’re big enough to benefit” from the service, says Eliyon spokesman Brian Payea. Clients include Fortune 500 companies and executive search firms, he says.

The service gives recruiters another way to locate candidates and reduces search costs by speeding up the sourcing process, according to Eliyon.

“Say we’re looking for a vice president of merchandising,” says Carl Lopes, vice president of corporate employment at office supply chain Staples Inc. “With Eliyon, you can put in ‘vice president of merchandising, Best Buy,’ and it’ll come out with a lot of names. There’s a start.”

Eliyon’s service isn’t perfect. It can’t find people who are not mentioned on the web. Its information is not always accurate—there may be two Bob Joneses, for example. But it beats surfing the Internet, which is time-consuming and inefficient, Lopes says. “The problem is you’ll come up with 17,000 hits. … Eliyon narrows the information,” he says.

Eliyon’s service, along with Staples’ own full-time recruiting researcher, allows Staples to minimize the use of executive search firms, according to Lopes. Staples pays Eliyon $9,000 per year for unlimited use, compared with a contingency search fee that can easily run $25,000 for just one job, he adds. Based in Framingham, Mass., Staples has 58,000 employees.

Resisting the Data Revolution

For now, data-mining software and services remain pie in the sky for most recruiters. Plenty of barriers to adoption remain, staffing experts say, starting with a general ignorance of high-end technology. “I don’t think [recruiters] understand the power of data,” including its strategic uses, Chambers says.

“There’s a lot of old-line human resources people saying, ‘I know [the best candidates] when I see ’em.’ That’s a major problem in this industry. A lot of potential to learn about people is going wasted,” adds Wendell Williams, managing director of ScientificSelection.com, an Atlanta-based hiring and performance-management consulting firm.

Many in HR don’t have a good handle on the information they do have, let alone the ability to massage it. “The data is sitting there, but it’s not in a format that’s conducive to data mining,” says Shakthi Kumar, executive director of information architecture in the Hackensack, N.J., office of information-technology services provider Ness Technologies Inc.

Taleo’s Schoenick adds that recruiters have a lot on their plates. The job’s daily demands, coupled with the long-term changes wrought by the Internet and globalization, leave recruiters little, if any, time to focus on long-range technology solutions, he notes.

Also, data mining is beyond many HR budgets and usually ends up being implemented by HR only when other parts of an organization are using it already.

Eventually, data-mining technology will catch on with more employers, SAS’s Zeid predicts, because competition for labor will force recruiters to become more proactive than they are now. “They can’t afford to wait and react to problems,” he says.

Additional Resources

  • Economics of Candidate Relationship Databases, iLogos Research, 2002.
  • Managing Data Mining: Advice from Experts, Stephan Kudyba, IRM Press, 2004.
  • Data Mining and Business Intelligence: A Guide to Productivity, Stephan Kudyba and Richard Hoptroff, Idea Group Publishing, February 2001.


Share with:


archive, Recruiting