artificial intelligence in big data

Jump Over Artificial Intelligence “Big Data” Requirements

Jeremy Artificial Intelligence, Data, Products & Technology

Jobscience provides recruiting and workforce activity management systems for recruitment agencies, staffing firms, and corporate employers on a global scale in 30 languages.  As the Company’s Chief Data Scientist, I work with companies at the forefront of artificial research and development.

Our attraction is that, in total, Jobscience manages large amounts of recruiting and employment data.  So-called “big data” is often a requirement for proof of concept testing.  (This is one reason that Google has launched Google Jobs and Google Hire.)

Many of our customers, both agencies, and corporate accounts lack the size to produce enough data to “move the needle” for most artificial intelligence tools to learn and adapt to the needs of these businesses.

Typically, a sample of at least 200 data points is required to provide statistical confidence.   When you are trying to determine significant classification factors and predict outcomes (often through regression analysis) we are considering hundreds of factors times 200 data points could be required.

Although a small firm may not produce data of this magnitude and data of this magnitude may be required to identify patterns that can be updated (learn) as more data is collected, we are exploring the feasibility of using “blind pools” of similar businesses in similar regions to assemble the magnitude of data that is required.  Participants would subscribe to the service and permit Jobscience to use their data, for example, to develop more accurate key performance indicator (KPI) reporting.

KPIs are widely reported by industry associations, consultants, and vendors.  At best they are averages that provide little credibility as a tool for managing your specific business.  Time-to-fill is common KPI that is widely used.   I have read that corporate recruiters average 70 days to fill an open position and permanent placement agencies average 53 days.  What relevance do these (national) averages have in evaluating your firm’s performance?  They are unverifiable and verge on superstition.

Using the time-to-fill example, consider the possibility of compiling enough data on enough factors from enough permanent placement agencies in the Northeastern United States to rank the top performers by quartile and determine whether there are significant differences between the top quartile and the bottom quartile for such KPIs and time-to-fill.

We did something similar using publicly-available budget data compiled by the State of Washington on each hospital licensed by the State to first rank them by profitability and then correlate different KPIs with the hospitals in the top profit quartile.  (There was a 600 basis-point difference between the top quartile and the bottom quartile so the differences were significant.)  We found that the most important KPI factor that correlated with profitability was contingent staff utilization.  The more they could utilize their on-demand workforce, and the less they had to rely on an under-utilized full-time workforce, the higher the chances the hospital was in the top quartile.  Time-to-fill (and have to rely less on overtime to fill in for an opening) was far down the line in important factors related to profitability.

It is generally believed that it will take “big data” to identify patterns and apply that knowledge to the management process being automated.  The implication is that our biggest customers will have the most benefit from the application of artificial intelligence tools because they have the most data.  A workaround solution is clearly for smaller customers to pool data so that, in this example, we can begin to determine which KPIs are the most important for our customer’s businesses.  This analysis can then be embedded in the process flow and learning tools can be added to incorporate future data into improving the workflow logic.

The purpose of these articles is to show how artificial intelligence tools can be incorporated into Jobscience’ products to address specific customer issues and to promote suggestions.  If you have suggested areas within recruitment and workforce management that could benefit from applying the emerging artificial intelligence toolbox, please contact me.  If you are a Jobscience customer, please use the IDEAs community site.  If you are not my email is