The 2020-2021 global pandemic has fundamentally changed the way we live and work. While we have all experienced at least some temporary suffering – some much more seriously than others - how we work has changed in ways that are unlikely to be reversed. The data scientists are one profession that has seen substantial transformation that is very likely to be permanent. With most business organizations continuing to view their data science needs as unique and highly specialized, tools such as Varicent’s augmented intelligence solution, Symon.AI, will continue to see strong demand growth.
The Data Science Decline is Only Temporary
Like all workers over the past year, data scientists have seen a fall-off in demand. However, unlike other professions, the decline is likely only temporary. Table 1 shows the data science job postings by industry that in 2019 accounted for 75% of all open positions. Despite the overall decline in 2020 job posts, in a few critical industries demand continued to grow - pharmaceuticals, e-commerce, information systems, and technology consulting.
Table 1
Source: Burning Glass Technologies
Employment services firms conducted very few job searches in 2020, but many other industries reduced their demand for data scientists only slightly. Despite last year’s fall off, the eight-year trend of steadily rising, robust demand continued. See Figure 1. With fewer that 1,500 job posts in 2012, a peak of nearly 40,000 positions were open in 2019. As vaccinations take hold and immunity spreads, the return to strong aggregate demand will renew demand for data scientists.
Figure 1
AI and ML Continues to Transform
Not only has the demand for data scientists continued to grow, but the focus on artificial intelligence (AI) and machine learning (ML) applications has continued to shift and transform.
At least in part as a result of the pandemic, new use cases have emerged. While engaging with customers has been an important AI/ML application in recent years, social distancing, lock downs, and quarantines have made automated customer service and AI/ML support of customer service representatives even more important.
Financial Planning and Budgeting More Important Now Than Ever
With the challenges faced by many businesses as a result of shutdowns and lock downs, financial planning and budgeting has also become increasingly important. In addition, the demand for data science tools has also expanded rapidly over the year. The continuing shortage of data science skills persists but is changing only slowly. Also, the importance of regulatory compliance is only increasing, not decreasing.
The Need for Smarter Security Soars
The threat and risk of cybersecurity incidents has created demand for AI/ML applications that can foresee such incidents. With a very large proportion of the global labor force working remote, demand for smarter, more secure systems has exploded. For years, if not decades, security management has been on site, but without humans on site, much smarter systems are needed.
Despite these rapidly changing needs and demands of business organizations, many still want to build their own systems. It appears that building and installing pre-made applications is still years, if not decades, away.
The Demand for Natural Language Processing, Business Process Automation, and Edge Computing
Significant differences across firms and industries continue to create demand for natural language processing (NLP). External benchmarks, document understanding, semantic parsing of text, and other idiosyncratic differences make the business context critical for NLP applications.
Business process automation is also, generally, unique across business organizations. As a result, very specialized AI/ML business process applications are most often required. Finally, trust and explainability are also extremely important, especially in regulated environments with bias detection as a priority. With a growing number of AI deployments consisting of thousands, and in some millions of equations, model management, explainability, and lifecycle management has become a challenging requirement.
Finally, as edge computing evolves, there will be a need to push the training of AI models out to the edge where data are created and consumed. It’s not economical to move data to the models. The data, computing, storage, and user interface all need to occur on the same cloud deployment.
This is the third blog in my “The Future of” series, which leverages data from Burning Glass Technologies. Be sure to read the first blog in the series, “The Future of Sales: How Selling Is Changing and What to Do About It.
