Data Management Job Market Projections

An exploration of how AI is redefining data-related roles.

Author:  Data Strategy Professionals team  |  Post Date:  Jan 30, 2024  |  Last Update:  Mar 28, 2024  |  Related Posts

The past year has seen an explosion in the use and capabilities of AI, especially Large Language Models (LLMs) such as OpenAI's GPT-4, Anthropic's Claude, Google's Bard, and Facebook's Llama-2. In this writeup, we aim to help you understand what the AI gold rush means for the Data Management job market.

Contents

• Impact of New Technologies
• Projections
• Complexities
• Conclusion
• Related Posts

Impact of New Technologies

AI capabilities are becoming more impressive with each new release, so it's worth considering: how replaceable is the work of Data Management practitioners? Right now, AI tools are more an asset than a threat to Data Management practitioners' job security; it seems like AI is strengthening the demand for Data Strategy work. One industry observer notes:

Here's a list of the ways AI supports various Data Management roles:

• Data Governance – automatically draft documentation for business terms
• Data Architecture – data asset discovery with complete context
• Metadata Management – anomaly detection based on automated data lineage
• Data Modeling – generate synthetic data for training, testing and validating models when actual data is limited, sensitive, or biased
• Data Quality – data labeling at or above a human level of accuracy (with much greater efficiency)
• Data Science – summarizing or extracting structured data from unstructured sources

There are still significant hurdles that limit AI's ability to perform a Data Management role in full. Hallucinations threaten the reliable accuracy of data. Limited strategic planning and autonomy undermine the ability to make decisions and manage data across its lifecycle. There is a sizable trust gap between AI adoption and the 60% of people who feel current safeguards are insufficient.

Given trends in the advancement of AI capabilities, it's possible we're only one or two GPT releases away from machine learning tools that can overcome these barriers. Studies by the AI Policy Institute (AIPI) and OpenAI indicate that around 20% of current US jobs and tasks could be "significantly exposed to AI automation in the near future." AI increasingly demonstrates the capacity to generate high-quality outputs across various domains.

Yet even if AI can support or take on more and more Data Management tasks in the long term, the near-term demand for data practitioners is fueled by the increasing volume and significance of data as a source of value for the business.

Projections

It's clear that the services of data-related professions are very much still in demand. The Future of Jobs Report 2023 from the World Economic Forum predicts a 40% increase in the number of AI and machine learning specialists by 2027, a 30-35% rise in demand for roles like Data Analysts and Data Scientists, and a 31% increase in demand for Data Security professionals. Across these roles, the net growth totals 2.6 million jobs.

Over the next decade (2022-32), the US Bureau of Labor Statistics estimates that:

• The profession of Database Administrators and Architects will grow 8% (faster than average),
• The profession of Computer and Information Systems Managers will grow 15% (much faster than average), and
• The profession of Data Scientists will grow 35% (much faster than average).

Complexities

Predictions about the rate and precise direction of AI capability development are highly uncertain. There are significant limitations in what job market projections can reliably tell us about the future of data-related jobs.

The UK's Government Office for Science highlights how meaningful gaps in model interpretability and our understanding of how to measure progress in artificial intelligence make it difficult to monitor the potential capabilities and impacts of new models. Anthropic, the creators of Claude, further highlight the added difficulty in predicting "qualitatively different, specific capabilities [which] can appear abruptly and discontinuously."

It's possible the labor market projections we have discussed so far do not fully account for the impact AI is currently having on jobs that heavily involve writing and programming. Already, generative AI is significantly impacting ways of working for data strategists, analysts, and engineers.

Another limitation is that there seems to be a strong assumption at both World Economic Forum and US Bureau of Labor Statistics that increases in demand for labor directly correspond to increases in the total amount of new human labor, which may not hold true over the long term.

However, just because AI can perform an increasing number of different tasks doesn't mean that the job market for data practitioners won't simultaneously grow. We may see increased demand for roles that require human judgment, communication, and soft skills, or entirely new opportunities to work with data may spring up.

Data Management practitioners can adapt to the dynamic job market by studying best practices, upskilling, and staying aware of developments in AI.

Conclusion

It's safe to say that the impact of generative AI is not just disruptive, but transformative. In the near term, AI creates new opportunities and productivity enhancements across activities such as drafting documents, conducting research, performing analytics, creating images, and planning.

Despite the advancements in AI and its growing capability to assume more complex tasks, the demand for skilled Data Management professionals remains strong. The need for data practitioners is fueled by the increasing demand for data-driven decision-making, coupled with the understanding that Data Management is a critical foundation for other initiatives that require the use of data.

The projections from the World Economic Forum and the US Bureau of Labor Statistics depict a future where data-related roles continue to expand. The core tasks of a Data Management professional require qualities such as judgment, emotional intelligence, and the ability to take ethical considerations into account with dealing with sensitive data assets. The future of Data Management will entail learning new things and using the tools at our disposal to make the best use of our organizations' data assets.