Data preparedness has become the next critical obstacle in a long queue of generative AI-implementation roadblocks. But unlike the skills gap, responsible use, inaccuracies, job threats and other hurdles we’re still working on getting past, this one looks to be more of a systemic issue deep within the wiring of company infrastructures that in many cases could take years to unravel. And new research from data observability firm Monte Carlo reveals that while nearly all data leaders are building generative AI applications, most don’t believe their data estate is prepared to support them.
The firm’s new State of Reliable AI survey, in partnership with Wakefield Research and which polled 200 data leaders and professionals in April 2024, comes as data teams are grappling with the adoption of generative AI. More threatening is that a full 100 percent of data pros surveyed feel pressure from their leadership to implement a GenAI strategy and/or build GenAI products despite these shortfalls.
Among other findings are several statistics that indicate the current state of the AI race and professional sentiment about the technology:
- 91 percent of data leaders (VP or above) have built or are currently building a GenAI product
- 82 percent of respondents rated the potential usefulness of GenAI at least an 8 on a scale of 1-10, but 90 percent believe their leaders do not have realistic expectations for its technical feasibility or ability to drive business value
- 84 percent of respondents indicate that it is the data team’s responsibility to implement a GenAI strategy, vs. 12 percent whose organizations have built dedicated GenAI teams
While AI is widely expected to be among the most transformative technologies of the last decade, these findings suggest a troubling disconnect between data teams and business stakeholders: Data leaders clearly feel the pressure and responsibility to participate in the GenAI revolution, but some may be forging ahead in spite of more primordial priorities—and in some cases, against their better judgment.
The state of reliable AI infrastructure
Even before the advent of GenAI, organizations were dealing with an exponentially greater volume of data than in decades past. Since adopting GenAI programs, 91 percent of data leaders report that both applications and the number of critical data sources has increased even further—deepening the complexity and scale of their data estates in the process.
“Data is the lifeblood of all AI—without secure, compliant, and reliable data, enterprise AI initiatives will fail before they get off the ground,” said Lior Solomon, VP of data at Drata, in a news release. “Data quality is a critical but often overlooked component of ensuring ethical and accurate models, and the fact that 68 percent of data leaders surveyed did not feel completely confident that their data reflects the unsung importance of this puzzle piece. The most advanced AI projects will prioritize data reliability at each stage of the model development life cycle, from ingestion in the database to fine-tuning or RAG.”
What’s more, the survey revealed that data teams are using a myriad of approaches to tackle GenAI, suggesting that not only is the volume and complexity of data increasing, but that there’s no one-size-fits-most method for getting these AI models customer-ready.
How data teams are approaching AI:
- 49 percent are building their own large language models (LLMs)
- 49 percent are using model-as-a-service providers like OpenAI or Anthropic
- 48 percent are implementing a retrieval-augmented generation (RAG) architecture
- 48 percent are fine-tuning models-as-a-service or their own LLMs
As the complexity of the AI’s architecture—and the data that powers it—continues to expand, one perennial problem expands with it: data quality issues.
The key question: Is your data GenAI-ready?
Data quality has always been a challenge for data teams. However, survey results reveal that the introduction of GenAI has exacerbated both the scope and severity of this problem.
Our findings suggest that while the data estate has evolved rapidly over the last few years to accommodate AI and other novel use cases, data quality management has not. In fact, many respondents still rely on tedious and unscalable data quality methods, such as testing and monitoring, with more than half (54 percent) of data professionals surveyed depending exclusively on manual testing.
This lack of automated, resolution-focused solutions is reflected in the data, with two-thirds of respondents experiencing a data incident in the past 6 months that cost their organization $100,000 or more. This is a shocking figure when you consider that 70 percent of data leaders surveyed reported that it takes longer than 4 hours to find a data incident. What’s worse, previous surveys commissioned by Monte Carlo reveal that data teams face, on average, 67 data incidents per month.
“In 2024, data leaders are tasked with not only shepherding their companies’ GenAI initiatives from experimentation to production, but also ensuring that the data itself is AI-ready, in other words, secure, compliant, and most of all, trusted,” said Barr Moses, co-founder and CEO of Monte Carlo, in the release. “As validated by our survey, organizations will fail without treating data trust with the diligence it deserves. Prioritizing automatic, resolution-focused data quality approaches like data observability will empower data teams to achieve enterprise-grade AI at scale.”
