We are midway through the fall conference cycle, and it is time to take stock on where vendors are in the uptake of generative AI… Here are eight questions to ask the vendor to separate the hype from the true progress towards generative A.
When you hear - “We have been doing AI for a long time…” – it maybe trouble.
It is human nature to downplay something new, especially when caught by surprise… so, when a vendor says above, take caution. The previous approaches to AI were not using large language models (LLMs) and were not able / typically did not leverage unstructured data. They were – if true AI – limited use case applications, more likely using ML and account for a large part being powered by predictive analytics. That doesn’t make any of these offerings not valuable, but they are not generative AI. And vendors resorting to these may give an insight on potentially not getting their heads and arms around generative AI.
WHen you hear - “We have unparalleled data…” - don't get fooled
Many vendors often quote their data advantage, but that is likely limited to transactional data. There are almost no vendors that can say they have brought together all relevant structured and unstructured data in their automation domain… AND made it available for generative AI algorithms. And while that vendor may practically exist but has not come across my briefing and event schedule in Fall of 2023.
Does the vendor have a data lake, lakehouse etc.?
AI needs data and a lot of it. And it needs to be stored cheaply, while being able to query it in any possible way – as the digestion and preparation for AI learning is not standardized. A lakehouse is the right approach as it can store both structured and unstructured data of the enterprise. A data lake is fine as well. Or at least any Big Data / Hadoop powered data storage. If none of that exists – the vendor has a lot of catching up to do.
Is the vendor running AI in the public cloud?
If the answer to this is not a clear Yes! this spells trouble. Only the abundance of cheap compute in the cloud can make generative AI feasible. And as a customer you do not want a vendor to invest double digit millions into an on premise / private cloud supercomputer for model training. It will not get the utilization and the CAPEX is spent better in software (like for generative AI).
How does the vendor get the data?
Before the cloud, AI offerings would rely on getting sample data, train the model and then federate it back to the customer instances. Of course, there was / is a delay for this – and that can make the whole AI model irrelevant / inadequate and worse – wrong. Today the best practice is to have all customer data and continuously train models – in a fully automated approach. You want your vendor at least to plan for this.
Can the vendor explain their 2024 AI platform?
There are many approaches to AI – but a vendor needs to be able to explain – even to a business audience – what their AI platform in 2024 will be. Kudos if that is the same as in 2023 – and they can explain that one today. Understand where data, where compute is coming from. Understand what AI models and training is used. Understand how models are being trained and then implanted into enterprise applications for AI powered outcomes. If the vendor is using any of the standard LLMs – which ones are they?
Can the vendor explain their foundation model strategy?
Foundation models are key to make generative AI work. General knowledge and conversation skills come from a cloud vendors LLMs… but then it comes back to how vendor nomenclature, vertical nomenclature, and lingo and finally customer lingo is inserted into the AI models. This requires an efficient foundation model and model merge architecture. Understand the models being used, how they are merged or call each other and how an enterprise can e.g., insert / create / add their own model.
Can the vendor access transactional data?
The dirty secret of the GenAI hype is – transformer models cannot (or at least easily) access transactional data. Unfortunately -that is where all relevant enterprise data gets stored – in transactional systems. There are workarounds and approaches to bring documents and transactional data together – but they are not yet established / proven. Ask your vendor how they are going to address this issue – the biggest challenge we have with AI going into 2024.
MyPOV
There is a lot of potential for generative AI. It is the
first technology innovation that was more familiar to the consumer than the
enterprise… thanks to how Microsoft / OpenAI launched ChatGPT & co. It is
also the first technology innovation that promised so much potential, that
enterprises were ok with their vendors dropping their roadmaps and figuring out
what to do instead… a first in the industry…
Nonetheless vendors need to understand the technology, adopt
it in their tech stacks and then let generative AI make a difference for their customers…
it will take some time. For now it is all first use cases, the big challenge
will be the integration of document and transactional data. 2024 will be the
big year for generative AI – in the meantime I hope the above debunk questions
help you figure out what is hype and what is real in the fall of 2024.
Let me know if I missed something, totally possible. And don’t
be shy to provide any other feedback. Thank you!