THE TECHNOLOGY INTERVIEW
others . We provide the best platform for workloads and for scheduling and for operations , and keeping that operation model the same , regardless of it ’ s an AI / ML workload or a traditional workload or a modern app workload . Now that IT needs to be in charge of that , it ’ s not just data scientists anymore . That ’ s why Cloud Foundation is so important .
“ It ’ s about making sure that our platform is ready to easily automate and accept those workloads . Most AI / ML workloads are deployed as containers , so Tanzu Kubernetes grid is built right into Cloud Foundation , so that ’ s a pretty natural step forward .”
Q . TELL US ABOUT THE IMPORTANCE OF SPECIALISM WHEN IT COMES TO BUILDING LLMS
» “ Performance is important to us , cost is important to us and sustainability is important to us , and all those things are related . There ’ s a good analogy that I heard that I like to repeat , where if you ask ChatGPT , ‘ how do I make toast ?’, that prompt and that query return might cost you a dollar . That ’ s a really expensive slice of toast .
“ If I have a domain specific model that ’ s only for making toast , and so it ’ s much smaller , it doesn ’ t require as many resources to run , it costs less . So when I ask it how to make toast , maybe it ’ s two cents and that ’ s much more appropriate .
“ So , from a sustainability perspective , it ’ s not taking as much power and cooling , and also from a cost perspective , I don ’ t need as much horsepower , or as many resources . And so that is really important .
“ There ’ s also the other aspect that if you just have a general model that hasn ’ t been fine-tuned on your domain
specific data , you ’ re not going to get as helpful responses . Being able to have that domain specific data is important for business value , but also because to create an LLM , it takes months and tens of thousands of GPUs and most organisations don ’ t have that , and that ’ s not going to happen .
“ Being able to take an LLM that is trained originally with data that they trust , but that open data set ’ s going to be important and then be able to put that next to their data that they ’ re going to fine tune with is going to be important . And then that will make sure they have the best responses , but also the lowest cost , the best performance and more sustainable choice .”
24 November 2023