🔵 Serving Fine-Tuned Models in Production

noravonthenen — Fri, 25 Oct 2024 14:52:51 GMT

When it comes to serving of fine-tuned LLMs it is essential in production systems to share the base models, while hosting the LoRA parameters in the same container. This reduces the costs, however poses new problems like providing tenant isolation, scaling and dynamic loading of new adapters. In this talk we will cover the different aspects of the SAP Generative AI Hub and how the serving issues can be solved.

Speaker: Karim Mohraz, SAP

Validation tutorial: https://developers.sap.com/tutorials/devtoberfest2024-week4-ai-session-validation.html

Link to download presentation: https://d.dam.sap.com/a/6eBffpm/Devtoberfest_Serving%20Fine-Tuned%20Models%20in%20Production.pdf?inline=true&rc=10&doi=SAP1137793

Re: 🔵 Serving Fine-Tuned Models in Production

guydebruyn — Thu, 24 Oct 2024 11:38:02 GMT

Slides mentioned "RAG vs. parameter fine tuning". But looking at the description, was the intention "Prompt engineering vs. parameter fine tuning"?

rss.livelink.thread@place:occasion

🔵 Serving Fine-Tuned Models in Production

Re: 🔵 Serving Fine-Tuned Models in Production