In-House LLM Serving at Netflix (opens in new tab)
In-House LLM Serving at Netflix -- 2 Listen Share By AI Platform’s Model Runtime team and Inference team Introduction Most organizations consume LLMs through hosted APIs. Netflix went further — we run the full stack ourselves, from model deployment through inference, inside our…