Podcast: Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM
Kaynak: InfoQ ·
In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI inference using llm-d and vLLM on Kubernetes. Edoardo explains how llm-d manages cache locality (KV and prefix caches), request routing, and batch inference across Kubernetes clusters to maximize expensive GPU utilization. By Edoardo Vacchi
Sıkça Sorulan Sorular
Podcast: Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM nedir?
Podcast: Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM, Makale kategorisinde yer alan ve InfoQ üzerinden sunulan bir keşiftir. In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI infere
Nasıl erişebilir veya deneyimleyebilirim?
Bu içeriğe "Aslına Git" butonuna tıklayarak doğrudan orijinal platformunda (InfoQ) erişebilirsiniz.
Bu içerik nereden geliyor?
Bu içerik InfoQ üzerinden OnSuggest topluluğu ve küratörleri tarafından internetin kaliteli köşelerinden seçilmiştir.
Benzer keşifler
Tüm Teknoloji keşifleri →From Cloud Native to AI Native: What Four 2026 Conferences Told Me
Four 2026 conferences, one shift to AI native.
Tesla to refund many Cybertruck deposits before production ends, Musk says
Tesla will have to refund deposits to many Cybertruck customers because orders far exceed what it can build, chief executive Elon Musk said on Monday, in a post on X. The Cybertruck order rate far exceeds our production rate. We will try to make as many as possible before shutting down the line in about […] This story continues at The Next Web
EU sues IBM, Atos and Leonardo over border system delays, FT reports
The European Union has sued IBM, French IT group Atos and Italian defence company Leonardo over years of delays in developing its new biometric border system, the Financial Times reported on Monday. The case was brought by eu-LISA, the EU agency responsible for large-scale IT systems used in border management and security. According to its […] This story continues at The Next Web
Meta’s Dave Wehner leaves to become CFO of AI start-up Fireworks
Dave Wehner has spent 14 years at Meta, more than eight of them as chief financial officer. On Friday, he announced that he is leaving his role as chief strategy officer to become CFO of Fireworks, an AI inference start-up. The Information first reported the move on Friday evening. Wehner joined Facebook in 2012 as […] This story continues at The Next Web
Apple Intelligence registered in China for use on iPads and Macs
Apple’s AI service, Apple Intelligence, has been cleared for use on tablets and laptops in China. The cyberspace regulator in Shanghai made the change on Monday, Reuters reported. The regulator said Apple’s iPads and Mac laptops had been added as two new uses in the service’s filing. It made the change under China’s rules for […] This story continues at The Next Web
Google reportedly adds two TPU variants for secure and long-context AI
Google is introducing two new versions of its in-house AI chips, known as tensor processing units (TPUs), Bloomberg News reported on Monday. The chips are called the TPU 8tx and the TPU 8i Max. Amin Vahdat, Google’s chief technologist for AI infrastructure, told Bloomberg they are reworked versions of existing chips, built for workloads that […] This story continues at The Next Web