Senior Product Manager - ML Platform

Description Senior Product Manager - ML Platform & Heterogeneous Inference Serving Location: Ra'anana or Tel Aviv Hybrid work schedule #LI-Hybrid DriveNets is a leader in large-scale networking solutions for AI infrastructure and service providers. The company’s disaggregated networking architecture transforms the economics of large-scale infrastructures while maximizing performance, utilization, and operational efficiency. Its high-performance AI fabric maximizes GPU utilization and accelerates deployments by optimizing the AI stack end-to-end, resulting in higher tokens-per-second and lower cost-per-token. DriveNets’ solutions power production networks for global tier-1 operators like AT&T and Comcast, and scale multi-vendor AI infrastructures at foundation model labs, NeoClouds, and enterprises. The role DriveNets is seeking a Senior Product Manager focused on ML Platform to be a key member of our Product Management team. Join a dynamic and forward-thinking company at the forefront of AI infrastructure. We leverage advanced technologies to develop innovative solutions that drive efficiency, scalability, and exceptional compute performance. Collaborate with the industry's best as we partner with hyperscalers, emerging NeoClouds, and enterprises building large-scale AI clusters, shaping the future of heterogeneous AI infrastructure. Our environment fosters creativity, teamwork, and growth, and offers you the opportunity to make a meaningful impact while working on groundbreaking projects. As a Senior Product Manager for the ML Platform, you will own the strategy, roadmap, and feature definition of DriveNets' heterogeneous inference serving platform - a system designed to enable efficient inference across diverse and mixed compute environments. You will work directly with our R&D teams and end customers to shape the product, engage compute and storage partners to co-define reference architectures, and serve as an internal expert on performance benchmarking and collective communication tuning in support of DriveNets' cluster and performance engineering teams. Responsibilities Lead product definition and roadmap for DriveNets' heterogeneous inference serving platform, enabling customers to deploy and operate large-scale inference workloads across mixed GPU and accelerator environments. Work closely with R&D teams and end customers to translate real-world inference requirements into concrete product capabilities - spanning serving engine integration, runtime scheduling, memory management, and multi-node execution. Engage with compute and storage partners (GPU vendors, accelerator OEMs, storage providers) to co-define and publish mutual reference architectures and design proposals, enabling validated, production-ready deployments. Act as an expert on ML performance benchmarking - defining benchmarking methodologies, analyzing throughput and latency results, and translating findings into product requirements and competitive positioning. Drive CCL (Collective Communication Library) tuning strategy in collaboration with DriveNets' cluster and performance engineering teams, providing guidance on NCCL/RCCL configuration, topology-aware communication patterns, and their impact on inference and training performance. Define and innovate on KV-cache management and tiering strategies, including disaggregated prefill/decode architectures, CPU/storage offload, and their effects on TTFT and cost-per-token at scale. Create, manage, and present product roadmaps and technology solutions, ensuring alignment with market needs, customer requirements, and partner capabilities. Demonstrate proof-of-concept solutions to customers and partners, showcasing performance, efficiency, and operational advantages of our platform. Work effectively with sales teams and other internal groups within DriveNets to support customer engagements and drive business success. Requirements What we need to see: 5+ years of experience in the HPC or AI/ML industry, with deep hands-on technical expertise across the AI compute stack. Deep understanding of inference serving architectures for heterogeneous compute - including serving engines (vLLM, SGLang, or equivalent), support for mixed accelerator environments, and the scheduling and memory challenges they introduce. Solid knowledge of multi-node inference, tensor and pipeline parallelism, and the trade-offs involved in scaling large models across heterogeneous GPU and accelerator clusters. Solid knowledge of KV-cache management and tiering, including disaggregated prefill/decode architectures, CPU/storage offload, and their operational implications at scale. Experience with performance benchmarking of ML workloads - defining methodologies, running experiments, interpreting throughput/latency/cost trade-offs, and communicating results to both technical and business audiences. Familiarity with CCL tuning (NCCL, RCCL) and the impact of collective communication configuration on inference and training efficiency across large GPU clusters. Familiarity with storage systems relevant to ML workloads - including high-throughput distributed file systems (e.g., Lustre, VAST, WekaIO), object storage, and checkpoint/model weight loading strategies under tight latency budgets. Experience engaging technology partners (compute, storage, silicon vendors) to define joint reference architectures and go-to-market proposals. Clear written and oral communication skills with the ability to effectively collaborate with executives, engineering teams, and external partners. Ability to write extensive technical content (white papers, technical briefs, reference architectures) for external audiences with a balance of technical accuracy and clear messaging. Travel as needed. Ways to stand out from the crowd: Hands-on experience with heterogeneous inference deployments - mixing GPU types, accelerators, or memory tiers within a single serving cluster. Proven experience working directly with end customers to gather requirements, validate solutions, and close feedback loops with engineering. Deep familiarity with disaggregated prefill/decode and its production trade-offs (routing, SLO management, resource utilization). Experience working with silicon or accelerator vendors (NVIDIA, AMD, Intel Gaudi, etc.) on joint solution definition or benchmarking programs. Background in training infrastructure and distributed training frameworks, and how they intersect with inference serving platform design. Understanding of AI-relevant networking technologies (RoCEv2, InfiniBand, lossless Ethernet) as they relate to collective communication performance across large-scale clusters. Proven experience with one or more Tier-1 Clouds (AWS, Azure, GCP, or OCI) or emerging NeoClouds and their ML platform offerings. Experience with observability and monitoring for ML workloads (DCGM, Prometheus, Grafana, OpenTelemetry, etc.). Familiarity with Kubernetes-based ML orchestration and how scheduling interacts with heterogeneous serving efficiency. EDUCATION BS/MS/PhD in Electrical/Computer Engineering, Computer Science, Physics, or other Engineering fields, or equivalent experience. Requirements Requirements What we need to see: 5+ years of experience in the HPC or AI/ML industry, with deep hands-on technical expertise across the AI compute stack. Deep understanding of inference serving architectures for heterogeneous compute - including serving engines (vLLM, SGLang, or equivalent), support for mixed accelerator environments, and the scheduling and memory challenges they introduce. Solid knowledge of multi-node inference, tensor and pipeline parallelism, and the trade-offs involved in scaling large models across heterogeneous GPU and accelerator clusters. Solid knowledge of KV-cache management and tiering, including disaggregated prefill/decode architectures, CPU/storage offload, and their operational implications at scale. Experience with performance benchmarking of ML workloads - defining methodologies, running experiments, interpreting throughput/latency/cost trade-offs, and communicating results to both technical and business audiences. Familiarity with CCL tuning (NCCL, RCCL) and the impact of collective communication configuration on inference and training efficiency across large GPU clusters. Familiarity with storage systems relevant to ML workloads - including high-throughput distributed file systems (e.g., Lustre, VAST, WekaIO), object storage, and checkpoint/model weight loading strategies under tight latency budgets. Experience engaging technology partners (compute, storage, silicon vendors) to define joint reference architectures and go-to-market proposals. Clear written and oral communication skills with the ability to effectively collaborate with executives, engineering teams, and external partners. Ability to write extensive technical content (white papers, technical briefs, reference architectures) for external audiences with a balance of technical accuracy and clear messaging. Travel as needed. Ways to stand out from the crowd: Hands-on experience with heterogeneous inference deployments - mixing GPU types, accelerators, or memory tiers within a single serving cluster. Proven experience working directly with end customers to gather requirements, validate solutions, and close feedback loops with engineering. Deep familiarity with disaggregated prefill/decode and its production trade-offs (routing, SLO management, resource utilization). Experience working with silicon or accelerator vendors (NVIDIA, AMD, Intel Gaudi, etc.) on joint solution definition or benchmarking programs. Background in training infrastructure and distributed training frameworks, and how they intersect with inference serving platform design. Understanding of AI-relevant networking technologies (RoCEv2, InfiniBand, lossless Ethernet) as they relate to collective communication performance across large-scale clusters. Proven experience with one or more Tier-1 Clouds (AWS, Azure, GCP, or OCI) or emerging NeoClouds and their ML platform offerings. Experience with observability and monitoring for ML workloads (DCGM, Prometheus, Grafana, OpenTelemetry, etc.). Familiarity with Kubernetes-based ML orchestration and how scheduling interacts with heterogeneous serving efficiency. EDUCATION BS/MS/PhD in Electrical/Computer Engineering, Computer Science, Physics, or other Engineering fields, or equivalent experience. More About DriveNets Based in Israel with extended teams located in the US, Japan, and Romania, DriveNets operations cover more than twelve countries globally. Powering production networks for global tier-1 operators, DriveNets is a leader in large-scale networking solutions for AI infrastructure and service providers. Visit our website to learn more: https://drivenets.com/company/ If your experience is close but doesn’t fulfil all requirements, please submit your application. DriveNets is on a mission to build a special company comprised of individuals with different backgrounds, perspectives, and experiences. DriveNets is an equal opportunity employer. We do not discriminate based on upon race, religion, national origin, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with disability, or other applicable legally protected characteristics.