Releases · pytorch/serve · GitHub
November 5, 2024ai_discoveryinfo
Highlights Include GenAI updates No code LLM deployments with TorchServe + vLLM & TensorRT-LLM using ts.llm_launcher script OpenAI API support for TorchServe + vLLM Integration of TensorRT-LLM engine Stateful Inference on AWS Sagemaker (see blog) Support for linux-aarch64 CI & nightly regression added Publish docker & KServe images PyTorch updates Support for PyTorch 2.4 Deprecation of TorchText PyTorch Updates upgrade to PyTorch 2.4 & deprecation of TorchText by @agunapal in #3289 Resnet152 bat
Read more →