Ant Group and vLLM Jointly Open Source Humming Quantitative Inference Kernel

Ant Group and the vLLM community have jointly open-sourced Humming, a high-performance quantitative inference operator library. Humming supports a wide range of devices, including various GPUs and devices compatible with CUDA programming. The library offers rich quantization support, enabling a combination of multiple weight and activation formats in Dense and MoE through a unified architecture, covering various integer and floating-point types, bit widths, and scaling methods. Humming significantly outperforms existing implementations, such as vLLM Marlin, FlashInfer, and DeepGEMM, in various typical workloads.

This content is provided for general informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online promotions, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets. Crypto assets are highly volatile and may result in loss. The availability of WEEX services, products, and related events may vary by region. You are responsible for ensuring that your participation is in accordance with applicable local laws and regulations.