Repository History
2 repositories tagged with CUDA
TensorRT-LLM: Optimizing Large Language Model Inference on NVIDIA GPUs
TensorRT-LLM is an open-source library by NVIDIA designed to optimize inference for Large Language Models (LLMs) and Visual Generation models. It offers a user-friendly Python API, state-of-the-art optimizations, and specialized kernels to ensure efficient performance on NVIDIA GPUs. This powerful tool enables developers to deploy LLMs with high throughput and low latency, from single-GPU setups to multi-node deployments.
Numba: A Just-In-Time Compiler for Numerical Python Functions
Numba is an open-source, NumPy-aware optimizing compiler for Python, leveraging the LLVM project to generate machine code. It significantly accelerates numerical functions, offering support for automatic parallelization, GPU-accelerated code, and ufuncs. This tool is essential for Python developers seeking high-performance computing capabilities.