Our take
Activeregular commits and releases- Activity
- 59
- Community
- 69
- Issues
- 21
- Pull requests
- 58
- Merges more pull requests than 82% of the projects we track
Tair KVCache is an actively developed infrastructure project with a substantial contributor base, but its components have different adoption scopes and HiSim’s documented support is specific.
Good fit if
- You are building LLM inference infrastructure and need centralized KV-cache metadata, capacity controls, or storage integration.
- Your engine is among the listed connector targets, or your simulation workload matches HiSim’s documented SGLang and Qwen3 Dense setup.
- You can evaluate a C++ project with CI and recent releases, and are prepared to check the available test coverage yourself.
Look elsewhere if
- You need a general-purpose inference simulator with documented support across engines, models, or hardware beyond HiSim’s stated scope.
- You need a ready-to-use end-user cache product rather than components to integrate into inference infrastructure.
- Your deployment depends on a specific storage backend or engine that is not covered by the repository’s stated integrations.
All health signals
| Last commit | 2026-10-28 (today) |
|---|---|
| Commits, last 90 days | 95 |
| Releases, last 12 months | 2 (latest binary-dependency-0.0.2, 2026-02-12, pre-release) |
| Contributors | 24 (top contributor: 30% of commits) |
| Issues closed, last 90 days | 0+ |
| Pull requests merged, last 90 days | 70 (typically merged in 1 day) |
| Project age | 10 months |
Checked on 2026-10-28 with the GitHub API.
Overview
Alibaba Cloud Tair KVCache is a high-performance KVCache system for large language model inference. Its open-source components address two needs: managing KV-cache metadata and storage across inference engines, and using HiSim, a CPU-based simulator, to estimate inference performance from workload traces without GPU resources.
The Manager and its client/connectors are intended for inference infrastructure teams integrating engines such as vLLM, SGLang, RTP-LLM, and TRT-LLM with storage backends. HiSim currently focuses on SGLang v0.5.6.post2 with Qwen3 Dense models on H20 GPUs, so its stated simulation scope is narrower than the broader Manager integrations.
Key Features
- Centralized management of KV-cache metadata, queries, and storage capacity.
- Cache matching options include prefix, sliding-window, and KV matching.
- Two-phase writes and metadata persistence support reliable cache operations.
- Storage abstraction supports backends including HF3FS, Mooncake, and NFS.
- Capacity controls and background eviction manage quotas and backend water levels.
- Client/connectors support KV-cache transfer for vLLM, SGLang, RTP-LLM, and TRT-LLM.
- HiSim replays inference traces to estimate metrics such as TTFT, TPOT, throughput, cache hit rate, and capacity consumption.
Use Cases
- Inference platform teams that need shared KV-cache metadata and capacity management across engines and storage backends.
- Teams using supported inference engines that want to connect engine-side workloads to KV-cache storage through the project’s client/connectors.
- Infrastructure planners who want to replay workload traces with HiSim before committing GPU resources, within its documented model and engine scope.
- Operators tuning cache capacity or eviction settings who want to compare simulated cache behavior and resource use.
What you need
Detected in the repository
- C++ (main language on GitHub)
- Automated checks on GitHub Actions
License in plain words
Apache-2.0permissive
- Commercial use: yes
- Modify and redistribute: yes
- You must keep: the license, the NOTICE file and a note of your changes
- Share your changes: no
- Includes an explicit patent grant from the contributors:
A summary, not legal advice: the LICENSE file is what applies.
Getting Started
Start with the repository README to choose between the Manager and HiSim and follow their component-specific setup instructions. For HiSim details, see the HiSim README.
Considerations
The repository is active, with regular commits and releases over the past year, and development is spread across multiple contributors rather than concentrated in one person. However, the supplied project data does not indicate a detected test suite, so teams should validate the components and workflows they intend to deploy. HiSim’s documented configuration is specific to SGLang v0.5.6.post2, Qwen3 Dense models, and H20 GPUs; confirm that this scope matches your workload before relying on its estimates. The Manager’s usefulness also depends on integrating its clients and storage abstractions with your chosen inference and storage systems.
Found this useful?
Share it with someone who would like tair-kvcache.