DevOps & Infrastructure
Sets up a shared L2 KV cache for self-hosted LLM inference on Amazon ElastiCache for Valkey, so GPU servers load prefill KV state over EFA RDMA instead of recomputing it. Checks fit, estimates savings with KVTrace, builds a business case, creates the cache, connects vLLM or SGLang (also as NVIDIA Dynamo or llm-d workers) through an LMCache server, tunes for cost or latency, and troubleshoots. Use when a builder serves open-weight models on GPUs (for example H100 or B200, on Amazon EC2 or Amazon EKS) for multi-turn chat, agents, RAG, or long contexts and asks about KV cache offloading, prefix cache sharing, prompt caching for self-hosted models, LMCache with Valkey, or lower TTFT or GPU cost. Not for Bedrock prompt caching, LLM response or semantic caching, or general application caching.
Publisher / source
aws/agent-toolkit-for-aws (found on skills.sh)
Upstream record
Version
Version 1
Published
2026-11-20
Publication review
Approved for publication
Content fingerprint
a2dc62729c8159caa0b5c83635f1f62b249ac03ecc790cc12f4bed60ac8340ad
Review before you use it
Publication approval is a recorded review state, not a guarantee of safe execution. Check the upstream license, required tools and permissions, and test in your intended client.
Sign in to inspect the full instructions and download the SKILL.md file. This public page contains metadata only.
Source: https://prompeteer.ai/skills-hub/406b26cd-7468-490b-81c7-c5b1b1c6ed5c