Prefix Cache Reuse
Where a configuration finds its prompt tokens as concurrency rises: served from the chip’s HBM cache, from the host tier behind it, or recomputed. Each stacked bar is one measured row, read from the runtime’s own cache counters.
Where a configuration finds its prompt tokens as concurrency rises: served from the chip’s HBM cache, from the host tier behind it, or recomputed. Each stacked bar is one measured row, read from the runtime’s own cache counters.