Embedding table
In plain English
An embedding table stores learned vectors in rows that the model retrieves using input-derived identifiers.
Technical definition
An embedding table is a collection of learned vector parameters indexed by discrete identifiers, such as token IDs or identifiers derived from multi-token patterns.
Engineering details
A lookup selects rows instead of multiplying by the entire table. Engram extends ordinary token embeddings with learned multi-token lookups. The table must be stored somewhere, but sparse access allows its storage tier to differ from that of frequently used dense weights.
Why it matters
Table capacity, fetched bytes, and the cost of retrieving those bytes are separate quantities. A large table need not consume accelerator memory if the serving path can access its rows efficiently from another tier.
How to read it in InferenceX
The article places the same Engram table in HBM, pinned DRAM, or an SSD-backed mapping. Its comparisons measure complete serving configurations, including coordination and transfer costs, rather than treating table size alone as a prediction of throughput.