NCM models generate an entire answer in parallel rather than sequentially by storing words in collections of layers. At the topmost layer, there is a very rough outline of a concept, which the researchers call codes. In each subsequent layer, the concept gets progressively refined, and only the refinements are stored in that layer as additional codes. So, every subsequent layer has more codes than the preceding one, continuing until the very last layer where every code represents an output token.

To get an answer for a given prompt, the NCM model is inferenced once per layer with each one representing a parallel prediction of the entire output at a given level of fidelity. As the fidelity progressively increases from layer to layer, the final pass in the bottom layer gives you output text of the highest fidelity as the answer to the prompt. In this way, the model is inferenced the same number of times as the number of layers that the model contains, rather than once per output token.

The trade-off is that evaluating those codes requires more processing power because there are more codes than there are output tokens at the bottom layer. So, NCM architecture makes a trade reducing the amount of memory bandwidth needed at the cost of more processing power.