you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 3 months ago* (last edited 3 months ago)

It's pretty much neutral for me also aha, and yeah I meant running up one of several TPUs of several servers of several pods of several clusters of my region's copy of Google Gemini.

I did a bit of research on the scale of GeminiAs of last year, each TPU has 32GB of HBM VRAM; there are 8 TPUs in a host server; 32 hosts in a 'pod'; in a mid-sized Europe hub of which there are several dozen, around 24 pods make up an inference cluster, which does one of image generation, coding etc.; several inference clusters in a regional cluster, and that is Gemini. Each regional cluster/hub around the world is essentially a livesynced copy of a Gemini instance.

The EU hub I researched had, last year, the rough equivalent of 250TB of VRAM and 1.4PB of system RAM.

  • source
  • parent