llm's are random text generators. each one is a different architecture, different training data, or different training recipe.

first off: you probably dont need to fill up your harddrive with every good model in existance. often a url link is sufficient.(unless the owner deletes it. but quants are often still there)

optional second questions: how many links do you have to decent models(excluding low value experiments) how many would you estimate that you have?

optional third question: how much is your divirsity of model usage? assuming you had the compute, how many tested models would you utilize for a single query across different models? average. max.

fourth question: do you still rely on inference compute for non-locally hosted models?