It's less about maximizing in volume in the way you seem to be thinking.
20 tomato plants planted too close together are going to produce X tomatoes of shit quality.
10 tomato plants planted too close together will produce X tomatoes of a slightly better quality.
6 tomato plants at this grouping would produce potentially normal quality fruits.
I'm almost positive those pepper plants won't produce anything.