IMO vaccine and evolutionary biology is very nuanced, and depends a lot on the individual genetics, type of pathogen, type of vaccine, etc. The net result from people dying off might be moot, and could even be harmful.
Immune science is often taught as an arms race, but that model tends to imply that both sides are constantly gaining beneficial traits. That's true in some cases, like the fever response, which is a beneficial trait we gained at some point, and it continues to be useful.
Meanwhile, other phenotypes are very context dependent for whether they are helpful or harmful. HLA (human leukocyte antigen) for example, that's how our T-cells identify between 'self' and 'foreign' particles. We rely on the tremendous diversity of HLA alleles in the human population in order to survive new diseases. Someone's HLA alleles can be a poor match for a current disease, but very helpful for a future disease. Having them die off now would be a bad thing. Similarly, someone with an HLA combination that makes them more effective against a current disease, may be ineffective against a future disease. Another simpler one is the ABO blood types, where different pathogens (ex. malaria, cholera, smallpox) are better/worse at infecting cells with certain blood types, evidenced by the different proportions of blood types in regions endemic to such diseases.
Evolution is messy, and the evolution of the immune system is messier still. Even if we only look at it from a simplified Darwinian evolution perspective, having genetic diversity might be more important than any shedding of 'weaker' alleles from people dying off because their natural immunity couldn't handle a particular infection.