No, it is about their presence being wanted or not. I don't want to go to a swamp because is infested with mosquitoes for example, mosquitoes were always there.
https://dictionary.cambridge.org/dictionary/english/infestation
https://dictionary.cambridge.org/dictionary/english/infest