I wonder if you need to explicitly prompt it to check if a function really exists before suggesting it? Think about how a human brain works... we are constantly evaluating whether or not things are really true based on info in our heads... but we are not telling the models to do the same thing and instead they just yolo some shit that is confidently-wrong (not unlike many humans, admittedly).