Did a quick search, in higher doses stevia may cause mutagenic effects and regulatory bodies initially classified it as safe over the objections of some researchers who say it needs more studies on this effect. Stevia also lowers blood pressure and can cause light headedness and fainting in larger doses. The effect on blood pressure is potentially fatal in small children and people with certain health complications.

In less refined forms stevia also carries a high risk of allergic reactions because it's a form of ragweed but the stuff used as a dietary sweetener is highly refined so that shouldn't effect the recommended limit.