Food grade nitrous oxide from the smoke shop is safe, aerosols from the grocery store are not. Always use it sitting down, and don't do stupid shit like put a bag over your head to make the high last longer and you'll be fine. Even if you pass out, you will eventually come to so long as your airway remains unobstructed.
The problem with nitrous is that it's mentally addictive, and tolerance is permanent. Knowing this, I tried to be careful. I have used it maybe 20 times in the past 20 years and I still had to quit forever because it completely lost its mystique. In the beginning it was great for going on a quick 5 minute spiritual journey, but now I need real psychedelics for that.
So if you do decide to get into it, don't be an idiot, use it once every 3-5 years* tops, and enjoy it while it lasts. Then quit when it stops being trippy. Also, don't waste your time with the individual crackers. Just get the big $25 bottle + a pressure regulator and it'll be a lot less work for you.