It's at the high end of what you might expect in the US, but it's not far outside the range. This page estimates that a typical visit costs between $150 and $450. But, the cost of an ER visit without insurance is in the multiple thousands.

What you did if you went to see a random doctor in Sweden is probably more similar to an ER visit, not because it's an emergency, but because it it's not the easy case of someone just visiting their family doctor. A doctor in the US is probably used to seeing uninsured patients and has a standard way of billing them. But, a doctor in Sweden is probably not used to dealing with patients who aren't covered under the Swedish system (or under an EU country's system that has a reciprocal agreement with Sweden). If you lived there long-term, you'd be integrated into their system. So, I assume you weren't seeing your regular doctor, you were seeing a doctor who had to handle a special case of an American without travel insurance. So, they have to figure out how to do all the required paperwork, and pay all the people who need to get paid when a doctor sees a patient, but without the standard procedures. So, you pay extra to deal with that one-off paperwork.

So, the expense isn't because healthcare in Sweden is expensive in general. It's expensive for you because dealing with uninsured Americans is not something they normally do, and it's a pain in the ass, so it has a high price.