Sure it was rough, but it wasn't exactly the end of civilization either. And remember that wasn't policies meant to reduce oil consumption, that was just a side effect. What would happen if we actually put a similar amount of effort specifically targeted at reducing oil consumption?
Unlike the pandemic, we wouldn't be avoiding public transit, we wouldn't need to close restaurants and public gatherings. And we wouldn't need to drop it by 20% in a day. Also unemployment wouldn't be a problem if people were working on installing heat pumps, building EVs and solar and wind generation.
Nobody starved to death because of the pandemic, civilization didn't end. It proved that a significant percentage of oil consumption is not required to provide necessities. So we know that we are heading towards a climate change catastrophe (which could end civilization) because we don't want to give up burning oil as a luxury.