If they were just mashed potatoes, they were probably ok to leave out. However, the issue is usually what was mixed into the mashed potatoes milk & butter can be a problem with dairy more so than olive oil or margarine.

Here are the general guidelines:

Once cooked and left out at room temperature holding hot food needs to be 135°F or above and holding cold food should be at 41°F or below.

Is A Boiled Potato Left Out All Night Safe For Consumption?

Also from a web site on food safety called stilltasty.com “Big offenders include rice salads, pasta salads and that bowl of mashed potatoes, if it's not put into the fridge a couple of hours after serving it. These starchy dishes are breeding grounds for Bacillus Cereus, a microbe that can cause severe vomiting or disabling diarrhea.”

All Things Mashed Potatoes HERE