Villa took the lead in the first half as an unmarked Trezeguet got on the end of John McGinn's set-piece to slot home comfortably after a jaded start from the visitors.
Liverpool completed their comeback in injury time as Mane glanced Trent Alexander-Arnold's corner at the near post into the back of the net to give the Reds all three points.
Asked about his emotions after Mane's winning goal, Klopp said: “It’s the best possible [feeling] in football. It’s really satisfying.
“We still have to learn and improve and sometimes you have to get knocks and we got knocks. For little mistakes we got knocks but in the moment we could sort it.
“It’s not that we didn’t deserve to win, I’m fine with the performance but there were moments I didn’t like.
“We made it difficult for ourselves. We started playing football well but not exactly how we should have. Aston Villa were ready for a proper fight and a proper battle.
“We conceded a goal and it’s not so easy to change this wrong path immediately.
“When you watch the game one team deserved to win and it’s us. I can imagine how bad Aston Villa will feel, with all they did they will have deserved something.”
“Managers get sacked for losing football games but I don’t want to make it bigger than it is. I don’t think anyone is doing it on purpose. At the end we need to sort a couple of things,” he added.