The Reds dominated the ball in the early stages but often lacked the final ball. However, they went into the break 1-0 up after Sadio Mane ran on to Milner's lofted ball and finished expertly five minutes before the break.
Brendan Rodgers' side grew into the game as Liverpool's intensity declined during the second half. They were rewarded ten minutes from time as substitute Ayoze Perez found James Maddison, who fired between Adrian's legs.
It looked like the home side would see their flawless start to the Premier League come to an abrupt end in front of a stunned Kop. However, a tackle from Marc Albrighton brought down Mane in the box, with the referee pointing to the spot.
VAR confirmed the referee's initial decision as Milner stepped up and coolly dispatched the penalty past Kasper Schmeichel to give Liverpool a late 2-1 win.
Jurgen Klopp's side have now won their first eight Premier League games of this season. Whilst that looked in jeopardy at times, once again, it shows Liverpool's ability to adapt to any situation and grind out a positive result.
Slide through our gallery below for all the best photos from Anfield.