Alex Oxlade-Chamberlain cut the deficit for Liverpool after a miserable start saw Jamie Vardy bag a brace in front of the Kop, but a James Maddison strike from distance gave the Foxes a 3-1 lead at the break.
Liverpool upped the ante and dominated the second half as substitute Diogo Jota halved the visitors' advantage with 20 minutes left before Takumi Minamino excellently finished in stoppage time to make it 3-3 and force the game to penalties.
Caoimhin Kelleher saved twice from the spot, including in sudden-death after Minamino had squandered a chance to seal the game, with Jota stepping up for the Reds and converting to send Liverpool into the semi-finals, where they will meet Arsenal.
Jurgen Klopp named a much-rotated side, as expected, with ten changes to the starting eleven from the weekend. Billy Koumetio made his competitive debut alongside Joe Gomez, whilst skipper Jordan Henderson returned to partner Tyler Morton and Alex Oxlade-Chamberlain in midfield.
Roberto Firmino made his first start since returning from injury as he spearheaded an attack consisting of Takumi Minamino and usual right-back Neco Williams. The likes of Diogo Jota and Naby Keita were named on the bench.
The visitors had the first opportunity of the game as a misplaced pass by Gomez allowed Leicester to free Patson Daka, who burst into the box and sent a powerful strike at goal, but Caoimhin Kelleher was on hand to turn it behind.
It was enough encouragement for the Foxes as they pressed ahead in search of an opener, which was duly provided by Jamie Vardy, who ran on to James Maddison's pass and powered a low finish across Kelleher and into the back of the net to make it 1-0.
Things went from bad to worse just a few minutes later as the visitors remained in the ascendancy and soon doubled their advantage. Kiernan Dewsbury-Hall slipped through Daka, who showcased his pace and squared the ball for Vardy to tap home his second of the game with less than 13 minutes on the clock.
But the Reds pulled one back soon after as Conor Bradley headed the ball down for Firmino inside the box, with the Brazilian turning away from goal and feeding Oxlade-Chamberlain, who picked the bottom-left corner with a well-placed finish to make it 2-1.
Liverpool looked to build ahead of steam in search of an equaliser, but Leicester undoubtedly looked sharper of the two sides and they made it 3-1 in stunning fashion through a strike from Maddison, who unleashed a powerful effort into the back of the net from the best part of 25 yards out to regain the Foxes' two-goal advantage.
Klopp's side were unable to get going and could have found themselves further behind as an error by Gomez allowed Vardy through on goal before his shot beat Kelleher in goal, but he was denied a first-half hat-trick by the woodwork, much to the Reds' relief.
Dewsbury-Hall could have extended Leicester's lead after he got in behind the Reds defence and cut inside before getting a low shot at goal, but Kelleher was up to the task as he got down well and turned the ball behind.
Caglar Soyuncu was forced into making a crucial intervention to deny Firmino from close range after the Brazilian was found by Minamino's square pass. Liverpool were beginning to gain momentum and Henderson soon made way for Keita, with the boss looking to go all out in search of a second goal.
Liverpool finally broke Leicester's resilience with a little over 20 minutes left on the clock as Minamino slid through Jota, with the Portuguese forward taking the ball into his stride and finishing across Kasper Schmeichel to make it 3-2.
Owen Beck replaced Tsimikas as the Reds continued to push for an equaliser in the final ten minutes, with Jota redirecting Keita's long-range strike towards the top corner, but Schmeichel was on hand to put the ball behind.
Beck should likely have done better after a well-worked move saw Keita release the left-back just inside the box, but the youngster wasn't up to the speed of the ball as he found himself smothered by the Foxes defence.
But Liverpool finally completed their comeback as Milner's long ball into the box was missed by the Leicester defender before being controlled by Minamino, who kept his cool and finished brilliantly to take the game to penalties in stoppage time.
Youri Tielemans, Maddison and Marc Albrighton slotted in the visitors' first three penalties whilst Milner, Firmino, Oxlade-Chamberlain returned the favour for the Reds, before Leicester blinked first as Luke Thomas's spot-kick was saved by Kelleher.
Keita stepped up to make it four out of four before Kelechi Iheanacho converted from the spot, with Takumi Minamino given the responsibility to fire the Reds into the semi-finals with the final kick.
However, Minamino hit the crossbar and fired over before Kelleher was up to the task again to deny Ryan Bertrand with the first kick of sudden death. Jota stepped up for the Reds and made no mistake as he slotted the ball home to book Liverpool's place in the semi-finals.
Liverpool: Kelleher, Bradley, Koumetio, Gomez, Tsimikas, Morton, Henderson, Oxlade-Chamberlain, Williams, Minamino, Firmino
Leicester: Schmeichel, Ricardo, Ndidi, Soyuncu, Thomas, Soumare, Dewsbury-Hall, Tielemans, Maddison, Daka, Vardy
MOTM: Alex Oxlade-Chamberlain