The Reds were hardly in the game during the first half and trailed 2-1 and although the arrival of captain Steven Gerrard just before the hour-mark changed the momentum he could not produce another miraculous recovery as they went down 3-2.
"We are bitterly disappointed. We started the first half poorly and we started the second half and got punished," he said. "I was disappointed with the young players today I've got to be honest."
He added: "We are trying to give them experience and let them see what it was like to play for Liverpool.
"It is not just about playing for Liverpool, you have to be competitive and be part of a group that is going to challenge for trophies.
"I thought how we reacted to that and how we performed, knowing that is what we are looking for, was disappointing.
"It is not enough to play for Liverpool and get up for the Arsenal and Manchester City games (later this week) because if we are going to win trophies you need to be able to come to places like Oldham and win and we weren't good enough for that."
Latics boss Paul Dickov hailed his side's performance as two goals from Matt Smith and a further striker from Reece Wabara set up a fifth-round tie with Everton.
He said: "I'm proud of the players and the performance they put in. To put that in against a team of world-class players was fantastic.
"I know what they are capable of. I just thought the desire they had to win the game was the reason we won it. I've won a European medal, I was involved in FA Cup finals, played for my country, scored at Wembley but after everything that has gone on recently that was right up there at the top."