Paul Dickov admitted his 10-man Oldham side were poor in the final third in their 1-0 loss to Charlton, who had two men sent off.
Charlton's Danny Hollands was shown a straight red card for his late tackle on Kieran Lee and 11 minutes later, referee Trevor Kettle dismissed Jean Yves M'voto for a second booking. Charlton winger Scott Wagstaff became the third sending off, booked for a second time in the 76th minute.
Oldham struggled to create chances and are still not clear of relegation worries after eight defeats in 10 games.
Dickov said: "We should be a lot higher up the table, but this is the time when you find out the character of your team.
"It's about the players rolling their sleeves up and getting the job done - if they don't they won't be here next season.
"We are on a dodgy run and we need to show more guile and craft in the final third of the pitch. There's no lack of effort or application, but you could see why Charlton are top and we are in our position."