Boro Display Deserves Credit!
And Herita Ilunga's goal, with just seven minutes left on the clock, made Southgate's side pay the ultimate price.
Southgate told the BBC: "I was pleased with how we played, the quality was good and I was pleased how we opened them up. We will take more from it than the result itself."
The only side to beat West Ham in recent months is Manchester United, which shows the sort of form the Hammers have produced under their new manager.
WEST HAM'S RECENT FORM:
West Ham 0 Man Utd 1
Arsenal 0 West Ham 0
West Ham 2 Hull 0
(FA Cup) Hartlepool 0 West Ham 2
West Ham 3 Fulham 1
Newcastle 2 West Ham 2
(FA Cup) West Ham 3 Barnsley 0
West Ham 2 Stoke 1
Portsmouth 1 West Ham 4
FA CUP RESULTS:
Watford 1 Chelsea 3
Sheffield United 1 Hull 1
Blackburn 2 Coventry 2
West Ham 1 Middlesbrough 1
Swansea 1 Fulham 1
WEST HAM UTD 1 MIDDLESBROUGH 1
WEST HAM: Green, Neill, Collins, Upson, Ilunga, Collison, Parker, Noble (Nsereko 74), Boa Morte (Sears 49), Di Michele, Cole (Tristan 33).
Subs Not Used: Lastuvka, Lopez, Kovac, Spector.
BORO: Jones, Hoyte, Huth, Wheater, Pogatetz, Adam Johnson, O'Neil, Digard, Arca (Walker 77), Downing, Alves (Sanli 74).
Subs Not Used: Turnbull, Emnes, Bates, McMahon, Bennett.
Att: 33,658
West Ham full-back Herita Ilunga headed a late equaliser against Middlesbrough at Upton Park to earn a replay from their FA Cup fifth-round clash.
Boro had led through Stewart Downing's first goal of the season and Gareth Southgate's men looked good value for their lead until Ilunga struck.
The Congo defender has a taste for the cup - both his goals this campaign have come in the competition and it keeps alive Gianfranco Zola's hopes of winning the trophy as a manager.
Zola had FA Cup pedigree as a player, winning it twice with Chelsea and beating Boro at Wembley both times, including getting the better of Southgate in the 2000 final.
The Hammers also went into today's clash in good form - their defeat against Manchester United last Sunday came after eight unbeaten matches - but it was the visitors who took the initiative in the early stages.
Boro boss Southgate was forced into a change of formation due to Marlon King being cup-tied, meaning Downing was employed in a central position in midfield.
It was Downing who helped create the first chance, after Jack Collison lost the ball.
Downing raced down the left flank, his cross was cleared as far as Adam Johnson and the midfielder dragged his shot wide.
Another mistake, this time from David Di Michele, led to another Boro chance after 11 minutes. Didier Digard cut inside from the left and offloaded to Downing, whose shot from 20 yards went just past Robert Green's upright.
The Hammers were hoping for a repeat of the 2006 season when they beat Boro on the way to the final - and before falling behind they applied pressure.
Collison wriggled free on the edge of the area and laid the ball off, but Brad Jones was alert to the near-post shot from Mark Noble.
Di Michele failed to connect with a scissor-kick, James Collins forced a save with a long-range drive and a handball shout against David Wheater was turned down.
Despite the encouraging attacks, it was Boro who took the lead in the 22nd minute.
Gary O'Neil picked the ball up on the right flank and curled in a dangerous ball that Matthew Upson missed and Collins managed to allow to bounce through his legs, leaving Downing to stoop down and head home.
O'Neil's right boot almost created a second goal but Robert Huth could not direct his header.
West Ham attempted to reply and a dipping volley from Di Michele was just off target - but they received another blow when in-form England striker Carlton Cole was forced off with an ankle injury. Diego Tristan came on.
Boro goalkeeper Jones kept his side on level terms with a save in the 38th minute.
Luis Boa Morte had driven into the penalty area and Noble got a shot away, with Jones swatting away after being wrongfooted by a deflection. Collison then blasted the rebound over the crossbar.
Di Michele got behind the Boro defence and squared the ball to Tristan - but the finish was ugly.
Zola waited three minutes into the second half before bringing on Freddie Sears and taking off Boa Morte, with the former Fulham winger getting jeered by his own supporters.
Sears had a shot from 25 yards that flew over the crossbar as West Ham attempted to find a way back into the match with a three-man strikeforce.
Downing released Afonso Alves on the hour mark and the Brazilian's angled drive went wide of Green's post. Green was also down to parry an Alves effort shortly after.
Downing could have added another but Green stopped a long-range drive hit with the England man's unfavoured right foot.
Di Michele missed a glorious chance when Ilunga crossed and the Italian skied the finish.
The leveller came after Savio Nsereko, on as a substitute, swung over a free-kick with seven minutes left and Ilunga headed in at the far post after Collins flicked on.