Jimmy Floyd Hasselbaink looked to have set Steve McClaren's side on their way when he netted early in the second half.
But Stern John equalised in the 55th minute and Coventry created a string of chances but failed to find the vital goal.
Coventry started positively enough when Gary McSheffrey managed to force Mark Schwarzer into a diving save in the third minute when he let fly from 20 yards.
Middlesbrough, however, exerted the early pressure but with Micky Adams men sitting deep, they were unable to get behind the Coventry defence and really hurt them.
Jimmy Floyd Hasselbaink did expose the home rearguard when McSheffrey let a pass go astray, but the Dutchman's shot lacked control and flew over the top.
Don Hutchison, in for the suspended Dennis Wise, was showing his quality and Coventry looked far more impressive whenever they played through the veteran.
Boro were also finding Dele Adebola a handful and he caused problems midway through the first half when he moved powerfully across the face of the Middlesbrough area.
Hutchison teed up the best chance of the half with a cross which fell nicely for John, but he failed to get a shot on target.
The Championship side ended the half confidently but a crucial slip 40 seconds into the second half gift-wrapped a goal for Boro.
Marcus Hall opted not to pass the ball back to keeper Marton Fulop, but his clearance stuck Yakubu and fell for Hasselbaink who coolly slotted it home.
It needed an immediate response and it came 11 minutes later courtesy of John.
McSheffrey kept chasing a long pass from Michael Doyle and managed to keep it in by hoisting the ball into the area.
John was the first to react, brought the ball under control and lashed home a stunning volley.
Suddenly it was Middlesbrough who were rocking and Coventry pinned them back for long periods.
McSheffrey, operating down the Coventry left was causing all sorts of problems. He set up James Scowcroft with a cross from the left but his stabbed effort flew over the top.
Just minutes later McSheffrey skipped past two defenders only for his driven centre to fly across the face of goal and out of danger.
It looked Coventry's best chance of scoring but eventually their storm blew itself out and it was the Premiership side who ended the stronger of the two.
Doriva forced Fulop into action with a weak shot, but Ady Williams and Richard Shaw held firm.