Jagielka powered forward late on and after latching onto a clearance by the flustered Boro rearguard, let fly from fully 30 yards to smash home the sweetest of strikes.
It was the Blades' first top-flight victory since sinking another North East side in Newcastle way back in April 1994, and was greeted with ecstatic scenes of celebration from the home fans.
Blades manager Neil Warnock recalled Paddy Kenny, Leigh Bromby, Alan Quinn and Michael Tonge for the crunch encounter, and fielded a 4-5-1 system aimed at stifling Boro.
Visiting boss Gareth Southgate, who has asked for fans to be patient after his side's disappointing start, brought back Stewart Downing, Mark Viduka and Lee Cattermole to his starting line-up, and it was the visitors who made the more assured start.
The Teessiders dominated possession early on with Fabio Rochemback warming the palms of Kenny with a seventh-minute free-kick.
Cattermole then robbed Tonge close to the byline, but saw his dangerous cross cleared by Claude Davis before the visitors were left to kick themselves after missing two glorious chances to take the lead in the space of 60 seconds.
The first culprit was Yakubu, who was put clear after slick play involving Viduka and James Morrison, but after rounding Kenny, he fired his shot against the post.
Unbelievably, Boro spurned an equally golden chance moments later when skipper Jonathan Woodgate, in splendid isolation in the box, side-footed a volley wide with the goal at his mercy from eight yards out following Downing's cross.
Barely able to believe their luck, United made the all important breakthrough in the 35th minute and it was top scorer Rob Hulse who was on the mark again.
The striker latched onto Colin Kazim-Richards' pass after Boro were caught cold with men upfield following a corner, and he coolly held his nerve to slot the ball past the advancing Mark Schwarzer.
Boosted in confidence, Hulse and Kazim-Richards went close before the break, but the Blades were rocked on their heels when Boro levelled four minutes into the second half.
Emanuel Pogatetz cleared a punt forward by Davis into the path of unmarked Yakubu and the striker raced clear before drilling home a crisp shot past Kenny - ending a team goal drought of over six hours.
Undeterred, the hosts pushed forward in a bid to break their Premiership duck, with Keith Gillespie seeing his stinging drive beaten away by Schwarzer.
Boro sent on Gaizka Mendieta in a bid to inspire a victory charge 18 minutes from time - and he almost set up Viduka, but Chris Armstrong saved the day with an excellent challenge.
Despite plenty of possession, Boro's final ball let them down in the second period and they were duly punished in stoppage-time when Jagielka's screamer flew past the bemused Schwarzer to extend Boro's winless streak to six matches.