It was vice-captain Lee Wallace that won the crucial spot kick when he was brought down in the box by Black and Whites stopper Sean Crighton after a powerful run.
But McCulloch showed his leadership qualities by stepping up when his side needed it most and his composed finish sent keeper Joe Malin the wrong way.
At times it looked likely this hard-fought battle would finish goalless as both teams worked hard on a tricky surface but could not convert the few goalscoring chances created.
But with 20 minutes to go the Light Blues made the breakthrough and despite Ian Black’s sending off late on McCulloch’s strike takes Rangers one step closer to winning the Third Division title.
With Kyle Hutton suspended, David Templeton still recovering from an Achilles injury and Dean Shiels expected to be out for the season Ally McCoist made considerable changes to his team for the 90 minutes against Ross Jack’s side.
McCulloch moved back to the centre of defence alongside Emilson Cribari with Seb Faure, Robbie Crawford and Fraser Aird in midfield and Fran Sandaza up top as the lone striker in a 4-5-1 formation.
Having suffered the huge disappointment of a 2-1 to Annan Athletic at home Rangers looked to start this match positively and they certainly did that.
The league leaders, wearing an all-blue kit and black armbands for Irene Love, looked hungry and determined, knowing full well they would have to match the battling qualities their Moray opponents would display.
After just four minutes a great cross from Lee Wallace found Sandaza but his glancing header was acrobatically saved by Malin.
With the early stages of the match being played in Elgin’s half Crawford then tried his luck with a powerful shot which went wide with the keeper scrambling across his line.
Sandaza looked up for it and he showed great skill to beat his man and get into the 18-yard box. His cutback to Little was intelligent but the in-form Ulsterman could not find the net from close range.
It was proving to be another intriguing contest with both teams giving 100 per cent but on 26 minutes Elgin’s towering forward Paul Millar, believed to be the tallest player in Scottish football, showed too much enthusiasm when tackling Wallace and his name was the first to be scribbled into referee David Somers’ notebook.
With nine minutes of the first half remaining Elgin were given the chance to test Neil Alexander from a free-kick situation after Cribari had climbed on to Millar’s back when challenging for a header. Daniel Moore stepped up and his left foot curler beat the wall and narrowly missed the postage stamp corner to the disappointment of the home fans.
Elgin were now enjoying their best spell in the first half but on 42 minutes Black was left in pain on the turf when Mark Nicolson caught him on the face with his right arm as the pair battled for the ball.
In the end neither team could get their noses in front at the break so it was all to play for in the second period.
With no changes in terms of personnel for the restart Ian Durrant could be seen urging his players to push forward and five minutes in Crawford stung Malin’s palms with a powerful drive which was straight at the keeper.
On 53 minutes a challenge from Black resulted in the ref pulling out his yellow card for the second time and the decision looked harsh with Elgin’s Stuart Leslie making the most of the challenge to win a free-kick and ease the pressure off his defence.
With the rain now lashing down the bobbly Borough Briggs playing surface was getting greasy and before the hour mark a shot from Black from 25 yards skidded off the grass but was well held by Malin.
On 65 minutes, and with Barrie McKay getting ready to replace Sandaza, Little whipped a great cross in from the right but with the Spaniard preparing to head the ball in Jamie Duff managed to get a toe on it.
Crighton’s late challenge on Wallace, however, resulted in the ref rightly pointing to the spot and McCulloch stepped up and sent Malin the wrong way with a composed finish to give his side a narrow lead at a crucial stage of the game.
With four minutes remaining Black saw red for a second bookable offence on substitute Gavin Morrison, a decision which enraged boss Ally McCoist.
Nevertheless Rangers held on to their advantage until the final whistle and it was a deserved victory given the hard work the players put in to secure three hugely valuable points.
ELGIN CITY: Malin; Niven, Duff, Crighton, McMullen (Beveridge 90); Gunn, Nicolson, Cameron (Morrison 84), Leslie, Moore; Millar (Harkins 76)
SUBS: McLean, Black
BOOKED: Millar, Crighton, Gunn
RANGERS: Alexander; Hegarty, McCulloch, Cribari, Wallace; Little (Argyriou 90), Black, Faure, Crawford, Aird; Sandaza (McKay 68)
SUBS: Gallacher, Naismith, Hemmings
BOOKED: Black, Aird, Alexander
SENT OFF: Black
REF: David Somers