Jim McInally’s men arrived in Glasgow on the back of five league wins on the bounce and having scored 17 goals without conceding.
This run had catapulted the Blue Toon up into second place in the league and they put in another spirited display to beat the Light Blues on their home patch.
The early signs looked good for Ally McCoist’s side when Lee McCulloch’s tenacious play led to an early goal but on 23 minutes the game was level when Ross headed home at the back post.
The visitors then went ahead after the break when McAllister hit his 21st goal of the campaign but the Rangers players and McCoist will no doubt be frustrated after a disappointing second-half display.
The manager made two changes to the team that beat Clyde 2-0 seven days ago with Andy Little joining skipper McCulloch in attack and David Templeton shifting out to the right wing.
With Emilson Cribari unavailable because of a groin injury Chris Hegarty returned to the team after a two-game suspension with the Ulsterman starting alongside Seb Faure in the middle of the Gers defence.
It was McCulloch, however, who netted his 26th goal of the season with just 11 minutes on the clock.
The skipper used his power to latch on to a slack back pass before firing the ball into the roof of the net with former Gers keeper Graeme Smith unable to stop him.
The goal was just what the game needed and on 20 minutes Rangers came close to doubling their advantage. Kyle Hutton slipped a great ball in behind the Peterhead backline to McCulloch who took it past Smith but could only hit the post from a tight angle.
This proved costly as, just three minutes later, Peterhead were back on level terms. After a short corner David Cox whipped in a dangerous cross to the back post and centre half Ross was there to power a header in from close range.
The game was now opening up and on 26 minutes Rangers should have regained their lead. Little produced a neat flick to Templeton who, with just the keeper to beat, smacked the post with his shot when he really should have scored.
After a slow start it was turning into an intriguing first half albeit the action was unnecessarily halted when Ian Black, David Cox and Ryan Strachan were booked following a brief altercation in front of the Gers technical area.
McCoist stayed with the same 11 players for the start of the second half but he did change his tactics slightly with Templeton joining McCulloch up top and Little moving to the right wing.
It was Peterhead though that came out of the traps quickly and a superb left-foot volley from Andy Rodgers clipped Alexander’s crossbar with the keeper well beaten.
Sloppy is the only word that can be used to describe Rangers’ play after the break and on 55 minutes they were punished.
McAllister’s excellent first touch took him away from his marker and his accurate finish from 18 yards was his 10th goal in six games.
Kal Naismith was introduced before the hour mark in place of Fraser Aird as the Light Blues tried to get back into the game.
Peterhead, however, continued to push forward with intent and another back post header from Ross from a Strachan corner had to be watched carefully by Alexander as it narrowly missed the target.
On 68 minutes Daniel Stoney went on for Templeton and the 16-year-old almost had an instant impact when he fired a shot over the bar at the Copland Road end of the stadium.
Naismith was also seeing plenty of the ball and the number 12’s dipping free-kick from 25 yards produced a good save from Smith.
At the other end McAllister was a constant threat and with 10 minutes remaining his instinctive left foot volley was inches away from finding the postage stamp corner of the net.
As the final minutes ticked away Peterhead put every man behind the ball to preserve their lead and after Smith palmed a powerful drive from McCulloch over the bar the visitors were able to celebrate their shock win at Euan Norris’ final whistle.
RANGERS: Alexander; Mitchell, Hegarty (Gasparotto 75), Faure, Wallace; Templeton (Stoney 68), Black, Hutton, Aird (Naismith 57); Little, McCulloch
Subs: Smith, Crawford
PETERHEAD: G. Smith, Sharp, Noble, Ross, R. Smith, Cowie (McCann 62), Cox (Gilfillan 62), Redman, McAllister, Rodgers (McGlinchey 79), Strachan
Subs: Jarvie, Fitzgerald
Booked: Cox, Strachan, McAllister
REF: Euan Norris