Wong scored on Yadier Molina's sacrifice fly just beyond the infield to tie it in the fourth. He bunted for a hit and drew a wild throw in St. Louis' go-ahead, two-run rally off Aaron Harang (4-4) in the sixth.
The Cardinals matched their season high with their fourth straight win.
Miller (6-2) allowed a run and five hits in seven innings, one inning longer than his previous high this season. He matched his season best with seven strikeouts, fanning B.J. Upton three times.
Trevor Rosenthal fanned two in the ninth inning, with Upton striking out a fourth time, to earn his 13th save in 14 chances.
Braves manager Fredi Gonzalez was ejected in the fifth inning for arguing a call that led to a bunt double play. He could face a fine for charging out of the dugout a second time.
Andrelton Simmons had three hits for Atlanta, which lost its third in a row and scored fewer than three runs for the 19th time in 41 games this season.
Cardinals center fielder Peter Bourjos bunted twice for hits, the second time squeezing home a run in the seventh to make it 4-1.
Miller has won six straight decisions in seven starts after losing his first two of the season. The last three St. Louis starters have worked seven innings.
In the fifth, Harang stayed in the batter's box after his attempt to sacrifice Simmons to third didn't go far. Molina pounced on the ball and threw to third, with shortstop Jhonny Peralta covering, for a tag play. Peralta had an easy relay to first to finish the double play.
Gonzalez vigorously protested home plate umpire Ron Kulpa's ruling of a fair ball and was ejected. Seconds after leaving the field, Gonzalez bounded out of the dugout again and was headed off by first base umpire Eric Cooper, hustling to the plate.
Molina's sacrifice fly was a pop-up just outside of the infield. Second baseman Tyler Pastornicky backpedaled to make the catch, and Wong raced home ahead of the relay.
The Braves capitalized on shoddy defense to take the early lead.
Simmons scored from first on Harang's two-out single - the pitcher's first hit of the year. Harang took third after left fielder Matt Holliday threw behind him, then trotted home after Wong's wild throw from second.
NOTES: Cardinals lefty Jaime Garcia will make his first start since injuring his shoulder last May in the series finale, with Gavin Floyd (0-1, 2.70) starting for Atlanta. ... The attendance of 44,981 was the Cardinals' eighth sellout. ... B.J. Upton made a nice running shoe-top catch on Allen Craig's sinking fly ball to start the second. ... The Braves are 1-4 against the Cardinals and 8-1 against the rest of the NL Central.