Pettine said he's not going to hit the panic button after one loss. Cleveland scored just six points against Jacksonville in a 24-6 defeat, and were held scoreless in the second half.
"Brian is still firmly our starter," said Pettine, according to Mary Kay Cabot of the Cleveland Plain Dealer.
Pettine said the Browns briefly considered putting in rookie Johnny Manziel against the Jaguars, but ultimately decided against it.
"It was discussed briefly, but it was a situation where we still wanted to...at least try to end the game with Brian on a positive note,'' he said.
The Browns are 3-3 this season, and had won two straight before Sunday's loss. Hoyer recently led Cleveland to the biggest comeback win in NFL history, overcoming a 25-point deficit to defeat the Titans on Oct. 5.
During training camp, Hoyer won the starting quarterback job over Manziel. Manziel has briefly appeared in two games, throwing one pass, and Pettine said the team decides each week whether to run a package of plays for the Heisman Trophy winner.
Hoyer has completed 55.8 percent of his passes in 2014, throwing for 1,439 yards, seven touchdowns and two interceptions. He leads the league in yards per pass completion.
Cleveland hosts the Oakland Raiders (0-6) on Sunday.
- Molly Geary