BEREA, Ohio (AP) Browns general manager Ray Farmer was back in his office Monday after serving a four-game NFL suspension.
''Not to speak for him, but I know he's thrilled to be back,'' coach Mike Pettine said.
Thrilled might be presumptuous. Farmer, who was penalized for sending text messages to the sideline last season, has a pile of work to catch up on.
While he was away, the Browns (1-3) staggered through what's supposed to be the easier portion of their schedule, underwent two quarterback changes, had one of their big free-agent signings fail to catch a pass in four games and watched their defense, which was supposed to be the team's strength, give up too many yards and too many big plays.
But a day after a gut-wrenching 30-27 loss to the San Diego Chargers, Pettine said he has no plans to overhaul a defense ripped by the Chargers for 438 yards, just a week after Oakland rolled up 468.
''One thing we're not going to do is anything wholesale and hit the panic button,'' Pettine said. ''We know we're close, but there are things we have to do better.''
Last week, Pettine bemoaned missed tackles against the Raiders. That wasn't as much of an issue against the Chargers, who drove 57 yards in the final two minutes to set up rookie Josh Lambo's 34-yard field goal as time expired. Lambo got a second chance to make his kick after he missed an attempt but Browns cornerback Tramon Williams was called for offside.
The Browns' defense - the league's most expensive unit - is making mistakes, costly ones, and Pettine said they've got to stop.
''Football is a game of a million little things,'' he said at his Monday news conference. ''Right now, defensively, we're not doing enough of those little things right, and we have to get them crossed off the list and make sure that they don't come back to the list. We have to be that team that if a mistake is made, we learn from it, we lock it away and we don't make it again.''
Pettine said he has no plans to spend more time with the defense and expressed confidence in second-year coordinator Jim O'Neil, whose defense was ranked last in the league against the run in 2014 and is at No. 32 again.
''We know what we need to do and we need to do it better,'' Pettine said. ''I have a lot of faith in Jim O'Neil and the rest of that staff. We've showed that we're very capable of playing well. We just need to do it on a more consistent basis.''
Following Sunday's loss, Williams and linebacker Paul Kruger said the Browns' defense was so predictable that Chargers quarterback Philip Rivers knew what was coming. When the Browns blitzed, Rivers simply threw to the vacated area for big gains.
Pettine attributed some of his players' comments to ''competitiveness.''
''It is hard to disguise everything,'' he said. ''I don't put too much stock into it. I think a player hears the quarterback call something out one time and it becomes `They know what we are doing.' As a staff, we don't get too wrapped up into that.''
As for fixing Cleveland's defense, Williams said players and coaches have to figure out what's working and what isn't. They can't let egos get in the way and he's confident O'Neil can get the job done.
''He's one of the guys open to anything,'' said Williams, who spent nine seasons with Green Bay before signing with the Browns. ''I can go up to Jim and talk to him about anything and vice versa, we do the same. We're in this together. It's no finger-pointing. At the end of the day, as long as we stick together, we'll pull it together.''
Cleveland's defense was not at full strength Sunday as Pro Bowl cornerback Joe Haden sat out with a broken finger sustained against Oakland. Haden's loss was a damaging blow to an underperforming secondary already missing starting nickel back K'Waun Williams.
Pettine said Haden, who practiced all week, was scratched before kickoff.
''We left it up to Joe,'' Pettine said, ''and he made the decision that he didn't want to go.''
NOTES: Pettine did not have an update on DB Tashaun Gipson's injured ankle. ... WR Brian Hartline has a bruised thigh and could be limited ''early in the week.'' ... WR Dwayne Bowe, who has yet to catch a pass after signing a two-year, $12.5 million contract, made a brief appearance in the locker room but left quickly before talking to reporters.
AP NFL website: www.pro32.ap.org and www.twitter.com/AP-NFL