Down
enlarge
Super Bowl 100: Virtual Reality is already changing football
5:45 | NFL
Super Bowl 100: Virtual Reality is already changing football
Wednesday February 3rd, 2016

SAN FRANCISCO — It should come as no surprise that Bill Walsh saw how technology would affect the game he loved the most. Walsh was a consultant and speaker for Silicon Valley firms after his historic tenure as the 49ers’ head coach came to an end in 1989, and in 2001, he spoke with Bain & Company, a leading management consulting firm, to get advice on how to better scale the NFL draft. At that time, Walsh was the 49ers' Vice President and General Manager, and out of those talks with Bain came the hire of Paraag Marathe as the team's COO. It was a high-tech, Moneyball move. Long before computers were common, Walsh sought to run his team with a technological level of perfection. He often referred to the offenses he developed as a “machine,” and at their best, they certainly seemed like that.

In truth, football and computers have had an alliance for decades, but it's only recently that the partnership has been obvious. Today, the league strives to be as technically conversant as possible, but the start of this story was far more humble.  

• The coming revolution in football strategy and game-planning 

The Past: “I bow to the machine”

The Los Angeles Rams of the late 1950s had a pretty hefty front office: There was a PR guy named Pete Rozelle, there was general manager Tex Schramm, and there was a scout by the name of Gil Brandt. When Rozelle moved on to become NFL Commissioner in 1960, Schramm and Brandt took over the league’s newest team, the Dallas Cowboys. It was tough sledding at first, as it is for most expansion teams—Dallas posted an 0–11–1 mark in that first year and didn't have a winning season until 1966 (they finished 7–7 in ’65), but wouldn't have another losing one until 20 years later, in ’86. Schramm and Brandt left the franchise when Jerry Jones bought the team in 1989, but their legacy was secure: 18 playoff seasons, two Super Bowl wins, and an enormous roster of Hall-of-Famers from Bob Lilly to Roger Staubach to Tony Dorsett. How did they do it? Among other things, it was the franchise’s belief in the power of data that put them ahead of the pack for a very long time. And it all started when Schramm did some TV work in 1960.

Doug Farrar/Sports Illustrated

“Tex was finishing his work at CBS, and they were putting the Winter Olympics on at Squaw Valley,” Brandt recently told me. “IBM put a chip in a ski to tell you how fast a skier was going downhill. So, when Tex came back in March, he said, ‘We're going to try and do something with this.’ We went to IBM, and they sent us to a place called Service Bureau. We started from scratch, and I was laughing when we were going down to Santa Clara the other day, saying, 'I've been down this street so many times,' because we owned a computer company there in Palo Alto.”

As Brandt remembers, the part that was the toughest—“starting from scratch”—was the definition of a system that would accurately combine the most important characteristics for every position. He compared the system to the ways in which certain restaurant chains pick out ideal locations based on traffic, so the question was, how were the Cowboys going to decide what the ideal traffic counter was for this? Schramm and Brandt asked fifty college coaches to describe the ideal football player—that was the start. The Cowboys' brain trust extrapolated the ideal characteristics from those lists based on what the coaches deemed essential for every player, regardless of position.

NFL
OK Computer: How computers will—and won't—change the NFL

Then, they combined those characteristics into bundles. For example, quickness, agility and balance were three separate things, and the Cowboys combined them into one stat. Strength and explosiveness were two, and the Cowboys combined them into one. And so on. After Schram and Brandt did that, it was time to come up with a grading scale. Not only for athletes, but for scouts and coaches and executives. Brandt was smart enough to see early on that his team would have to alter and adjust the curve for the particular biases of every evaluator. Take the guy who grades everyone too high, and the guy who grades everyone too low, and bend their evaluations to the mean. A DVOA for scouts, if you will.

The grading scale started at 3 and went up from there, and different positions had different requirements. From there, the Cowboys started to redefine how scouting was done. Or, to be more exact, they beat the system, and in doing so, redefined the system.

“Let’s say you’re a quarterback, and you get a seven in mental alertness, and two in strength and explosion for a total of nine,” Brandt said. “I get a four in mental alertness and a six in strength and explosion, but I’m a defensive lineman, so I’m not anywhere near as highly-ranked as you are, because you are at the most important position.”

As Brandt told me, you can do a ton more on any laptop now than the Cowboys did on the old IBM System/360, but someone had to get the ball rolling. And the key was not only to discern where the best players might be in hidden areas near the bottom of the draft, but to figure out how players from other sports might be best attuned to his game. Bob Hayes was a champion sprinter who became a top receiver. Cornell Green played basketball at Utah State, but the Cowboys turned him into a five-time Pro Bowl defensive back. Peter Gent, who later wrote the classic book North Dallas Forty, was a basketball player at Michigan State, but the Cowboys saw him as a receiver. Kicker Toni Fritsch was discovered in Austria during a European scouting tour in the early 1970s. Just as in the Moneyball ethos of today, it was about beating the table—finding a new way to count cards and take home all the money. 

NFL
How far are athletes willing to go to gain an edge on the gridiron?

Of course, not everyone in the building was on board at first. It took time for old school and new school to meet, even back then. Brandt told me an amazing story about a meeting just before the 1977 draft, when Dallas had made a trade with the Seahawks and were sitting pretty with the second overall pick. The Cowboys were in the market for a running back. Brandt wanted Pitt's Tony Dorsett, while scout Red Hickey—the former 49ers head coach and the man who basically brought the shotgun into pro football—was on board with Ricky Bell from USC. 

There was a final meeting, about 36 hours before the draft, when only three people in the organization knew the trade had been made, and the trade only went through if the player Dallas wanted was there with the second pick.

“I had this big book,” Brandt recalled of the massive computer printouts he used to carry. “And I could tell you what chance you had of being an All-Pro, a Pro Bowler, a starter, or a backup, by the grades. So, Red does his thing, and he loves Ricky Bell. I do my thing, and I'm the opposite. So, Coach Landry says, ‘Get the book out, Gil.’ I got the book out, and I read the numbers. Ricky Bell had a 62% chance of starting, and he has a 0% chance of being All-Pro. I read Dorsett’s thing. Dorsett has a 100% chance of starting, and 100% chance of being an All-Pro.  

“Red Hickey says, ‘I bow to the machine.’”

NFL
The future of football equipment

The Buccaneers took Bell with the first overall pick, so Dallas had their guy. Sadly, Bell died in 1984 from heart failure due to dermatomyositis. He rushed for 3,063 yards and 16 touchdowns on 822 carries in an abbreviated career that ended in 1982. He never made the Pro Bowl, despite a 1,263-yard season in 1979. Dorsett rushed for 12,739 yards and 99 touchdowns on 2936 carries, made the Pro Bowl four times, and has a bust in Canton. Not that every combination of tech and scouting went that well, but the Cowboys were the first NFL team to bow to the machine, and it paid enormous dividends for them—and eventually, for every team. Even guys like Vince Lombardi, who were making wisecracks about computers in the early ’60s, were asking Brandt a few years later how they could get in on the action.

Brandt may be 82 now, but he still gets around just fine, and he’s sharp as a tack. He writes for NFL.com and works with STATS, Inc. When I met with him at the media center during Super Bowl week, he had a huge folder of stat breakdowns for college quarterbacks that broke everything down to the minutiae—basically, what STATS, or Football Outsiders, or Pro Football Focus, would do of their own accord.

None of it would be possible, however, without Squaw Valley, and IBM, and Schramm and Brandt who saw things in a whole new way.

The Present: Surfaces replace staples

Jeff Lewis/AP

​​The process of getting information to players on the sidelines for in-game analysis wasn’t much different in 2012 than it was in 1982. That started to change when the NFL and Microsoft patterned to make the Surface the official tablet of the NFL in time for the 2014 season. I recently attended a Microsoft panel hosted by FOX’s Erin Andrews, with some heavy hitters on the stage.

“I’m Joe Montana, and I think I represent the past.”

“Drew Brees, the present.”

“I guess I’m in-between, huh? Reggie McKenzie, general manager, Oakland Raiders.”

That’s how the intros went at the Future of Football summit on Tuesday of Super Bowl week, at the Super Bowl 50 media center at Moscone Hall. In addition to Montana, Brees and McKenzie, there was NFL VP of Media Brian Rolapp and Mike Nichols from Microsoft. The purpose of the summit was to discuss the progress of the Surface in the NFL as it pertains to everything from sideline pictures to scouting.

For Montana, it was an exercise in “what if”—what if one of the most cerebral and successful quarterbacks in NFL history, guided as he was by super-genius Walsh, had all this technology? The mind reels. What if he could study a Surface while traveling like players do these days? What if access to information was as easy and fast when he was playing as it is now? 

NFL
NFL Films embraces digital cameras to keep nostalgia alive

“We used to send the photos [overhead photos of in-game action], and they had these big metal clips, and there was a wire that came from the top. And the pictures slid down to our bench. Things have changed a lot, as fast as they get stuff on the sideline now. There are different views—it’s a lot better, I would imagine, especially for a quarterback; usually they’re the ones who watch the most film. I would have loved to spend more time [studying] on the plane rides, because we were on the plane a lot...it was refreshing not to have to do it, but it would have helped.”

More to the point was that unless you were like Montana’s one-time teammate Jack “Hacksaw” Reynolds and were such a film maven that you insisted on stocking up on film at the facility, there was nothing available to you. No internet, of course, so no All-22 on your laptop. You were in the dark unless you sought the light as much as you could.

Montana’s career went from 1979 through ’94. Brees’s NFL career began in 2001. And when Brees spoke about the advancements in technology in the space of those years, it was surprising (and somewhat appalling) how little had changed until very recently. Brees said that until three years ago, players were getting basically the same pictures thrown down from above. Only now, there were more of them, stapled together haphazardly, which made the logistics even worse.

“After you go on a 15-play drive, somebody hands you about 40 pages worth of stuff,” the Saints QB said. “The staple doesn't go all the way through, so the pictures are dangling, and one falls out, and you don't know where it goes. You look through two plays, and all of a sudden, you have to go back out on the field. You have no idea what just happened the series before.”

Scott Boehm/AP

“Now, we walk to the sideline, and it's seamless. Here’s your Surface—it has every series that's taken place throughout the game. Just click on that first play, and up pops four images. I get a pre-snap photo, a post-snap photo, and I get two photos after that, that might be a half-second later. So, you're able to see the development of a play, and what the defense did—the front, the coverage, and all those things. Once I see that, I swipe to the next play. Swipe to the next one. If there’s something important for guys to see, or you need to make an adjustment, I can show it to them. I can draw with the pen. ‘I need your angle on that post route to be a little skinnier.’ Whatever it might be. So, your ability to see it in such a clear way, and make those adjustments and take it right to the field, it's really incredible what you can do, and it gets me excited about where we're going.”

There isn’t the element of in-game video yet due to competitive concerns with the league, and it’s something the Competition Committee will discuss at the league meetings in March, but Brees shared one anecdote about how sideline video, held in the quarterback's hand, on-demand, could further advance game strategy on the fly. The NFL has given teams the ability to use video at the Pro Bowl, and during it, Brees had a fascinating interaction with Steelers receiver Antonio Brown.

NFL
How will NFL improve concussion problem over the next 50 years?

“Last year, I was watching a video of a series down in the red zone, and there’s a route where he’s running across the middle. He goes behind the defender instead of in front of him, and I see that. I immediately go over to him and ‘Hey, check this out—we’re going to come back from this play, and it’s a great play, but I need to go in front of the defender, and we’ll score. Two series later, we go back out there and call the same play, he crosses the defender’s face, and catches the touchdown. Without the video, I would not have been able to see that and communicate it to him. It does a lot for our preparation and our ability to adapt.”

Brees said that there are some old-school coaches who have had issues adapting to the new technology, but when they see the differences, they come around. When it comes to convenience and accessibility, Brees says, the difference is truly amazing. As it was in Brandt’s day, it’s all about the ability to get the most possible information in the shortest amount of time.

And from a personnel perspective, it could be argued that the new tech has helped GM McKenzie and his fellow executives even more.

“We use it extensively,” McKenzie said of the Surface. “With the scouting department, you have a tablet for viewing [players], a tablet for player information, a tablet for writing reports, and this totally encompasses everything—all in one. You can get all the stat information, you can get all the report-writing done, you can get all the viewing from tapes, college and pro. You can travel with it—you scout in Florida, or New York—you can go anywhere and take this as your office, and still continue to do everything...

“To echo what Drew was saying, as far as on the field [now], when our quarterback or a linebacker comes back to talk to the coaches, that interaction after the coaches have looked at the plays, and they can go right to the player... there's no downtime. It's all about real time.”

When asked how many hours of film McKenzie has on his tablet, he brought it back to a story—the day before, he’d brought all his scouts in to start the Raiders’ draft meetings. They started at 7 a.m. and McKenzie left four hours later, when he got in his car to head to the summit. McKenzie said that his scouts would be working on film until 6 p.m., all off the tape they had on their tablets. The Raiders, like every other NFL team, have made the tablet the go-to accessory for the study of their own players and the players on every other team. Whatever tape the Raiders have at their facility can easily be downloaded and accessed from anywhere.

NFL
Roundtable: What will media coverage of Super Bowl look like in 50 years?

“Our scouting department can now see what our team is like,” McKenzie concluded. “And when I was beginning in the scouting department, before I became a general manager, I was on the road, and you don't know what your team is doing, how they’re playing, until you get in-house at the end of the week. Now, they can download tape and see it at their hotel. It’s a lot of film-watching, but it’s fun. When it’s made easy and accessible, that makes it even better.”

I spoke with Brees a bit after the summit, and he wanted to make one extra point—the Surface malfunctions that happened in the AFC Championship Game between the Broncos and Patriots were a product of a network cable issue.

“Imagine all the feeds that are coming into a stadium on gameday—from the television networks and this and that,” Brees said. “In many cases, the headsets will go down... people ask me, ‘Why does Coach Payton look like such a lunatic on the sideline; what is he doing?’ He’s usually yelling at the headset guy. But that’s a network feed issue, or a cable feed issue, or something is faulty and they get it fixed. But as far as the performance and functionality of the Surface, it’s never given us problems. In the 100,000-plus hours it's been used on the sidelines for games over the last couple of years, there have been zero issues with the Surface.”

From players to coaches to scouts to the top executives, the tech revolution started by Schramm and Brandt all those years ago has taken hold more firmly than ever. The fan experience is the next thing to undergo a major renovation.

The Future: Tabletop Football in a new way

It’s tough to express what this version of televised football is like, and how different it is, except to compare it to going from checkers to the real-life intergalactic creature chess at the cantina in Star Wars.

The Hololens, a 3-D holographic headset on the Windows 10 platform, is what Microsoft is betting on for its future, as much as Windows was the foundation to its establishment as a technological power, and the Xbox and Surface are the structures of today. CEO Satya Nadella has said that the Hololens experience is “like the first time you used Excel on a PC with a mouse and a keyboard,” and the clear hope is that it will reverse recent failures the company has had in hardware, from the Zune MP3 player to the Microsoft phone, that never really got off the ground. The Hololens is Microsoft’s chance to do what Apple has done so successfully throughout the Steve Jobs eras—control the hardware and software channels for devices the public deems indispensable.

At this time, Microsoft’s team is using the Hololens to try a lot of different things, from anatomy to space exploration to 3D rendering. Its best use as a consumer device down the road, however, may be how it changes the ways in which the public sees its favorite spectator sport. 

Microsoft/used by permission

The day after the summit, I was on Bryant Street in San Francisco, in an old Korean auto parts shop that Microsoft had converted into a lofty showspace. There, me and a handful of other NFL dorks would get a taste of what the combination of the Hololens and NFL football could do. Among my colleagues were Sam Farmer of the L.A. Times and NFL RedZone host Scott Hanson (me to Hanson at one point: “Can you imagine RedZone with this? Oh my God!”)

After a quick briefing, we were escorted into a small room that's made to look like the standard Man Cave, with an HDTV plastered to the wall, and team paraphernalia everywhere. We sat on comfy leather couches and prepared to have our minds blown.

NFL
Examining the future of virtual reality and the impact it will have on football

You put the Hololens on your head, at a bit of an angle, and you want the lens mechanism itself resting slightly above your nose. When the operating system first comes on, it looks like a standard HD television screen. Then, the flyout screens show on either side for stats, fantasy information and other sidebars. Then, you can adjust the virtual screen to play at the size of a living room wall.

That’s all cool, but the real experience comes when the tabletop part of the experience comes in. Basically, this is the new-world version of the old tabletop football games of a generation ago, except that now, you’re seeing 3-D versions of the players in live game action. And the control over the viewer experience is otherworldly. I started to imagine the All-22 pieces I could do with this technology, and I had to go sit down for a minute.

Using your phone, you can alter the angle, zoom in and out, and make the experience whatever you'd like it to be. You can go over the top of the action just as easily as you can stand on a virtual sideline. It’s tough to express what this version of televised football is like, and how different it is, except to compare it to going from checkers to the real-life intergalactic creature chess at the cantina in Star Wars. When you see it, the way you watch the game will change forever. Yes, that sounds like bogus ad copy, but it's true. 

So, you may ask, when do we get this stuff in our hands? At this time, the Hololens and the NFL are dating—the relationship isn’t quite ready for marriage. A Microsoft Hololens spokesperson told me that the idea with the first rollout is to get it into the hands of developers and expand what the device can do. At this point, Microsoft sees the Hololens as a glimmer of what could be, and there's a “Would it be cool if...?” aspect to the whole thing. The Developers’ Edition of the hardware starts at $3,000, and you have to apply for one.

All I got was a tantalizing taste, but I am now wondering, along with everyone else in this story, what more can be done.

SI Apps
We've Got Apps Too
Get expert analysis, unrivaled access, and the award-winning storytelling only SI can provide - from Peter King, Tom Verducci, Lee Jenkins, Seth Davis, and more - delivered straight to you, along with up-to-the-minute news and live scores.