Instead, he is the leading scorer for No. 3 Gonzaga, and couldn't be happier.
''I saw a great fit here,'' Wiltjer said. ''The system here is tailored to a player like me who plays inside and outside.''
The 6-foot-10 Wiltjer, who sat out last year because of transfer rules, has been a key reason that Gonzaga (17-1, 6-0 West Coast) is off to a great start.
Wiltjer is averaging 16 points and 5 rebounds per game. He's shooting 53 percent from the floor, including 43 percent from 3-point range.
The junior forward forms a potent front court with center Przemek Karnowski.
''I love playing with big Shem,'' Wiltjer said of the 7-foot-1 Karnowski, who has blossomed this season. ''He takes up so much space up there.''
''He's scoring in bunches,'' said point guard Kevin Pangos.
But on a deep Gonzaga squad with numerous scorers, Wiltjer is far from the only weapon. Five Gonzaga players average double figures.
''He has benefited from the team's willingness to pass and share the ball,'' coach Mark Few said.
Wiltjer said his desire to play with Pangos, a teammate on the Canadian national squad, was one big reason he decided to leave Kentucky.
''I really like to play with him,'' Wiltjer said.
He decided to play for Kentucky, spurning Gonzaga among others, and averaged 5 points per game as a freshman for a team that won the 2012 national title.
As a sophomore, he was the SEC Sixth Man of the Year, averaging 7.3 points and 2.8 rebounds per game.
But he grew disenchanted by a lack of playing time at Kentucky, and became enamored of the story of Kelly Olynyk, who used a redshirt season at Gonzaga to change himself from a role player into an NBA lottery draft pick.
''I'm choosing to transfer to Gonzaga because I feel they have a proven plan in place to help develop players similar to me,'' Wiltjer tweeted in 2013. ''I believe with hard work I can maximize my development at Gonzaga.''
''I just thought the year off would be a great way to get better,'' Wiltjer said this week.
Wiltjer is a threat both inside and from 3-point range, and this week was named one of 25 finalists for the Wooden Award, given to the best player in college basketball.
He is one of three newcomers making a big impact for the Bulldogs this season.
Byron Wesley, who led Southern California in scoring and rebounding last year, decided he wanted a chance to play in the NCAA tournament and transferred to Gonzaga. Because he had already graduated, he did not have to sit out a season. Wesley is averaging 11.6 points and 5.5 rebounds per game.
''He's a big, physical guard who can defend,'' Wiltjer said. ''He opens up things for me as a shooter.''
Meanwhile, Domantas Sabonis - the son of former NBA star Arvydas Sabonis - is averaging 10 points and 5.8 rebounds as a freshman, despite playing only 20 minutes per game.
''We have so many weapons, we knew we were going to be a special team,'' Wiltjer said.
The Zags have a goal this year and it is to advance beyond the first week of the NCAA tournament for the first time since 2009.
''We've got to get past that first weekend,'' Wiltjer said.