Toure will leave Manchester City at the end of the season after eight years in England, but has stated his intention to remain in the Premier League next season.
The Ivorian has faced many great players during his time at City but says than none have been tougher to play against than the tireless Matic.
"I hate him, that bastard!" laughed Toure, quoted by the Daily Mail. "He's such a difficult player to play against. Tall and strong as well. It's been fun to play against him, even though it's difficult. He's powerful, he's tall.
"To be able to find an opponent to play against me and stop me...there's few of them, but Matic was one of the close ones who was able to give me problems. He can run. He can fall back. He can track. He can fight. He's a good, good player."
Matic has enjoyed an impressive debut season since joining United from Chelsea last summer, but the same cannot be said of Pogba. Now in his second season at Old Trafford, the Frenchman is still struggling to live up to the £89m that United paid Juventus for his services.
But Toure compared Pogba to himself, both in terms of ability and hype, and backed the 25-year-old to come good.
"I always loved to prove people wrong," Toure told BBC Sport. "I was put under a lot of pressure, as Paul Pogba, who has similar characteristics to me, is now.
"I believe Pogba will become the perfect player. And if they are honest, the people who said things about me will say they were wrong."
Toure will make his final home appearance for Manchester City against Brighton on Wednesday as he brings his time at the club to an end after more than 300 appearances.
City need two more points and two more goals in their remaining games to eclipse Premier League records on both counts.