In an interview with the Louisiana Courier Journal, Col. Sanders’ grandson Trigg Adams called any accusation that his grandfather used the N-word or any other racial slur “an absolute lie.” Adams called Schnatter “a weasel” who lied when he suggested Col. Sanders used racial slurs.

The controversy started when Schnatter, the founder of Papa John’s, used the N-word during a conference call and suggested that Col. Sanders used the word. In a radio interview after the incident, Schnatter said that he was “just talking the way the colonel talked.”

Sanders’ family has responded sharply to the claims, saying the KFC founder was in no way a racist. They also accused Schnatter of trying to use their grandfather and great-grandfather to protect himself amid the controversy.

“For Mr. Schnatter to use the colonel as a scapegoat for his own horrible, disgusting mouth and racist beliefs is inexcusable,” Cindy Wurster Sjorgen, Sanders’ great-granddaughter, said in a statement.

For his part, Schnatter has resigned from Papa John’s and the company has tried to distance itself from him by removing him from all advertising materials and scrubbing him from the office. They’ve also banned him from talking to media outlets.

KFC, which is owned by YUM Brands, has increasingly used Colonel Sanders’ likeness in advertising to promote its food chain.