I figure it's part of Dief's ability to influence thoughts - he doesn't exactly "speak" to Fraser, just implants what he's thinking into Fraser's mind. Fraser may sometimes interpret things a little too much toward his own point, which makes Dief grumble a lot. Sometimes, Fraser does this on purpose.