He wants to tell her that she's not the one who kept steering the conversation this way. If anyone's putting a damper on it, it's him. But the point seems moot.

And she makes him laugh.

"I am one hundred percent certain that that isn't true," He says, laughing anyway. Too late, he's already told her. "...But I can see where it would help."

He points his cane up and outwards. "To the kitchens!"