It isnt just him, I get them and so do many others.
It is just a phishing technique, they can send out the letters for a very small cost and if it works on a few people they dont give a hoot about all the non-customers that they annoy.
I have similar letters from a debt company (for somebody I have never heard of or ever lived at this house).
To answer the original question, I have had a visit from an inspector and he didnt even knock on the door or look around outside, he just stuck a card through the door saying that he had visited.
Dont be too hard on these poor sods who are working for Capita, they probably got sent there by the DWP on threat of losing their Unemployment Benefit and they are working on a commission for every person they get to pay the fee. Obviously took one look at my house and decided it wasnt worth the effort.
Many people claim that a mention of fetching a camera to film them when they visit makes them move on rather quickly.