I finally managed to set up an SMB share to test this on, and I do see the problem.
It smells just a little bit like samba itself being very buggy, though. If that's the case, I can try and work around it but I can't guarantee I will have much success.
(For fun, try this: delete the picture you're viewing from inside Xee. Then browse forward a couple of images and then back to the deleted image. For me, it's still there and can be loaded, although it's disappeared from the Finder!)