There is very likely a way to clone a neighbor pixel over the bad pixel. This would be superior to masking.
I don't know the Avisynth script needed off-hand, and a sample clip would be required. Either sanlyn here at dFAQ knows, or jagabo, poisondeathray or manono over at VH. I know I've seen scripts from jagabo, to accomplish similar (and more complex) tasks.
Start by posting a sample clip here. I'll give one of them a nudge, see if they can give you some scripting help.
... and this would be something good to know, if I ever have the need.