I found this great page whilst looking for something else. It gives some ideas about how to block brute force attacks using recent iptables.