More than once I have come across the issue where we have had a problem using an X509Cert from the certificate store. Everything is configured properly in the certificate store but when we attempt to create the signature we end up with a cryptographic exception for no apparent reason.
See CryptographicException: The handle is invalid post by Benoit Martin explains the problem and shows how this issue can be resolved.