Automating signing when using GPG

How to setup automatic signing when using GPG

We use Gnu Privacy Guard, aka gpg, to encrypt files that we send to banks, for example. GPG employs public/private key encryption. We use the banks' public key to encrypt the files the files we send to them, and they use their private key to decrypt the files. But bankds also want to be certain that the files they get from us really did come from us, so we also need to sign each file. To sign a file, gpg hashes the file and then encrypts that hash using our private key. This is the signature, which is bundled into the file. When the bank gets the file, they decrypt the signature using our public key, which gives them the hash; and then they hash the file and compare that number to the number from the decrypted signature. If they match, "it is almost certain the documents are identical", to use the words of the gpg manual.

The problem for us is how to automate the signing process, since the encrypting and sending of the files is usually handled through scripts called by cron. To use the private key, in this case for the signature, normally requires the use of a passphrase. To not have a passphrase on a private key would allow anyone to use that private key and would render the whole process pointless. GPG does have a --batch option, which tells gpg to not prompt the user for any info, but this just causes the gpg command to fail. The GPG FAQ does have an entry that addresses this problem, but it is so terse that it leaves out some significant details.

Here, then, is a shortmodified version of the FAQ answer, that includes the left-out parts; followed by a longer version that explains the process in greater detail.

Short Version

Create a separate signing key:
# gpg --edit-key "Petre Scheie"
Within gpg, use 'addkey' to generate a new key with a DSA key type.
Run 'gpg --list-keys' to see what the ID is for the key just added. In the sample below, the ID for the new signing subkey is 30B8F215.
gpg --list-keys
/home/pscheie/.gnupg/pubring.gpg -------------------------------- pub 1024D/05B27CE2 2004-03-12 Petre Scheie (created 10:43 3-12-04) sub 1024g/F8DEDB3D 2004-03-12 sub 1024D/30B8F215 2004-03-12
gpg --export-secret-subkeys --no-comment 30B8F215 >secring.auto
Copy secring.auto and the public keyring to a test directory.
Change to this directory.
Rename secring.auto to secring.gpg.
gpg --homedir . --edit 30B8F215 and use "passwd" to remove the passphrase from the subkeys.
Change back to your home directory.
Copy the .gnupg directory and its contents to .gnupg.insec
Replace ~/.gnupg.insec/secring.gpg with the secring file in the test directory above.
Use the following syntax to encrypt & sign your files:
gpg --homedir ~/.gnupg.insec -s -r "Mohammad Imam" --default-key 30B8F215 -o file.txt.gpg -e file.txt

Long Version

The key to automating signing (no pun intended) is to create a separate signing key that has no passphase. As such, GPG renders the key useless for anything but signing; that is, you can't use it to decrypt documents sent to you, but, as you'll see, that's why it is kept separate from your 'regular' set of keys.

Create a separate signing key. If you type 'gpg --list-keys' you'll see all the keys that you have, including your own keys and any public keys from other people that you've imported. You should see something like this:
/home/pscheie/.gnupg/pubring.gpg -------------------------------- pub 1024D/05B27CE2 2004-03-12 Petre Scheie (created 10:43 3-12-04) sub 1024g/F8DEDB3D 2004-03-12
pub 1024D/E2DBCA66 2003-04-11 Mohammad Imam (Oracle DBA) sub 1024g/217F8766 2003-04-11
pub 1024D/FEE0FDA8 2004-03-11 Petre Two (second ID for gpg testing) sub 1024g/C169F4E8 2004-03-11
In the above example, it's the first key pair, for Petre Scheie, that we are interested in. The first column indicates whether it's a primary (pub) or subordinate (sub) key. I'm not sure what the difference is, especially with the public keys from other people that have been imported, in this case, Mohammad Imam and Petre Two. The second column indicates the size of the key; in this case, they're all 1024-bit. The third column is the important one, and is easy to overlook as it is the single letter right next to the size. In the example above, the key type is either D or g. The D stands for DSA and is for signing; the g stands for El Gamal and is for encrypting. The fourth column, which is separated from the third column by the slash, (why they use a separator here and nowhere else escapes me) is the key ID. The fifth column shows when the key was created. The sixth column has three parts: the person's name, a comment, and an email address, any of which can be used when you're telling GPG which key you want to edit. Generally, I find it easiest to use the key ID (column four) or the person's name in quotes six when telling GPG which key I'm interested in. In the GPG documentation, they seem to use the email address more frequently.
Run gpg with the --edit-key option and tell it which key you want to edit.
gpg --edit-key "Petre Scheie"
At the Command> prompt, type 'addkey'. It will ask for your secret key--this is to prevent someone else from adding keys to your ring. Next it will ask what kind of key to add. Choose DSA (sign only). And for the other questions, just use the default answers. Once back at the Command> prompt, type 'save' to save and exit. If you run 'gpg --list-keys' now, you'll see your new signing key:
cacsdv01:/home/pscheie >gpg --list-keys /home/pscheie/.gnupg/pubring.gpg -------------------------------- pub 1024D/05B27CE2 2004-03-12 Petre Scheie (created 10:43 3-12-04) sub 1024g/F8DEDB3D 2004-03-12 sub 1024D/30B8F215 2004-03-12
Now you need to export your secret keys into a separate file to be edited so the passphrase can be removed.
gpg --export-secret-subkeys --no-comment 30B8F215 >secring.auto
Your secret keys are stored in ~/gnupg/secring.gpg. The reason you need to export the secret ring, instead of just copying it and working on that copy, is that --export-secret-keys will strip out all the secret parts of your primary key, which are, near as I can tell, the first two entries you see when you list your keys; that is, the D key and the g key. This is important for security reasons. If you didn't strip those parts out, anyone could get access to all your keys, including those used to decrypt files and sign files, without a passphrase, which would defeat the whole purpose of encrypting the files.
Copy secring.auto and the public keyring to a test directory.
mkdir /tmp/gpg cp ~/secring.auto /tmp/gpg/ cp ~/.gnupg/pubring.gpg /tmp/gpg/
Change to the test directory.
cd /tmp/gpg
Rename secring.auto to secring.gpg. I'm not sure why the FAQ doesn't just tell you to call the exported file secring.gpg, but perhaps it's to avoid accidently stepping on the unexported version of the file.
Edit the keys to remove the passphrase from the signing key.
gpg --homedir . --edit 30B8F215
At the Command> prompt, use the 'key 2' command to indicate that it is the second subkey that you want to edit. (The pub key isn't counted, for whatever reason.) GPG will put an asterisk by the selected key.
Command> key 2 pub 1024D/05B27CE2 created: 2004-03-12 expires: never trust: u/u sub 1024g/F8DEDB3D created: 2004-03-12 expires: never sub* 1024D/30B8F215 created: 2004-03-12 expires: never (1). Petre Scheie (created 10:43 3-12-04)
At the prompt, type 'passwd'. First you'll be prompted for your private key passphrase, which is needed in order to gain the rights to change passwords. Then you'll be asked for the new passphrase for they signing key. Just hit Enter, and again when it asks you to confirm the new passphrase. GPG will tell you using a blank passphrase is a bad idea, which it is, but obviously you've already considered the potential perils and concluded it's a reasonable risk. ;-) Then type 'save' to save and exit.
Command> passwd Secret parts of primary key are not available.
You need a passphrase to unlock the secret key for user: "Petre Scheie (created 10:43 3-12-04) " 1024-bit ELG-E key, ID F8DEDB3D, created 2004-03-12
Enter the new passphrase for this secret key.
You don't want a passphrase - this is probably a *bad* idea!
Do you really want to do this? y
Command> save
Change back to your home directory and copy the .gnupg directory and its contents to .gnupg.insec.
cd cp .gnupg .gnupg.insec
Copy the secret ring from the test directory to the new insecure directory. Don't copy all the contents of the test directory, just the secret ring. If you were to copy all the files from the test directory, you'd get errors about gpg not being able to verify the authenticity of the public keys you've imported (see Troubleshooting below).
cp /tmp/gpg/secring.gpg .gnupg.insec/
You can now use the insecure signing key to sign files like so:
gpg --homedir ~/.gnupg.insec -s -r "Mohammad Imam" --default-key 30B8F215 -o outputfile.gpg -e file.txt
The --homedir option tells GPG to look in .gnupg.insec for the keys to be use; -s means sign the file with your private key so that the recipient will be able to verify that you actually created the file. -r indicates the recipient, that is, which public key to use to encrypt the file. --default-key says which key to use for the signing (see Troubleshooting below); this is a really important piece that the FAQ leaves out; if you don't specify the signing key, GPG will just use the default key, which won't work because all the necessary secret parts were removed from the default key in the export stage. -o indicates the name to be used for the encrypted file that will be created; and -e means encrypt the file.
This syntax allows you to keep your exisiting .gnupg directory to be used for more secure operations in which interacting with GPG to encrypt the files isn't a problem. In other words, the default operation of GPG will require the user to have passphrases, but for just signing a file within a batch script, the .gnupg.insec/ directory can be used.

Troubleshooting

When I try to sign and encrypt a file, I get "gpg: no default secret key: secret key not available".
Your forgot to use the --default-key option or you specified the wrong key with it. Make sure you use the keyID from the signing key you specifically created for this process, and not the default signing key that is created when you first gen the key pair.
When I try to sign and encrypt a file, I get "gpg: C169F4E8: There is no indication that this key really belongs to the owner" and then it prompts me as to whether to really use this key.
You copied the pubring.gpg file from /tmp/gpg to the .gnupg.insec/ directory. Don't do this. Only copy the secring.gpg file from /tmp/gpg. Also, make sure you have signed (fingerprinted) any public keys on your keyring before you export the secret keys. I'm not certain this matters, that is, I didn't test it; but it seems to me that gpg wouldn't be able to know a given key was authentic unless it had been signed, and so presumably it would say so, which of course will cause problems in scripts.
I can create a signed and encrypted file, but when the recipient of the file runs 'gpg --verify' on it, it says 'gpg: verify signatures failed: unexpected data'.
Don't use --verify unless you have a detached signature (which isn't covered in this doc, so you probably don't have one). Instead, using --decrypt to decrypt the file will also verify the signature. When you do this, you should get something like
gpg: Signature made Fri Mar 12 14:05:18 2004 CST using DSA key ID 30B8F215 gpg: Good signature from "Petre Scheie (created 10:43 3-12-04) "
in the output. Or you can use --verify-files, which seems to produce output only if there are errors.