Security in an error-prone world

Posted Nov 3, 2015 22:26 UTC (Tue) by nix (subscriber, #2304)
Parent article: Security in an error-prone world

Oh God they recommend using gpg-agent as an ssh-agent replacement? I tried to get this working, but eventually gave up: it's devastatingly buggy. Among other problems, forwarding of the agent to other machines is only just barely kinda sorta working in GnuPG 2.1 and has no chance of working in 2.0; the eternal GPG_TTY bugs (are you sure you're resetting that in *every single* startup script? Without exception?) routinely lead to you being asked to unlock your key using a pinentry running on the wrong console, causing an apparent hang; even if that's fixed, if you ssh or su to another user and want it to use your smartcard, you suddenly find that you can't because the pinentry can't appear on that tty since it's owned by another user and not writable by the one the gpg-agent is running on... the list goes on and on. It's a mass of barely functional components held together with baling wire and twine.

To make things worse, the YubiKey is a multifunction device, but it can only do one thing at once: so if you use it for OTP passwords or U2F, every time you touch the key's touchpad or ask for a U2F password it'll disconnect from the gpg-agent, and neither pcscd nor the builtin ccid driver in GnuPG 2.1 is remotely expecting this. (pcscd is a whole other mass of crawling horror, running right down to the fact that the author has recently removed all autospawning support for non-systemd configurations for poorly-stated reasons, meaning that if you don't run systemd you are forced to keep a systemwide pcscd running just in case a user plugs a smartcard in: if you want this configuration to be remotely secure, you have to use a very recent JS-infested PolicyKit. What a mess.)

PIV support using the yubico-piv-tool does work much better -- but it relies, again, on pcscd to do the heavy lifting, and this has no idea that slot 9a allows reauthentication without a password. Since nothing anywhere stores the PIN and there is nothing like automatic reconnection, whenever you ask for an OTP, your SSH key vanishes until you do an ssh-add -e and -a again, and type in the PIN (again). Half the time you'll get stuck in some unclear state where doing *either* of these just tells you 'agent refused operation' while the logs scream 'process_remove_smartcard_key: pkcs11_del_provider failed'.

There is a debugging interface for pcscd: it involves *renaming the shared library* and making a new symlink. I've got some debug logs out of this ridiculous system but haven't yet analyzed them to see what's going wrong (I need to learn about the protocol it's trying to talk, first).

This whole thing is not remotely ready for prime-time. I'd love to recommend SSH and GPG key storage on the Neo and its use everywhere, but I just can't. I use PIV keys myself and am frequently bitten by the smartcard connection being lost because I had the temerity to use the Neo like a Yubikey and get an OTP out of it, and much of the time I can't get it back again.

to post comments

Security in an error-prone world

Posted Nov 3, 2015 23:11 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (9 responses)

> (are you sure you're resetting that in *every single* startup script? Without exception?)

Well, my zshrc sets it and it isn't in systemctl --user show-environment, so…yes? :)

> if you ssh or su to another user

Eh, not clearing your environment in such situations is probably not the best idea anyways.

> and want it to use your smartcard

Why would you want to use an SSH or GPG key as another user? Genuinely curious (though I disable agent forwarding; no need to have a malicious server slurp keys when I connect to it).

> very recent JS-infested PolicyKit

To be fair, this happened a while ago. Sad as it is :( .

So it seems that my automounted usb key is still the most flexible setup for me if this is the state of things.

Security in an error-prone world

Posted Nov 4, 2015 0:23 UTC (Wed) by wahern (subscriber, #37304) [Link] (6 responses)

Why would you want to use an SSH or GPG key as another user? Genuinely curious (though I disable agent forwarding; no need to have a malicious server slurp keys when I connect to it).

1) A server would normally only be able to use the key, not read it. Maybe you meant something else. 2) If the client SSH program is buggy, yes, a malicious server could read the key, but that's because you're not using a smartcard. Personally, I never understood the appeal of putting a private key on a USB device. It's not much different than using a password encrypted key stored on your computer. It makes it easier to use on other computer, but that seems even more risky, because now the security of the key is a function of the least secure computer you use it on. Though I don't know your precise reasons, and am not trying to judge them specifically.

Even though I use a smartcard, I only enable authentication forwarding on a case-by-case basis. Still, I've always thought it would be useful to configure a card to require a physical key press before performing the signing operation. This seems like a more useful feature than a PIN, IMO, especially for contexts like banking. Somebody can hack my computer and steal my PIN, then use the card without my knowledge--it's plugged in throughout most of the day. Whereas if it required a physical confirmation, they couldn't. PINs address the wrong attack scenario--the biggest threat is somebody hacking my computer, not stealing my smartcard. If somebody steals my smartcard I'll know about it, or at least the damage be circumscribed. And maintaining physical custody and security of a key is much easier as a practical matter, especially in terms of the threats I and most people face. Whereras much like a password, if they steal my PIN I'll have no idea. And the universe of people that could access my PIN is, as a practical matter, any sufficiently knowledge hacker on the face of the planet.

This is why when people use the term 2-factor, I cringe. Even supposed professionals are enamored with this phrase, without giving much consideration to the _real_ threat scenarios, and to the relative costs and benefits of these factors. Yes, a coworker stealing your smartcard for 5 minutes when you're at lunch is a real threat. But the state of computer security is _so_ utterly abysmal that the threat absolutely pales in comparison to remote threats. Combined with the implementation and interoperability problems that something as simple as a PIN can cause (it's 2015 and, with all its problems, the Yubikey NEO is as good as it gets), this and similar features shouldn't be considered a requirement if you want to improve organizational security.

I have it on my TODO list to hack the Yubikey NEO OpenPGP applet to support personal data objects. Perhaps I should look into support for requiring a key press before signing.

Security in an error-prone world

Posted Nov 4, 2015 0:46 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

far too many systems have been hacked by getting a copy of someone's ssh key and hopping from machine to machine as that user (and picking up other privileges along the way)

I don't care how complex your pin is, if someone can see your keystrokes they can get your pin.

I agree that you really want your two factor authentication to be something that requires affirmative action to use (either typing in the result, or at least hitting a button on the key itself)

Security in an error-prone world

Posted Nov 4, 2015 13:39 UTC (Wed) by nix (subscriber, #2304) [Link] (1 responses)

They can get your pin, but they cannot get your identity, and as soon as you take your smartcard out (which you should do whenever you step away from the machine for more than a few minutes) that pin is useless.

In fact, capturing it is more or less useless anyway: it only serves to *unlock* the smartcard, and if you plug it in, you're probably going to do that anyway. An attacker doesn't need to capture your PIN: it just needs to wait for you to type it in yourself, then query the unlocked smartcard as usual. Getting your key remains impossible, as is doing authentication operations when the smartcard is not plugged in. So you've basically restricted a successful attacker to only attacking when you're around and can potentially spot attacks. (You probably won't, but still.)

Security in an error-prone world

Posted Dec 1, 2015 13:37 UTC (Tue) by dany (guest, #18902) [Link]

as an update, new yubikey4 has feature "touch" which if enabled, requires to touch button before any interesting operation on GPG key. you can enable it for any key (sign, auth, decryption) and you can force this feature so it cannot be disabled, until new private key is placed on card. so even if attacker knows your PIN, they cannot use your key in any meaningful way.

Security in an error-prone world

Posted Nov 4, 2015 1:32 UTC (Wed) by mricon (subscriber, #59252) [Link]

> Perhaps I should look into support for requiring a key press before signing.

"Touch to sign" is part of the OpenPGP Card v3 spec and I expect it will be supported by upcoming versions of yubikey NEO.

Security in an error-prone world

Posted Nov 4, 2015 2:27 UTC (Wed) by mathstuf (subscriber, #69389) [Link]

> Personally, I never understood the appeal of putting a private key on a USB device. It's not much different than using a password encrypted key stored on your computer.

The setup is a key with what I need day-to-day (SSH keys, keepass database, SSL client certs) with a passphrase I can actually type. There are other copies with everything I need (main GPG private key, SSL cert backups, TOTP recovery keys, etc.) without the daily typeable passphrase and instead a much longer passphrase.

> It makes it easier to use on other computer, but that seems even more risky, because now the security of the key is a function of the least secure computer you use it on.

Well, no different than other setups, really. Just don't use it on machines I don't trust.

Security in an error-prone world

Posted Nov 4, 2015 13:37 UTC (Wed) by nix (subscriber, #2304) [Link]

Quite. You use agent forwarding in clustered environments, where you have lots of systems within the same security boundary but you still don't want someone who gets into one of them to be able to ssh completely freely around in there. A good clue if you have a setup like that is if you're using networked filesystems across them.

My local setup has agent forwarding turned on to the clustered machines (which also share $HOME filesystems via NFS) but the firewall host does not have any of that, so an attacker will be stuck on there, unable to ssh in to the cluster even if I'm sshed into the firewall, because that SSH connection is *not* accompanied by an agent forwarding, so the smartcarded key I used to get into the firewall and across the cluster is still inaccessible.

Security in an error-prone world

Posted Nov 11, 2015 21:20 UTC (Wed) by nix (subscriber, #2304) [Link]

> Eh, not clearing your environment in such situations is probably not the best idea anyways.

Alas, it's not related to that. The problem is that the connection to the agent is forwarded over ssh back to the gpg-agent, and it communicates the value of $GPG_TTY back over that (Assuan) connection: gpg-agent then tries to kick up a keychain on your TTY, and oh look it's a different user and I bloody hope the gpg-agent isn't running as root, so it can't do it. Now you're in trouble.

As far as I can tell, gpg-agent is only designed to work in a situation in which each Unix user has his own keychain, and only one Unix user has a connected smartcard, and no other user ever wants to use it. It actively militates against a scheme where you use multiple uids to separate your concerns (I use a different uid for work and non-work, for instance), and it always will. The problem is that the keychain is forked by the gpg-agent to ensure that nothing else can spy on the passphrase as it passes through -- but the keychain *cannot* be forked by the gpg-agent without causing the problem above!

I see no way to fix this :(

Security in an error-prone world

Posted Nov 11, 2015 21:31 UTC (Wed) by nix (subscriber, #2304) [Link]

Why would you want to use an SSH or GPG key as another user? Genuinely curious

I consider a smartcarded SSH key to be 'something I have' combined with 'something I know': proof that I have physical access to the smartcard (though not quite as much proof as a touch-to-generate one-time password) and proof that I know the PIN. As such, it's quite safe to use it for multiple users, if what you're using those users for is separation of concerns and to stop programs running as one from accidentally smashing programs running as the other. Identities are not the same as Unix uids!

Security in an error-prone world

Posted Nov 3, 2015 23:40 UTC (Tue) by wahern (subscriber, #37304) [Link] (1 responses)

I've been using a Yubikey NEO with the OpenPGP applet on OS X with MacPGP2 for over a year, and I have no such bad experiences. Except for a brief stint when Apple borked their pcscd fork, it's been smooth sailing.

SSH authentication forwarding shouldn't be problematic--it's the same ssh instance talking to the agent when authenticating the first hop as when you're authenticating the next hop. Certainly I've never had a problem. I presume you're talking about GnuPG protocol-specific forwarding, which does seem to be buggy. But that's irrelevant for SSH authentication, it's just a bummer when you use mutt to read your e-mail on a remote server (as I do) and would like to be able to use PGP for e-mail.

MacPGP2 is using GnuPG 2.0, FWIW. But MacGPG's secret sauce is their GUI PIN entry program, so maybe they've fixed other problems as well.

Finally, my Yubikey NEO will work in OpenPGP mode _and_ HOTP mode just fine, although I do have to re-enter my PIN after generating an HOTP. Multifunction worked both before and after Apple's pcscd problems, but notably Apple's bug made pcscd lose track of the card state. Something similar (though I doubt identical, because it's forked) seems to be the problem here.

I agree things could be better. But compared to the way things were just a few years ago, particularly with SSH it's like night & day.

Ideally somebody will devise a scheme to use U2F keys for SSH, and OpenSSH will gain native U2F support, removing the need for all the middleware.[1]

[1] Ludovic Rousseau is one of the hardest working and most capable FOSS developers out there, but the deck is simply stacked against him. There's only so much one person (or a whole team of people) can do to wrangle the horrendously complex state of smartcard interfacing and management. The OpenPGP smartcard spec works because it simplifies many things, leaves less room for optional crap, and specifies basic management capabilities. U2F simplifies things even further. Heck, they could've probably just ditched the PIN requirement altogether. A "1-factor" pubkey smartcard without a PIN is still an unfathomably better state of affairs than using passwords when it comes to remote authentication, and even better than password-in-all-but-name schemes like HOTP, TOTP, and biometrics.

Security in an error-prone world

Posted Nov 4, 2015 13:56 UTC (Wed) by nix (subscriber, #2304) [Link]

I've been using a Yubikey NEO with the OpenPGP applet on OS X with MacPGP2 for over a year, and I have no such bad experiences.

Excellent! That means I'm probably just doing something stupid wrong -- though the fact remains that there are lots of ways to get it wrong, and the way described on the LF site is one way to do it (because that's what I tried, and it didn't work).

SSH authentication forwarding shouldn't be problematic--it's the same ssh instance talking to the agent when authenticating the first hop as when you're authenticating the next hop. Certainly I've never had a problem.

That works until you use your Yubikey to do anything else (e.g. OTP). If you were using native SSH, you could use ssh-add -e / -s to sever the smartcard connection and restart it, and everything would mostly be fine -- but using GPG, well, as soon as the connection is severed, the gpg-agent (and, if you're using it, pcscd) hang, hard. You have to kill -9 and restart them, and as soon as you do that the authentication forwarding is severed: you have to restart all your ssh sessions too! This is very far from optimal.

notably Apple's bug made pcscd lose track of the card state. Something similar (though I doubt identical, because it's forked) seems to be the problem here.

Almost certainly. Possibly this is the ill-defined 'issues' which caused pcscd autostarting without systemd to be removed, but I doubt it: it was only half a dozen lines, and in particular nothing changed about smartcard state tracking: pcscd still exits when idle in both cases, presumably losing track of card state as it does so.

But compared to the way things were just a few years ago, particularly with SSH it's like night & day.

Good God that's horrifying. :)

Ideally somebody will devise a scheme to use U2F keys for SSH, and OpenSSH will gain native U2F support, removing the need for all the middleware.[1]

Agreed! I'd be oh so very happy with that. U2F looks much easier to wrangle than PKCS#11, enough so that adding support is something that does not fill me with horror... oh no I haven't just given myself another spare-time project that'll never get done, have I? ... you'd think I'd learn.

In particular, it's stateless, so if U2F stops working for a second while we do an OTP authentication, nothing bad happens (and it's physically impossible to do both at once, since both involve a button press).

Presumably it would be done similarly to how PKCS#11/PIV support already is, only rather than a PKCS11Provider, you'd specify an URL to an authentication server (obviously in some new ssh_config option), and if you wanted to forward things, you'd use an SSH agent and have ssh-add and agent forwarding do the work of getting to where your smartcard is actually plugged in.

Security in an error-prone world

Posted Nov 5, 2015 15:18 UTC (Thu) by apoelstra (subscriber, #75205) [Link] (2 responses)

> Oh God they recommend using gpg-agent as an ssh-agent replacement? I tried to get this working, but eventually gave up: it's devastatingly buggy. Among other problems, forwarding of the agent to other machines is only just barely kinda sorta working in GnuPG 2.1 and has no chance of working in 2.0; the eternal GPG_TTY bugs (are you sure you're resetting that in *every single* startup script? Without exception?) routinely lead to you being asked to unlock your key using a pinentry running on the wrong console, causing an apparent hang; even if that's fixed, if you ssh or su to another user and want it to use your smartcard, you suddenly find that you can't because the pinentry can't appear on that tty since it's owned by another user and not writable by the one the gpg-agent is running on... the list goes on and on. It's a mass of barely functional components held together with baling wire and twine.

Thanks for this. For years now I've just thought gpg-agent just "usually doesn't work" but never had an idea of what was going wrong or how to look into it. This paragraph provides many hints.

Security in an error-prone world

Posted Nov 5, 2015 22:15 UTC (Thu) by flussence (guest, #85566) [Link]

gpg-agent has *one* useful function in my daily routine: to work around gpg itself being broken out of the box. I run `gpg -s -o /dev/null /dev/null` once, then thereafter I can actually make git commits without gpg barfing with TTY errors.

It's completely FUBAR. I wish OpenBSD's alternative would catch on.

Security in an error-prone world

Posted Nov 10, 2015 16:25 UTC (Tue) by nix (subscriber, #2304) [Link]

I dug into this because, surely, if I fixed all the problems, I could get to the golden ideal of having all my SSH and GPG auth, on whatever machine, all handled via the same set of smartcarded keys, derived from the same offline master!

Sadly, this golden ideal remains unattainable :(