Should users be allowed to use any special character they want when creating a password?

  • I came across a number of login configuration settings where there is a list of allowable special characters and was wondering:

    Does this limitation cater for a specific security or usability need?

    Example: A list of special characters supported by Oracle Identity Manager and Microsoft Active Directory for password field :

    enter image description here

    Update:

    Thanks everyone for the generous response!

    Every time I have asked a question that involves security and usability there seems to be a clear divide between proponents on each side. However this need not be as this is one area that requires a lot of compromises and trade-offs… UX depends on it!

    Consider masked password entry. Since it is password it will go through rigorous testing. By limiting it to certain characters the number of test cases is more manageable. Clearly you don't want to allow beep or tab down. There are a number of control type charters that don't belong in passwords.

    The limitation generally caters for implementer-laziness, because they're worried that allowing arbitrary characters might break something elsewhere (which usually means they have bigger problems). Limiting passwords to specific characters drives me crazy, and there's no (good) reason for it ever (unless you're worried that not doing it will lead to injection attacks, in which case, fix the code you're worried about injection attacks in, and *then* fix your password validation).

    UX people, developers are not the enemy (nor lazy). For something like a password would wire up a test cases for all possible combinations. There are only so many CPU cycles in a day. @neminem What would be the purpose of allowing a beep or tab down? What would be the purpose of a character not on the keyboard?

    @Blam The purpose would be not limiting your password, without having to hardcode anything. Who's going to put a non-keyboard symbol in, and if they tried, so what? By contrast, if I were, say, Chinese, and had a Chinese keyboard (or for that matter, French with a French one), I'd be sort of annoyed if my password had to be in English (because Chinese symbols are "not on the [standard American] keyboard", either.) I would argue you really don't have to test every possible password, just test "it works in Unicode", done. (And make sure you can't get sql injection attacked.)

    p.s. I'm not a UX person, I'm a developer. :p But more importantly, I'm a *user*. It drives me crazy when sites do this - just that, as a developer, I know why they're usually doing it. :p

    @neminem Just test in unicode? Unicode has 60 some thousand characters. Really you know why Microsoft Active Directory is limited. You know for a fact it is because they were lazy. And if you were a developer you would know you don't protect from SQL infection attack by limiting characters - you do it with parameterized queries. And a hash is binary - it is not even character aware.

    @Blam Right, obviously you don't have to test every possible unicode character, you test a random sampling of them, same as you don't have to test every string length, you test 0, 1, a few, and a random large number. And yes, *smart* people don't protect from injection attacks by limiting characters... but that doesn't mean that people don't still try protecting from injection attacks by limiting characters (usually, as Nathan Rabe points out, the same people also most likely to end up on plaintextoffenders.)

    @neminem First it was test unicode and now it is test a sample. Really you would allow control characters like beep and down tab. You would allow smart quotes when most people don't know what they are. You would allow visual characters that have more than one unicode. You would allow ½ when some normalization would break that out. Maybe YOU don't fully test what goes in a password but I do.

    @neminem: Unicode has a lot of tricky rules and edge cases. In the old days, code could be character-set agnostic for any sequence of bytes that didn't contain nulls, but that's no longer the case. For usages other than passwords, the fact that a database sometimes represents "mañana" as six code points and sometimes represents it as seven may be an annoyance, but it can be resolved by having a search for "mañana" look for both forms. Even if a string has many characters that could appear in multiple forms, it may be possible to search for all forms simultaneously.

    @neminem: Unfortunately, such an approach doesn't really work for passwords. If a password contains eight characters, each of which might appear in two different forms, the only way to find out if it is equivalent to a particular hashed password is to try hashing all 256 combinations of characters to see if any of them work. Nasty.

    Microsoft Active Directory does allow "!".

    "The grave accent cannot be reproduced in this document" --- seriously? What half-baked markup language are they writing it in?

    If there is a password character restriction, there is a very good chance that they are storing your password unhashed.

    One VERY important thing to keep in mind when allowing arbitrary characters is NORMALIZATION. If two strings contain the same characters, but are normalized differently, their hashes will be different.

    @FedericoPoloni: Actually, that makes sense. TRWTF is the fact that they chose a grave accent in the first place, not to mention the following statement that it's "also known as the backquote character". What nonsense. A backtick and an accent on a character are two very different things. They should have written "backtick" in the first place and stuck with it... and _then_ they'd be able to reproduce it. :)

    @Blam Unicode is a 21-bit character set. It has far more than "60 some thousand characters", although most of the 2^21 code points are not allocated, and may never will be.

    @DavidConrad Yes, so. It is bigger than 128. A lot of the lookups are like in the 60000

    From my school: "The dis-allowed characters are mostly to prevent cleverly-constructed passwords from causing harm to Unix-based systems. [...] There are, of course, other ways to guard against those sorts of problems, but keeping certain characters out of passwords in the first place is the simplest, and can be controlled centrally. That's in stark contrast to the way Unix/Linux systems that use netid authentication are managed - by numerous schools, departments or individuals. It would be prohibitively difficult to ensure that each application & server had all the right safeguards in place."

    The more characters available to a user, the harder it is to guess their password. If you were trying to brute force a password and knew that the user couldn't use the @ symbol, you could automatically rule out every string that contains an @ symbol, whereas with the @ symbol accepted there's a lot more combinations the brute forcer would have to go through.

    If we are following the principle that four dictionary words is enough entropy to create a strong password, then all you need is lowercase letters (maybe not even space). Tell everyone: "type four uncommon words without spaces, caps, digits, special characters, etc and it will be good enough." That solves all the problems.

  • DaveAlger

    DaveAlger Correct answer

    6 years ago

    If the user can type it then it should be allowed in their password.

    Telling someone what they can and can't use in their password always feels wrong to the user. Passwords are currently the most universal way to authenticate. Preventing users from entering anything is, in essence, telling them who they can or can't be.

    1. Any printable character that a user inputs should be allowed.

    The following characters are okay...

    'A', 'a', 'á', 'Æ', 'æ', 'Ñ', 'ñ', '-', '_', ' ' (space), '\t' (tab), '\n' (newline), ...
    

    Just because I don't know how to submit a TAB or ENTER character as part of my password doesn't give me the right to prevent others from doing so.
    (Don't worry, few people will try to submit an ENTER character as part of their password but allowing the few that do will earn their respect.)

    2. Keys that don't display a printable character should not be allowed.

    The reasons for this should be obvious but for completeness I will mention the following keys which can be detected but are reserved for other actions. For example, a password input shouldn't record that the shift was hit multiple times...

    [ctrl], [alt], [shift], [arrow keys], [apple key], [windows key], etc.
    

    3. Not allowing certain characters makes users question your security.

    When you prevent users from putting certain characters in their password it not only annoys people but causes many of them to question what else you are doing that isn't secure.

    You may as well be saying...

    "Hey we don't want to fix our application to properly deal with special characters so would you mind helping us out by making your password less secure?"

    The rules below will allow for secure input while preventing a user from ever getting stuck:

    • Don't show the characters that the user is typing in password fields (there are some exceptions on mobile)

    • Having the user type in their password twice is usually sufficient in letting them know that they got it right (i.e. didn't accidentally add unintended white space etc.)

    • Having a password reset mechanism is important to handle any cases of accidental lockout.

    4. Encouraging a user to add more isn't the same as prohibiting characters.

    One way to help a user come up with a secure password is to make a game out of it...

    password strength indicator

    5. The future of authentication

    "The Tech That Will Kill Passwords Dead" is a pretty good gizmodo article discussing the problems we all face with passwords. It also talks about some new patterns that could possibly replace passwords one day.

    Many mobile applications are starting to allow users to show or hide passwords in plain text in order to increase ease of use and remove one barrier to entry. I would still avoid this because the problem it creates is worse than the problem it solves.

    Even with a very intuitive mechanism for showing/hiding a password 60% of users still say it feels wrong to see passwords in clear text.

    According to that same article it appears that Touch ID is on the right track and easily wins as far as usability is concerned. Touch ID still has some major problems that make it impractical. The biggest being that it only works on select devices and has issues with one person controlling multiple accounts.

    Facial recognition is another contender as an increasing number of high pixel density cameras make their way into the world but this approach often leaves people worrying about privacy.

    The one problem shared by all of these new authentication attempts is this: You are the password.

    It's actually a lot easier to fake who you are than what you know. In addition, once you've been compromised it's nearly impossible to change (your fingerprints for example)

    Passwords are the authentication mechanism of choice for a good long while so...

    Please don't place arbitrary character limitations on my password. Thanks!

    Sites that only allow certain characters always make me worry that they are storing/transmitting the text of my actual password instead of a hash.

    @NathanRabe I would think the opposite. If they are confining it I would think more controls in place.

    @Blam If a site says I can't use a character like @, %, $, or ; in my password, to me that means some script will be reading my password and they know those characters will mess something up. Even if the only time they parse the characters is to generate a hash, it seems like bad coding if it was possible to inject code via a password and all they did to fix it was disallow certain characters.

    @NathanRabe It was just a comment and my opinion. I am they - I am a developer.

    @NathanRabe Once there was a site that didn't allow '&' because the password was sent via GET, and it fucked up the URL string. It was surprising when the 'dev' told me he didn't know of `encodeURIComponent`... even though the password _was_ stored hashed.

    Even tabs or other non printable characters should be allowed; most users can't type them and so won't use them, but if someone is copy pasting or it's an automated script which uses those then let them do so.

    Just because it's a key doesn't mean it's a character (ctrl, shift, arrows, etc.); but if it's a character it should generally be allowable (including tab). If I want `§™╔┌çÅ╡Θ` as my password, I should be permitted to do so. I would probably make exception for non-printing characters such as ␀, ␇, ␌ and such (and the backspace character, because we want people to be able to edit a mistyped password!), but "non-printing" does _not_ include the likes of ␉! (horizontal tab)

    Updated: I agree that even `tab` shouldn't be prevented if you can figure out how to get that character in there then go for it!

    @AndréDaniel What should the we do with this password: `12345\b67890` where `\b` is the backspace character? What about this one: `12345\067890` where `\0` is the null character, which is used to *terminate strings*? Control characters are intended to do weird things with text that are *not* appropriate in a password. If a user copy/pastes something that contains strange non-printable characters, they may not even be aware that they're present. Best to exclude non-printing characters. Then the user knows exactly what's in the password, and whoever wrote that bogus nonsense can fix their stuff.

    @DaveAlger It's fine to exclude characters that are used for behavior on a web page, such as tab which will move focus to another field. I'd exclude newline characters for this reason, too, since Enter should usually submit a form. Excluding characters that will actively detract from a normal user experience (by making it overly difficult to type the password) is fine. At a minimum, I'd say warn the user that it could cause problems.

    Really anything I can type? How about duplicates? These are not the same character - ☉⊙. How about diacritic (can be broken out)? Characters not supported in HTML? How about normalization in some environments that breaks out ½ into 1/2. A password from a character set of 20 is not less secure than a password from a character set of 2000 - just might need to use more characters. Numbered swiss bank accounts are numbered. A 32 character hash of a password is represented as 0-9 A-F. Actually the hash is binary - 0 or 1. Allow characters that are problematic is not good UX.

    @jpmc26 it doesn't matter whether the characters can be typed or produce weird stuff with text. If the user submitted them successfully then it means he is able to type/enter them and the system should accept that password just fine. The system should accept any block of data as the password, the only constraint must be length so that attackers can't upload 1GB files and DoS the server by forcing it to hash that enormous file.

    @AndréDaniel Yeah, until they go to type the password without the control characters and can't log in because they didn't know the invisible characters were there. That sounds like great UX, doesn't it? You're not talking about arbitrary data in a file. You're talking about designing your system for *typing* the password in. Control characters are simply not helpful for that. Sure, if you wanna use key files that they upload or something, go for it, but as long as you're talking typing, control characters are not helpful.

    @DaveAlger agree with your response, except that having a function to unmask the password typed instead of asking user to confirm their password would be particularly helpful in this case _as users make more errors when they can't see what they're typing_ ultimately this helps in avoiding login failures.

    @DaveAlger would you know if smartphone soft-keyboards have all the characters that exist on the desktop keyboards? If they are not the same, then it might be useful to exclude those characters that are not available in the soft-keyboard on the smartphones.

    @jpmc26: At most, you might warn the user that their password could be hard to type. But I, for example, use a password manager. I don't type passwords! If I have to type a password, *something has already gone wrong in UX*. The sorts of power users that copy-paste generated passwords are often those that then use the same automated tools to enter those passwords; forbidding weird characters is a misguided attempt at coddling those who are deliberately taking security and usability into their own hands.

    @jpmc26 great UX is not when the system makes me loose time just to accommodate some idiot who has weird characters in his password (and I really don't see how this can happen; if he's typing the pass each time then he has no way of entering those chars when setting the password, and if he has a "passwords.xls" file then he's copy/pasting and even if there are weird chars they'll still get copied just fine). The system should consider whatever was submitted when setting the pass as a key, and require that exact same key when logging in; even binary data should be accepted.

    +1 except for limiting password length to 64 characters. As the owner of the server, there is really only one concern which should limit password length: the time & memory it takes to process. Using modern computers and hashing algorithms means limiting the password size to 1000 characters or even more should not have any recognizable impact on your service.

    I'm open to update the max password length with some data backing it. Limiting the length is certainly not to help the user it is done to protect yourself. A lot of attacking bits can fit in 1000 characters. Microsoft says that 16 characters has been the limit for years though I think this is stupidly short. -- http://thenextweb.com/microsoft/2012/09/21/this-ridiculous-microsoft-longer-accepts-long-passwords-shortens/

    Thanks for the help fleshing out this answer everyone. I'm always looking for a more user friendly way of identifying you as you. Passwords are what we have right now. Limiting passwords without good reason is like telling someone who they can or can't be.

    @AndréDaniel Never let your inability to envision a user doing something fool you. All that would be required is that they copy/paste the password from one place and then try to type it in when the original source is unavailable, thinking they know what they typed in. In today's multi-device world, that could easily happen. I like this quote: "So why would a user do that? Because the user could do that." -Why Would a Baseball Player Do That? Also, if you have binary, you can just convert it to hex for the password.

    What's a standard keyboard? - I don't have most of the characters you mentioned on mine (e.g. Ñ).

    @daveAlger Just read the article in your response, really interesting! Guess the devil is in the detail! so it all boils down to how password masking is implemented.

    @NathanTuggy I would never set a password that wasn't typeable on both a PC and a phone. There are always edge cases like websites that won't let you download pdf etickets on their mobile site (and the button is off-screen if you force the desktop version) so you have to log on to someone else's machine and print. If you use e.g. KeePass you can easily display the password on your phone and retype on the borrowed box. You've entered a password on an untrusted box, but not a master password of any kind. (I have seen all the fails I mention, luckily not all at once).

    "Chances are anyone doing this knows more than you" This is an assumption, and a bad one at that. I've seen plenty of ways for a user to accidentally/unintentionally include whitespace characters when setting up a password, and thus later when they try to enter the password they don't know why it doesn't work. I certainly agree with the *spirit* of your answer in terms of not wanting to get in the way of what the user wants to do, but there are certain limitations in passwords intended to avoid common problems.

License under CC-BY-SA with attribution


Content dated before 7/24/2021 11:53 AM