Fixed
Status Update
Comments
js...@chromium.org <js...@chromium.org> #2
[Empty comment from Monorail migration]
js...@chromium.org <js...@chromium.org> #3
I know when I looked into this four years ago the TLDs were extremely permissive, and whitelisting didn't seem viable. However, if Mozilla has seen a marked improvement then we should certainly consider using their whitelist approach.
I do think there's still value in falling back to a decent anti-spoofing heuristic for non-whitelisted TLDs. Ideally that would improve IDN coverage without adding any risk, since we've been comfortable with the security of the current heuristic for the last several years.
I do think there's still value in falling back to a decent anti-spoofing heuristic for non-whitelisted TLDs. Ideally that would improve IDN coverage without adding any risk, since we've been comfortable with the security of the current heuristic for the last several years.
js...@chromium.org <js...@chromium.org> #4
[Comment Deleted]
pk...@chromium.org <pk...@chromium.org> #5
What do you think of the heuristic they describe on that wiki page, which is based on character sets rather than UI languages?
js...@chromium.org <js...@chromium.org> #6
Ah, I stopped reading too soon. So, they still enforce a character blacklist, then they use the TLD whitelist, then they fall back to TR-39. It seems reasonable, but I haven't actually read through TR-39.
ge...@gmail.com <ge...@gmail.com> #7
Do let us know if we can help with understanding the algorithm.
We are fans of this approach because it has the significant advantage that IDNs that work, work everywhere - no matter which version or locale of Firefox (or, soon, Chrome :-) you are using. Without this property, IDNs will always remain a niche sideshow, because no site owner is going to risk some unknown percentage of his customers getting gibberish in the URL bar.
Gerv
We are fans of this approach because it has the significant advantage that IDNs that work, work everywhere - no matter which version or locale of Firefox (or, soon, Chrome :-) you are using. Without this property, IDNs will always remain a niche sideshow, because no site owner is going to risk some unknown percentage of his customers getting gibberish in the URL bar.
Gerv
mp...@chromium.org <mp...@chromium.org> #8
[Empty comment from Monorail migration]
pk...@chromium.org <pk...@chromium.org> #9
[Empty comment from Monorail migration]
pk...@chromium.org <pk...@chromium.org> #10
[Empty comment from Monorail migration]
pk...@chromium.org <pk...@chromium.org> #11
Unassigning. I still think this is important and should happen, but I'm not personally going to get to it in the foreseeable future.
js...@chromium.org <js...@chromium.org> #12
I'm going to look into this soonish because we definitely need to improve our IDN display policy (I thought I had filed a bug along the line of this bug, too, but couldn't find it ).
pa...@chromium.org <pa...@chromium.org> #13
jshin: Any update?
Adding talo for internationalization interest.
Adding talo for internationalization interest.
js...@chromium.org <js...@chromium.org> #14
I'm planning to start working on this late next week. Assigning to myself.
mg...@chromium.org <mg...@chromium.org> #15
[Empty comment from Monorail migration]
js...@chromium.org <js...@chromium.org> #16
Finally got here (much later than expected). ICU's uspoof API does most of the heavy lifting for us so that I can get rid of most of our codes.
I'll use 'highly restricitve' rather than 'moderately restrictive' (see http://unicode.org/reports/tr39) for now. What I really want to do is 'moderately restrive' with 'ASCII-only Latin' in place of 'Latin'.
I'll use 'highly restricitve' rather than 'moderately restrictive' (see
js...@chromium.org <js...@chromium.org> #17
The CL is almost ready except for one unresolved issue, namely, what to do about the whole-script confusable. I don't feel very comfortable doing what Mozilla does (that is, allowing them uncondtionally).
Currently, we do allow them if they're in a script (more exactly, they're covered by the exemplar character set of one of user's languages) as mentioned in
http://www.chromium.org/developers/design-documents/idn-in-google-chrome
-------
For example, the first component of http://сахар.ru is entirely Cyrillic, while the first component of http://caxap.ru is entirely Latin. If you claim to speak Russian or another Cyrillic language, both of these will be shown in IDN form.
----------
One possibility is to allow them only for white-listed TLDs. Either we can skip any spoof check at all for TLDs in the whitelist or we can allow whole-script confusable for TLDs in the whitelist after doing other checks. The former is simpler than the latter implementation-wise, but the latter is a bit more conservative ( The policies of some registrars linked at https://www.mozilla.org/en-US/about/governance/policies/security-group/tld-idn/ are not as stringent as the algorithm I'm planning to implement while other registras' policies are more restrictive.)
Another possibility is allowing a whole-script confusable label in non-Latin only if the exemplar set of one of 'Accept-Languages' covers all the letters in that label. This would make our IDN display policy still depend on A-L and still has the same issue as mentioned above about http://сахар.ru (in Cyrillic) vs http://caxap.ru (in Latin) with 'ru' (other languages written in Cyrillic whose exemplar sets include all the letters in the label) in the A-L list.
Currently, we do allow them if they're in a script (more exactly, they're covered by the exemplar character set of one of user's languages) as mentioned in
-------
For example, the first component of
----------
One possibility is to allow them only for white-listed TLDs. Either we can skip any spoof check at all for TLDs in the whitelist or we can allow whole-script confusable for TLDs in the whitelist after doing other checks. The former is simpler than the latter implementation-wise, but the latter is a bit more conservative ( The policies of some registrars linked at
Another possibility is allowing a whole-script confusable label in non-Latin only if the exemplar set of one of 'Accept-Languages' covers all the letters in that label. This would make our IDN display policy still depend on A-L and still has the same issue as mentioned above about
pk...@chromium.org <pk...@chromium.org> #18
I would do the simplest and least conservative thing of the choices you mention, but whatever is done, we should coordinate this discussion with Mozilla so we can mutually agree on the route forward, as they wanted our feedback and both of us want to be in sync going forward.
rs...@chromium.org <rs...@chromium.org> #19
Yeah, with Peter on that. Especially not keen on whitelisting TLDs - there's enough hassle going on with the PSL and ICANN that makes me think that with the new gTLD landrush, there's going to be greater and greater churn on IDN policies, which makes a whitelist either frustrating or unmanagable long-term (especially considering other applications' at Google desire to align on IDN policies).
I'm curious if you could explain more the disagreement on whole-script confusables, as Mozilla's position on the matter seems to make ample sense, and reflects the thoughts I had.
I'm curious if you could explain more the disagreement on whole-script confusables, as Mozilla's position on the matter seems to make ample sense, and reflects the thoughts I had.
js...@chromium.org <js...@chromium.org> #20
Mozilla currently does use whitelist TLDs, for which they skill all the checks they'd do otherwise.
They said that they'd deprecate it, but has been keeping the list (even though they also said that the list wouldn't be updated further).
The patchset #6 at https://codereview.chromium.org/1258813002/ does the following:
1. Use Highly restrictive rules for script mixing [1] with an added
restriction on Latin.
- Script mixing is only allowed with ASCII-Latin (instead
of any Latin) + Han + {Hangul, Bofomopo, Hiragana+Katagana}.
2. Allow the recommended and inclusion sets from UAX 39 [2]
3. Allow 5 aspirational scripts from UAX 31 [3]
4. Do not allow labels with two or more numbering systems mixed.
5. Do not allow invisible characters or a sequence of the same
combining mark.
6. Turn off whole script confusable check. It'd block 'pax' because
p,a, or x has a Cyrillic look-alike. The same is true of
'bücher' (German) or 'färgbolaget' (Swedish).
7. Block 4 Katakanas surrounded by other scripts because they could be
mistaken as a slash.
8. Block repeating Japanese combining marks (U+3099 - U+309C) even
when different ones come in sequence. If the same one comes in
a row, they'll be blocked by rule #5 above.
9. Allow two additional characters (U+2665 - Black Heart Suit and U+272A - Circled white star). Yeah, this is rather arbitrary (copied from Google's internal spoofing/email address filtering routine).
[1] http://www.unicode.org/reports/tr39/#Restriction_Level_Detection
[2] http://www.unicode.org/reports/tr39
http://www.unicode.org/Public/security/latest/xidmodifications.txt
[3] http://www.unicode.org/reports/tr31/#Aspirational_Use_Scripts
-----------------------
Comparison with Mozilla's :
a. Does not use whitelist TLDs (we may end up using : see https://crbug.com/chromium/336973#c16 above). All the IDN labels are subject to the same rules regardless of TLDs
b. Rules #2 ~ #6 are identical
c. Rule #1 (script mixing)
- Mozilla has not yet utilized new Unicode script extension properties to count
# of scripts or detect script mixing (apparently). We get this free from
ICU's uspoof API.
- We use 'highly restrictive' as opposed to 'moderately restrictive'. The difference is that we allows only the following script sets to be mixed with Latin: {Han, Bopomofo}, {Han,Hiragana, Katakana}, {Han, Hangul}. With moderately restrictive, all other scripts (single script) other than Greek, Cyrillic and Cherokee are allowed to mix with Latin.
- Latin allowed for script mixing above is limited to ASCII-Latin in my CL as opposed to any Latin.
- Mozilla has a user preference to select between highly restrictive and moderately restrictive. The default is moderately.
d. Rules #7 and #8 are only in my CL. They're kept from our current implementation. Those rules were added in response to security reports (of sort). I'm planning to propose them to the UTC (Unicode Technical Committee) for consideration.
e. Rule #9 (two additional characters) is only in my CL (copied from Google's internal code).
PS#6 does not do anything about the wholescript confusion (it's disabled). If you look at the Cyrillic code chart (a subset of Cyrillic in the Unicode) at http://www.unicode.org/charts/PDF/U0400.pdf, a lot of them look like Latin. I'm weary of disabling 'whilescript confusability' check all together (uncoditionally as Mozilla does and my PS#6 does ), which is why I wrote https://crbug.com/chromium/336973#c16.
They said that they'd deprecate it, but has been keeping the list (even though they also said that the list wouldn't be updated further).
The patchset #6 at
1. Use Highly restrictive rules for script mixing [1] with an added
restriction on Latin.
- Script mixing is only allowed with ASCII-Latin (instead
of any Latin) + Han + {Hangul, Bofomopo, Hiragana+Katagana}.
2. Allow the recommended and inclusion sets from UAX 39 [2]
3. Allow 5 aspirational scripts from UAX 31 [3]
4. Do not allow labels with two or more numbering systems mixed.
5. Do not allow invisible characters or a sequence of the same
combining mark.
6. Turn off whole script confusable check. It'd block 'pax' because
p,a, or x has a Cyrillic look-alike. The same is true of
'bücher' (German) or 'färgbolaget' (Swedish).
7. Block 4 Katakanas surrounded by other scripts because they could be
mistaken as a slash.
8. Block repeating Japanese combining marks (U+3099 - U+309C) even
when different ones come in sequence. If the same one comes in
a row, they'll be blocked by rule #5 above.
9. Allow two additional characters (U+2665 - Black Heart Suit and U+272A - Circled white star). Yeah, this is rather arbitrary (copied from Google's internal spoofing/email address filtering routine).
[1]
[2]
[3]
-----------------------
Comparison with Mozilla's :
a. Does not use whitelist TLDs (we may end up using : see
b. Rules #2 ~ #6 are identical
c. Rule #1 (script mixing)
- Mozilla has not yet utilized new Unicode script extension properties to count
# of scripts or detect script mixing (apparently). We get this free from
ICU's uspoof API.
- We use 'highly restrictive' as opposed to 'moderately restrictive'. The difference is that we allows only the following script sets to be mixed with Latin: {Han, Bopomofo}, {Han,Hiragana, Katakana}, {Han, Hangul}. With moderately restrictive, all other scripts (single script) other than Greek, Cyrillic and Cherokee are allowed to mix with Latin.
- Latin allowed for script mixing above is limited to ASCII-Latin in my CL as opposed to any Latin.
- Mozilla has a user preference to select between highly restrictive and moderately restrictive. The default is moderately.
d. Rules #7 and #8 are only in my CL. They're kept from our current implementation. Those rules were added in response to security reports (of sort). I'm planning to propose them to the UTC (Unicode Technical Committee) for consideration.
e. Rule #9 (two additional characters) is only in my CL (copied from Google's internal code).
PS#6 does not do anything about the wholescript confusion (it's disabled). If you look at the Cyrillic code chart (a subset of Cyrillic in the Unicode) at
js...@chromium.org <js...@chromium.org> #21
> I'm curious if you could explain more the disagreement on whole-script
> confusables, as Mozilla's position on the matter seems to make ample sense,
> and reflects the thoughts I had.
Perhaps, the difference is how much we can trust the registrars for doing the due diligence and how 'scared' I am about some of examples I created off-line :-).
I read the IDN policy of a few registrars and '.museum' policy deals with whole-script confusability, but '.com' policy (Verisign) does not mention it.
If everybody implements what '.museum' specified in its policy, it'd be great.
Anyway, a couple of alternatives (other than what I wrote in https://crbug.com/chromium/336973#c16) are:
1) Safe-browsing : Add spoofing check to our safe-browsing feature (server-side). Safe-browsing team can maintain a (huge) list of good domains and alert users when a user tries to navigate to look-alike (confusable) domains.
2) Spoof-checking against good domains (a smaller list than the server-side safe-browsing implementation) can be done as a part of our display policy implementation.
I think 1) is much better than 2) and my two other ideas in https://crbug.com/chromium/336973#c16.
While 1) is worked on, we may as well take a more conservative approach (like the first in https://crbug.com/chromium/336973#c16. I want the policy to be language-independent so that I'd rather avoid the second in https://crbug.com/chromium/336973#c16.). Once 1) is done, we can turn off whole-script confusability check.
> confusables, as Mozilla's position on the matter seems to make ample sense,
> and reflects the thoughts I had.
Perhaps, the difference is how much we can trust the registrars for doing the due diligence and how 'scared' I am about some of examples I created off-line :-).
I read the IDN policy of a few registrars and '.museum' policy deals with whole-script confusability, but '.com' policy (Verisign) does not mention it.
If everybody implements what '.museum' specified in its policy, it'd be great.
Anyway, a couple of alternatives (other than what I wrote in
1) Safe-browsing : Add spoofing check to our safe-browsing feature (server-side). Safe-browsing team can maintain a (huge) list of good domains and alert users when a user tries to navigate to look-alike (confusable) domains.
2) Spoof-checking against good domains (a smaller list than the server-side safe-browsing implementation) can be done as a part of our display policy implementation.
I think 1) is much better than 2) and my two other ideas in
While 1) is worked on, we may as well take a more conservative approach (like the first in
js...@chromium.org <js...@chromium.org> #22
Well, I might be a paranoid given that Mozilla has implemented their policy without whole-script confusability check since 2013 and there's no (major) spoofing attempt.
I'll talk to Mozilla folks (Gerv and Simon). I've already sent an email to the safe-browsing team.
I'll talk to Mozilla folks (Gerv and Simon). I've already sent an email to the safe-browsing team.
js...@chromium.org <js...@chromium.org> #23
oh, Gerv was here :-). Any input on my comments #16, 19, 20.
@jschuh : do you have any take ?
@jschuh : do you have any take ?
ma...@google.com <ma...@google.com> #24
(copied from Google's internal spoofing/email address filtering routine).
I'd suggest just allowing the emoji characters: I can point you to the data
file.
Mark Davis | Int’l Arch | markdavis@google.com | +1 650-450-9291 / +41
44-668-1282
I'd suggest just allowing the emoji characters: I can point you to the data
file.
Mark Davis | Int’l Arch | markdavis@google.com | +1 650-450-9291 / +41
44-668-1282
js...@chromium.org <js...@chromium.org> #25
Mark, I like to get the data. Can you upload it here (or give me a pointer if it's at unicode.org) or send me offline?
In the meantime, do you have any take on whole-script confusability handling?
In the meantime, do you have any take on whole-script confusability handling?
ma...@google.com <ma...@google.com> #26
The data is at http://unicode.org/Public/emoji/1.0/emoji-data.txt (ignore
the DRAFT, that's a typo that will be corrected soon).
The information is not in ICU; I filed a ticket for that:
http://bugs.icu-project.org/trac/ticket/11802
About the whole-script confusable: I think we need to support a more
refined API, because whether it has a whole-script confusable or not
depends on any character restrictions in play.
For example, the sequence "ցօօց1℮" is confusable with google, if displayed
in Noto Sans Armenian. Here's a screen shot showing that:
[image: Inline image 1]
That's because of the ℮, which is a symbol that may not be within the set
of allowed
identifier
characters.
Mark
Mark Davis | Int’l Arch | markdavis@google.com | +1 650-450-9291 / +41
44-668-1282
the DRAFT, that's a typo that will be corrected soon).
The information is not in ICU; I filed a ticket for that:
About the whole-script confusable: I think we need to support a more
refined API, because whether it has a whole-script confusable or not
depends on any character restrictions in play.
For example, the sequence "ցօօց1℮" is confusable with google, if displayed
in Noto Sans Armenian. Here's a screen shot showing that:
[image: Inline image 1]
That's because of the ℮, which is a symbol that may not be within the set
of allowed
identifier
characters.
Mark
Mark Davis | Int’l Arch | markdavis@google.com | +1 650-450-9291 / +41
44-668-1282
ma...@google.com <ma...@google.com> #27
Missing screen shot that was in email:
js...@chromium.org <js...@chromium.org> #28
"ցօօց1℮" with Armenian ℮ would be blocked because I'm planning to use 'highly restrictive' instead of 'moderately restrictive'. One reason for going with 'highly rest.' is that 'moderately restrictive' allows 'Latin + Armenian' mixing.
As for allowing all or a significant subset of Emoji, I'd rather not deal with that in this bug but deal with in in a follow-up bug.
As for allowing all or a significant subset of Emoji, I'd rather not deal with that in this bug but deal with in in a follow-up bug.
js...@chromium.org <js...@chromium.org> #29
oops. Sorry about the off-the-mark https://crbug.com/chromium/336973#c27.
I should have printed out the code points of ցօօց1℮.
They're U+0581, U+0585, U+0585, U+0581, U+0031, U+212e.
This particular example (fortunately?) would be flagged even without whole_script check because U+212E is not in the set ( recommended set + alowance set) I'm allowing.
Nonetheless, this just shows how tricky it can be.
I should have printed out the code points of ցօօց1℮.
They're U+0581, U+0585, U+0585, U+0581, U+0031, U+212e.
This particular example (fortunately?) would be flagged even without whole_script check because U+212E is not in the set ( recommended set + alowance set) I'm allowing.
Nonetheless, this just shows how tricky it can be.
js...@chromium.org <js...@chromium.org> #30
[Empty comment from Monorail migration]
js...@chromium.org <js...@chromium.org> #31
[Empty comment from Monorail migration]
mp...@chromium.org <mp...@chromium.org> #32
[Empty comment from Monorail migration]
js...@chromium.org <js...@chromium.org> #33
[Empty comment from Monorail migration]
pk...@chromium.org <pk...@chromium.org> #34
Jungshik, is https://crbug.com/chromium/505262 related to this as well? Your triage of that would be appreciated.
bu...@chromium.org <bu...@chromium.org> #35
The following revision refers to this bug:
https://chromium.googlesource.com/chromium/src.git/+/62a928390ba06db29576bbb32606696b3e16a66c
commit 62a928390ba06db29576bbb32606696b3e16a66c
Author: jshin <jshin@chromium.org>
Date: Fri Mar 18 18:42:39 2016
Implement a new IDN display policy
The new policy is language-indepedent, implemented with
ICU's uspoof API and is as following:
1. Use moderately restrictive rules for script mixing [1] with additional
restrictions on mixing with Latin.
- Script mixing is only allowed with ASCII-Latin (instead
of any Latin) + another script allowed at the moderatate
restriction level
2. Only allow the recommended sets from UTS 39 [2] and inclusion sets from
UAX 31 [3]. This is equivalent to [:IdentifierStatus=Allowed:] [4].
3. Allow 5 aspirational scripts from UAX 31 [5]
4. Do not allow labels with two or more numbering systems mixed.
5. Do not allow invisible characters or a sequence of the same
combining mark.
6. Turn off whole script confusable check. It'd block some common
domain labels like рф (IDN ccTLD for '.ru'),
'bücher' (German) and 'färgbolaget' (Swedish).
7. Keep ON 'mixed script confusable' check. This is different/separate
from 'script mixing restriction' and will catch cases like 'gօօgle'
with 'օ' (U+0585; Armenian Small Letter OH) [6] that would be otherwise
allowed by rules #1 ~ #5.
8. Block 4 Katakanas surrounded by non-Japanese scripts because they could be
mistaken as a slash. (this has been in place for a few years and is kept.)
9. Labels with any of four deviation characters (IDNA 2003 vs IDNA 2008)
encoded in punycode/ACE are always shown in Punycode. This is to make
the display policy consistent with our prior decision to use UTS 46
'transitional' processing (map or drop the 4 deviation characters.). [9]
10. Character black list (Mozilla's : [8]) is trimmed down to two characters.
Note that this is almost identical to Mozilla's IDN display algorithm
[7] except for #7, #8, and an additional restrictions in #1. #9 is another difference
because of Mozilla's use of UTS 46 'non-transitional' processing and our use of UTS 46 'transitional' processing.
Most of domains filtered out in ".com" TLD is filtered due to the
character set restrictiction (#2 and #3) that accounts for 94% (2,050)
of IDNs filtered out (0.2% of ~ 1 million IDNs in com TLD).
All the IDN TLDs are shown in Unicode. So are all the IDNs in the
effective TLD list, ".рф" (~ 860k), and ".みんな" (~25k).
48 out of 200k in ".xyz" and 3 out of 25k in ".jp" are filtered and shown
in punycode.
P.S. This CL keeps 'languages' parameter for the public APIs. I'll follow up
this CL with another to get rid of that parameter and adjust callers.
P.S.2: http://dev.chromium.org/developers/design-documents/idn-in-google-chrome will be updated after this CL is landed.
[1] http://www.unicode.org/reports/tr39/#Restriction_Level_Detection
[2] http://www.unicode.org/reports/tr39
http://www.unicode.org/Public/security/latest/xidmodifications.txt
[3] http://www.unicode.org/reports/tr31/#Table_Recommended_Scripts
[4] http://goo.gl/L3WD1s
[5] http://www.unicode.org/reports/tr31/#Aspirational_Use_Scripts
[6] http://unicode.org/cldr/utility/confusables.jsp?a=o&r=None
[7] https://wiki.mozilla.org/IDN_Display_Algorithm
[8] http://kb.mozillazine.org/Network.IDN.blacklist_chars : Most of them
are blocked or mapped any way by other restrictions/mechanism in place.
See https://bugzilla.mozilla.org/show_bug.cgi?id=1257108
[9] This is to "fix" https://crbug.com/chromium/595263
BUG=336973,595263
TEST=components_unittests --gtest_filter=*IDN*, --gtest_filter=UrlForm*,
--gtest_filter=*Puny*
Review URL: https://codereview.chromium.org/1258813002
Cr-Commit-Position: refs/heads/master@{#382029}
[modify] https://crrev.com/62a928390ba06db29576bbb32606696b3e16a66c/components/omnibox/browser/history_url_provider_unittest.cc
[modify] https://crrev.com/62a928390ba06db29576bbb32606696b3e16a66c/components/url_formatter/url_formatter.cc
[modify] https://crrev.com/62a928390ba06db29576bbb32606696b3e16a66c/components/url_formatter/url_formatter.h
[modify] https://crrev.com/62a928390ba06db29576bbb32606696b3e16a66c/components/url_formatter/url_formatter_unittest.cc
[modify] https://crrev.com/62a928390ba06db29576bbb32606696b3e16a66c/url/url_canon_unittest.cc
commit 62a928390ba06db29576bbb32606696b3e16a66c
Author: jshin <jshin@chromium.org>
Date: Fri Mar 18 18:42:39 2016
Implement a new IDN display policy
The new policy is language-indepedent, implemented with
ICU's uspoof API and is as following:
1. Use moderately restrictive rules for script mixing [1] with additional
restrictions on mixing with Latin.
- Script mixing is only allowed with ASCII-Latin (instead
of any Latin) + another script allowed at the moderatate
restriction level
2. Only allow the recommended sets from UTS 39 [2] and inclusion sets from
UAX 31 [3]. This is equivalent to [:IdentifierStatus=Allowed:] [4].
3. Allow 5 aspirational scripts from UAX 31 [5]
4. Do not allow labels with two or more numbering systems mixed.
5. Do not allow invisible characters or a sequence of the same
combining mark.
6. Turn off whole script confusable check. It'd block some common
domain labels like рф (IDN ccTLD for '.ru'),
'bücher' (German) and 'färgbolaget' (Swedish).
7. Keep ON 'mixed script confusable' check. This is different/separate
from 'script mixing restriction' and will catch cases like 'gօօgle'
with 'օ' (U+0585; Armenian Small Letter OH) [6] that would be otherwise
allowed by rules #1 ~ #5.
8. Block 4 Katakanas surrounded by non-Japanese scripts because they could be
mistaken as a slash. (this has been in place for a few years and is kept.)
9. Labels with any of four deviation characters (IDNA 2003 vs IDNA 2008)
encoded in punycode/ACE are always shown in Punycode. This is to make
the display policy consistent with our prior decision to use UTS 46
'transitional' processing (map or drop the 4 deviation characters.). [9]
10. Character black list (Mozilla's : [8]) is trimmed down to two characters.
Note that this is almost identical to Mozilla's IDN display algorithm
[7] except for #7, #8, and an additional restrictions in #1. #9 is another difference
because of Mozilla's use of UTS 46 'non-transitional' processing and our use of UTS 46 'transitional' processing.
Most of domains filtered out in ".com" TLD is filtered due to the
character set restrictiction (#2 and #3) that accounts for 94% (2,050)
of IDNs filtered out (0.2% of ~ 1 million IDNs in com TLD).
All the IDN TLDs are shown in Unicode. So are all the IDNs in the
effective TLD list, ".рф" (~ 860k), and ".みんな" (~25k).
48 out of 200k in ".xyz" and 3 out of 25k in ".jp" are filtered and shown
in punycode.
P.S. This CL keeps 'languages' parameter for the public APIs. I'll follow up
this CL with another to get rid of that parameter and adjust callers.
P.S.2:
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
are blocked or mapped any way by other restrictions/mechanism in place.
See
[9] This is to "fix"
BUG=336973,595263
TEST=components_unittests --gtest_filter=*IDN*, --gtest_filter=UrlForm*,
--gtest_filter=*Puny*
Review URL:
Cr-Commit-Position: refs/heads/master@{#382029}
[modify]
[modify]
[modify]
[modify]
[modify]
js...@chromium.org <js...@chromium.org> #36
I'll update the document. Moreover, I'll get rid of an unused input argument (language) in the public API and fix callers.
bu...@chromium.org <bu...@chromium.org> #37
The following revision refers to this bug:
https://chromium.googlesource.com/chromium/src.git/+/1fb76469391f6e7e2a846c27a095d57b54b50738
commit 1fb76469391f6e7e2a846c27a095d57b54b50738
Author: jshin <jshin@chromium.org>
Date: Tue Apr 05 22:13:03 2016
Drop |languages| from FormatUrl*() and ElideUrl()
In http://crrev.com/382029, IDN display policy was revised to be
independent of Accept-Languages (A-L). Therefore, IDNToUnicode(), FormatUrl*()
and ElideUrl() do not need |languages| argument any more.
( ElideUrl() can pass A-L down to gfx string drawing/measuring API help
their font selection, but there's no gfx string drawing/measuring API taking
into account A-L values in the font selection atm.)
Drop it in the API and fix all the call sites. There is a huge ripple effect
and a lot of files have to be fixed up.
BUG=336973
TEST=Builds on all platforms and pass all the tests pass
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_site_isolation
CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel,mac_blink_rel,win_blink_rel
Review URL: https://codereview.chromium.org/1841653003
Cr-Commit-Position: refs/heads/master@{#385299}
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/android_webview/browser/aw_javascript_dialog_manager.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/android_webview/browser/aw_javascript_dialog_manager.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/android/fullscreen/fullscreen_infobar_delegate.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/android/omnibox/autocomplete_controller_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/android/omnibox/autocomplete_controller_android.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/android/password_ui_view_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/android/url_utilities.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/autocomplete/in_memory_url_index_factory.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/autocomplete/search_provider_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/bookmarks/bookmark_model_factory.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/chrome_quota_permission_context.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/content_settings/content_settings_usages_state_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/content_settings/tab_specific_content_settings.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/custom_home_pages_table_model.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/dom_distiller/tab_utils_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/download/download_query.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/download/notification/download_item_notification.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/engagement/site_engagement_service_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/extensions/api/bookmarks/bookmarks_api.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/extensions/api/passwords_private/passwords_private_delegate_impl.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/extensions/api/passwords_private/passwords_private_delegate_impl.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/extensions/api/sessions/sessions_api.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/geolocation/geolocation_infobar_delegate_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/geolocation/geolocation_infobar_delegate_android.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/history/android/android_provider_backend_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/history/history_service_factory.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/history/history_tab_helper.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/interstitials/security_interstitial_page.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/media/media_stream_capture_indicator.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/media/midi_permission_infobar_delegate_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/media/midi_permission_infobar_delegate_android.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/media/protected_media_identifier_infobar_delegate_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/media/protected_media_identifier_infobar_delegate_android.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/notifications/notification_permission_infobar_delegate.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/notifications/notification_permission_infobar_delegate.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/password_manager/account_chooser_dialog_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/pepper_broker_infobar_delegate.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/pepper_broker_infobar_delegate.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/permissions/permission_bubble_request_impl.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/permissions/permission_bubble_request_impl.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/permissions/permission_context_base.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/permissions/permission_queue_controller.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/push_messaging/push_messaging_notification_manager.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/renderer_context_menu/render_view_context_menu.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/safe_browsing/incident_reporting/last_download_finder_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/safe_browsing/incident_reporting/off_domain_inclusion_detector_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/search_engines/ui_thread_search_terms_data.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/search_engines/ui_thread_search_terms_data.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ssl/bad_clock_blocking_page.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ssl/captive_portal_blocking_page.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ssl/ssl_blocking_page.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/android/bluetooth_chooser_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/android/usb_chooser_dialog_android.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/bookmarks/bookmark_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/bookmarks/bookmark_utils.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/browser.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/cocoa/bookmarks/bookmark_editor_controller.mm
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/cocoa/status_bubble_mac.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/cocoa/status_bubble_mac.mm
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/cocoa/status_bubble_mac_unittest.mm
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/content_settings/content_setting_bubble_model.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/content_settings/content_setting_bubble_model_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/login/login_handler.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/passwords/manage_passwords_view_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/passwords/password_manager_presenter.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/passwords/password_manager_presenter.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/passwords/password_manager_presenter_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/search_engines/search_engine_tab_helper.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/status_bubble.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/toolbar/back_forward_menu_model.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/toolbar/chrome_toolbar_model_delegate.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/toolbar/chrome_toolbar_model_delegate.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/bookmarks/bookmark_bar_view.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/bookmarks/bookmark_bar_view.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/bookmarks/bookmark_editor_view.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/bookmarks/bookmark_menu_delegate.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/omnibox/omnibox_view_views_browsertest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/status_bubble_views.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/status_bubble_views.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/views/website_settings/permissions_bubble_view.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/website_settings/website_settings.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/webui/browsing_history_handler.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/webui/browsing_history_handler.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/webui/options/password_manager_handler.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/browser/ui/webui/options/password_manager_handler.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/common/net/x509_certificate_model.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/renderer/net/net_error_helper.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/test/base/testing_profile.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/chrome/test/base/ui_test_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/app_modal/javascript_dialog_manager.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/app_modal/javascript_dialog_manager.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_expanded_state_tracker_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_index.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_index.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_model.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_model.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_utils.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/browser/bookmark_utils_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/managed/managed_bookmarks_tracker_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/bookmarks/test/test_bookmark_client.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/browser_sync/browser/profile_sync_service_bookmark_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/content_settings/core/browser/content_settings_usages_state.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/content_settings/core/browser/content_settings_usages_state.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/error_page/common/localized_error.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/error_page/common/localized_error.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/error_page/renderer/net_error_helper_core.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/error_page/renderer/net_error_helper_core.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/error_page/renderer/net_error_helper_core_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_backend.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_backend.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_backend_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_querying_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_service.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_service.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/history_service_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/top_sites_impl_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/typed_url_syncable_service_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/browser/url_database.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/test/history_backend_db_base_test.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/test/history_service_test_util.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/history/core/test/history_service_test_util.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/nacl/browser/nacl_process_host.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_controller.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_match.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_match.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_match_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_result.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_result.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/autocomplete_result_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/base_search_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/bookmark_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/bookmark_provider.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/clipboard_url_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_quick_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_quick_provider.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_quick_provider_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_url_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_url_provider.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/history_url_provider_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/in_memory_url_index.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/in_memory_url_index.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/in_memory_url_index_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/scored_history_match.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/scored_history_match.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/scored_history_match_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/search_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/search_suggestion_parser.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/search_suggestion_parser.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/shortcuts_backend_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/shortcuts_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/shortcuts_provider.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/shortcuts_provider_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/url_index_private_data.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/url_index_private_data.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/omnibox/browser/zero_suggest_provider.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/affiliation_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/affiliation_utils.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/affiliation_utils_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/password_ui_utils.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/password_ui_utils.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/password_manager/core/browser/password_ui_utils_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/keyword_table.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/search_terms_data.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/search_terms_data.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/template_url.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/template_url.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/template_url_parser.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/template_url_service.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/search_engines/template_url_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/bad_clock_ui.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/bad_clock_ui.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/common_string_util.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/common_string_util.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/ssl_error_ui.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/security_interstitials/core/ssl_error_ui.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/sync_bookmarks/bookmark_data_type_controller_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/toolbar/toolbar_model_delegate.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/toolbar/toolbar_model_impl.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/elide_url.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/elide_url.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/elide_url_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/url_fixer.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/url_formatter.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/url_formatter.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/components/url_formatter/url_formatter_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/content/browser/download/save_package.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/content/browser/download/save_package.h
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/content/browser/download/save_package_unittest.cc
[modify] https://crrev.com/1fb76469391f6e7e2a846c27a095d57b54b50738/content/browser/frame_host/navigation_controller_impl_unittest.cc
[modify] https://crrev.com/1
commit 1fb76469391f6e7e2a846c27a095d57b54b50738
Author: jshin <jshin@chromium.org>
Date: Tue Apr 05 22:13:03 2016
Drop |languages| from FormatUrl*() and ElideUrl()
In
independent of Accept-Languages (A-L). Therefore, IDNToUnicode(), FormatUrl*()
and ElideUrl() do not need |languages| argument any more.
( ElideUrl() can pass A-L down to gfx string drawing/measuring API help
their font selection, but there's no gfx string drawing/measuring API taking
into account A-L values in the font selection atm.)
Drop it in the API and fix all the call sites. There is a huge ripple effect
and a lot of files have to be fixed up.
BUG=336973
TEST=Builds on all platforms and pass all the tests pass
CQ_INCLUDE_TRYBOTS=tryserver.chromium.linux:linux_site_isolation
CQ_INCLUDE_TRYBOTS=tryserver.blink:linux_blink_rel,mac_blink_rel,win_blink_rel
Review URL:
Cr-Commit-Position: refs/heads/master@{#385299}
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
[modify]
la...@chromium.org <la...@chromium.org> #39
Security>UX component is deprecated in favor of the Team-Security-UX label
[Monorail components: -Security>UX]
[Monorail components: -Security>UX]
is...@google.com <is...@google.com> #40
This issue was migrated from crbug.com/chromium/336973?no_tracker_redirect=1
[Multiple monorail components: UI>Browser>Omnibox, UI>Internationalization]
[Monorail blocking: crbug.com/chromium/496473, crbug.com/chromium/575007]
[Monorail mergedwith: crbug.com/chromium/286212, crbug.com/chromium/551479, crbug.com/chromium/90364, crbug.com/chromium/93537]
[Monorail components added to Component Tags custom field.]
[Multiple monorail components: UI>Browser>Omnibox, UI>Internationalization]
[Monorail blocking:
[Monorail mergedwith:
[Monorail components added to Component Tags custom field.]
Description
Mozilla spent a while defining and then implementing what they hope is a better algorithm. This is covered by
In the interests of browser consistency, and because I think Mozilla's algorithm is better for users due to relying less on user-configured, and often erroneous, language settings, we should implement this algorithm.
If we find tweaks or problems, we can dialog with Mozilla about them, but I'd like to get to an end state where we can agree on a common algorithm and then see that implemented across other browsers as well.
Interested parties should feel free to take this bug. I've assigned to myself so as not to lose it, but I may not get to this any time soon, and I think with the rapid upcoming growth in IDNs, this is going to become more important.