maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/test/Lexer/unicode.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

89 lines
3.6 KiB
C
Raw Normal View History

Add test that we correctly allow some non-letter unicode characters in identifiers, and extend existing test to also cover C++. llvm-svn: 248079
2015-09-19 10:14:12 +08:00
// RUN: %clang_cc1 -fsyntax-only -verify -x c -std=c11 %s
// RUN: %clang_cc1 -fsyntax-only -verify -x c++ -std=c++11 %s
Use a more general test here. The interesting bit about that triple isn't the architecture, it's the fact that ps4 implies C99 as the standard rather than a newer C mode. Specify the language standard rather than the triple so the test is a bit more general.
2021-08-18 21:30:36 +08:00
// RUN: %clang_cc1 -std=c99 -E -DPP_ONLY=1 %s | FileCheck %s --strict-whitespace
Do not emit diagnostics for invalid unicode characters in preprocessing mode This amends 4e80636db71a1b6123d15ed1f9eda3979b4292de with a fix for https://lab.llvm.org/buildbot/#/builders/139/builds/8943
2021-08-18 21:10:34 +08:00
// RUN: %clang_cc1 -E -DPP_ONLY=1 %s | FileCheck %s --strict-whitespace
As an extension, treat Unicode whitespace characters as whitespace. llvm-svn: 173370
2013-01-25 04:50:50 +08:00
// This file contains Unicode characters; please do not "fix" them!
extern int x; // expected-warning {{treating Unicode character as whitespace}}
extern int　x; // expected-warning {{treating Unicode character as whitespace}}
Don't warn about Unicode characters in -E mode. People use the C preprocessor for things other than C files. Some of them have Unicode characters. We shouldn't warn about Unicode characters appearing outside of identifiers in this case. There's not currently a way for the preprocessor to tell if it's in -E mode, so I added a new flag, derived from the PreprocessorOutputOptions. This is only used by the Unicode warnings for now, but could conceivably be used by other warnings or even behavioral differences later. <rdar://problem/13107323> llvm-svn: 173881
2013-01-30 09:52:57 +08:00
// CHECK: extern int {{x}}
// CHECK: extern int　{{x}}
Lexer: Don't warn about Unicode in preprocessor directives. This allows people to use Unicode in their #pragma mark and in macros that exist only to be string-ized. <rdar://problem/13107323&13121362> llvm-svn: 174081
2013-02-01 03:48:48 +08:00
#pragma mark ¡Unicode!
#define COPYRIGHT Copyright © 2012
#define XSTR(X) #X
#define STR(X) XSTR(X)
static const char *copyright = STR(COPYRIGHT); // no-warning
// CHECK: static const char *copyright = "Copyright © {{2012}}";
Don't warn about Unicode characters in -E mode. People use the C preprocessor for things other than C files. Some of them have Unicode characters. We shouldn't warn about Unicode characters appearing outside of identifiers in this case. There's not currently a way for the preprocessor to tell if it's in -E mode, so I added a new flag, derived from the PreprocessorOutputOptions. This is only used by the Unicode warnings for now, but could conceivably be used by other warnings or even behavioral differences later. <rdar://problem/13107323> llvm-svn: 173881
2013-01-30 09:52:57 +08:00
#if PP_ONLY
Lexer: Don't warn about Unicode in preprocessor directives. This allows people to use Unicode in their #pragma mark and in macros that exist only to be string-ized. <rdar://problem/13107323&13121362> llvm-svn: 174081
2013-02-01 03:48:48 +08:00
COPYRIGHT
// CHECK: Copyright © {{2012}}
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
CHECK : The preprocessor should not complain about Unicode characters like ©.
Don't warn about Unicode characters in -E mode. People use the C preprocessor for things other than C files. Some of them have Unicode characters. We shouldn't warn about Unicode characters appearing outside of identifiers in this case. There's not currently a way for the preprocessor to tell if it's in -E mode, so I added a new flag, derived from the PreprocessorOutputOptions. This is only used by the Unicode warnings for now, but could conceivably be used by other warnings or even behavioral differences later. <rdar://problem/13107323> llvm-svn: 173881
2013-01-30 09:52:57 +08:00
#endif
Add test that we correctly allow some non-letter unicode characters in identifiers, and extend existing test to also cover C++. llvm-svn: 248079
2015-09-19 10:14:12 +08:00
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
int _;
[Lex] Don't assert when decoding invalid UCNs. Currently if a lexically-valid UCN encodes an invalid codepoint, then we diagnose that, and then hit an assertion while trying to decode it. Since there isn't anything preventing us reaching this state, remove the assertion. expandUCNs("X\UAAAAAAAAY") will produce "XY". Differential Revision: https://reviews.llvm.org/D125059
2022-05-06 07:44:41 +08:00
extern int X\UAAAAAAAA; // expected-error {{not allowed in an identifier}}
int Y = '\UAAAAAAAA'; // expected-error {{invalid universal character}}
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
#ifdef __cplusplus
extern int ༀ;
extern int 𑩐;
extern int 𐠈;
extern int ꙮ;
Support Unicode 14 identifiers This update the UAX tables to support new Unicode 14 identifiers.
2021-09-17 01:20:35 +08:00
extern int \u1B4C; // BALINESE LETTER ARCHAIC JNYA - Added in Unicode 14
extern int \U00016AA2; // TANGSA LETTER GA - Added in Unicode 14
[Clang][C++23] P2071 Named universal character escapes Implements [[ https://wg21.link/p2071r1 | P2071 Named Universal Character Escapes ]] - as an extension in all language mode, the patch not warn in c++23 mode will be done later once this paper is plenary approved (in July). We add * A code generator that transforms `UnicodeData.txt` and `NameAliases.txt` to a space efficient data structure that can be queried in `O(NameLength)` * A set of functions in `Unicode.h` to query that data, including * A function to find an exact match of a given Unicode character name * A function to perform a loose (ignoring case, space, underscore, medial hyphen) matching * A function returning the best matching codepoint for a given string per edit distance * Support of `\N{}` escape sequences in String and character Literals, with loose and typos diagnostics/fixits * Support of `\N{}` as UCN with loose matching diagnostics/fixits. Loose matching is considered an error to match closely the semantics of P2071. The generated data contributes to 280kB of data to the binaries. `UnicodeData.txt` and `NameAliases.txt` are not committed to the repository in this patch, and regenerating the data is a manual process. Reviewed By: tahonermann Differential Revision: https://reviews.llvm.org/D123064
2022-04-04 18:41:12 +08:00
extern int _\N{TANGSA LETTER GA};
extern int _\N{TANGSALETTERGA}; // expected-error {{'TANGSALETTERGA' is not a valid Unicode character name}} \
// expected-note {{characters names in Unicode escape sequences are sensitive to case and whitespace}}
Support Unicode 14 identifiers This update the UAX tables to support new Unicode 14 identifiers.
2021-09-17 01:20:35 +08:00
// This character doesn't have the XID_Start property
extern int \U00016AC0; // TANGSA DIGIT ZERO // expected-error {{expected unqualified-id}}
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
extern int 🌹; // expected-error {{unexpected character <U+1F339>}} \
expected-warning {{declaration does not declare anything}}
extern int 👷; // expected-error {{unexpected character <U+1F477>}} \
expected-warning {{declaration does not declare anything}}
extern int 👷‍♀; // expected-warning {{declaration does not declare anything}} \
expected-error {{unexpected character <U+1F477>}} \
expected-error {{unexpected character <U+200D>}} \
expected-error {{unexpected character <U+2640>}}
#else
Add test that we correctly allow some non-letter unicode characters in identifiers, and extend existing test to also cover C++. llvm-svn: 248079
2015-09-19 10:14:12 +08:00
// A 🌹 by any other name....
extern int 🌹;
int 🌵(int 🌻) { return 🌻+ 1; }
Use functions with prototypes when appropriate; NFC A significant number of our tests in C accidentally use functions without prototypes. This patch converts the function signatures to have a prototype for the situations where the test is not specific to K&R C declarations. e.g., void func(); becomes void func(void); This is the sixth batch of tests being updated (there are a significant number of other tests left to be updated).
2022-02-10 06:13:53 +08:00
int main (void) {
Add test that we correctly allow some non-letter unicode characters in identifiers, and extend existing test to also cover C++. llvm-svn: 248079
2015-09-19 10:14:12 +08:00
int 🌷 = 🌵(🌹);
return 🌷;
}
Warn if we find a Unicode homoglyph for a symbol in an identifier. Specifically, warn if: * we find a character that the language standard says we must treat as an identifier, and * that character is not reasonably an identifier character (it's a punctuation character or similar), and * it renders identically to a valid non-identifier character in common fixed-width fonts. Some tools "helpfully" substitute the surprising characters for the expected characters, and replacing semicolons with Greek question marks is a common "prank". llvm-svn: 320697
2017-12-14 21:15:08 +08:00
int n; = 3; // expected-warning {{treating Unicode character <U+037E> as identifier character rather than as ';' symbol}}
int *n꞉꞉v = &n;; // expected-warning 2{{treating Unicode character <U+A789> as identifier character rather than as ':' symbol}}
// expected-warning@-1 {{treating Unicode character <U+037E> as identifier character rather than as ';' symbol}}
int v＝［＝］（auto）｛return～x；｝（）; // expected-warning 12{{treating Unicode character}}
PR38870: Add warning for zero-width unicode characters appearing in identifiers. llvm-svn: 341700
2018-09-08 03:25:39 +08:00
int ⁠xx‍;
// expected-warning@-1 {{identifier contains Unicode character <U+2060> that is invisible in some environments}}
// expected-warning@-2 {{identifier contains Unicode character <U+FEFF> that is invisible in some environments}}
// expected-warning@-3 {{identifier contains Unicode character <U+200D> that is invisible in some environments}}
int foo​bar = 0; // expected-warning {{identifier contains Unicode character <U+200B> that is invisible in some environments}}
int x = foobar; // expected-error {{undeclared identifier}}
Don't emit "will be treated as an identifier character" warning for UTF-8 characters that aren't identifier characters in the current language mode. llvm-svn: 343040
2018-09-26 06:34:45 +08:00
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
int ∣foo; // expected-error {{unexpected character <U+2223>}}
Don't emit "will be treated as an identifier character" warning for UTF-8 characters that aren't identifier characters in the current language mode. llvm-svn: 343040
2018-09-26 06:34:45 +08:00
#ifndef PP_ONLY
#define ∶ x // expected-error {{macro name must be an identifier}}
#endif
Implement P1949 This adds the Unicode 13 data for XID_Start and XID_Continue. The definition of valid identifier is changed in all C++ modes as P1949 (https://wg21.link/p1949) was accepted by WG21 as a defect report.
2021-08-18 19:33:14 +08:00
#endif