[clang-format] spacesRequiredBetween is not honouring clang-format off/on

https://github.com/llvm/llvm-project/issues/52881

It seems that clang-format off/on is not being honoured in regard to adding spaces.

My understanding of clang-format off/on is that it marks the token as finalized based on whether formatting is currently enabled or disabled.

This was causing a space to be added between the `<` and `<<`  in the Cuda kernel `foo<<<1, 1>>>();`

This if doesn't solve this actual issue but ensure that clang-format is at least honoured.

Reviewed By: curdeius, owenpan

Differential Revision: https://reviews.llvm.org/D116494

clang/lib/Format/TokenAnnotator.cpp

@@ -3294,6 +3294,11 @@ bool TokenAnnotator::spaceRequiredBefore(const AnnotatedLine &Line,
     return Right.WhitespaceRange.getBegin() != Right.WhitespaceRange.getEnd();
   };
 
+  // If the token is finalized don't touch it (as it could be in a
+  // clang-format-off section).
+  if (Left.Finalized)
+    return HasExistingWhitespace();
+
   if (Right.Tok.getIdentifierInfo() && Left.Tok.getIdentifierInfo())
     return true; // Never ever merge two identifiers.
 

clang/unittests/Format/FormatTest.cpp

@@ -21156,6 +21156,16 @@ TEST_F(FormatTest, SpacesInAngles) {
   verifyFormat("A< A< int > >();", Spaces);
   verifyFormat("A<A<int > >();", Spaces);
   verifyFormat("A< A< int>>();", Spaces);
+
+  Spaces.SpacesInAngles = FormatStyle::SIAS_Always;
+  verifyFormat("// clang-format off\n"
+               "foo<<<1, 1>>>();\n"
+               "// clang-format on\n",
+               Spaces);
+  verifyFormat("// clang-format off\n"
+               "foo< < <1, 1> > >();\n"
+               "// clang-format on\n",
+               Spaces);
 }
 
 TEST_F(FormatTest, SpaceAfterTemplateKeyword) {