View source for Lupdate clang-based c++ parser

== Introduction ==
lupdate is part of qttools/linguist module. It allows parsing of the code in order to retrieve the text to be translated. It has a customised c++ parser, with its limitations.
In order to overcome those limitations and modernize the parser, a new
clang-based c++ parser is added to lupdate.
The new parser is available as an option, accessible with -clang-parser.

The new clang-based lupdate c++ parser is based on clang tooling.
Using clang tooling instead of libclang gives access to the AST in full
and allows the construction of avery customized parser.
Information about llvm and clang:
Information about AST:

Here is documented some of the steps taken in order to implement this parser.

== Clang-environment and Qt configuration ==

In order to add clang related code to lupdate,
clang needs to be available in the environment.
QDoc is already checking for clang availability, at qt configuration level,
in configure.pri and configure.json.
Those files were originally located in qttools/src/qdoc.

Changes:
Those files have been placed somewhere accessible to both qdoc and lupdate.
Since lupdate clang-based c++ parser needs other libraries than qdoc, lupdate
dedicated checks have been added.

Note: if the clang environment is not adapted to the new lupdate clang-based parser,
nothing is breaking, the new parser option is simply not be available.
Nothing is changing for qdoc.

=== How to make clang-environment checks available for both qdoc and lupdate? ===
Simply moving the configure.json and configure.pri files upstream,
at qttools level does NOT work (because the configuration headers
don't end up in a place that can be accessed from anywhere)

Solution:
Creating a "header only" module:
*name = tools
*location = qttools/src/global
The configure.json and configure.pri are now there.
A "header only" module is a module that only provides header files.
In our case: qttools-config.h and qttools-config_p.h.
Those files are created at configuration time and placed in
builtDIR/qttools/src/global/

To have those files available in a reliable manner at compilation time:
add "QtTools" => "$basedir/src/global" in qttools/sync.profile
which creates:
builtDir/qtbase/include/QtTools/qttools-config.h
builtDir/qtbase/include/QtTools/QT_VERSION/QtTools/private/qttools-config_p.h

and now we can access those headers from the code using
\#include <QtTools/private/qttools-config_p.h>

=== lupdate clang-based parser special needs ===

The new lupdate clang-based parser is based on clang tooling,
which requires clang-c++ and
llvm libraries. Also requires clang version 7.0.0 or higher, otherwise
llvm/Support/InitLLVM.h header is missing.
So, unlike qdoc, lupdate needs clang-c++ and llvm libraries.
They are not part of the libclang which contains only clang-c
related libraries.

For qdoc the feature clang, checking for clang environment
(directory, version, and libclang aka libclang-c) needs to be ON.
For lupdate, a new feature clang_cpp, checking for
additional clang libraries (clang c++ and llvm libraries) needs
to be ON.
This new feature is added in configure.json.
It is ON if:
*the clang feature is ON, technically if the libclang-test associated with the clang feature is passed
*the variable 'has_clangcpp' is equal to true. This variable is set within the libclang-test.
This new clang_cpp feature does not rely on its own test, but on the already existing
libclang-test which is expended for lupdate clang-based parser needs, without modifying
it's behavior for qdoc.

=== LLVM/clang libraries availability===
Unlike libclang (the c related clang libraries needed for qdoc), the clang c++ and llvm
libraries availability changes from one build to another and from one pre-built
package to an other.
Having a valid clang environment for qdoc does not imply having one for lupdate new
clang-based parser.
Let's try to give an overview of the situation:
When building LLVM, clang-c++ and llvm libraries are
by default installed as static (only since may-june 2019, an option
exist to  export the clang-c++ libraries as shared using:
CLANG_LINK_CLANG_DYLIB=ON, https://reviews.llvm.org/D61909).

When installing an official pre-built LLVM package (http://releases.llvm.org/download.html), 
clang-c++ libraries are either:
* not available (Windows)
* available as static (all the rest)
There is also the possibility for the user to install a pre-built llvm
package from http://download.qt.io/development_releases/prebuilt/libclang/.
In those packages, all the clang libraries
are available as static for all platforms.

==== To summarize ====
in the configure.pri, to fit lupdate new clang-based parser needs,
we need to check for the existence of clang c++ and llvm libraries,
as static or dynamic, and if none are present,
the feature clang_cpp needs to be turned off.
Additionally, one needs to check that clang version is at least 7.0.0 in order to 
have the proper headers available.

Basic squeleton of the c++ parser clang-tool or how to built a clang tool

here summarize the basic things needed to built a clang tool