CMake Port/Porting Guide

Porting Notes

General porting guide

There is a python script called pro2cmake.py in qtbase/cmake/utils.

It takes a .pro file as input, and generates a CMakeLists.txt file in the same folder. You need to have python3 installed and a few packages from pip (pyparsing, sympy) to use the script.

Example:

python3 qtbase/util/cmake/pro2cmake.py qtbase/src/corelib/corelib.pro

The script can handle .pro files that add modules, plugins, examples, tests, and partial support for .pro files that just include other .pro files (subdirs).

The script does a good chunk of the conversion process for you, but you'll sometimes need to do manual fixes to the file. Make sure to mark those manual changes with a "# special case" marker. Example:

SOURCES
 foo.cpp
 bar.cpp # special case

This way when you re-run the script, you won't lose your manual modifications. You can also use block special case markers:

# special case begin
LIBRARIES
  Qt::Gui
# special case end

When you add a special case, and rerun the script you might notice a new .prev_CMakeLists.txt file is created and staged in your git repo. Make sure to commit that file together with your special case changes in the main file. This prev file is used by the pro2cmake.py script to correctly reapply any special cases you had before. You can generally just commit and ignore any changes in that file.

There is also another script called run_pro2cmake.py which runs the first script recursively on all .pro files in the given folder. A good place to use it would be on the examples folder, or on the whole repository you are porting: Example:

python3 qtbase/util/cmake/run_pro2cmake.py qtbase/examples
python3 qtbase/util/cmake/run_pro2cmake.py qtsvg

If a directory has a configure.json, you'll want to run a script called configurejson2cmake.py to generate a configure.cmake file. These files should not be modified manually, but rather the script should be fixed to handle your specific case.

python3 qtbase/util/cmake/configurejson2cmake.py qtbase/src/corelib

Porting a new repository

• Request a wip/cmake branch for your repository from the dev branch
• Copy the root CMakeLists.txt file from either qtsvg/CMakeLists.txt or qtimageformats/CMakeLists.txt into the root of your repository
• Change the project name and description
• Adjust the find_package() calls to import the required Qt Components (Core, Gui, Widgets, Test, Network, Xml, etc.) and make sure BuildInternals component is listed as well
• Run either pro2cmake.py individually or run_pro2cmake.py on the whole repo.
• Try to build it against an installation of qtbase, for example:

cmake ../qtsvg -DQT_USE_CCACHE=1 -GNinja -DCMAKE_INSTALL_PREFIX=/home/foo/qt/qt5_cmake/qtbase_installed && ninja

• Fix something in the CMakeLists.txt file and rebuild again

General info

The Qt CMake port uses custom CMake macros and functions that wrap regular functions like add_executable / add_library, etc. You should use them over the CMake provided ones, unless strictly necessary.

Modules like Gui or Widgets are created in CMake with add_qt_module.

Plugins with add_qt_plugin.

Tools with add_qt_tool.

Tests with add_qt_test.

Examples with add_qt_executable (consistent, right?).

3rd party packages should be found using qt_find_package() instead of find_package(), and you should specify the targets that the package provides with the PROVIDED_TARGETS option. You can grep that token for examples. But when specifying Qt specific packages that have to be found when building qtsvg for example, use find_package, not qt_find_package().

Most of the macros and common CMake functionality are in the files QtBuild.cmake and QtFeature.cmake under qtbase/cmake.

There are also custom Find modules that are used by qt_find_package(), for 3rd party projects that do no provide their own Find modules or Config files. It's very probable you'll have to write many of these by yourself for libraries that are defined in configure.json.

Tools CMake Packages (used for cross compilation)

Tools that we build as part of Qt, like moc, rcc, uic, qfloat16-tables, qmake, are associated with a specific module like Core or Widgets, and we create a Tools CMake package for each module that has tools, so CoreTools, WidgetTools.

This is done to allow CMake only to import tools and not modules when doing cross-compilation. So we find_package(Qt5Tools) to get moc, so we can cross compile to Embedded Linux for example.

The tool <-> module association is done in add_qt_tool via the TOOLS_TARGET option.

Feature system

When building qt5 using the configure script, you can enable or disable features by passing something like -opengl dynamic or -developer-build.

In CMake the equivalent is to pass -DFEATURE_opengl_dynamic=ON or -DFEATURE_developer_build=ON. An example of disabling a feature: pass -DFEATURE_dlopen=OFF.

The list of features supported in CMake can be found by inspecting the converted configure.cmake files (from configure.json).

Each entry you find there of the form qt_feature("eventfd") means you can pass -DFEATURE_eventfd=ON or -DFEATURE_eventfd=OFF.

If you inspect the generated CMakeCache.txt file in your build directory, among other things, it contains the explicit values you passed for the above features.

For each FEATURE_foo, the cache file will also contain a QT_FEATURE_foo. This is used for the internal implementation, so you can mostly ignore those values. The internal implementation uses those to discern if something was passed explicitly on the command line or not.

If you wish to reconfigure qtbase or some other repository with a different set of enabled features: - make sure to remove the CMakeCache.txt file before calling cmake with the different feature parameters - remove all generated moc_foo.cpp files.

There is currently a limitation in upstream CMake (https://bugreports.qt.io/browse/QTBUG-74521) regarding automoc not noticing updated define values in dependent headers. This can happen when you reconfigure with a feature being suddenly disabled.

Developer builds

When porting a new repo, it might be a hassle to constantly have to run make install in qtbase.

For example you modify the qtbase/cmake/QtBuild.cmake file, and you need that updated file to be used while building qtsvg. You would have to run make install in qtbase before the file is picked up by qsvg.

You can configure your qtbase build with -DFEATURE_developer_build=ON, and then you don't have to install anything, only run make / ninja.

To build another repo against such a qtbase, pass the build directory of qtbase as the CMAKE_INSTALL_PREFIX, e.g.:

cmake ../qtsvg -DQT_USE_CCACHE=1 -GNinja -DFEATURE_developer_build=ON -DCMAKE_INSTALL_PREFIX=/home/foo/qt/qt5_cmake/qtbase_built && ninja

Handling 3rd party libraries

In Qt5, configure.json lists many 3rd party libraries that can be used by Qt when building Qt. For example: system-zlib, zstd, system-pcre2, double-conversion etc. These are listed in the libraries section of configure.json.

There a few ways of specifying where to find these libraries and how to test that they exist:

1. just passing -lfoo, and hoping it's found in the linker search path
2. using pkg-config
3. using qmake makeSpec, e.g. QMAKE_LIBDIR_FOO, QMAKE_LIBS_FOO and QMAKE_INCDIR_FOO
4. using some own custom qmake function.

In CMake, finding 3rd party libraries is handled with find_package(). find_package operates on two different file types: Find modules (FindZLIB.cmake) and Config package files (ZLIBConfig.cmake).

Usually if a library is built with CMake, it exports a Config file (the latter option described above), and then find_package would find and use that Config file.

In case if a library is built with some other build tool, the Find module approach allows finding and using such libraries. The find modules have to be written manually for every third party library / package. General documentation on writing Find modules can be found here https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-To-Find-Libraries .

CMake already ships many Find modules (see https://cmake.org/cmake/help/v3.0/manual/cmake-modules.7.html ).

For libraries that Qt uses, and don't have Find modules, we ship our own custom written Find modules in qtbase/cmake. You can use them as a template for writing your own Find modules.

Once the Find module is written, or you confirmed that a 3rd party library you want to use ships its own Config file, you need to update qtbase/cmake/utils/helper.py to let the conversion script know about that library.

You want to add a new entry to _library_map = {}.

1. The first key is the qmake library name / soname, for example: "libjpeg".
2. The second key is the package name CMake will look for in a find_package call. For example JPEG. That means CMake will look for either a FindJPEG.cmake file or a JPEGConfig.cmake file.
3. The third key is the CMake target exported by that package, e.g. JPEG::JPEG. This means that a Qt Gui plugin can link against JPEG::JPEG, which informs CMake both about the library location and the library include directory.

After modifying helper.py, just rerun configurejson2cmake.py on the configure.json file that uses the third party library you just added, and then rerun pro2cmake.py on any .pro file that uses that library.

Debugging qmake parser in pro2cmake.py

It might happen that when you run the pro2cmake.py script on a .pro file, the script can fail with a parsing exception. This is most likely because there is some qmake syntax quirk that isn't handled by the python parser at the moment.

The script uses the pyparsing library to define a grammar with which to parse .pro files. You can pass --debug to the script, to get debug output about the token matching done by the parser.

You should then try to bisect your .pro file by removing half of the code, and again and again, until you find the specific qmake construct that makes the parser fail. The token matching output should then be short enough that you can manually trace what is being matched and correlate that with the grammar described in _generate_grammar() method.

Figure out what's missing, and amend the grammar to properly handle that case. After that you should add a test to qtbase/util/cmake/tests which checks for that specific qmake syntax.

You can run the the tests by going into the qtbase/util/cmake/tests and running "pytest". To execute a specific test, run "pytest -k test_foo". To get print output from the test, pass "-s".

Common qmake <-> CMake constructs

qmake	CMake
qtHaveModule(foo)	if(TARGET Qt::foo)
qtConfig(foo)	if (QT_FEATURE_foo)
QT += widgets	find_package(Qt5 COMPONENTS Widgets) target_link_libraries(my_lib PRIVATE Qt::Widgets)
LIBS += zlib	target_link_libraries(my_target PRIVATE ZLIB::ZLIB)
find a 3rdparty package	qt_find_package(Cups REQUIRED PROVIDED_TARGETS Cups::Cups)