Qt for Python Development Notes 2019

2019

19. December 2019

- Python 3.8.1 is out, which simplifies certain issues.
- The missing COIN variable QTEST_ENVIRONMENT=ci was restored. The impact on PySide was fortunately only two missing functions.
- PYSIDE-1158 "qApp: None != None" will probably be documented only. Change is hard and not worth the effort.
- PYSIDE-1135 "embedded QApplication subclass" fix was confirmed to work correctly and will be checked in.
- PYSIDE-1079 clarified. "Missing signal" support not really possible: They have no signatures.
- PYSIDE-939 continued effort explaining the needed inversion.

12. December 2019

- Qt 5.14 is out, but we will wait until next Monday for Python 3.8.1 to be out. More information to come.
- QThread/threading freezing issue are a high priority task now.
- Issues with Python Enums to be work on (Q_ENUM related)
- Documentation update: 5.14 will have new things, and we are still working on them.

5. December 2019

- About dev branch:  will be 5.15 at the moment, until feature freeze.
- Python 3.8 status for 5.14:  it was a hard to find solution, and it is currently based on a flag to enable the fix for 3.8 only.
- New documentation: fixing issues with the new structure.

28. November 2019

- 5.14 to be close soon, last set of patches to be merge.
- Python 3.8, first part of the fix is ready for review. Mainly the issues regarding the refcounting. The second part will be more tricky, and it is related to Python itself.
- shared_ptr on linux is working, namespaces features for Shiboken waiting for review,
- new documentation incoming for 5.14,
- positive feedback from the Qt Contributors Summit.
 - Qt_Contributors_Summit_2019_Program/Qt_for_Python_and_beyond
 - Qt_Contributors_Summit_2019_Program/Qt_for_Python_Documentation
 - Qt_Contributors_Summit_2019_Program/Qt_Machine_Learning_and_Math

14. November 2019

- 5.12.6 Release: issues with the 5.12 merge. Some issues with Ubuntu.
- 5.14 Release hopefully in a couple of weeks.
- Qt Contributors Summit happening next week.
- NumPy integration documentation soon.
- qApp issue with embedded subclasses: PYSIDE-1135
- Python 3.8 refcount issues PYSIDE-939 soon to be solved.
- Shiboken Shared Pointers support, Namespaces, MSVC2017 issues solved.
- Building docs on macOS seems really hard to achieve.

07. November 2019

- Documentation improvements are coming, more people involved.
- Qt World Summit feedback
 - First time we had a Qt for Python booth,
 - Request to improve the Python+QML story
 - Suggestions to have a better experience in QtCreator
 - In general the people really liked the integration with modules like matplotlib, pandas, opencv, scikit-image
 - The voice assistant also was really well received, this involve many other modules like Deepspeech, rasa, mozilla tts technologies, etc.
- Documentation will be written for the NumPy compatibility patch.
- Planned work on a Python 3.8 leak. 
- After the chain of patches improving the namespace in shiboken, there are some issues with Python 2.7
- New template for Qt for Python on QtCreator will be merge soon.

24. October 2019

  - Release Qt for Python 5.13.2
    - 5.13.2 branch exists -> testing and final build on the way
    - aiming for release on 1st Nov 2019
  - 5.15 branch is still broken
  - further requirements mgmt/grooming ongoing
  - name spacing modification
    - hiding namespaces like QtCharts from Python and adding ability to shiboken to work with inline namespaces
    - https://bugreports.qt.io/browse/PYSIDE-990
    - https://bugreports.qt.io/browse/PYSIDE-1074
  - removed pyside-uic and pyside-rcc -> tooling is 100% Qt
  - QSettings related bug fix caused regression, more fixes were needed
  - Qt World Summit 2019 preparations (demos, shipping etc)
  - translator script from C++ to Python being done
    - translates 90% of the code (enough to simplify the snippet conversion process)
  - Numpy work complete for now
  - started working on Python 3.8 (https://bugreports.qt.io/browse/PYSIDE-939)
  - for now Python 3.5 support stays as is

24. October 2019

  - struggling with coin and therefore none of the scheduled releases went out
  - worked on multiple \brief attributes appearing in the docs
  - discussions started how to improve the general Qt for Python doc index page
  - investigating hiding C++ inline namespaces (e.g. Qt3D and QtCharts name spaces)
  - https://bugreports.qt.io/browse/PYSIDE-795 -> complete for 5.14 (merge pending)
   -> pysequence can be handled
  - Adopting Python 3.8 and removing support for 3.5 in Qt for Python 5.14
  - converter from C++ to Python worker approaching point of being usable for doc snippet conversion
  - moving some docs from wiki to official QtfP docs

17. October 2019

  - Qt for Python 5.14 release -> not build-able with current Qt 5.14 release
    -> Qt Declarative needs change to enable QtfP binaries
  - Qt for Python 5.15 
     -> currently not possible in CI
  - Qt for Python 5.13.2
     -> in principle 5.13 branch is releasable (except there is an elaborate set of doc patches)
  - split of QtfP project modules continues
    -> dependencies defined via yaml
    -> adds the ability to also sign produced wheel.
  - more example work
  - working on a semi translator to translate C++ snippets to Python
  - visit to PyConDE
  - QSharePointer related fixes being worked on
  - numpy proof-of-concept (PYSIDE-795) is available
  - removed uic and rcc from pyside-setup since the Qt tools support Python now
  - exploring the idea to include designer in the Python wheels
  - discussion on supported Python versions (can we add 3.8 and remove 3.5?)
     -> more comms on the mailing list can be expected

26. September 2019

  - Porting guide (C++ -> Python) ready for staging
  - Further snippet handling improvements
     - being split out and translating them to Python
  - general bug cleanups/triaging
  - working on new template for Python to create ui files
  - PYSIDE-795 (numpy) progressing
  - OPCUA viewer example port completed
  - added more Qt 5.14 API to Qt for Python 5.14
  - PYSIDE-1087 in progress

19. September 2019

  - Roadmapping session for Qt for Python 5.15 & 6.0 on 8. October
  - experimenting with new configuration changes in coin and how they affect Qt for Python
  - Porting tutorial (C++ -> Python) was pushed for review
  - Progress on porting the C++ snippets to Python
     -> will be a longer process
  - PYSIDE-795 (numpy) progressing
    -> streamline-ing some code sections in the process
  - focus on validating existing bugs 
  - otherwise the team dealt with impact of a cmake based Qt build system
  - porting QtOpcua to Qt for Python
     - particular work went into to opcuaviewer example

12. September 2019

  - Work on QtOpcUa continued PYSIDE-952
  - Work on example for Qt Summit
  - Work on PYSIDE-1057
  - Prototyped QStringView support
  - PYSIDE-1087 huge speed patch ready for review
  - PYSIDE-795 (numpy) progressing

5. September 2019

  - Qt for Python 5.13.1 & 5.12.5
    - RTA for 5.13.1 released
    - 5.12.5 released next week
  - Branching
    - challenge is the building of Qt 6 while maintaining a stable Qt for Python build
    - one solution might be a QfP build for selected modules only
  - Team is planning a roadmapping session for QfP 5.15 release
  - PYSIDE-795 (numpy) progressing
  - quite a few bugs have been fixed e.g. PYSIDE-1051
  - Looking at QStringView exposure in Python
  - Continued work on uic/rcc tooling
  - General tooling improvements when working with QtCreator and Python
  - Updated some build instructions for QfP on Fedora

29. August 2019

  - Releases (5.12.5 & 5.13.1) upcoming
  - Release candidate wheels under test for both releases (Linux & macOS, Win is outstanding)
     - last patch for 5.13.1 is PYSIDE-1079
  - PYSIDE-795 continued progress
  - new look at our example basket
  - continued work on PYSIDE-1033
  - uic and rcc can output python code -> needs deployment changes in Pyside2 -> target release Qt for Python 5.14
     - adding support in QtCreator being able to deal with the above python tools

22. August 2019

  - Work on QtOpcUa started  PYSIDE-952
  - Solved PYSIDE-1079 partially, need the structure
    of 5.14 to implement the typing.Optional feature.
  - Solved and closed PYSIDE-1077 wrong init code
  - Solved PYSIDE-1073, waiting for review
  - Refined and closed PYSIDE-945.

15. August 2019

  - Partially solved PYSIDE-1052 (constants missing).
  - Solved at the same time PYSIDE-945.
    Solved a QtCore.Slot() bug that dates back to 2010-11-03.
  - Work on PYSIDE-1073 trying to fix the bug.
  - Work on PYSIDE-1051 trying to fix the bug.
  - Merges/work on continuous integration infrastructure (preparing 5.13.1)

8. August 2019

  - Licenses on Pypi
    - pypi is used for commercial and open source users
    - currently only publishing the open source license -> will add second "commercial" license
  - Releasing
    - 5.13.1 release is being prepared
    - 5.12.5 follows later this month
  - Documentation changes
    - Qt for Python Logo updated
    - CSS updates -> (lots of suggestions/bug reports from the community)
  - PYSIDE-951 (primitive type support) merged and done
  - PYSIDE-795 (Numpy support progressing)
  - ported simpletreewidget from C++ to python
  - restructuring cmake setup with the aim to have a more modular build selection

1. August 2019

  - (Summer break)
  - PYSIDE-795 (numpy support is progressing)
  - PYSIDE-951 (primitive pointer types ready for review)
  - Fixed pyside-setup 5.14 branch building (addressing lots of deprecation changes)

25. July 2019

  - (Summer break)
  - continuing on PYSIDE-795: documentation, 
    - supporting the iterable protocol (will become sub-task)
    - augmenting incomplete classes (QtPrintSupport)
    - checking and correcting function signatures according to PYSIDE-951

18. July 2019

  - (Summer break)
  - Some progress related to Bug fixing is being done.
  - bug fixing on signatures accessing "__doc__" on a module function.
  - continued on PYSIDE-795

11. July 2019

  - visiting EuroPython
  - discussed roadmap for 5.14 release (check out Pyside Jira project with FixVersion 5.14)
  - continued CI configuration setup for pyside-setup
  - continued on shared pointer bugs(PYSIDE-1024 & PYSIDE-1043)
  - assert triggering for certain types (PYSIDE-1037)
  - further changes for PYSIDE-951
  - numpy enhancements (PYSIDE-795) (documentation and tests improvements)

04. July 2019

  - Qt for Python 5.13.0 released
  - Experimented with cross compiling
    - not easily possible -> build system needs major work
  - New control system for Qt for Python (isolates Qt4Python CI configuration from coin)
    - more flexible way of working as not dependent on coin release
  - PYSIDE-951 finish
  - PYSIDE-795 work continues
  - fixed QFlags handling
  - had trouble merging pyside to dev branch
  - some discussions around smart pointer handling in Qt4Python
  - https://bugreports.qt.io/browse/PYSIDE-634 ready to be merged
  - https://bugreports.qt.io/browse/PYSIDE-1010

27. June 2019

  - about to finish PYSIDE-951
  - fixed "Fix qApp import of QCoreApplication"
  - cleanup formatting of all source code and modern C++11 features
  - final re-working of "Example: Move the SQL statements", improving the C++ code as well

20. June 2019

  - Release 
    - Qt for Python 5.12.4 released today
    - Qt for Python 5.13.0 wheels done -> still need testing
  - Coin configurations to be separated from qt5.git (will become part of pyside-setup.git
  - SQL example was updated (to be more pythonic)
  - Shared pointer related fixes being done
  - reopened https://bugreports.qt.io/browse/PYSIDE-1033 (how to build shiboken) after reviewing the current situation
  - macOS build issue being addressed
  - https://bugreports.qt.io/browse/PYSIDE-951
  - team got commit rights for pyinstaller project repository
  - TQtC is sponsoring PyCon DE

12. June 2019

  - Example polish, introduced QSaveFile, fixed tabbed browser PYSIDE-1022
  - Fixed PYSIDE-1017, named parameters in added functions
  - Started to look at some issues with smart pointers, PYSIDE-1024, PYSIDE-1016 related to PYSIDE-454
  - Making progress in the embedded support story.
    - After trying the setup.py approach, there was a lot of modifications involve in our build process, so it was not considered a good option.
    - a CMake approach is currently being used, and shiboken generator and the python module are successfully cross-compiled, but still, there is work to do.
  - Final solution of qApp embedding PYSIDE-571. After a long Odyssey of breaking solutions, qApp macro works now as expected with embedding.
  - PyInstaller maintainer still did not reply as promised on 2019-06-04 phone call.

6. June 2019

  - Summer holidays affecting releasing? Any estimate when we would have:
    - Qt for Python 5.13.0 (Qt still on RC0)
    - Qt for Python 5.12.4 (in the next few weeks)
  - discussed options how to sign wheel content
  - cross compilation of Qt for Python (work in progress)
    -> investigating options on how to achieve that in an easy way
  - fixed build on CentOS 
  - https://bugreports.qt.io/browse/PYSIDE-1017
  - https://bugreports.qt.io/browse/PYSIDE-1016
  - https://bugreports.qt.io/browse/PYSIDE-1024
  - https://bugreports.qt.io/browse/PYSIDE-1019
  - https://bugreports.qt.io/browse/PYSIDE-795 (NumPy)
  - discussed commit issues with pyinstaller maintainer
  - QtC will have presence on PyCon DE

23. May 2019

  - Qt 6 (e.g. Pyside 6.0?) https://bugreports.qt.io/browse/PYSIDE-904
  - Pyinstaller issues being a bottle neck at the moment for the deployment story
  - Qt for Python 5.12.4 release
    - patches are accumulating
    - TODO: branching, changelog, testing
  - Qt for Python 5.13 release status
    - need cleanup in Jira of fix versions
    - Qt 5.13 blog post
    - start generating 5.13 release wheels for testing
  - Discussed Python 2 EOL on 1.1.2020
    - Current proposal is dropping Python 2 support in Qt for Python 6
    - Python 2 users can continue to use Qt for Python 5.15

16. May 2019

  - Team member attended PyCon US
  - Work on tutorials
  - PYSIDE-816 PYSIDE-1003 Final fix checked in
  - PYSIDE-735 Make generate_pyi crash on real ImportErrors, only!
  - PYSIDE-1004 writable signatures implemented but needs to change

9. May 2019

 - Final fixes to uic to be able to generate Python PYSIDE-797
 - Refactorings in libpyside
 - Work on SQL book tutorial
 - Research into using Clang C++ API
 - PYSIDE-816 PYSIDE-1003 Final fix is waiting for approval
 - Viewer tool for PYSIDE-803 nearly ready for browsing code

2. May 2019

 - 5.12.3 released
 - Looking into OpenGL version functions PYSIDE-955
 - Improved shiboken2 progress message output
 - Sped up shiboken2 class generation significantly
 - Improved code generated for multiple inheritance
 - First look at 5.14 API
 - Fixed Problems with PyInstaller after removal of Coin glitches
 - Wrote viewer tool to finally support PYSIDE-803

25. April 2019

- Preparation for Qt for Python 5.12.3 release.
 - Replaced QtXmlPatterns by libxml2/libxslt for documentation generation
 - Testing and final polishing of Python output mode of uic 5.14
 - New signatures related to PYSIDE-989
 - Looking into OpenGL version functions PYSIDE-955
 - Improving the usability of the "--reuse-build" option
 - fixing new PyInstaller problem with Python 2.7.16

18. April 2019

- Usage of QtXmlPattern in doc tool chain of Qt for Python
  - xml pattern deprecated in Qt 5.13
  - working on libxml2 replacement (has issues on platforms w/o libxml2 support)
  - in the long term, the qdoc generation process should really produce python docs too
  - updated coin to continue to include QtXmlPattern for Qt for Python builds
- PYSIDE-996 fixed on SuSE but not yet RedHat
- working on docs to provide hints when working with pyinstaller
- PYSIDE-955 in progress
- New attempt on PYSIDE-803
- shiboken PYSIDE-995 call operator support
- custom scene graph rendering node in Python (related to PYSIDE-955 PYSIDE-989)
- improving primitive type conversion (by the various OpenGL related bugs like PYSIDE-989 or PYSIDE-955)

11. April 2019

- PyInstaller problems identified as COIN bugs PYSIDE-996
- continuing with PYSIDE-795
  - handling pointers which are arrays arrays (patch defunct after too many other changes)
  - identifying remaining bad spots that need improving
- Started work on PYSIDE-955
- Started diagnosis of PYSIDE-995
- Continued on work enabling extending namespaces across modules

4. April 2019

- working in Wiki page on how to port a C++ application to Qt for Python (over time should turn into tutorial)
- further discussion how to handle snippets
- more tutorials (especially for PYSIDE-901)
- Further work on cross compilation for ARMv7
- Various fixes for Shiboken to handle namespaces in Qt better
- Improve Shiboken documentation composition/layout
- Bugfixing related to Qt file handling

28. March 2019

- Qt for Python 5.12.2 released last on 22. March 2019
- started to write up documentation on how to deploy with pyinstaller
  - finding quite a few issues when using pyinstaller on Windows (and particular Python 2)
  - macOS & Linux less problematic
- worked on documentation issues (like missing links for easier browsing)
- investigated issues where shiboken didn't handle C++ namespaces across modules
- handle nullptr usage properly in docs
- more translation of C++ snippets to Python (is a never ending task as there are thousands of C++ snippets)

21. March 2019

- Qt for Python 5.12.2 release activities
 - some bugs and general CI issues kept delaying the generation of the final release wheels
 - most bugs were related to pyinstaller and other deployment tools
 - one last cxfreeze bug being integrated
 - next/subsequent set of wheels will be 5.12.2 release
- Lots of installer related discussions in the team
 - question is which installer/deployment tool to use/recommend
 - current decision is to provide guides/documentation for at least fbs and pyinstaller
- review of project's current release practices (in particular in the area of QA)
 - no outcome at this stage
- addressed issues in dev branch related to changes in Qt dev branch
- Qt for Python team joined Qt doc team to talk about ways how to improve Qt for Python docs

14. March 2019

- Fixed broken documentation formatting; added deprecation information
- QTBUG-74424 5.13 Release blocker affecting doc generation occurred
- PYSIDE-932/PYSIDE-942: The embedding patch is mostly
  working, but needs some refinement to be complete:
  - The .pyc generation contradicts the Limited API and must be changed to 
    use source code.
  - The code isolation must be made explicit and complete, so that installers 
    don't recognize that there is something.
    Right now this was done in the wrong direction.
  Not much code and time necessary, just fixing the right bits.
- PYSIDE-934 The Buffer protocol for Python 2 needed a special flag to be compatible.                 
- Added more features for the Python project wizards on QtCreator https://codereview.qt-project.org/#/c/255686/
- Created a new tutorial for a workshop https://github.com/cmaureir/qtforpython_workshop

7. March 2019

- Qt for Python 5.12.2 release
 - waiting for final patch before release (PYSIDE-942/PYSIDE-932)
- Setting up test envs for Python 3.7 & 3.5 (so far testing was always done on Python 3.6) -> work in progress
- QML tutorial under review (https://codereview.qt-project.org/#/c/254044/10)
- how to integrate deprecation infos from Qt to the Python side
- Python output to rcc PYSIDE-855
- PYSIDE-95 fixed
- Experimented with MicroPython
- Found the reason for PYSIDE-966: A memory leak was fixed by ctypes.

28. February 2019

- Discussion on adding more python testing environments to be certain 3.5+ versions are properly working.
- PYSIDE-953 fixed
- PYSIDE-934 almost done
- PYSIDE-931 almost done
- refcount bug fixes mathplotlib crashes:
  - PYSIDE-928
  - PYSIDE-937
  - PYSIDE-943
- WIP: Embedding to solve signature problems dependency of support directory since 5.12.1 . Will solve  
  - PYSIDE-942  missing support, PyInstaller  (was solved but involves PyInstaller change)
  - PYSIDE-932  missing support, cx_freese

21. February 2019

- research on how to do ad hoc releases of Pyside 2 for bug fixes like last weeks meta data change (and otherwise no further source change)
- Qt 5.12.2 about to be released -> should have a Qt for Python 5.12.2 release
- discussed installer related crash bug following 5.12.1 release PYSIDE-932 etc.
- tutorials works (based on recent Qt for Python youtube videos)
- prepared discussions with doc team meeting next week (on Python support)
- some more work on PYSIDE-795
- work to distinguish array types from pointer variables
- PYSIDE-946 under development
- PYSIDE-934 void* type handling
- PYSIDE-938 under review
- Current Creator 4.9 beta released today added Python project templates
- PEP 518 (project format for Python projects) -> needs Qt Creator support

14. February 2019

- 5.12.1 wheels had meta data issues and were redone with fixed meta data
- discussed improving the integration of Python example code into Qt for Python docs
- PYSIDE-921
- improved signature matching for custom types
- PYSIDE-816 fixed and merged
- PYSIDE-922 fixed
- PYSIDE-45 fixed
- looking at crashes when importing pandas & numpy (the order of imports matters when working around it)

7. February 2019

- Fixed 5.13 deprecation warnings (which unearthed that we had a dependency on QString::isNull() vs isEmpty() in shiboken)
- Fixed a large number of shiboken code generation warnings (revealing some issues)
- Provided s solution for one issue mentioned at PYSIDE-725 (apparently Sharon Woods from US is involved in some consulting?)
- Will start formal release tag marking in Jira (matching Qt style)
- Data visualization tutorial fixes
- Talking about further integration of existing examples into general docs
- PYSIDE-803 further investigation - no conclusion yet
- PYSIDE-921 fixed
- PYSIDE-816 investigated - fix seems at hand
- PYSIDE-924 fixed issue where Q_PROPERTY flags were not properly passed on to Qt
- restructured cmake files adding a master cmake file (simplifies opening of project in IDEs)
  - Now it's possible to open the whole project (PySide2 + Shiboken2) on QtCreator using the global CMakeList.txt
- worked on issues related to the setup.py and how to execute it.

31. January 2019

- Preparing 5.12.1 release
  - Packages are looking good, likely to be released at same time as Qt 5.12.1 release
- Submitted class bindings of Qt RemoteObjects for PYSIDE-862
- Added classes QWebEngineHistory, WebEngineHistoryItem with required fix PYSIDE-906
- Have calls to method annotated as deprecated issue a warning
- Fixed warnings, prepared Qt 5.13 bindings (based on current 5.13 API additions)
- Looked into PYSIDE-816, PYSIDE-803
- Data visualization tutorial added
- working on ways of substitution of C++ specific terms in docs against Python equivalents (e.g. nullptr and bool)

24. January 2019

- Release branch for 5.12.1 exist
  - There was some issues in CI with 5.12 branch earlier this week and it was pretty much blocked. All issues should be solved now.
- Prepared class bindings of Qt RemoteObject for PYSIDE-862 for review
- Some creativity to keep integrations going due to COIN issues
- Investigation of PYSIDE-803 is ongoing. The issue needs deep analysis to isolate the problem.

17. January 2019

- Fixed PYSIDE-908, further investigation into PYSIDE-803 (free threading)
- QfP 5.12.1 release being prepared
- PYSIDE-797 (uic change ready for review)
- PYSIDE-862 (Qt RemoteObject module support)
- PYSIDE-634 being fixed
- started to look at true cross compilation (PYSIDE-802)
- Improving Qt Creator project template support (QTCREATORBUG-21824)

10. January 2019

- Scope for Qt for Python 5.13 release
  -> relevant tasks marked with Fixversion 5.13 in bugreports.qt.io
  -> feature freeze for Qt for Python 5.13 (April 2019)
- looked into case of corrupted meta data of wheel
- supporting named arguments for signals
- some minimal support in Qt Creator to create a Qt for Python project
- bug hunting in 5.12.0 release
- working on Pyside-797
- further analysis of PYSIDE-803, working on an implementation plan

3. January 2019

- Completed: Complete The Signature Introspection (PYSIDE-510)
- After the release there has been a couple of new reported bugs that we are working on:
 - There was some outdated documentation issue regarding Slots()
 - A couple of issues with multiple inheritance like: PYSIDE-868
 - and with QPlainTextEdit PYSIDE-886
- Restarted investigation of PYSIDE-803