Qt wiki will be updated on October 12th 2023 starting at 11:30 AM (EEST) and the maintenance will last around 2-3 hours. During the maintenance the site will be unavailable.

Qt Metrics 2 Backlog: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
(Created page with "This is the backlog for Qt Metrics 2 site currently under development. In case of proposals or improvement ideas please contact Juha Sippola (report builder, da...")
 
m (Status of the items updated to reflect implementation v1.0)
 
(20 intermediate revisions by the same user not shown)
Line 1: Line 1:
This is the backlog for Qt Metrics 2 site currently under development.
This is the backlog for Qt Metrics 2 site that is currently under development.
 
The priority **** items are the core functionality and would be the target prior to launch or wider use.
 
The priority *** items are considered content/functionality for basic use as requested by the key stateholders.


In case of proposals or improvement ideas please contact [[User:Juha|Juha Sippola]] (report builder, database) or Tony Sarajärvi (parser).
In case of proposals or improvement ideas please contact [[User:Juha|Juha Sippola]] (report builder, database) or Tony Sarajärvi (parser).


=== 1. Data reliability and completeness (database and parser) ===
=== Data reliability and completeness (database and parser) ===


{| class="wikitable sortable"
{| class="wikitable sortable"
Line 9: Line 13:
! Status !! Priority !! Item
! Status !! Priority !! Item
|-
|-
| in progress || *** || Get the actual testset parent (not the Qt5 where it was run)
| DONE || **** || Show the actual testset parent (not the Qt5 where it was run) ([https://codereview.qt-project.org/#/c/114432/ Code Review])
|-
| DONE || **** || Change build_number (int) to build_key (string) for the new-CI ([https://codereview.qt-project.org/#/c/115251/ Code Review]); UPDATE: type changed to 64bit int, BIGINT in MySQL ([https://codereview.qt-project.org/126506 Code Review])
|-
|-
| idle || *** || Change build_number (int) to build_key (string) for the new-CI
| DONE || *** || Get testfunctions from xml ([https://codereview.qt-project.org/126148 Code Review])
|-
|-
| idle || ** || Get testfunctions from xml
| DONE || *** || Get testrows from xml ([https://codereview.qt-project.org/126148 Code Review])
|-
|-
| idle || ** || Get testrows from xml
| DONE || *** || Change testset_run.testcases_xxx to total_xxx (terminology change) ([https://codereview.qt-project.org/#/c/115251/ Code Review])
|-
|-
| idle || ** || Database indexing (for performance)
| DONE || *** || Database indexing and query optimization (for performance)
|-
|-
| idle || ** || OS grouping (currently platform.os has quite many values, should we group e.g. all windows platforms into one on home page)
| idle || * || Database normalisation: result fields in the *_run tables should have a separate table instead of the enums (could also make the naming more flexible)
|-
|-
| idle || ** || Change testset_run.testcases_xxx to total_xxx (terminology change)
| idle || * || Database normalisation: db_status table does not have a primary key (table holds just one row with the latest data)
|-
| idle || * || Database normalisation: conf.features is not perfect design, instead a separate feature table should be used (how this information will be used in the end)
|}
|}


=== 2. Pages and views ===
=== Pages and views ===


{| class="wikitable sortable"
{| class="wikitable sortable"
Line 30: Line 38:
! Status !! Priority !! Item
! Status !! Priority !! Item
|-
|-
| in progress || *** || Home page modifications (enable buttons)
| DONE || **** || Home page modifications (implemented as part of the Overview, Project, Platform and Testset page changes)
|-
|-
| idle || ** || Summary graph
| DONE || **** || Overview page ([https://codereview.qt-project.org/#/c/115677/ Code Review])
|-
|-
| idle || *** || Overview page
| DONE || **** || Project pages for build project (Qt5) and testset projects ([https://codereview.qt-project.org/#/c/115677/ Code Review])
|-
|-
| idle || *** || Project page
| DONE || *** || Project pages: Link to log file ([https://codereview.qt-project.org/122768 Code Review])
|-
|-
| idle || *** || Project page: Link to log file
| DONE || **** || Platform page ([https://codereview.qt-project.org/#/c/115679/ Code Review])
|-
|-
| idle || ** || Project page: Way to expand the range of builds, include dates to listed builds
| DONE || *** || Platform page: Link to log file ([https://codereview.qt-project.org/122768 Code Review])
|-
|-
| idle || *** || Platform page
| DONE || ** || Platform page: Configuration flag identification ([https://codereview.qt-project.org/122759 Code Review])
|-
|-
| idle || *** || Platform page: Link to log file
| idle || * || Platform page: Identification of configurations where insignificant or force success could be removed
|-
|-
| idle || ** || Platform page: OS grouping
| idle || * || Platform page: Filtering mechanism
|-
|-
| idle || ** || Platform page: Configuration flag identification
| DONE || *** || Configuration page ([https://codereview.qt-project.org/122759 Code Review])
|-
|-
| idle || * || Platform page: Identification of configurations where insignificant or force success could be removed
| DONE || *** || Configuration page: Link to log file ([https://codereview.qt-project.org/122768 Code Review])
|-
| idle || * || Platform page: Filtering mechanism
|-
|-
| idle || * || Branch page
| idle || * || Branch page
|-
|-
| done || *** || Top failing testsets page
| DONE || **** || Top failing testsets page ([https://codereview.qt-project.org/#/c/113921/ Code Review / UI], [https://codereview.qt-project.org/#/c/114211/ Code Review / parser])
|-
|-
| done || *** || Top flaky testsets page
| DONE || **** || Top flaky testsets page (see Code Review links for Top failing testsets page)
|-
|-
| in progress || *** || Testset page
| DONE || **** || Testset page ([https://codereview.qt-project.org/#/c/115675/ Code Review])
|-
|-
| idle || *** || Testset page: Link to log file
| DONE || *** || Testset page: Link to log file ([https://codereview.qt-project.org/122768 Code Review])
|-
|-
| idle || *** || Testset page: Testset flag identification
| DONE || *** || Testset page: Testset flag identification ([https://codereview.qt-project.org/122763 Code Review])
|-
|-
| idle || ** || Testset page: Testset run times
| DONE || ** || Testset page: Testset run times ([https://codereview.qt-project.org/#/c/115675/ Code Review])
|-
|-
| idle || * || Testset page: Filtering mechanism
| idle || * || Testset page: Filtering mechanism
Line 72: Line 78:
| idle || * || Testset page: Identification of testsets where insignificant could be removed
| idle || * || Testset page: Identification of testsets where insignificant could be removed
|-
|-
| idle || *** || Testfunction page/information
| DONE || *** || Testset page: Test function information ([https://codereview.qt-project.org/126149 Code Review], [https://codereview.qt-project.org/126151 Code Review])
|-
| DONE || *** || Testset page: Test function flag identification ([https://codereview.qt-project.org/126149 Code Review])
|-
| DONE || ** || Page to identify test functions where blacklisting could be removed ([https://codereview.qt-project.org/126176 Code Review], [https://codereview.qt-project.org/126294 Code Review])
|-
| DONE || ** || Page to identify test rows where blacklisting could be removed ([https://codereview.qt-project.org/126295 Code Review])
|-
| DONE || ** || All pages: Link to build directory and xml file ([https://codereview.qt-project.org/126166 Code Review])
|-
| DONE || *** || Test function page: Test row information ([https://codereview.qt-project.org/126152 Code Review])
|-
|-
| idle || *** || Testfunction page/information: Testfunction flag identification
| DONE || ** || Top failed testfunctions page ([https://codereview.qt-project.org/126170 Code Review])
|-
|-
| idle || ** || Testfunction page/information: Identification of testfunctions where blacklisting could be removed
| DONE || ** || Top testset execution duration page ([https://codereview.qt-project.org/126505 Code Review])
|-
|-
| idle || ** || Testfunction page/information: Link to xml file
| DONE || ** || Top test function execution duration page ([https://codereview.qt-project.org/126505 Code Review])
|-
|-
| idle || * || Testfunction page/information: Find skipped testfunctions
| idle || ** || Trends for top failed/flaky lists
|-
|-
| idle || *** || Testrow page/information
| idle || ** || Failing tests in branches filtered by platform os (target configuration)
|-
|-
| idle || ** || Top failed testfunctions page
| idle || ** || Project/Platform/Testset pages: Way to expand or filter the range of builds (shows all by default), or allow configuration to be visible when scrolling horizontally
|-
|-
| idle || Example || Example
| idle || * || Testfunction page/information: Find skipped testfunctions
|}
|}


=== 3. Graphs ===
=== Graphs ===


{| class="wikitable sortable"
{| class="wikitable sortable"
Line 95: Line 111:
! Status !! Priority !! Item
! Status !! Priority !! Item
|-
|-
| idle || ** || Build phases by configuration (like in the old metrics page)
| DONE || ** || Summary graph (dashboard) of current CI status ([https://codereview.qt-project.org/126628 Code Review], [https://codereview.qt-project.org/126629 Code Review])
|-
| idle || ** || Build phases by configuration (like on the old metrics page)
|-
|-
| idle  || * || Build phases by configuration: Tick box to select if idle times are included or not in the Build phases graph
| idle  || * || Build phases by configuration: Tick box to select if idle times are included or not in the Build phases graph
Line 110: Line 128:
|}
|}


=== 4. Other features ===
=== Other features ===


{| class="wikitable sortable"
{| class="wikitable sortable"
Line 116: Line 134:
! Status !! Priority !! Item
! Status !! Priority !! Item
|-
|-
| idle || *** || Web security
| DONE || **** || Web security ([https://codereview.qt-project.org/#/c/115252/ Code Review])
|-
|-
| idle || ** || Utilise Ajax for better user experience
| DONE || *** || Performance and user experience <br />
utilising Ajax for selected pages [https://codereview.qt-project.org/125354 Code Review]<br />
other changes: database query change [https://codereview.qt-project.org/121193 Code Review], HTML compression [https://codereview.qt-project.org/121197 Code Review], HTML minification [https://codereview.qt-project.org/121198 Code Review]
|-
|-
| idle || * || Progress bar when loading page or data
| DONE || *** || Take template engine into use ([https://codereview.qt-project.org/115659 Code Review])
|-
| DONE || *** || Align the UI with Qt brand guidelines ([https://codereview.qt-project.org/126510 Code Review], [https://codereview.qt-project.org/126779 Code Review])
|-
| DONE || *** || Admin: Remove old data from the database (timescale fixed or selectable), keeping the referential integrity in mind ([https://codereview.qt-project.org/126144 Code Review])
|-
| DONE || *** || Admin: Remove branch, keeping the referential integrity in mind ([https://codereview.qt-project.org/126144 Code Review])
|-
| DONE || ** || Admin: Archive a branch by tagging it archived so that it is not shown on the views without deleting the data ([https://codereview.qt-project.org/126164 Code Review])
|-
| DONE || ** || Indication on content loading ([https://codereview.qt-project.org/121194 Code Review])
|-
| DONE || ** || Indication on database refresh / log parsing ([https://codereview.qt-project.org/126155 Code Review])
|-
| DONE || ** || Save the external libraries used as git submodules ([https://codereview.qt-project.org/126720 Code Review], [https://codereview.qt-project.org/126714 Code Review])
|-
|-
| idle || * || Link(s) to other related Qt Metrics
| idle || * || Link(s) to other related Qt Metrics
Line 143: Line 177:
|}
|}


=== 5. Documentation ===
=== Documentation ===


{| class="wikitable sortable"
{| class="wikitable sortable"
Line 149: Line 183:
! Status !! Priority !! Item
! Status !! Priority !! Item
|-
|-
| idle || ** || Implementation solution description
| in progress || *** || Implementation solution description
|-
| in progress || *** || Database design
|-
| DONE || ** || Site map ([https://codereview.qt-project.org/122764 Code Review])
|-
|-
| in progress || ** || Database design
| DONE || ** || Documentation links into header ([https://codereview.qt-project.org/126157 Code Review])
|-
|-
| in progress || * || Class documentation
| idle || * || Class documentation
|}
|}

Latest revision as of 07:45, 1 October 2015

This is the backlog for Qt Metrics 2 site that is currently under development.

The priority **** items are the core functionality and would be the target prior to launch or wider use.

The priority *** items are considered content/functionality for basic use as requested by the key stateholders.

In case of proposals or improvement ideas please contact Juha Sippola (report builder, database) or Tony Sarajärvi (parser).

Data reliability and completeness (database and parser)

Status Priority Item
DONE **** Show the actual testset parent (not the Qt5 where it was run) (Code Review)
DONE **** Change build_number (int) to build_key (string) for the new-CI (Code Review); UPDATE: type changed to 64bit int, BIGINT in MySQL (Code Review)
DONE *** Get testfunctions from xml (Code Review)
DONE *** Get testrows from xml (Code Review)
DONE *** Change testset_run.testcases_xxx to total_xxx (terminology change) (Code Review)
DONE *** Database indexing and query optimization (for performance)
idle * Database normalisation: result fields in the *_run tables should have a separate table instead of the enums (could also make the naming more flexible)
idle * Database normalisation: db_status table does not have a primary key (table holds just one row with the latest data)
idle * Database normalisation: conf.features is not perfect design, instead a separate feature table should be used (how this information will be used in the end)

Pages and views

Status Priority Item
DONE **** Home page modifications (implemented as part of the Overview, Project, Platform and Testset page changes)
DONE **** Overview page (Code Review)
DONE **** Project pages for build project (Qt5) and testset projects (Code Review)
DONE *** Project pages: Link to log file (Code Review)
DONE **** Platform page (Code Review)
DONE *** Platform page: Link to log file (Code Review)
DONE ** Platform page: Configuration flag identification (Code Review)
idle * Platform page: Identification of configurations where insignificant or force success could be removed
idle * Platform page: Filtering mechanism
DONE *** Configuration page (Code Review)
DONE *** Configuration page: Link to log file (Code Review)
idle * Branch page
DONE **** Top failing testsets page (Code Review / UI, Code Review / parser)
DONE **** Top flaky testsets page (see Code Review links for Top failing testsets page)
DONE **** Testset page (Code Review)
DONE *** Testset page: Link to log file (Code Review)
DONE *** Testset page: Testset flag identification (Code Review)
DONE ** Testset page: Testset run times (Code Review)
idle * Testset page: Filtering mechanism
idle * Testset page: Identification of testsets where insignificant could be removed
DONE *** Testset page: Test function information (Code Review, Code Review)
DONE *** Testset page: Test function flag identification (Code Review)
DONE ** Page to identify test functions where blacklisting could be removed (Code Review, Code Review)
DONE ** Page to identify test rows where blacklisting could be removed (Code Review)
DONE ** All pages: Link to build directory and xml file (Code Review)
DONE *** Test function page: Test row information (Code Review)
DONE ** Top failed testfunctions page (Code Review)
DONE ** Top testset execution duration page (Code Review)
DONE ** Top test function execution duration page (Code Review)
idle ** Trends for top failed/flaky lists
idle ** Failing tests in branches filtered by platform os (target configuration)
idle ** Project/Platform/Testset pages: Way to expand or filter the range of builds (shows all by default), or allow configuration to be visible when scrolling horizontally
idle * Testfunction page/information: Find skipped testfunctions

Graphs

Status Priority Item
DONE ** Summary graph (dashboard) of current CI status (Code Review, Code Review)
idle ** Build phases by configuration (like on the old metrics page)
idle * Build phases by configuration: Tick box to select if idle times are included or not in the Build phases graph
idle * Build phases by configuration: Configuration result to the Build phases graph (e.g. to conf title)
idle * Build phases by configuration: Add tooltips into Build phases graph (e.g phase name and value)
idle * Visualize CI performance via testset execution time (to visualize sequential and parallel runs)
idle * Visualize CI performance via the trend of phase duration (one, or several if feasible) across the last 20 builds in selected Project Configuration
idle * Improvement progress trend graphs, e.g. on build success rates, build execution times, Configuration and Autotest force success and insignificant flag removals.

Other features

Status Priority Item
DONE **** Web security (Code Review)
DONE *** Performance and user experience

utilising Ajax for selected pages Code Review
other changes: database query change Code Review, HTML compression Code Review, HTML minification Code Review

DONE *** Take template engine into use (Code Review)
DONE *** Align the UI with Qt brand guidelines (Code Review, Code Review)
DONE *** Admin: Remove old data from the database (timescale fixed or selectable), keeping the referential integrity in mind (Code Review)
DONE *** Admin: Remove branch, keeping the referential integrity in mind (Code Review)
DONE ** Admin: Archive a branch by tagging it archived so that it is not shown on the views without deleting the data (Code Review)
DONE ** Indication on content loading (Code Review)
DONE ** Indication on database refresh / log parsing (Code Review)
DONE ** Save the external libraries used as git submodules (Code Review, Code Review)
idle * Link(s) to other related Qt Metrics
idle * Associate gerrit change ids with builds
idle * Include the bug id’s for test cases. This requires first a database or other solution where to get the data.
idle * Log file parsing helper (a way to find things in the last n logs)
idle * Register to receive error reports to a certain module (plus unregister)
idle * Report generator to create a detailed text & graph report from a view. This would be applicable for such data that takes long time to extract and format.
idle * Data export to excel/pdf from a view
idle * Identify which CI build machines failed most often and which CI builds take the longest time to execute
idle * Add Google Analytics to get information on the portal usage
idle * Support for https

Documentation

Status Priority Item
in progress *** Implementation solution description
in progress *** Database design
DONE ** Site map (Code Review)
DONE ** Documentation links into header (Code Review)
idle * Class documentation
Retrieved from "https://wiki-qt-io-staging.herokuapp.com/index.php?title=Qt_Metrics_2_Backlog&oldid=19639"