Qt-contributors-summit-2011-QDateTime

This article may require cleanup to meet the Qt Wiki's quality standards. Reason: Auto-imported from ExpressionEngine. Please improve this article if you can. Remove the {{cleanup}} tag and add this page to Updated pages list after it's clean.

QDateTime

This page details the discussions about QDateTime in the Date/Time and Localisation sessions.

This page was written by John Layt from KDE (jlayt@kde.org).

KDE would like to phase out using our own KDateTime, KLocalizedDate and KCalendarSystem implementations in favour of QDateTime but many changes would be required to do so. We believe most of these changes will be of use to other app developers, especially the QtMobility calendaring team. More details on the issues can be found on the KDE wiki at http://community.kde.org/KDE_Core/QtMerge/QDateTime. KDE code can be found at https://projects.kde.org/projects/kde/kdelibs/repository/revisions/master/show/kdecore/date.

It is proposed that Qt will fully implement the Unicode CLDR standard for date/time handling across all platforms.

Code Quality

Important code is inline and no d_ptr makes it impossible to change the implementation without breaking BC, this should be fixed.
Poor overflow checking in places returns incorrect results instead of an invalid date, fixing this in Qt4 was impossible as it is inline code.

John Layt to provide patches.

Calendar Systems

Calendar System support is standard in Windows and Mac and fully integrated into their date localization systems. Unicode CLDR defines Calendar System support and ICU implements it. Qt does not properly localise dates and date formats for those users who have selected an alternative Calendar System on these platforms. Linux does not have standardised support, but both KDE and ICU implement support.

Calendar	Win	Mac	CLDR	KDE
Gregorian	X	X	X	X
Hebrew	X	X	X	X
Islamic Civil	X	X	X	X
Persian / Jalali	X	X	X	X
Japanese Era	X	X	X	X
RoC / Taiwan / Minguo	X	X	X	X
Thai / Buddhist Era	X	X	X	X
Coptic		X	X	X
Ethiopian		X	X	X
Ethiopian Amete Alem		X	X	X
Indian National		X	X	X
Islamic Lunar		X	X
Chinese Lunisolar	X		X
Islamic Um Al Qura	X
Koran / Tangun	X
East Asian Lunisolar	X
Japanese Lunisolar	X
Taiwan Lunisolar	X
Julian				X
ISO 8601
Hindu Lunar
North Korean / Juche

Note that some of these calendars are just the Gregorian calendar with different Epochs or Eras, i.e. ISO8601, Japanese Era, Thai, RoC, North Korean (same as RoC). The Coptic and Ethiopean are also the same calculation with different Epochs. The various Asian Lunisolar calendars are all derived form the Chinese Lunisolar. Windows uses separate implementations of the Gregorian for different digit sets or translations (Arabic, English, localized, etc) but others cover this using api.

Lars in favour of minimal set of calendars common to all platforms, but this will not solve the issue of say .Net and Qt apps running side-by-side and showing different format dates. Compromise may be that on Windows and Mac the date formatting/parsing internally uses Win/Mac system calendar system, but on Linux Qt only supports minimal set which are then available for explicit use on all platforms.

One consequence of this will be separate Calendar System support for Gregorian and Julian calendars, the current hybrid calendar would be removed and it would be up to the app to decide if/when they need to switch between the different calculations.

Proposed api:

1.  enum QLocale::CalendarSystem {LocaleCalendar, GregorianCalendar, JulianCalendar, ... };  // All possible calendar systems
2.  QList<QLocale::CalendarSystem> QLocale::calendarSystems();  // Returns the list of host system calendars
3.  QLocale::CalendarSystem QLocale::calendarSystem();  // Returns the locale calendar system
4.  QString QLocale::calendarSystemName(QLocale::CalendarSystem calSys=QLocale::LocaleCalendar);
5.  QLocale::CalendarSystem QDate::calendarSystem();  // Returns the QDate instance calendar system, usually the locale one
6.  void QDate::setCalendarSystem(QLocale::CalendarSystem calSys);  // Set the QDate instance calendar system, LocaleCalendar sets to locale calendar

The enum will list all possible calendars for all platforms with a calendarSystems() returning which ones are available on a given platform.

Implement QCalendarSystem which is embedded in QDateTime, possibly doesn’t need to be public? QCalendarSystem to provide names to QDateTime (but how does that affect the QLocale month/day methods?). KDE can sub-class and extend using platform backend for calendars Qt does not want to support directly?

John Layt will work on a design and merge request for this.

Time Zones

Proper timezone support is required to allow apps like calendaring to use QDateTime. While full support inside QDateTime would be ideal, it should at least be able to act as the data container with apps providing their own advanced manipulation tools. KDE has full cross-platform time zone support which could be adapted. Qt Mobility is also interested in this area and any solution should be coordinated with them.

Cross platform support is a major issue, apps need a consistent api and naming standard, for example to create a QTimeZone instance for Central European Time (CET). Lars in favour of a single implementation and database in Qt common to all platforms to give apps a consistent experience, with Qt providing a conversion layer to system time zone, especially on Windows.

Using a tz file shipped with Qt would quickly go out of date as the tz file is updated very frequently, sometimes even weekly. The system tz file should be preferred if available as it is more likely to be automatically kept up-to-date. Only if no system file is found should Qt fall back to its own file.

Unix tz zones.tab is regularly updated by OS, but no translations and no canonical ID’s.

Mac uses own tz file and translations. Mac api should provide inspiration for Qt api.

Windows does have its own time zone database stored in the registry and is regularly updated, but the names and definitions are very different to the Unix tz / Olsen zones. KDE implements a translation layer to convert Windows zones and names into Olsen zones and names which could be used.

CLDR time zones (http://www.unicode.org/reports/tr35/tr35-15.html#Timezone_Names) based on Olson zones.tab but provides canonical ID’s for each zone and translations for all zone names. Problem comes with lack of regular system updates.

Possible api:

• add enum value TimeSpec::TimeZone
• add new QTimeZone class
• add QDateTime timeZone() api to return the zone value if TimeSpec set to TimeZone, otherwise return null QTimeZone

Date/Time Span/Duration

QTime is restricted to a range from 00:00 to 23:59. However there is often a need for time durations such 52 hours or 52:00 which QTime is unable to represent or format. Either QTime should be extended to allow for a duration mode or a QDuration/QTimeSpan class created. Basically a period of x milliseconds, but able to be relative to a start date, and having various maths and formatting functions.

KDE has an implementation in KCalCore::Duration http://api.kde.org/4.x-api/kdepimlibs-apidocs/kcalcore/html/classKCalCore_1_1Duration.html using KDateTime

QTimeSpan is a merge proposal at https://qt.gitorious.org/qt/qt/merge_requests/1014.

Does QtMobility Organizer have a similar class?

Valid Date Range

The date is stored as a Julian Day Number in a uint with jd == 0 erroneously considered to be invalid. This restricts dates to a range of 2 January 4713BC to about 11 million AD. Changing the uint into an int32 or a int64 will make this date range more useful scientifically:

• int32 = 5.8 million BC to 5.8 million AD
• int64 = 25 Quadrillion BC to 25 Quadrillion AD
• Geological time: Age of Earth = 4.5 billion years old
• Astronomical time: Age of Universe = 13.75 billion years old

So an int32 is sufficient for geological time, but int64 seems overkill to gain astronomical time.

In theory every value of JD will be considered valid, although the lowest negative number may be designated the null value rather than using a bool.

Currently used Gregorian/Julian formulas do not reliably support converting JD’s to/from Dates outside of 8000BC to 8000AD, but this can be later extended.

isValid() will indicate if the current JD lies in a valid date range, i.e. one supported by the conversion formulas of the current Calendar System.

isNull() will indicate an uninitialised date or completely invalid date, e.g. where date maths has overflowed the JD storage size.

Nothing should stop setting Julian Days outside the isValid() date range as this will still be scientifically useful, you just won’t be able to convert them to Date form or do Date maths beyond adding days.

John Layt to submit patches.

Date Locale Functions

Some additional localization options could be added to QLocale to support date formatting, these are all defined by Unicode CLDR:

• Short Year Window
• Digit Sets
• Calendar System
• Week Number System (support US weeks as well as ISO)

Note Week Start Date and Working Week have been added for 4.8.

Extra Calendar/Date Attributes

Add additional standard date attributes defined in the CLDR such as quarter, months in year, etc, that are useful for calendar systems and date formatting support.

Proposed api:

1. int QDate::monthsInYear()
2. int QDate::weeksInYear()
3. int QDate::quarter()
4. int QDate::weekOfMonth()
5. int QDate::weekdayInMonth()

See also Name functions and Date Format Codes which implement string versions of some of these.

John Layt to submit patches.

Date Name formatting

The current QLocale and QDate api for month/weekday names is rather messy and mixed up due to incremental changes and is marked for clean-up in Qt5. The following new api based on CLDR is proposed:

1.  enum QLocale::NameFormat {LongName, ShortName, NarrowName}  //Replaces QLocale::FormatType
2.  enum QLocale::NameContext {SentanceFormat, StandaloneFormat}  //Replaces QDate::MonthNameType
3.  QString QLocale::monthName(int month, QLocale::NameFormat format = QLocale::LongName, QLocale::NameContext context = QLocale::SentenceFormat)
4.  QString QLocale::weekdayName(int weekday QLocale::NameFormat format, QLocale::NameContext context = QLocale::SentenceFormat)
5.  QString QLocale::quarterName(int quarter, QLocale::NameFormat format, QLocale::NameContext context = QLocale::SentenceFormat)
6.  QString QDate::monthName(QLocale::NameFormat format = QLocale::LongName, QLocale::NameContext context = QLocale::SentenceFormat)
7.  QString QDate::weekdayName(QLocale::NameFormat format, QLocale::NameContext context = QLocale::SentenceFormat)
8.  QString QDate::quarterName(QLocale::NameFormat format, QLocale::NameContext context = QLocale::SentenceFormat)

Alternatively NameFormat and NameContext could be merged but this is less future proof, i.e. if more contexts are later required:

1.  enum QLocale::NameFormat {LongName, ShortName, NarrowName, StandaloneLongName, StandaloneShortName, StandaloneNarrowName}

Date/Time Format Codes

The Qt to/from string format codes are mostly based on the Unicode CLDR formats but have a couple of problems:

• Only a small sub-set of CLDR codes are supported
• A few Qt format codes are not in the CLDR standard
• Currently 3 different implementations of parsers/formatters, need simplifying to one

These format codes also have equivalents in the standard Posix, Windows and OSX date/time formats which are used in localization, so by not supporting them Qt may not be correctly localizing dates to the users system date/time formats. Many apps also need more flexibility in formatting dates.

Most of the missing format codes are just string forms of existing Qt api.

Reference http://www.unicode.org/reports/tr35/tr35-15.html#Date_Format_Patterns

Qt currently supports the following format codes:

1. d    Day as a number without a leading zero (1 to 31)
2. dd   Day as a number with a leading zero (01 to 31)
3. ddd  Weekday abbreviated localized name (e.g. 'Mon' to 'Sun').
4. dddd Weekday long localized name (e.g. 'Monday' to 'Sunday').
5. M    Month as a number without a leading zero (1 to 12)
6. MM   Month as a number with a leading zero (01 to 12)
7. MMM  Month abbreviated localized name (e.g. 'Jan' to 'Dec').
8. MMMM Month long localized name (e.g. 'January' to 'December').
9. yy   Year as two digit number (00 to 99)
10. yyyy Year as four digit number. (-9999 to 9999)
11. h    Hour without a leading zero (0 to 23 or 1 to 12 if AM/PM)
12. hh   Hour with a leading zero (00 to 23 or 01 to 12 if AM/PM)
13. H    24 Hour without a leading zero (0 to 23)
14. HH   24 Hour with a leading zero (00 to 23)
15. m    Minute without a leading zero (0 to 59)
16. mm   Minute with a leading zero (00 to 59)
17. s    Second without a leading zero (0 to 59)
18. ss   Second with a leading zero (00 to 59)
19. z    Milliseconds without leading zeroes (0 to 999)
20. zzz  Milliseconds with leading zeroes (000 to 999)
21. AP   AM/PM uppercase
22. ap   AM/PM lowercase
23. A    AM/PM uppercase
24. a    AM/PM lowercase
25. t    Timezone (for example "CEST")

Note that:

• ddd and dddd formats are not valid CLDR forms but derived from Windows, they should be EEE and EEEE instead.
• AP and ap are not valid CLDR forms
• h and hh do not exactly match the CLDR definition
• t is not a valid CLDR form

The following CLDR codes should be supported:

1. ddddd Day narrow localized name (e.g. 'M' to 'S').
2. MMMMM Month narrow localized name (e.g. 'J' to 'D').
3. y     Year without a leading zero (1 to 9999)
4.  
5. D (1..3)
6. Day of Year, numeric with or without leading zeros.
7. Matches existing dayOfYear() api.
8.  
9. w (1-2)
10. Week of Year, numeric with or without leading zero.
11. Matches existing weekNumber() api.
12.  
13. Y (1..n)
14. Week year, e.g. for ISO Week Number year week falls in, forms to match y.
15. Matches existing weekNumber() api.
16.  
17. L (1..5)
18. Stand-alone Month
19.  
20. E (1..5)
21. Weekday, abbreviated (e.g. 'Mon'), long (e.g. 'Monday') and narrow (e.g. 'M')
22.  
23. c (1..5)
24. Stand-alone Weekday
25.  
26. G (1..5)
27. Era localized name, abbreviated (e.g. 'AD'), long (e.g. 'Anno Domini') and narrow (e.g. 'A')
28.  
29. z/Z/v/V
30. Timezones.  TODO.  Depends on QTime support.
31.  
32. Q (1..4)
33. Quarter in year, '2', '02', 'Q2' or '2nd Quarter'
34.  
35. q (1..4)
36. Stand-alone Quarter in year
37.  
38. W (1)
39. Week of Month, numeric.
40.  
41. F (1)
42. Day of Week In Month, e.g. if today is 2nd Monday in Month = 2
43.  
44. e (1..5)
45. Weekday, localized numeric (1 or 01), abbreviated (e.g. 'Mon'), long (e.g. 'Monday') and narrow (e.g. 'M').
46. Would require QLocale support for localized week number, i.e. US week as well as ISO week.

It was suggested to use a regex based fromString() method like in QTimeSpan. This can wait for 5.1 or later.

Agreed to remove non-standard format codes, implement as many standard CLDR codes as possible. Initial implementation may not be most efficient but can be improved later. Must be in 5.0 as changes Source Compatibility and changes behaviour.

John Layt to submit patches.

Date/Time Named Formats

QDateTime supports a number of named date/time formats:

1.     enum Qt:DateFormat {
2.         TextDate,      // default Qt
3.         ISODate,       // ISO 8601
4.         SystemLocaleDate, // deprecated
5.         LocalDate = SystemLocaleDate, // deprecated
6.         LocaleDate,     // deprecated
7.         SystemLocaleShortDate,
8.         SystemLocaleLongDate,
9.         DefaultLocaleShortDate,
10.         DefaultLocaleLongDate
11.     };

This enum is used by QDate, QTime and QDateTime so being called DateFormat is a misnomer and should be fixed, either by splitting into date/time/datetime enums or making the value names more generic.

The CLDR standard defines standard date formats for Full, Long, Medium, and Short, all of which are also supported by Mac and Windows. The extra Full and Medium formats should be added.

The difference between the System* and Default* formats should be investigated and if possible split to a second enum? The deprecated ones should be removed, and TextDate investigated to see if it is meaningful anymore.

The following common formats supported in KDE could also be added:

• ISOTime – ISO 8601 format time
• ISODateTime – ISO 8601 format date and time
• RFCDateTime – RFC 822/850/1036/1123/2822 format
• RFCDateTimeDay – RFC format including day of the week
• RFC3339DateTime – RFC 3339 format
• ISOWeekDate – ISO 8601 Week Date format
• ISOOrdinalDate – ISO 8601 Ordinal Date format

The ISO dates should accept both basic and expanded formats.

Note KDateTime uses regex’s for some of these in an inefficient way so don’t just copy the code.

Possible api:

1. enum QLocale::DateTimeFormat { DefaultFormat, FullFormat, LongFormat, MediumFormat, ShortFormat,
2.                                IsoFormat, RfcFormat, Rfc3339Format, WeekFormat, OrdinalFormat }

Where the context of the class will determine if it is a date, time or datetime version. Where a format is inappropriate the default format will be returned.

Alternative api

1. enum QLocale::DateTimeFormat { DefaultDateTime, FullDateTime, LongDateTime, MediumDateTime, ShortDateTime,
2.                                IsoDateTime, RfcDateTime, Rfc3339DateTime, WeekDateTime, OrdinalDateTime }
3.  
4. enum QLocale::DateFormat { DefaultDate, FullDate, LongDate, MediumDate, ShortDate,
5.                              IsoDate, RfcDate, Rfc3339Date, WeekDate, OrdinalDate }
6.  
7. enum QLocale::TimeFormat { DefaultTime, FullTime, LongTime, MediumTime, ShortTime,
8.                             IsoTime, RfcTime, Rfc3339Time, WeekTime, OrdinalTime }

Date Eras

Many locales have an optional Era system for dates, others such as Japan use an era system in its official date formats. Proper localisation support requires this to be added, especially if Calendar Systems are supported.

Proposed api:

1. QString QDate::eraName()
2. int     QDate::yearInEra()
3. QString QDate::eraName(QLocale::NameFormat format)bool
4. QDate::isValidEraDate(QString eraName, int yearInEra, int month, int day)
5. bool QDate::setEraDate(QString eraName, int yearInEra, int month, int day)

John Layt to submit patches.

ISO Ordinal Date

The ISO 8601 standard defines an Ordinal Date standard of yyyy-ddd for day ddd in year yyyy. Qt already provides api for dayOfYear(). Support could be added to to/from string using the standard Unicode format codes and a named date format, and api provided for directly setting the date using the ordinal date.

Proposed api:

1. bool QDate::isValidOrdinalDate(int year, int dayOfYear)
2. bool QDate::setOrdinalDate(int year, int dayOfYear)

John Layt to submit patches.

ISO Week Date

The ISO 8601 standard defines a Week Date standard of yyyy-Www-d for weekday d in ISO week ww of year yyyy. Qt already provides api for weekNumber(). Support could be added to to/from string using the standard Unicode format codes and a named date format, and api provided for directly setting the date using the week date.

Proposed api:

1. bool QDate::isValidIsoWeekDate(int year, int weekOfYear, int dayOfWeek)
2. bool QDate::setIsoWeekDate(int year, int weekOfYear, int dayOfWeek)

If support for week number systems other than ISO Weeks is added (e.g. US Weeks) then the api would be changed to remove the ISO form the name and optionally add the week number system as a parameter.

John Layt to submit patches

Date Maths

Qt currently only provides standard date math functions for adding years, months or days. Some more convenience functions could be added.

• Date difference (inverse of addYMD())
• Years difference (inverse of addYears())
• Months difference (inverse of addMonths())

The following are useful with Calendar System support:

• First day of year
• Last day of year
• First day of month
• Last day of month

Possible api:

1. void QDate::dateDifference(const QDate &toDate, int *years, int *months, int *days, int *sign)
2. int  QDate::yearsDifference(const QDate &toDate)
3. int  QDate::monthsDifference(const QDate &toDate)
4. int  QDate::daysDifference(const QDate &toDate)
5. QDate QDate::firstDayOfYear()
6. QDate QDate::lastDayOfYear()
7. QDate QDate::firstDayOfMonth()
8. QDate QDate::lastDayOfMonth()

Look in KCalCore and QtOrganizer to see what other date math / classes might be useful.

Widgets

The various widgets will need modification to match the new features and api and to resolve old bugs.

QDateTimeEdit needs to support null dates and extend the supported date range.

QCalendarWidget needs to use Start of Week and Working Week Start/End from QLocale, but be able to override if needed.

Some new widgets may be needed,perhaps based on KDE combo box versions.