QtAppMan-Intents

Part I -- Status Quo

Status quo after reviewing Eddy's comparison and looking into the specifics of using (and also providing) Intents on Android and Apple's new SiriKit/Intent Framework.

• Apple's SiriKit approach isn't really an open interface for loose coupling between applications, but a strongly typed IPC description that only Apple itself can extend.
• Android's Intents on the other side have a very flexible interface to send any data back and forth, but adds unnecessary complexity by giving the content provider multiple options to attach (meta)data to requests (this most likely is the result of Android's growth while trying to keep backwards compatibility). In addition Android imposes a tight coupling between Intents, background services, Activities and the implementation language (Java), which simplifies things a lot in the Android world, but which is also not something we want to carry over into the Qt world.

Just introducing Intents is not enough for that concept to be useful in real world applications though: there are at least two more concepts that need to be implemented:

1. content sharing: Android uses content:// URLs to return any type of data, which can be resolved by the caller to either a file:// URL or a (sort of) SQL query for structured data.
2. background services: This is not 100% necessary, but helps with overall resource usage and snappiness of the system, if not all applications that advertise intents have to be running all the time in full mode which may bind QML and even OpenGL resources.

As for content sharing, the question is, if returning 'structured data' via a SQL query cursor is really necessary, or if returning a JSON/YAML array would be sufficient for these cases. Returning pointers to existing files looks pretty basic, but will need additional trickery inside the runtime environment (e.g. QtApplicationManager) to support file access across applications running in containers. If those return values are gated behind a content:// URL like Android is doing it, it would be simple to even extend the system later on to support the SQL cursor-like returns, if just returning plain JSON/YAML turns out to be a problem.

Background services are an optimization step, that could very well be added later on, but it would be valuable to have a rough concept in place first. The idea here is that in order to avoid having the main UI process running all the time to answer Intent request, some (or all) of those Intents could also be handled by a light-weight, headless background process. An app could optionally have one or many background services, where each one would be responsible for handling one (or many) specific Intents: this would roughly mirror the iOS app extension model, but in a much more flexible way. In this case, each Intent description in the manifest would need an additional specifier for the service that would be responsible for answering.

Architecture wise, our idea would be to have the intent specifications inside the already existing info.yaml application manifest (just as Android does it). The benefit would be that (a) everything about an application is in one place and (b) it would be easy to reference 'capabilities' (for securing access to specific Intents) and potentially also define and reference background services.

As for API format of the parameters going into an Intent request and being returned back from one, a very simple, flexible and extendable solution would be to use JSON/YAML in a single field. Combined with content:// URLs this should enable sharing all kinds of data, while still keeping one consistent data-field that is provided to and returned from the Intent request. The required and returned fields in those JSON/YAML snippets have to be documented in a special Intent API document by the system manufacturer.

Part II -- Voice UI disambiguation example

One big issue that we are seeing here is the question of who should (or is even allowed) to display an UI as a result of an Intent request: this is especially relevant for light background services that would be handling Voice-UI data requests: if these services should do a needed disambiguation themselves, they either

• need to open a window (which will allocate QML, Wayland and OpenGL resources), or they
• could forward the request to the main app (which would negate any performance gain from light-weight/non-QML background services, plus adding additional IPC complexity), or they
• would supply QML files to the calling application (which would then defeat any security restrictions the system might have in place).

Putting the responsibility for showing a disambiguation UI and/or playing the voice prompt into a central daemon or the system-ui would require quite detailed Intent replies, but would also get rid of the problem space mentioned above altogether.

Part III -- QtCS discussion

Discussions at Qt contributor summit and with QtAuto partners.

Sadly there wasn't much feedback at QtCS regarding Intents, although there were quite a few people attending the session. The session was mostly me talking and presenting high-level concepts plus quite some affirmative nodding from the audience (which I guess is a good sign nonetheless).

One interesting topic that came from this was that KDE already has a very simple Intent-like framework called "Purpose" (https://github.com/KDE/purpose). It's pretty basic and currently only used to implement some "Share with" like functionality in selected KDE applications. Implementation wise, it is built around Qt plugins and their JSON meta-data: the Qt plugin loader is used as the shared central registry for the available intent handlers/plugins. Although the implementation is quite elegant, it leaves a lot to be desired security wise, since apps and the System-UI would have to load arbitrary shared libraries, that are provided by the app(s) providing intents, into their address space.

Another suggestion was that the central intent repository as well as the sharing of that data across all applications could be done by registering intent handlers as D-Bus interfaces that follow a specific naming scheme.

Part IV -- Example manifest with Intents

Example info.yaml file with Intent definitions:

formatVersion: 2
formatType: am-application
---
id:      'com.pelagicore.radio'
icon:    'icon.png'
code:    'Main.qml'
runtime: 'qml'
name:
  en: 'Radio'
  de: 'Radio'

categories: [ 'media' ]
## (not needed anymore) mimeTypes: [ 'x-scheme-handler/x-radio' ]

intents:
- intent: io.qt.openUrl  # standard mime-type handler replacement for x-scheme-handler/...
  schemeHandler: 'x-radio'

  ## parameters:
  ##   url

- intent: io.qt.openFile # standard mime-type handler replacement
  mimeType: 'audio/mpeg'
  
  ## parameters:
  ##   file
  
# custom intent for Neptune3, can be used for Voice UI
- intent: io.qt.neptune3.playRadioStation

  requiredCapabilities: [ radioControl ]
  
  ## parameters:
  ##    frequency: '98.5' or
  ##    stationName: 'Radio Foo' or
  ##    next: yes or
  ##    previous: yes

# custom intent for Neptune3, can be used for Voice UI
- intent: io.qt.neptune3.queryRadioStations
  
  requiredCapabilities: [ radioControl ]
  
  ## returns:
  ##    stations: list of station names