Qt wiki will be updated on October 12th 2023 starting at 11:30 AM (EEST) and the maintenance will last around 2-3 hours. During the maintenance the site will be unavailable.
Openmax-decoding-of-images: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
AutoSpider (talk | contribs) (Decode HTML entity names) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
= | {{Cleanup | reason=Auto-imported from ExpressionEngine.}} | ||
[[Category:QtEcosystemTeamTasklist]] | |||
= OpenMAX decoding of images = | |||
== | == Description == | ||
# Investigate OpenMAX IL | Add OpenMAX IL as a backend for image decoding. Use hardware decoder to directly decode into graphics-memory (zero-copy-decoding). | ||
Additional we need to keep [[Openmax backend for mm kit]] in mind, as this is the next logical step in improving performance/features. | |||
== Sub tasks == | |||
# Investigate OpenMAX IL API and concept | |||
# Implement a prove of concept in application space | # Implement a prove of concept in application space | ||
# First, small benchmark application to compare implementation with | # First, small benchmark application to compare implementation with "normal" decoding | ||
# Integrate implementation into Qt image-loading code | # Integrate implementation into Qt image-loading code | ||
# Improve benchmark application; Run sophisticated tests | # Improve benchmark application; Run sophisticated tests | ||
| Line 15: | Line 20: | ||
# white-paper | # white-paper | ||
==Optional sub tasks== | == Optional sub tasks == | ||
This sub tasks might or might not be covered, based on the findings and decision happening. | This sub tasks might or might not be covered, based on the findings and decision happening. | ||
* Implement Qt-OpenMAX_IL-wrapper | * Implement Qt-OpenMAX_IL-wrapper | ||
* build/port OpenMAX AL for the rpi | * build/port OpenMAX AL for the rpi | ||
* build/port gst-openmax | * build/port gst-openmax | ||
This sub tasks are in relation with [[Openmax backend for mm kit]] | |||
==Logbook== | == Logbook == | ||
===28. February | === 28. February - Thomas Senyk === | ||
* Reading OpenMAX IL Specification (and building my first Qt5 for rpi) | * Reading OpenMAX IL Specification (and building my first Qt5 for rpi) | ||
* Searching for example code | * Searching for example code | ||
===29. February | === 29. February - Thomas Senyk === | ||
* Further searching for example code … found gst-openmax, but gst code | * Further searching for example code … found gst-openmax, but gst code isn't great as a reference ;) | ||
* Got some special code underhand | * Got some special code underhand :) | ||
===1. March | === 1. March - Thomas Senyk === | ||
* First look at the handed OpenMAX IL example indicates the following: | * First look at the handed OpenMAX IL example indicates the following: | ||
** OpenMAX IL is a micro-controll | ** OpenMAX IL is a micro-controll API (as suspected) | ||
** OpenMAX IL is a async c api, with callback, state-control, ect. (as suspected) | ** OpenMAX IL is a async c api, with callback, state-control, ect. (as suspected) | ||
* The conclusion is that there are 3 options to continue: | * The conclusion is that there are 3 options to continue: | ||
# implement a image-decoder-plugin which directly uses openmax il, don't do much abstraction, just use it. | |||
# write a proper Qt/c++ API to access basic OMX IL interfaces (components, tunnels, callbacks, ect.). … and based on that, write a image decoder plugin. | |||
# As there is already a "convenience" API "for" OpenMAX IL: Try to get OpenMAX AL running. | |||
Option 3 seams to be the most effective and cleanest approach, it's preferable but depends on how easy it is to build/intergrate or of rpi will eventually support it out-of-the-box | |||
===2. March | === 2. March - Thomas Senyk === | ||
* Further investigation of options | * Further investigation of options | ||
* Started proof-of-concept implementation | * Started proof-of-concept implementation | ||
===13. March | === 13. March - Thomas Senyk === | ||
* Finished implementation. | * Finished implementation. It's "running", but it's only producing errors so far. | ||
* The problem now is to understand the behavior, expectation and communication with OpenMAX IL. | * The problem now is to understand the behavior, expectation and communication with OpenMAX IL. | ||
* It could take a while to figurer everything out and get something to the screen | * It could take a while to figurer everything out and get something to the screen | ||
===14. March | === 14. March - Thomas Senyk === | ||
* Got more understanding of the behavior and expectations of | * Got more understanding of the behavior and expectations of OMX IL | ||
* Sorted a few setup things out | * Sorted a few setup things out | ||
* reimplemented the thread-sync code | * reimplemented the thread-sync code | ||
* got to a state where a got a few specific errors which need so be solved | * got to a state where a got a few specific errors which need so be solved | ||
* | * THE main error I got is that I can't set: OMX_SetParameter(''', OMX_IndexParamPortDefinition, &portSettings) on any of my 2 components (decoder and renderer) … which sucks because I eventually need to set the EGL_Display | ||
''' Next thing to do: a lot of cleaning, reading/removing my left-over 'TODO's | |||
* Then: fix problems | * Then: fix problems | ||
* After that: world domination! | * After that: world domination! | ||
===15. March | === 15. March - Thomas Senyk === | ||
* A lot of cleanup | * A lot of cleanup | ||
| Line 74: | Line 80: | ||
* Code seams to be ok | * Code seams to be ok | ||
* Had successful run of image-decoding … seams to be very unstable … probably some check/wait/additional property missing … | * Had successful run of image-decoding … seams to be very unstable … probably some check/wait/additional property missing … | ||
* | * couldn't get anything to screen … I hate opengl es 2.0 … need to fix painting, then retry | ||
===16. March | === 16. March - Thomas Senyk === | ||
* very quickly fixed the painting … was a stupid error | * very quickly fixed the painting … was a stupid error | ||
* clean up of the code | * clean up of the code | ||
* created repository | * created repository | ||
* | * it's seams to work now, but texture doesn't change … I need to reboot everytime I start the application … at the 2nd execute (without reboot) the program hangs on OMX_FillThisBuffer(mRenderHandle, eglBuffer); … the 3rd time it can't even init EGL. | ||
I need help to fix this … going to get in contact with the xbmc guys. | I need help to fix this … going to get in contact with the xbmc guys. | ||
===20. March | === 20. March - Thomas Senyk === | ||
* some communication with xbmc guys … but the tips so far | * some communication with xbmc guys … but the tips so far didn't help | ||
* still stuck … no clue how to proceed | * still stuck … no clue how to proceed | ||
===21. March | === 21. March - Thomas Senyk === | ||
* xbmc guys still silent | * xbmc guys still silent | ||
* posted on khronos forum: | * posted on khronos forum: http://www.khronos.org/message_boards/viewtopic.php?f=31&t=4851 … hope that brings some light into the problem | ||
Latest revision as of 17:28, 12 March 2015
| This article may require cleanup to meet the Qt Wiki's quality standards. Reason: Auto-imported from ExpressionEngine. Please improve this article if you can. Remove the {{cleanup}} tag and add this page to Updated pages list after it's clean. |
OpenMAX decoding of images
Description
Add OpenMAX IL as a backend for image decoding. Use hardware decoder to directly decode into graphics-memory (zero-copy-decoding). Additional we need to keep Openmax backend for mm kit in mind, as this is the next logical step in improving performance/features.
Sub tasks
- Investigate OpenMAX IL API and concept
- Implement a prove of concept in application space
- First, small benchmark application to compare implementation with "normal" decoding
- Integrate implementation into Qt image-loading code
- Improve benchmark application; Run sophisticated tests
- Upstream
- white-paper
Optional sub tasks
This sub tasks might or might not be covered, based on the findings and decision happening.
- Implement Qt-OpenMAX_IL-wrapper
- build/port OpenMAX AL for the rpi
- build/port gst-openmax
This sub tasks are in relation with Openmax backend for mm kit
Logbook
28. February - Thomas Senyk
- Reading OpenMAX IL Specification (and building my first Qt5 for rpi)
- Searching for example code
29. February - Thomas Senyk
- Further searching for example code … found gst-openmax, but gst code isn't great as a reference ;)
- Got some special code underhand :)
1. March - Thomas Senyk
- First look at the handed OpenMAX IL example indicates the following:
- OpenMAX IL is a micro-controll API (as suspected)
- OpenMAX IL is a async c api, with callback, state-control, ect. (as suspected)
- The conclusion is that there are 3 options to continue:
- implement a image-decoder-plugin which directly uses openmax il, don't do much abstraction, just use it.
- write a proper Qt/c++ API to access basic OMX IL interfaces (components, tunnels, callbacks, ect.). … and based on that, write a image decoder plugin.
- As there is already a "convenience" API "for" OpenMAX IL: Try to get OpenMAX AL running.
Option 3 seams to be the most effective and cleanest approach, it's preferable but depends on how easy it is to build/intergrate or of rpi will eventually support it out-of-the-box
2. March - Thomas Senyk
- Further investigation of options
- Started proof-of-concept implementation
13. March - Thomas Senyk
- Finished implementation. It's "running", but it's only producing errors so far.
- The problem now is to understand the behavior, expectation and communication with OpenMAX IL.
- It could take a while to figurer everything out and get something to the screen
14. March - Thomas Senyk
- Got more understanding of the behavior and expectations of OMX IL
- Sorted a few setup things out
- reimplemented the thread-sync code
- got to a state where a got a few specific errors which need so be solved
- THE main error I got is that I can't set: OMX_SetParameter(, OMX_IndexParamPortDefinition, &portSettings) on any of my 2 components (decoder and renderer) … which sucks because I eventually need to set the EGL_Display
Next thing to do: a lot of cleaning, reading/removing my left-over 'TODO's
- Then: fix problems
- After that: world domination!
15. March - Thomas Senyk
- A lot of cleanup
- Better understanding of what happens when/how/where
- Code seams to be ok
- Had successful run of image-decoding … seams to be very unstable … probably some check/wait/additional property missing …
- couldn't get anything to screen … I hate opengl es 2.0 … need to fix painting, then retry
16. March - Thomas Senyk
- very quickly fixed the painting … was a stupid error
- clean up of the code
- created repository
- it's seams to work now, but texture doesn't change … I need to reboot everytime I start the application … at the 2nd execute (without reboot) the program hangs on OMX_FillThisBuffer(mRenderHandle, eglBuffer); … the 3rd time it can't even init EGL.
I need help to fix this … going to get in contact with the xbmc guys.
20. March - Thomas Senyk
- some communication with xbmc guys … but the tips so far didn't help
- still stuck … no clue how to proceed
21. March - Thomas Senyk
- xbmc guys still silent
- posted on khronos forum: http://www.khronos.org/message_boards/viewtopic.php?f=31&t=4851 … hope that brings some light into the problem