Qt partners can deliver support for Qt on their hardware boards a lot easier. We have improved and created features for Qt Board Support Package (QBSP) and ability to create Qt commercial builds on Yocto. The enablers will be part of both Qt 5.9 LTS and Qt 5.12 LTS.

As a partner, building and delivering open source versions of Qt under (L)GPL is a relatively simple affair: All the sources are available and the open source licensing allows the distribution of the assets as long as the open source licenses are attained with the packages.

Less so with a commercial license. If there was no out-of-box support as part of Qt for Device Creation partners or their customers have had to swap and add files manually. In addition, integrating developer tooling such as Qt Creator or Qt for Visual Studio had to be done manually with the help of documentation.

Sounds like a lot of hassle?

We agree.

That’s why we’ve been working hard to make it easier for Qt partners to create support for Qt embedded targets and test them as part of their own continuous integration and quality assurance processes.

QBSP for a super-easy developer tooling setup

The QBSP contains all the IDE configurations for deployment, debugging and profiling, as well as the toolchain and embedded target software required for embedded development. In other words, a developer can set up an embedded target on their IDE by simply installing a QBSP package instead of clicking around and manually configuring the environment.

QBSP has been out an about for some time but the feature hardening and documentation have been… … fragmented. We spent a fair amount of time this summer to make the feature simple to use for external parties. We have done the initial documentation of QBSP with Yocto-based embedded Linux as a target, but it can just as easily be used with any other target OS.

In addition, we are also using the QBSP feature ourselves in order to deliver to a wider range of Qt for Device Creation embedded targets (but that is another blog post).

Building commercial Qt from Git

We have also enabled an easy way to create commercial Qt builds from Git. This is primarily an enabler intended for operating system and hardware vendors integrating Qt as part of their own continuous integration and testing. In all simplicity, one can now create Qt images with a -commercial switch.

For verification that the Qt build for the hardware was a success and QBSP for the tooling works, we will initially provide partner documentation. The further planned evolution is to provide partners with additional test applications and mechanisms to improve automated testing capability on partner side.

Partner agreement required

The Qt License Agreement 4 explicitly forbids the distribution of commercial Qt or parts of it without a valid license. The rights to distribute Commercial Qt require a partnership agreement between you and The Qt Company. If you are interested in such a partnership agreement, please get in touch with us through https://www.qt.io/contact-us/

The post Board Support Package and Commercial Build Enablers for Qt Partners appeared first on Qt Blog.

In Qt for Device Creation 5.12.0, we have enabled additional content installation via QBSPs (Qt Board Support Packages). A QBSP combines toolchain, target device images and set of Qt Creator configurations for particular device into a single file that can be easily shared and installed using the Qt online installer or the Maintenance tool. Technically a QBSP is an archived repository created by the Qt Installer Framework, and it’s creation is now fully integrated into the Yocto builds that are used to create the Boot to Qt Software Stack content.

For all the target devices currently supported in the meta-boot2qt layer, you can create a QBSP simply by issuing a bitbake command:

bitbake meta-b2qt-embedded-qbsp

This will first build both the toolchain and the image, and then package those into a QBSP file with the required Qt Creator configurations. The resulting QBSP file is located in tmp/deploy/qbsp/ folder in your Yocto build environment. The QBSP packaging respects the SDKMACHINE variable, so that if you use environment variable SDKMACHINE=i686-mingw32, a Windows toolchain is packaged into the QBSP.

The Yocto integration is implemented in two classes that can be used even if your target device has not been otherwise integrated into the meta-boot2qt layer. By inheriting the classes and setting up the required variables, you can have a QBSP with your own image for your own target device.

qbsp-image.bbclass

The QBSP will need a suitable device image to include in the package and to achieve this you will need to inherit qbsp-image.bbclass in the image recipe you want to use. You can control the content of the package with variable QBSP_IMAGE_CONTENT. By default Boot to Qt images include a .img file and a .conf file used by the Flashing Wizard.

inherit qbsp-image

QBSP_IMAGE_CONTENT = "\
  ${IMAGE_LINK_NAME}.img \
  ${IMAGE_LINK_NAME}.conf \
  "

qbsp.bbclass

qbsp.bbclass is the main class that handles the creation of the QBSP and you can control the various aspects of it through variables. Most important ones are the dependencies to the toolchain and image task using QBSP_SDK_TASK and QBSP_IMAGE_TASK variables.

For the Boot to Qt Software Stack, this is done in the meta-b2qt-embedded-qbsp.bb recipe:

inherit qbsp

VERSION_SHORT = "${@d.getVar('PV').replace('.','')}"
QBSP_NAME = "Boot2Qt ${PV}"
QBSP_MACHINE = "${@d.getVar('MACHINE').replace('-','')}"
QBSP_INSTALLER_COMPONENT = "embedded.b2qt.${VERSION_SHORT}.yocto.${QBSP_MACHINE}"
QBSP_INSTALL_PATH = "/${PV}/Boot2Qt/${MACHINE}"

QBSP_SDK_TASK = "meta-toolchain-b2qt-embedded-qt5-sdk"
QBSP_IMAGE_TASK = "b2qt-embedded-qt5-image"

Rest of the variables are then used to control how the installer shows the component and where they are installed. You can optionally also, e.g., add an EULA to the package, which the user would need to accept before they can install the components. You can find this and all the other information about the QBSP specific variables in the documentation .

The post How to create QBSPs from Yocto builds appeared first on Qt Blog.

5 years ago today, this happened:

commit 11a94957dc038fc27c5ff976197ad2b2d0352d20
Author: Florian Bruhin 
Date:   Sat Dec 14 22:15:16 2013 +0100

    Initial commit

That's how qutebrowser looked a day after that (and that commit still seems to run!): https://imgur.com/a/xoG1r4G

Exactly a year later, things …

Wayland is a display server protocol used on modern Linux systems, the Qt Wayland platform plugin lets Qt applications run on Wayland display servers (compositors).

Continuing the trend from the Qt 5.11 release, the Qt 5.12 release contains a substantial amount of improvements.

Window decorations

Old window decorations

We’ve received quite a few complaints about the looks of our window decorations over the last few years. The message is usually: “They are ugly and don’t look native.”

On Wayland, window decorations are supposed to be drawn by the client by default. Consequently, Qt’s window decorations will probably never look identical to what other toolkits draw. I.e. there is no “native” look with client-side decorations. That is, however, no excuse for them to be ugly.

In 5.12, I’ve updated our default decoration plugin so it looks like something from this decade. It still won’t be “native” looking, but it looks closer much closer to what the other toolkits draw.

New window decorations

The default window background color is now used for the decorations, so it should follow your Qt theme quite nicely.

Decorations follow the theme of the window.

Furthermore, we’ve now also added support for support for the Wayland extension, xdg-decoration unstable v1. Compositors implementing this extension can now tell Qt clients that decorations will be drawn server-side instead of client-side. I.e. the decorations for Qt applications and other toolkits can be identical after all. This is also good news for people running tiling compositors. It’s no longer needed to set the environment variable QT_WAYLAND_DISABLE_WINDOWDECORATION on compositors that implement this extension. Toggling window decorations at runtime is now also possible.

The server-side-decoration QtWayland Compositor example. The bar at the top with the close button is drawn and handled by the compositor.

xdg-shell stable

On Wayland, the shell extension is the part of the protocol that gives meaning to surfaces, i.e. it communicates things like: This surface is a popup-menu, this is a toplevel window, this is its window title and application id, this is how it should be maximized, resized minimized etc.

In other words, a pretty important part of the protocol. The problem is that, while Wayland has been stable for many years, there has been no stable shell extension for the desktop. wl-shell is considered deprecated, and development has continued with the unstable xdg-shell. Unstable protocols have backwards incompatible changes in every release, this means that when a new version is released, it’s essentially a new, although similar, protocol. If we were to simply update our code to the protocol described in the new version, Qt clients would stop working on compositors not supporting the new version.

We’ve solved this by using a plugin architecture for loading a shell integration at runtime depending on what the compositor supports. In other words, each time a new breaking version of xdg-shell is released, we create a new plugin for it. The exciting news this time, is that xdg-shell finally turned stable in December last year, and with it broke backwards compatibility for the last time.

Consequently, Qt 5.12 adds the last and final shell plugin for xdg-shell. This means we can finally deprecate the old unstable xdg-shell plugins and concentrate on making the one stable plugin great.

High DPI cursors

Support for high DPI cursors has been added. Cursors are now also loaded lazily to significantly reduce the memory footprint on embedded devices that don’t really need them.

Handling maximization and fullscreen on xdg-shell

Implementing fullscreen and maximization properly on xdg-shell required a huge refactor in how resizing works in Qt Wayland. That refactor is now in place, and Qt applications on all supported versions of xdg-shell should now properly switch between fullscreen, maximized and windowed mode.

Forcing a logical DPI of 96

Sometimes compositors report 1px=1mm (a DPI of 25.4) when they don’t have better data available, causing physical DPI to be unreliable.

Quite a few things in Qt depend on the logical DPI, most notably point sized fonts. Previously, we’ve calculated the logical DPI using the physical size and resolution of the screen. This works great in most cases, but sometimes the physical dimensions that compositors provide are wrong, which usually results in tiny unreadable fonts. So in Qt 5.12, we switched to forcing a DPI of 96 by default. The old behaviour can still be restored by setting QT_WAYLAND_FORCE_DPI=physical in the environment.

Fractional scaling

Support for xdg-output unstable v1 was added. It’s a protocol extension for communicating additional information about screens and is needed to implement fractional window scaling on the compositor side. Read the details in David Edmundsons blog post.

The post What’s new with the Wayland platform plugin in Qt 5.12? appeared first on Qt Blog.

We are happy to announce the release of Qt IFW 3.0.6. With Qt IFW you can create your own installers for the supported desktop Qt platforms: Linux, Microsoft Windows, and macOS.

3.0.6 is a bug fix release, here are some of the fixes:

• Fix crash in updater if component does not exist anymore
• Decrease metadata dowload amount
• Fix admin query retry in Windows
• Register virtual component for uninstall

The full list of bug fixes can be found from ChangeLog.

Precompiled binaries for the Installer Framework can be downloaded from Qt Online Installer, sources and binaries can be found also in Qt Download page (open source) or in your Qt Account.

The post Qt Installer Framework 3.0.6 Released appeared first on Qt Blog.

With Qt for Python released, it’s time to look at the powerful capabilities of these two technologies. This article details one solopreneur’s experiences.

The task

Back in 2016, I started developing a cross-platform file manager called fman. Its goal: To let you work with files more efficiently than Explorer on Windows or Finder on Mac. If you know Total Commander and Sublime Text, a combination of the two is what I am going for. Here’s what it looks like:

Picking technologies for a desktop app

The biggest question, in the beginning, was which GUI framework to use. fman’s choice needs to tick the following boxes:

• Cross-platform (Windows, Mac, Linux)
• Good performance
• Support for custom styles
• Fast development

At the time, Electron was the most-hyped project. It has a lot of advantages: You can use proven web technologies. It runs on all platforms. Customizing the UI of your application is trivial by means of CSS. Overall, it’s a very powerful technology.

The big problem with Electron is performance. In particular, the startup time was too high for a file manager: On an admittedly old machine from 2010, simply launching Electron took five seconds.

I admit that my personal distaste for JavaScript also made it easier to discount Electron. Before I go off on a rant, let me give you just one detail that I find symptomatic: Do you know how JavaScript sorts numbers? Alphabetically. ’nuff said.

After considering a few technologies, I settled on Qt. It’s cross-platform, has great performance and supports custom styles. What’s more, you can use it from Python. This makes at least me orders of magnitude more productive than the default C++.

Writing a GUI with Python and Qt

Controlling Qt from Python is very easy. You simply install Python and use its package manager to fetch the Qt bindings. Gone are the days where you had to set up gigabytes of software, or even Qt. It literally takes two minutes. I wrote a Python Qt tutorial that shows the exact steps and sample code if you’re interested.

You can choose between two libraries for using Qt from Python:

• PyQt is mature but requires you to purchase a license for commercial projects.
• Qt for Python is a recent push by Qt the company to officially support Python. It’s offered under the LGPL and can thus often be used for free.

From a code perspective, it makes no difference because the APIs are virtually identical. fman uses PyQt because it is more mature. But given that Qt for Python is now officially supported, I hope to switch to it eventually.

Deploying to multiple platforms

Once you’ve written an application with Python and Qt, you want to bring it into the hands of your users. This turns out to be surprisingly hard.

It starts with packaging. That is, turning your source code into a standalone executable. Special Python libraries exist for solving this task. But they never “just work”. You may have to manually add DLLs that are only required on some Windows versions. Or avoid shipping shared libraries that are incompatible with those already present on the users’ systems. Or add special handling for one of your dependencies. Etc.

Then come installers. On Windows, most users expect a file such as AppSetup.exe. On Mac, it’s usually App.dmg. Whereas on Linux, the most common file formats are .deb, .rpm and .pkg.tar.xz. Each of these technologies takes days to learn and set up. In total, I have spent weeks just creating fman installers for the various platforms.

Code signing

By default, operating systems show a warning when a user downloads and runs your app:

To prevent this, you need to code sign your application. This creates an electronic signature that lets the OS verify that the app really comes from you and has not been tampered with. Code signing requires a certificate, which acts like a private encryption key.

Obtaining a certificate differs across platforms. It’s easiest on Linux, where everyone can generate their own (GPG) key. On Mac, you purchase a Developer Certificate from Apple. Windows is the most bureaucratic: Certificate vendors need to verify your identity, your address etc. The cheapest and easiest way I found was to obtain a Comodo certificate through a reseller, then have them certify my business – not me as an individual.

Once you have a certificate, code signing involves a few more subtleties: On Windows and Ubuntu, you want to ensure that SHA256 is used for maximum compatibility. On Linux, unattended builds can be tricky because of the need to inject the passphrase for the GPG key. Overall, however, it’s doable.

Automatic updates

Like many modern applications, fman updates itself automatically. This enables a very rapid development cycle: I can release multiple new versions per day and get immediate feedback. Bugs are fixable in less than an hour. All users receive the fix immediately, so I don’t get support requests for outdated problems.

While automatic updates are a blessing, they have also been very expensive to set up. The tasks we have encountered so far – packaging, installers and code signing – took a month or two. Automatic updates alone required the same investment again.

The most difficult platform for automatic updates was Windows. fman uses a Google technology called Omaha, which is for instance used to update Chrome. I wrote a tutorial that explains how it works.

Automatic updates on Mac were much easier thanks to the Sparkle update framework. The main challenge was to interface with it via Objective-C from Python. Here too I wrote an article that shows you how to do it.

Linux was “cleanest” for automatic updates. This is because modern distributions all have a built-in package manager such as apt or pacman. The steps for enabling automatic updates were typically:

• Host a “repository” for the respective package manager. This is just a collection of static files such as .deb installers and metafiles that describe them.
• Upload the installer to the repository, regenerating the metafiles.
• Create a “repository file”: It imports the repo into the user’s package manager.

Because fman supports four Linux distributions, this still required learning quite a few tools. But once you’ve done it for two package managers, the others are pretty similar.

OS Versions

There aren’t just different operating systems, there are also different versions of each OS. To support them, you must package your application on the earliest version. For instance, if you want your app to run on macOS 10.10+, then you need to build it on 10.10, not 10.14. Virtual machines are very useful for this. I show fman’s setup in a video.

fman’s build system

So how does fman cope with all this complexity? The answer is simple: A Python-based build script. It does everything from creating standalone executables to uploading to an update server. I feel strongly that releasing a desktop app should not take months, so am making it available as open source. Check it out if you are interested!

Conclusion

Python and Qt are great for writing desktop apps: Qt gives you speed, cross-platform support and stylability while Python makes you more productive. On the other hand, deployment is hard: Packaging, code signing, installers, and automatic updates require months of work. A new open source library called fbs aims to bring this down to minutes.

If you liked this post, you may also be interested in a webinar Michael is giving on the above topics. Sign up or watch it here: Americas / EMEA

The post Python and Qt: 3,000 hours of developer insight appeared first on Qt Blog.

I’d like to talk about a common problem that we have seen come up in several projects, namely how to plot large datasets over time in an efficient manner using Qt. This seems to be a general issue, especially in embedded situations and is affecting many people. Because of this, I thought we’d share one approach that we have found to work well in some situations, maybe it helps you out in your project.

Problem: Waveform Graphs of Large Data Sets

We have a situation where we are accumulating more than 500.000 samples of sensor data. The data is appended over time to a huge array.

We want to visualize this growing amount of sensor information as a waveform graph:

To intelligently render this on low-profile hardware, we cannot visit all the data points in a single render pass, but we need to cheat. Instead of drawing lines between all the hundreds of thousands of data points, we draw each vertical pixel column as one line reaching from the minimum sample to the maximum sample.

With this method, we only need to render a few hundred lines instead of hundreds of thousands to reach the same visual result.

To pull off this trick, however, we need to query minmax(beginIndex, endIndex) to obtain the range of values for which to draw a line very efficiently and very often. For this, I developed a MinMaxTree data structure which can offer high speeds for this query. The effective API of such a data structure can look like this:

template  class MinMaxTree
{
public:
    explicit MinMaxTree();
 
    void append(T value);
    MinMax getMinMaxInRange(size_t rangeBegin, size_t rangeEnd) const;
// ...
};

This API allows you to insert a single data sample to store new sensor data. It also lets you retrieve the minimum and maximum given begin and end indices, which we can then use to render a line in our waveform.

Crafting an Append-Only Min–Max Tree as a Search Index

Remembering computer science classes, I figured that reusing prior calculations can save a lot of work (dynamic programming). In addition, given very large data sets, trees can cut your query times tremendously by intelligently arranging data or—to put it differently—by allowing us to skip a vast amount of data quickly.

The solution I explain in this blog post uses both approaches at their best.

Here is the tree I modeled on top of the data:

The tree summarizes ranges of data points in the underlying array. I designed the tree nodes to contain the minimum and maximum values of their range. Since parent nodes represent the data of their left and right children, they contain the minimum and maximum of their combined ranges. Finally, the root node contains the minimum and maximum values of the entire data set.

Because I want to keep the tree small and profit from caching effects, each node represents a sub-range or “bucket” of the array. The bucket size can be adjusted to match the cache sizes of the hardware best. This keeps the tree flat while still enabling fast linear scans inside the bucket.

Appending a Value, Updating the tree

There are two tasks that come with insertion: updating the tree and, optionally, extending the tree.

When appending a value, it needs to update the overlying tree structure. When inserted, the value needs to find and update its respective tree leaf, which, in turn, must inform its parent node and so on. I hope that it’s easy to see that if a node’s minimum or maximum do not change, it does not need to inform its parent node. Using this optimization, the average-case complexity of insertion is very low. The code snippet below illustrates this recursive “bubbling up” of a value trough the tree:

template;
void MinMaxTree::bubbleUpMinMax(T value, size_t treeIndex)
{
    //updates and returns true, if we altered minmax in the node
    auto minmax = [&](T value) -> bool
    {
        auto &node = m_treeOntopOfBuckets.at(treeIndex);
        const auto oldmin = node.min;
        const auto oldmax = node.max;
        node.min = std::min(value, node.min);
        node.max = std::max(value, node.max);
        return (value < oldmin || value > oldmax);
    };
 
    //we are at the root, break recursion
    if (treeIndex == m_treeCapacity/2)
    {
        minmax(value);
        return;
    }
    //update node and optionally bubble up further
    else
    {
        if (minmax(value))
            bubbleUpMinMax(value, parent(treeIndex));
    }
}

The second problem when inserting a value is that our tree structure might need to grow, because the new value breaches the capacity of the existing tree. Here, the tree needs to extend“sideways” to leave its complete old structure intact and form a new node on top. For this, I

1. double the trees size and mirror its current structure to extend its reach,
2. make a new root,
3. copy the data from the old root into the new root.

Now that we have doubled the tree size, we can again insert data until we need to expand again.

A note on the default values inside our nodes: I initialize all nodes with the highest possible value as minimum and lowest possible value as maximum.

template MinMax{
    T min = std::numeric_limits::max();
    T max = std::numeric_limits::lowest(); 
    //it’s not ::min(), this costed me hours
};

So when actual real values enter the array, they will change min and max, because they are different to these default extremes.

BE WARNED!std::numeric_limits::min() represents the smallest positive representation of a value, not the lowest possible number. I learned it the hard way.

Indexing: Squeezing the Tree into an Array

In our case, I wanted to optimize the tree accesses and its growth without using a pointer implementation that would link a node to its children using pointers.

Instead, I adopted a commonly used trick to squeeze heaps into arrays, by letting the left child of an element be at 2 × index and the right child at 2 × index + 1. While it seems that I could have used this approach for this project as well, growing the tree would require me to insert and make space at many places. Instead, I went with an infix indexing method, putting the tree into an array in a “left node–root node–right node” order like this:

This is nice for several reasons:

• It eliminates the need for the use of pointer chasing.
• Nodes and their parents are fairly close (in the lower tree).
• Expanding the tree is just doubling the array in size.
• The root node will be in the middle of the array.

To have convenient ways of accessing rightChild and leftChild as well as the parentNode from a certain index, we have to look at the binary representation of our indices. Note how the parent index, for instance, always is the next higher “pure” power of 2. It would be a bit cumbersome to explain all the magic bit in text form, instead the code tells it best:

leftChild Find lowest set bit, unset it, and set the one-lower bit

template
size_t MinMaxTree::leftChild(size_t index) const
{
    int lsb = ffs(index);
    index &= ~(1UL << (lsb-1));
    index |= 1UL << (lsb-2);
    return index;
}

rightChild Find lowest set bit and set the one-lower bit

template
size_t MinMaxTree::leftChild(size_t index) const
{
    int lsb = ffs(index);
    index |= 1UL << (lsb-2);
    return index;
}

parentNode Find lowest set bit, unset it, and set the one-higher bit

template
size_t MinMaxTree::parent(size_t index) const
{
    int lsb = ffs(index);
    index &= ~(1UL << (lsb-1));
    index |= 1UL << lsb;
    return index;
}

All Functions use the glibc-provided intrinsic ffs, which can be found in  (FindFirstSet to identify the lowest-significant set bit). On Windows, the intrinsic BitScanForward can be used to accomplish this. Similarly, I wrote a function to return the actual range in our data array a particular node covers.

Querying a Range

Now that we have all tools in hand, we can finally implement the range query itself.

We recursively query the tree nodes, starting from root downward. We check how a node’s range relates to our query range and query its children for the minimum and maximum, according to the following few cases:

Case 1: The node’s range is outside of  the query range → Return the uninitialized min/max.

Case 2: The node’s range is fully enclosed in the query range → That’s the best case, we just return the node’s min/max.

Case 3: The node’s range is partly inside, partly outside the query range or the node covers more than the range and we can split → Split, query left and right, and combine the results.

Case 4: The node’s range overlaps and we are at a leaf node → We need to scan through the bucket.

template
MinMax::queryNode(size_t nodeIndex, size_t rangeBegin, size_t rangeEnd) const
{
    // […] calculate Node Range, store in NodeBegin, NodeEnd  
 
    // node outside range, return empty MinMax()
    //               |------range----|
    //  |---node---|           or     |---node---|
    // this should never happen, but safe is safe
    if ((nodeEnd < rangeBegin) || (nodeBegin > rangeEnd))
        return MinMax
 
    // range spans node fully, return nodes' MinMax
    // |---------range-----------|
    //      |------node------|
    if ((rangeBegin <= nodeBegin) && (rangeEnd >= nodeEnd))
        return m_treeOntopOfBuckets[nodeIndex];
 
    // node cuts range left or right or both
    //       |------range----|
    //  |---node---| or |---node---| or
    //     |--------node-------|
    if ((nodeBegin <= rangeBegin ) || (nodeEnd >= rangeEnd))
    {
        const MinMax leftMinMax = queryNode(leftChild(nodeIndex), rangeBegin, rangeEnd);
        const MinMax rightMinMax = queryNode(rightChild(nodeIndex), rangeBegin, rangeEnd);
        MinMax resultMinMax;
        resultMinMax.min = std::min(rightMinMax.min, leftMinMax.min);
        resultMinMax.max = std::max(rightMinMax.max, leftMinMax.max);
        return resultMinMax;
    }
     
    // Node is leaf, re-scan its bucket for subrange
    //              |---------range-----------|
    //  |----node(leaf)----|   or   |----node(leaf)----|
    if( nodeIndex % 2 == 1)
    {
        MinMax scanResult;
        for(size_t i = std::max(nodeBegin, rangeBegin); i <= std::min(nodeEnd, rangeEnd); i++)
        {
            scanResult.min = std::min(m_dataInBuckets[i], scanResult.min);
            scanResult.max = std::max(m_dataInBuckets[i], scanResult.max);
        }
        return scanResult;
    }
}

Results

Visualizing 300.000 data points took more than 2 seconds with the naïve approach (drawing lines from point to point) on the embedded test hardware, while this approach brought down the time for queries to about 3 ms, such that the pure line rendering now even accounts for the bigger part of the cost (6 ms). We again have reached levels below the juicy target of 16 ms.

Further Optimization: Find Lowest Common Parent

In a similar setup, my colleague James Turner found that many of our node visits actually were searching down the tree for the lowest common node covering the full range. In these cases, we often ended up splitting the node in its two children, one of which containing the range fully and the other not containing the range at all.

This initial search down the tree took most of the node visits, while the actual desired min max scan (collecting all the nodes containing the range) was less than that.

So instead of searching from the top, beginning at the root node, I now calculate the left and right leaves, then determine the lowest common node spanning the full range, and perform the query from there (which is a simple loop on bit operations).

I hope, you found this useful. I am happy to hear your comments about this solution. At KDAB, we do these kind of optimizations specific to low-end hardware all the time, ask us if we can help you or check out our Profiling Workshop to learn how to find and tackle performance bottlenecks in your own projects.

The post A Speed-Up for Charting on Embedded appeared first on KDAB.

KDAB is presenting two great talks at Qt World Summit. At 13:30, you can get an in-depth look at a new concept for designers and developers with James Turner.

At 14:30, Milian Wolff will be presenting some of KDAB’s renowned opensource tools, some of which, like Hotspot, he created himself.

Streamlined integration of 3D content straight from 3DS Max and Blender into Qt3D

with James Turner

Qt 3D is now established as the ideal 3D rendering engine for cross platform applications needing to integrate 3D content with 2D interfaces in a seamless way. The scene graph lets developers import and control geometry and materials, while the frame graph can be used to build advanced rendering  effects with ease.

 However, just as in the case of 2D content, best results are achieved when designers and developer share a common experience and toolset. Yet the tools used to author 3D scenes, the geometry, the materials, the lighting, the animations, are all very different from what most developers are familiar with.

In this talk, we introduce techniques and principles which make the collaboration between designers and developers smoother. We identify the major pain points and highlight tools and patterns for addressing each of them. We will look at data formats, model conditioning, runtime integration and rendering effects in order to produce compelling applications integrating 3D content with a traditional 2D interface.

We will illustrate all these concepts with a running example.

KDAB’s Opensource Tools for Qt

with Milian Wolff

Richard Feynman is considered one of the greatest physicists, largely because he was able to build lots of practical mathematical tools that greatly simplify the analysis of complex phenomena, thus making them accessible and useful to many.

KDAB maintains its position as the leading Qt consulting firm, partly because of the huge effort KDAB’s top-class engineers put into R&D projects to build great tooling for Qt applications. This tooling has helped countless other engineers solve bugs in their code, fix performance issues and prevent more problems from slipping into codebases.

The cherry on top of all this Qt goodness is that KDAB publishes its tooling under Opensource licenses!

Come to this talk if you want to discover how KDAB’s awesome Opensource Tools for Qt can help you.

View the full agenda here. Sign up for Qt World Summit here.

The post KDAB Talks at Qt World Summit Berlin appeared first on KDAB.

Embedded Technology in Japan took place earlier this month and we are thrilled about our first participation in this event which attracted more than 400 exhibitors and over 25000 visitors.

We are also thankful to our partners SRA group and ISB Corporation for welcoming us heartily to their booths and for making this cultural experience possible.

IoT and embedded were at the center of the exhibition and we were proud to support our Japanese partners during the event by providing our expertise in C++, Qt and 3D.

If you couldn’t join us, here is some information about what we displayed :

– At the ISB Corporation booth

Nautical Navigation infotainment demo– our luxury sailboat dashboard demo

See our video here…

– At the Software Research Associates Inc. (SRA Group) booth

OPW mPro measuring station– customer showcase for the management, monitoring and automation of precision measurement systems in the context of quality control.

See more here…

Finally both booths will show our brand new 3D product that we premiered there:

Kuesa is a product that provides an integrated and unified workflow for designers and developers to create, optimize and integrate real time 3D content in a 3D or hybrid 2D/3D Qt application. Kuesa includes plugins, tools and libraries that are available for both embedded and desktop applications.

Read more about Kuesa here…

Thanks to everybody who joined us there and see you next year!

The post KDAB at Embedded Technology in Japan appeared first on KDAB.

Hello CES 2019!

One of the main events for automotive industry is less than a month away. Based on the early indicators, this year there will be more emphasis on creating a connected and intelligent vehicle than ever before. New connected experiences and multimodal inputs will be introduced by many companies showcasing at the event. In addition, I predict that we will see a wide range of new vehicle types, other than the standard four wheel vehicle, with displays.

This year The Qt Company and Qt based solutions can be found in multiple places in the event. You can meet The Qt Company with our demos in our private suite 2010 at the Westgate Hotel and at the Genivi event on Tuesday evening. In addition, we are very pleased to have been invited to join Luxoft’s booth, #3107 in the North Hall of the Las Vegas Convention Center, for meetings and quick demos. Together with our partners, we will present high performance visual implementations; integrations for virtual assistants; digital cockpits on different hardware and operating system configurations; and of course the tooling used for all of the solutions.

One thing is sure – our partners and customers will again surprise us with concepts built with Qt that the industry, including us, have not heard of or seen before. If you have a Qt based demo please let us know and remember to use “Built with Qt” icons and stickers!

See you in Las Vegas in January.

Want to schedule a meeting with us on site?  Fill out the form here.

The post Looking towards CES 2019: Predictions and where to find Qt appeared first on Qt Blog.

Another year has passed, winter has come so it’s time to celebrate the Qt Champions!

So without further ado:

@christian-ehrlicher was granted the *Rookie of The Year* title. Christian did an impressive number of submissions through this year while beginning only at the end of 2017. He wrote quite a lot of fixes for the model view system as well as test and documentation fixes.

@aha_1980 has again earned the *Quality Assurer* title due to his continuous services on the bug report system as well as doing quality reviews. He’s also actively improving Qt Creator and the QtSerialBus module.

This year’s *Ambassador* title goes to Kazuo Asano who’s a very active IRL Qt promoter in Japan. His various activities including study-sessions, seminar and now a book are doing wonder making Qt known in the empire of the rising sun.

Orgad Shaneh is our *Maverick Of The Year*. He’s the one we can thank for the now famous branch change bot on Gerrit. Bot he implemented as a side project to make the life of the sysadmins and developers easier.

This year’s *Fixer* is Alexander Volkov who’s done a lot of work on the X11/xcb backend that has no secret for him.

@VRonin various helper libraries and examples earned him the *Content Creator* title. His knowledge of Qt’s model view system is also well known and appreciated on the forum.

Due to his extensive presence on several fronts (IRC channel, forums, and more), @GrecKo has well earned the *Community Builder* title.

His many and to the point answers on the forum as well as continuous presence on it made @jsulm the second *Community Builder*.

And last but not least, his friendliness, continuous support, numerous examples and tests through these past years have opened him the door to the hall of fame. Congratulations to @mrjj who becomes the second Qt Lifetime Champion!

Let us all congratulate our Champions!

Many thanks for all the work done with and for the community!

The post Welcome to the 2018 Qt Champions! appeared first on Qt Blog.

As mentioned in the Qt 5.12 release blog we finally reached the point that we can announce a technology preview free first release of Qt for Python for Qt 5.12. As it has become custom for the project and expected by Python developers, you can install the release via:

Additionally, you can get the wheels from Qt’s official download server, or simply get the source code from our Git repository.

Note that Qt for Python is tightly bound to a particular Qt version and does not inherit the LTS flag from Qt 5.12. Our aim for the future is to release Qt for Python at the same time as the Qt version that Qt for Python is based on. Unfortunately, our release process have not reached this point quite yet.

To get started, take a look at the recent webinar ”Develop Your First Qt for Python Application’‘ on how to develop an application from scratch, based on Qt Widgets and different Python modules. You’ll also see some examples on how to continue developing with other Qt for Python components, such as QML and Shiboken.

PySide2 and Shiboken2

The release consists of two modules. The first module, PySide2, provides Python wrappers for most Qt APIs and the second module, Shiboken2, enables users to wrap their own C++ libraries and make them available to the Python world. Internally the PySide2 module depends on Shiboken.

As you might expect, dropping the Technology Preview (TP) label implies a certain compatibility guarantee. This compatibility statement is somewhat different for Qt for Python users compared to C++ guarantees though. It mostly relies on Qt’s source compatibility guarantee as we use a specific syntax to expose Qt’s C++ API. Any Python application written with Qt for Python 5.12 should continue to work with future releases. On the Shiboken side of things, we are still somewhat off target as we continue to carry the TP label.

When looking at the supported platforms, we currently support Python 2.7 and 3.5+ for the three major desktop platforms.

The Future of the Project

Our future roadmap is full of promising ideas and in January we’ll start sorting out the next steps, which will probably include:

• Simplifying the deployment of PySide2 applications,
• Provide a smoother interaction with other Python modules,
• Support other platform like embedded and mobile,
• and many others.

With this context in mind, we appreciate any input you may have for the next release. You can either post it here, in our IRC channel on Freenode (#qt-pyside) or directly to our issue tracker under https://bugreports.qt.io .

The post Qt for Python 5.12 Released appeared first on Qt Blog.

KDAB is offered eight superb Training Classes in Berlin, you can see the list below, which includes one run by our long-term collaborator, froglogic. All the rest were delivered by KDAB engineers.

There were five classes in our Introductory group, and three in the Advanced. Read the descriptions carefully to see which one you’d have liked to attend. An Introductory class can be just what you need for a refresher, even if you’re an experienced programmer, as our trainers are always up-to-date with the latest tips.

Introductory
Effective 3D in Qt
Introduction to CMake
Introduction to Qt/QML
Multithreading in Qt
Qt GUI Testing with Squish

Advanced
Modern C++ – What’s New in C++17?
Profiling and Debugging for Linux
QML Applications Architecture

Details

Introductory

Effective 3D in Qt

with James Turner

Target Audience: Qt developers wishing to integrate 3d technology in their application.

Prerequisites: The audience is expected to have familiarity with basic QtQuick and OpenGL concepts, but no in-depth knowledge of them is required.

Course Description

Starting with the utility and enabler classes (for OpenGL, Vulkan and the upcoming Metal support), we will look at low level support for using OpenGL with QOpenGLWindow for rendering and event handling. We will also look at the support in the Widgets module.

Later we will show the technologies available in Qt that allow deep integration of Qt Quick 2 scenes with custom drawn OpenGL content. We will discuss the possibility of simply providing a Qt Quick overlay for an OpenGL scene. The discussion will then proceed to the creation of custom Qt Quick Items drawn using raw OpenGL commands, which can then be used from QML. We will also illustrate how to manually drive Qt Quick’s own rendering if we need to be in complete control of how and when the rendering happens.

Finally, we will look at Qt 3D and how to use its scene graphs and frame graphs to create high performance 3d rendering without requiring the specialist knowledge required when accessing OpenGL directly. Along the way we will introduce the Qt 3D renderer and input systems and how they are built on top of a flexible, highly threaded, Entity Component System (ECS) architecture that scales very well and is ripe for future additions.

You will learn how to:

• create windows for 3d rendering
• add a Qt Quick based UI to an OpenGL application
• create custom high performance Qt Quick Items using OpenGL
• integrate your own OpenGL renderer with the Qt Quick Renderer
• construct a basic Qt 3D application
• make a scene graph, display 3D graphical content using geometry, materials, textures
• get Qt 3D maps onto the graphics pipeline
• extend Qt 3D to use your own custom geometry
• write custom materials and shaders
• completely control the Qt 3D renderer dynamically at runtime using the Frame Graph

James Turner

Senior Software Engineer and team lead at KDAB, James has been developing with Qt since 2002. He contributes to the current maintenance of Mac platform support as well as the development of OpenGL and 3D support in Qt. James has a background in user-interface, graphics and simulation development as well as a long history of development on OS-X and prior versions of Mac OS. He is a lead developer on FlightGear, the open-source flight simulator, and holds a BSc in Computer Science.

Introduction to CMake

with Kevin Funk

Target Audience: C and C++ Developers who are interested in how to build their code.

Prerequisite: Experience with build systems.

“The best thing a build system can do is not get in the way”.

Course Description

CMake is the de facto standard build system for C and C++ outside of frameworks that require their own. It has earned this place by supporting the situations and special cases that arise in real projects.

Support for CMake within Qt is being significantly improved and there are longer term plans to switch to CMake for building Qt itself. It’s currently a good alter- native if you hit limitations in qmake.

This course will teach the basics of creating and building projects with CMake.  In recent years, CMake has introduced some cleaner and more precise constructs. The course will focus on the new constructs where possible.

Why learn CMake? CMake has broad functionality that covers many real world problems. Learning CMake enables you to solve advanced build requirements. This includes cross-platform builds, feature detection based on platform or available libraries, built- time configurable feature switches and custom build steps. CMake is increasingly widely adopted across the industry.

Kevin Funk

Kevin has actively developed with Qt/C++ since 2006 and has a special interest in tooling and profiling. He’s an active contributor to KDAB’s GammaRay analyzer (a high-level Qt application debugger) and has a strong emphasis on state machine tooling. He is co-maintainer of the KDevelop IDE, a powerful C/C++ development environment backed by Clang, and is pushing for cross-platform success inside KDE. Kevin holds a Masters Degree in Computer Science.

Introduction to Qt/QML

with Jan Marker

Target Audience: Developers and managers interested in learning the autonomy of a QML application.

Prerequisite: Knowing the basics of Qt at C++ level is an advantage but not a requirement.

Course Description

This training is an introduction to Qt Quick. On the one hand it will teach you how to compose fluid user interfaces with slick animations using the QML language. On the other hand it will teach you how you hook the QML side up to your business logic in C++.

Course contents

• Connecting a QML UX with C++ business logic
• Complex list views including data provided from C++ models
• Custom objects implemented using Qt Quick scene graph
• Profiling and best practices for optimal performance

Why learn Qt/QML? Designed to take people new to Qt or QML, from the basics to a deep functional understanding of best practices, this Qt/QML training will equip you with the skills and know-how to boost your productivity at work.

Jan Marker

Software Engineer at KDAB, Jan has been using Qt since 2009 when he started contributing to the KDE project. Since joining KDAB he has worked on multiple large Qt and QML projects, while more recently also developing Wayland compositors. Besides in-depth knowledge of Qt and C++, Jan also has a deep interest in other technologies like NodeJS. He holds a BSc in Computer Science.

Multithreading in Qt

with Kevin Krammer

Target Audience: Qt Developers interested in multithreaded programming.

Prerequisite: Knowledge and experience programming with Qt and C++. A basic understanding of multithreaded programming is an advantage but not required.

Course Description

Multithreaded programming is essential for developers to create fast and responsive applications on computers, phones, and embedded devices all with an increasing number of cores. Qt offers several mechanisms for multithreading; however, it can be difficult to know which to use and how to steer clear of common pitfalls. This course offers guidance how to write safe and efficient multithreaded code with Qt.

Topics include:

• Basic multithreading concepts (threads, processes, data races, reentrency, shared data)
• Synchronization primitives (mutexes, semaphores, condition variables)
• Special concerns for Qt applications (cross-thread signals/slots, QObject thread affinity, the GUI thread)
• Low-level multithreading with Qt (QThread, QThreadPool, QMutex, etc)
• High-level multithreading with Qt (QtConcurrent)
• Threading with Qt Model/View
• A brief summary of atomic operations

Why learn about Multithreading with Qt? Multithreaded development is a complex subject where it is easy to write code that contains severe bugs yet looks correct. This training will provide a solid foundation for writing safe and effective multithreaded code in Qt applications.

Kevin Krammer

Senior Software Engineer and team lead at KDAB, Kevin has actively developed with Qt and contributed consistently to KDE since 2000. He is a founding member of the QtCentre website and has mentored at Google’s Summer of Code program for 10 years. One of KDAB’s most experienced trainers, Kevin keeps our training material up-to-date and has trained engineers from Blackberry, Lockheed Martin, Siemens and Safegate and many others. Kevin holds a BSc in Software and Communications Engineering.

Qt GUI Testing with Squish

with Tomasz Pawlowski

Prerequisites: The course is for developers and testers already familiar with the basic concepts of Qt.

Course Description

In order to achieve high quality applications during testing process all the functionality of the software shall be covered, including fully exercising GUI itself. For regression testing automating this process gives benefits, saving execution time and increasing accuracy. On the other hand, GUI Automation might be a challenge, as GUI may change significantly during a product life cycle.

In this course we learn how to design and implement cross-platform automated tests using Squish GUI Tester for Qt, QML & QtQuick applications that continue to work as your product evolves.

• Introduction to GUI Testing
• Squish Overview (installation, configuration)
• Test Script Creation and Execution
  • Recording and Replaying Test Scripts
  • Verification Points (Properties, Screenshot, Visual)
  • Test Results and Logging
  • Squish API
  • Image-Based Lookup
• Development of Test Cases at Business Level
• Set-Up and Tear-Down Functions
• Debugging Test Scripts
• Object Recognition
• Accessing Application Internals (Inspecting, Object Properties and Methods)
• Synchronisation and Event Handling
• Squish Command-Line Tools
• Working with Multiple Applications
• Hooking into Running Applications
• Squish Integration with CI

Tomasz Pawlowski

Software engineer at froglogic, Tomasz started the adventure with Squish and GUI Testing in 2011, designing and implementing automated tests for a Flight Planning solution at Lufthansa Systems. In 2014 he joined froglogic and is conducting Squish trainings and consulting for many companies in Europe, India and the USA. Additionally, Tomasz is implementing Squish integrations. Tomasz has a degree in computer science from Nicolaus Copernicus University in Poland.

Advanced

Modern C++ – What’s New in C++17?

with Giuseppe D’Angelo

Target Audience: C++ developers who want to know more about the new features introduced in C++17.

Prerequisite: Knowing the basics of C++11 is a requirement, though more advanced topics will be explained as needed.

Course Description

Starting with C++11 released in 2011, the C++ language and standard library have steadily evolved. At the end of 2017 the new C++17 standard was released, adding a sizable amount of useful new features. All major compilers already support most (if not all) of its features.

In this training, the most useful of the new features introduced in C++17 and its predecessor will be presented. In cases for which these features depend on features introduced in C++11 or C++14, these will be refreshed as well.

New library features being presented include the new types std::any, std::optional and std::variant, the new parallel algorithms, filesystem access, std::string_view and new operations on the container classes. The new language features range from ways to improve template code with fold expressions, constexpr if, and class template deduction over improvements of lambdas to structured bindings and initalizers in if and switch statements.

Why learn what’s new in C++17? C++ is the language that powers most applications written with Qt. To make the most out of the language, developers need to know its capabilities and pitfalls, and keep up with the incremental changes done in new releases. Doing so rewards them with ways to write easier, faster, cleaner and safer code

Giuseppe D’Angelo

Senior Software Engineer at KDAB, Giuseppe is a long-time contributor to Qt, having used Qt and C++ since 2000, and is an Approver in the Qt Project. His contributions in Qt range from containers and regular expressions to GUI, Widgets and OpenGL. A free software passionate and UNIX specialist, before joining KDAB, he organized conferences on opensource around Italy. He holds a BSc in Computer Science.

Profiling and Debugging for Linux

with Milian Wolff

Target audience: Developers who want to find and fix problems,

Prerequisite: Knowing the basics of C++ and Qt,

Course Description

This training introduces various tools, which help developers and testers in finding bugs and performance issues. This variant of the training focuses on Linux.

The tools presented cover a wide range of problems, from general purpose debugging and CPU profiling to Qt specific high-level analyzers. Often, it is relatively simple to run a tool, but interpreting the results, or even just using some of the more advanced tools, requires deep technical knowledge. The following tools will be covered:

Debugging

• General purpose debugger: GDB
• Record and replay, reverse debugging: RR
• Memory error detectors: AddressSanitizer
• Thread error detectors: ThreadSanitizer
• Various Qt built-in features
• GammaRay to investigate internals of Qt Applications

Static Code Analysis

• Compilers
• Clazy

Profiling

• CPU: Linux perf and hotspot
• Heap memory: heaptrack

Milian Wolff

Senior Software Engineer at KDAB, Milian leads the R&D in tooling and profiling in which he has a special interest. Milian created Massif-Visualizer and heaptrack, both of which are now used regularly to improve the performance of C++ and Qt applications. When not applying his knowledge to improving code-base performance for KDAB’s customers, Milian maintains QtWebChannel for the Qt Project and is co-maintainer of the KDevelop IDE. In 2015, Milian won KDE’s Akademy Award for his work on Clang integration. He has a Masters Degree in Physics.

QML Applications Architecture

with Tobias Koenig

Target audience: QML developers who want to learn about creating large-scale yet maintainable applications.

Prerequisite: Being comfortable with the basics of QML, as well as some familiarity with developing with C++ and Qt (QObject, signals & slots, properties, etc).

Course Description

QML is a great language for expressing user interfaces in a declarative, easy to understand way. It can however be difficult to scale up from small demo applications to fully featured, complex systems without paying too high a price in complexity, performance and maintainability. In this course, we explore different techniques to deal with these issues to enable you to scale up your applications while steering clear from the common pitfalls.

Topics include:

• Custom QML items
• C++ integration
• Declarative coding
• Multi-page application architectures
• Code organization

Tobias Koenig

Senior Software Engineer at KDAB, Tobias has actively developed with Qt since 2001 and has been an active KDE contributor during this time. His contributions have been mainly to the KDE PIM project and the KDE libraries, but also to other open source projects.

There’s still seats left for the One-Day Training at Qt World Summit in Berlin!

Sign up…

The post KDAB Training at Qt World Summit Berlin appeared first on KDAB.

KDAB is the main sponsor at Qt World Summit 2018.  In Berlin, we’ll be hosting a training day on December 5th  offering both Introductory and Advanced one day training classes and on December 6th you can listen to two talks from KDAB experts:

• Streamlined integration of 3D content straight from 3DS Max and Blender into Qt3D, with James Turner, and
• KDAB’s Opensource Tools for Qt with Milian Wolff.

KDAB will also be exhibiting on both days, starting at 5pm on Wednesday December 5th. Come to our booth and see:

Qt Quick Software Renderer

At Qt World Summit KDAB will show the Qt Quick Software Renderer in action on the very competitively priced NXP i.MX6 ULL platform. Thanks to some clever coding from KDAB, it provides a fluid 60fps touch UI and H.264 video decoding in spite of having neither GPU nor video decoding acceleration on the hardware.

You can now have the full feature set of Qt at your disposal, even at as little as 64MB of RAM and/or Flash memory.

• NXP i.MX6 ULL (no GPU/VPU/IPU)
• Fluid 60fps touch UI
• 64MB RAM/Flash
• H.264 video decoding using PxP acceleration
• Full Qt feature set available

See more….

Kuesa – 3D asset creation and integration workflow

Kuesa is a solution that provides an integrated and unified workflow for designers and developers to create, optimize and integrate real time 3D content in a 3D or hybrid 2D/3D software user-interface.

• Streamlined integration of 3D content from e.g. 3DS Max and Blender
• High-performance real-time rendering
• Uses Qt and Qt 3D engine
• Available on desktop and embedded

Find out more about Kuesa…

KDAB GammaRay

A high-level introspection tool for Qt applications, KDAB’s GammaRay allows you to examine and manipulate application internals at runtime, either locally or on an embedded target. Augmenting conventional debuggers, GammaRay leverages QObject to visualize application behavior at a high level, especially useful where complex Qt frameworks such as model/view, state machines, QGraphicsView or QTextDocument are involved. GammaRay is integral to the Qt Automotive Suite all-in-one package, where it also offers QtCreator integration.

• High-level introspection tool for Qt applications
• Insight into Qt Quick and Qt 3D scene graphs
• Visual state machine debugger
• Inspect models, layouts, rendering and much more

Find out more about GammaRay…

Clazy Static Code Analyzer

An opensource project spawned by KDAB’s R&D efforts for better C++ tooling, Clazy Static Code Analyzer is an LLVM/Clang-based static analyzer for Qt 5 that extends your compiler with approximately 50 Qt-oriented warnings.

Clazy Static Code Analyzer highlights Qt-related bugs, performance issues or unneeded memory allocations and performs a code rewrite for common tasks like porting to the Qt 5 connect syntax.

Clazy Static Code Analyzer integrates seamlessly with most existing build systems, and is bundled with the latest versions of QtCreator if you don’t want to compile it your self.

• LLVM/Clang-based static analyzer for Qt
• Highlights Qt-related bugs, performance issues or unneeded memory allocations
• Code rewrite for common tasks like porting to the Qt 5 connect syntax
• Seamless integration with most existing build systems

See our video about Clazy…

KDAB Hotspot Profiler

Created by KDAB’s Milian Wolff, who will be talking about this and other KDAB open source tools at Qt World Summit, KDAB Hotspot profiler does all the heavy lifting for you, when you need to analyze profiling data on Linux Perf.

• GUI for Linux Perf to analyze profiling data
• Bottom-Up and Top-Down tree views
• Caller/Callee list
• FlameGraph
• Time line

See how Hotspot works…

Qi – Cellular Tissue Imaging in Qt 3D

Demonstrating the power of Qt 3D, this shows stunning 3D visualizations of microscopic tissue samples that help researchers better understand cell pathology in the fight against cancer. The 3D images are created out of 2D images from electron microscopes in cutting edge clinical diagnostics data sets. The process enables real-time conversion to 3D from 30 image channels using Qt 3D.

• 3D visualization of microscopy of tissue samples
• 2D images from cutting edge clinical diagnostics data sets
• Real-time conversion to 3D from 30 image channels using Qt 3D

Find out more…

nanoQuill Interactive Wall

KDAB will be presenting the nanoQuill interactive display wall at Qt World Summit, where participants can literally help in the cure for cancer by coloring in images of cells. This is a collaboration between KDAB, the Qt Company, Quantitative Imaging Systems (Qi), and Oregon Health & Science University that crowd-sources electron microscope images of cancer cells for people to artfully color, thus #color4cancer.

Once photographed, completed images can be sent to the nanoQuill website where machine learning engages in discerning cell micro structures, helping us understand how tumour cells develop resistance to escape cancer-targeting drugs.

Find out more about nanoQuill….

The post KDAB demos at Qt World Summit, Berlin appeared first on KDAB.

Le Capitole du Libre, évènement de logiciel libre annuel à Toulouse auquel KDAB (France) était Sponsor Or pour la 8ème année consécutive, s’est achevé la semaine dernière après 2 jours de conférence intense proposant de nombreuses présentations en parallèle.

Les sujets allaient de l’IoT et l’embarqué au C++ et à la 3D en passant par les libertés et vie privée ainsi que le financement du logiciel libre pour n’en citer que quelques-uns.

Voyez le programme ici.

M. Kevin Ottens a animé‘Des blobs colorés pour l’analyse de données communautaires’ et M. Franck Arrecot a présenté ‘Introduction à Qt QML et à la création de composants’.

Si vous n’avez pas pu y assister, vous pouvez télécharger leur présentation ici:

• présentation de M. Kévin Ottens
• présentation de M. Franck Arrecot

Merci à tous ceux qui sont venus et à l’an prochain!

The post KDAB at Capitole du Libre appeared first on KDAB.

KDAB announces the release of Kuesa™, a solution that provides an integrated and unified workflow for designers and developers to create, optimize and integrate real time 3D content in a 3D or hybrid 2D/3D software user interface.

Kuesa provides an easy, integrated and unified workflow without any compromises for designers and developers giving:

• great performance on both desktop and embedded boards
• high quality real-time 3D scenes
• full expressiveness for designers, using professional 3D design tools
• full control of integration for developers
• reduced time to market.

For a practical demo, have a look at the tutorials created by KDABian Timo Buske, showing a complete workflow from the designer to the developer:

• How to prepare Qt 3D assets for Kuesa with 3ds Max® (designer focus)
• How to create a fully interactive 3D app with Kuesa (developer focus)

Kuesa is now available under both an AGPL and Commercial license. For more details about Kuesa and how to download it, visit the Kuesa web page.

Also, at this year’s Qt World Summit Berlin, KDAB’s James Turner will be giving a presentation on Kuesa titled, Streamlined Integration of 3D Content straight from 3DS Max and Blender into Qt3D. You can also view a demo of Kuesa at KDAB’s booth.

The post KDAB releases Kuesa™ for 3D asset creation and integration workflow appeared first on KDAB.

As mentioned in the Qt 5.12 release blog we finally reached the point that we can announce the first release of Qt for Python for Qt 5.12. As it has become custom for the project and expected by Python developers, you can install the release via:

Additionally, you can get the wheels from Qt’s official download server, or simply get the source code from our Git repository.

Note that Qt for Python is tightly bound to a particular Qt version and does not inherit the LTS flag from Qt 5.12. Our aim for the future is to release Qt for Python at the same time as the Qt version that Qt for Python is based on. Unfortunately, our release process has not reached this point quite yet.

To get started, take a look at the recent webinar ”Develop Your First Qt for Python Application’‘ on how to develop an application from scratch, based on Qt Widgets and different Python modules. You’ll also see some examples on how to continue developing with other Qt for Python components, such as QML and Shiboken.

PySide2 and Shiboken2

The release consists of two modules. The first module, PySide2, provides Python wrappers for most Qt APIs and the second module, Shiboken2, enables users to wrap their own C++ libraries and make them available to the Python world. Internally the PySide2 module depends on Shiboken.

As you might expect, dropping the Technology Preview (TP) label implies a certain compatibility guarantee. This compatibility statement is somewhat different for Qt for Python users compared to C++ guarantees though. It mostly relies on Qt’s source compatibility guarantee as we use a specific syntax to expose Qt’s C++ API. Any Python application written with Qt for Python 5.12 should continue to work with future releases. On the Shiboken side of things, we are still somewhat off target as we continue to carry the TP label.

When looking at the supported platforms, we currently support Python 2.7 and 3.5+ for the three major desktop platforms.

The Future of the Project

Our future roadmap is full of promising ideas and in January we’ll start sorting out the next steps, which will probably include:

• Simplifying the deployment of PySide2 applications,
• Provide a smoother interaction with other Python modules,
• Support other platform like embedded and mobile,
• and many others.

With this context in mind, we appreciate any input you may have for the next release. You can either post it here, in our IRC channel on Freenode (#qt-pyside) or directly to our issue tracker under https://bugreports.qt.io .

The post Qt for Python 5.12 Released appeared first on Qt Blog.

Good performance of 3D applications is essential for achieving the desired user experience. While the 3D assets are the most important items affecting the performance, the 3D engine itself needs to be highly efficient in what it does. We have been looking into ways for improving the performance of Qt 3D Studio and especially on how to reduce the CPU load and RAM consumption of 3D applications.

We released Qt 3D Studio 2.2 last week with a lot of cool features, explained in the release blog post. With this blog, the focus is on a couple of items provided by Qt 3D Studio 2.2 that significantly improve the performance and memory consumption. First, we’ll look into leveraging texture compression to reduce RAM consumption and to improve the startup time. Later in the blog post, we’ll dig into the new, still experimental, renderer provided by Qt 3D Studio 2.2. The new renderer, as well as the upcoming new animation system, is specifically developed to reduce the CPU load of Qt 3D Studio applications.

Using Texture Compression for 3D Assets

Qt 3D Studio 2.2 provides support for KTX texture format, enabling the use of various different compression algorithms, such as ETC and ASTC compression. Texture compression helps a lot in reducing memory consumption as well as application startup time. ASTC is a compression technique specified by the Khronos Group and stands for Adaptable Scalable Texture Compression. The ASTC compressed textures are then packaged into a KTX container (another Khronos Group standard, also used by Qt Quick) and used by the Qt 3D Studio. Use of compressed textures is enabled from the Qt 3D Studio Editor (see documentation). Currently, the compressed textures need to be created separately.

The graph below visualizes the impact of compressed textures for the RAM consumption.

The impact depends on the application. In these measurements, we have three different 3D usage scenarios, called Scene 1, Scene 2 and Scene 3. These are benchmark applications created specifically for testing purposes. They run continuously as part of our performance testing system to detect possible regressions (and improvements as well, of course). The impact of texture compression on a real-life Qt 3D Studio application depends upon the application, but the test applications (scenes) give a good impression on how applications of varying complexity can benefit from texture compression.

With ASTC texture compression (we have set it with 4×4 block size and linear rgb format) the memory consumption is reduced by 23MB (20%)  in the simplest Scene 1 test case, by 41 MB (28%) in the Scene 2 and by 56 MB (25%)  in the Scene 3 test (the most complex one of the three). On average, the tests using texture compression yield a 24% reduction in RAM consumption, which is quite a nice saving of that scarce RAM.

The graph below visualizes the impact of compressed textures for the startup time of the application.

The startup time improvement with texture compression is 57% (from 0,23 to 0,1 s) for Scene 1, 64% (from 0,55 to 0,2 s) for Scene 2 and 65% (from 0,85 to 0,3 s) for Scene 3. The time is measured from starting to load the presentation until the first frame is shown. On average, the startup time is reduced by 62%, so it really pays off to use compressed textures.

Note that all these can be directly achieved using the latest Qt 3D Studio 2.2 release and using the supported texture compression for the graphics assets.

Dragon – the New Renderer and Animation System for Qt 3D Studio

The default renderer and animation system of Qt 3D Studio 2.2 are still based on the same version as before. But behind the scenes we are also working on a new renderer and animation system codenamed “Dragon”, bringing performance improvements to the Qt 3D Studio Runtime. These are still experimental with Qt 3D Studio 2.2, but can be enabled to achieve some improvements. The intention is to make the “Dragon” renderer and animation system the default choice with the Qt 3D Studio 2.3 release (scheduled for March 2019).

To see the impact of the new renderer and the animation system on the CPU usage, we tested it on NVIDIA Jetson TX2 (64bit ARM embedded processor) and Intel i7-6700 equipped with NVIDIA Quadro P2000 graphics (i.e. a regular desktop PC), both running Linux as the operating system. Same test scenes as with the texture compression benchmark were used, but with no compression applied (as the texture compression is handled with the GPU, it’s not so relevant for the CPU benchmarking).

This time we compared the default Qt 3D Studio 2.2 with the Qt 3D Studio 2.2 running the new Dragon renderer (experimental with the 2.2 version) and the new animation system coming with Qt 3D Studio 2.3.

The CPU load is reduced on average by 45% with the NVIDIA TX2. The reduction varies a bit between the scenes, but is roughly on the same level (38-49% improvements in the different test cases).

Running the same comparison on an Intel i7 desktop PC, theCPU load reduction is 63% on average (59-67% improvements in the different test cases).

As seen from the results, the new Dragon renderer provides important optimization for the CPU utilization. The new renderer improves the CPU usage especially in scenes where there are many objects, but only a few of them animated between frames. It detects changes in different objects and ensures that interdependent jobs only process necessary intermediate values. The new renderer alone will not improve scenes where all objects are continuously animated, but it can significantly cut down the CPU usage in certain scenes.

The animation system has also been overhauled to minimize the processing performed for animated values. Previously, animations would be offloaded to a backend job, but this turned out to degrade performance due to additional copying and processing in the offloading step. By coupling the animation system more tightly to the renderer, we have managed to bring down the CPU usage in highly animated scenes as well. The tighter coupling has also made the data flow simpler, which makes the code easier to reason about and maintain.

To enable the new Dragon renderer set the Q3DS_DRAGON environment variable to 1 using the new Qt 3D Studio 2.2 release. The animation system is part of Qt 3D Studio 2.3 release, so to try that one out, use the master branch and enable the animation system by setting the environment variable DRAGONWINGS to 1. The animation system feature is already merged and will be part of the Qt 3D Studio 2.3 release in March 2019.

Getting started with Qt 3D Studio 2.2

The easiest way to get Qt 3D Studio 2.2 is through the Qt online installer. The Qt online installer and offline installers can be obtained from the Qt Download page and commercial license holders can find the packages via their Qt Account. Binary packages are available for Windows, Mac and Linux. If you are using Qt for Device Creation 5.12 images the Qt 3D Studio 2.2 Runtime and Viewer are already included in the images. Please also note that Qt 3D Studio runtime uses Qt 3D module for rendering which means that Qt 3D Studio 2.2 requires Qt 5.12 LTS.

The post Qt 3D Studio Performance Improvements appeared first on Qt Blog.

I’d like to talk about a common problem that we have seen come up in several projects, namely how to plot large datasets over time in an efficient manner using Qt. This seems to be a general issue, especially in embedded situations and is affecting many people. Because of this, I thought we’d share one approach that we have found to work well in some situations, maybe it helps you out in your project.

Problem: Waveform Graphs of Large Data Sets

We have a situation where we are accumulating more than 500.000 samples of sensor data. The data is appended over time to a huge array.

We want to visualize this growing amount of sensor information as a waveform graph:

To intelligently render this on low-profile hardware, we cannot visit all the data points in a single render pass, but we need to cheat. Instead of drawing lines between all the hundreds of thousands of data points, we draw each vertical pixel column as one line reaching from the minimum sample to the maximum sample.

With this method, we only need to render a few hundred lines instead of hundreds of thousands to reach the same visual result.

To pull off this trick, however, we need to query minmax(beginIndex, endIndex) to obtain the range of values for which to draw a line very efficiently and very often. For this, I developed a MinMaxTree data structure which can offer high speeds for this query. The effective API of such a data structure can look like this:

template  class MinMaxTree
{
public:
    explicit MinMaxTree();
 
    void append(T value);
    MinMax getMinMaxInRange(size_t rangeBegin, size_t rangeEnd) const;
// ...
};

This API allows you to insert a single data sample to store new sensor data. It also lets you retrieve the minimum and maximum given begin and end indices, which we can then use to render a line in our waveform.

Crafting an Append-Only Min–Max Tree as a Search Index

Remembering computer science classes, I figured that reusing prior calculations can save a lot of work (dynamic programming). In addition, given very large data sets, trees can cut your query times tremendously by intelligently arranging data or—to put it differently—by allowing us to skip a vast amount of data quickly.

The solution I explain in this blog post uses both approaches at their best.

Here is the tree I modeled on top of the data:

The tree summarizes ranges of data points in the underlying array. I designed the tree nodes to contain the minimum and maximum values of their range. Since parent nodes represent the data of their left and right children, they contain the minimum and maximum of their combined ranges. Finally, the root node contains the minimum and maximum values of the entire data set.

Because I want to keep the tree small and profit from caching effects, each node represents a sub-range or “bucket” of the array. The bucket size can be adjusted to match the cache sizes of the hardware best. This keeps the tree flat while still enabling fast linear scans inside the bucket.

Appending a Value, Updating the tree

There are two tasks that come with insertion: updating the tree and, optionally, extending the tree.

When appending a value, it needs to update the overlying tree structure. When inserted, the value needs to find and update its respective tree leaf, which, in turn, must inform its parent node and so on. I hope that it’s easy to see that if a node’s minimum or maximum do not change, it does not need to inform its parent node. Using this optimization, the average-case complexity of insertion is very low. The code snippet below illustrates this recursive “bubbling up” of a value trough the tree:

template;
void MinMaxTree::bubbleUpMinMax(T value, size_t treeIndex)
{
    //updates and returns true, if we altered minmax in the node
    auto minmax = [&](T value) -> bool
    {
        auto &node = m_treeOntopOfBuckets.at(treeIndex);
        const auto oldmin = node.min;
        const auto oldmax = node.max;
        node.min = std::min(value, node.min);
        node.max = std::max(value, node.max);
        return (value < oldmin || value > oldmax);
    };
 
    //we are at the root, break recursion
    if (treeIndex == m_treeCapacity/2)
    {
        minmax(value);
        return;
    }
    //update node and optionally bubble up further
    else
    {
        if (minmax(value))
            bubbleUpMinMax(value, parent(treeIndex));
    }
}

The second problem when inserting a value is that our tree structure might need to grow, because the new value breaches the capacity of the existing tree. Here, the tree needs to extend“sideways” to leave its complete old structure intact and form a new node on top. For this, I

1. double the trees size and mirror its current structure to extend its reach,
2. make a new root,
3. copy the data from the old root into the new root.

Now that we have doubled the tree size, we can again insert data until we need to expand again.

A note on the default values inside our nodes: I initialize all nodes with the highest possible value as minimum and lowest possible value as maximum.

template MinMax{
    T min = std::numeric_limits::max();
    T max = std::numeric_limits::lowest(); 
    //it’s not ::min(), this costed me hours
};

So when actual real values enter the array, they will change min and max, because they are different to these default extremes.

BE WARNED!std::numeric_limits::min() represents the smallest positive representation of a value, not the lowest possible number. I learned it the hard way.

Indexing: Squeezing the Tree into an Array

In our case, I wanted to optimize the tree accesses and its growth without using a pointer implementation that would link a node to its children using pointers.

Instead, I adopted a commonly used trick to squeeze heaps into arrays, by letting the left child of an element be at 2 × index and the right child at 2 × index + 1. While it seems that I could have used this approach for this project as well, growing the tree would require me to insert and make space at many places. Instead, I went with an infix indexing method, putting the tree into an array in a “left node–root node–right node” order like this:

This is nice for several reasons:

• It eliminates the need for the use of pointer chasing.
• Nodes and their parents are fairly close (in the lower tree).
• Expanding the tree is just doubling the array in size.
• The root node will be in the middle of the array.

To have convenient ways of accessing rightChild and leftChild as well as the parentNode from a certain index, we have to look at the binary representation of our indices. Note how the parent index, for instance, always is the next higher “pure” power of 2. It would be a bit cumbersome to explain all the magic bit in text form, instead the code tells it best:

leftChild Find lowest set bit, unset it, and set the one-lower bit

template
size_t MinMaxTree::leftChild(size_t index) const
{
    int lsb = ffs(index);
    index &= ~(1UL << (lsb-1));
    index |= 1UL << (lsb-2);
    return index;
}

rightChild Find lowest set bit and set the one-lower bit

template
size_t MinMaxTree::leftChild(size_t index) const
{
    int lsb = ffs(index);
    index |= 1UL << (lsb-2);
    return index;
}

parentNode Find lowest set bit, unset it, and set the one-higher bit

template
size_t MinMaxTree::parent(size_t index) const
{
    int lsb = ffs(index);
    index &= ~(1UL << (lsb-1));
    index |= 1UL << lsb;
    return index;
}

All Functions use the glibc-provided intrinsic ffs, which can be found in  (FindFirstSet to identify the lowest-significant set bit). On Windows, the intrinsic BitScanForward can be used to accomplish this. Similarly, I wrote a function to return the actual range in our data array a particular node covers.

Querying a Range

Now that we have all tools in hand, we can finally implement the range query itself.

We recursively query the tree nodes, starting from root downward. We check how a node’s range relates to our query range and query its children for the minimum and maximum, according to the following few cases:

Case 1: The node’s range is outside of  the query range → Return the uninitialized min/max.

Case 2: The node’s range is fully enclosed in the query range → That’s the best case, we just return the node’s min/max.

Case 3: The node’s range is partly inside, partly outside the query range or the node covers more than the range and we can split → Split, query left and right, and combine the results.

Case 4: The node’s range overlaps and we are at a leaf node → We need to scan through the bucket.

template
MinMax::queryNode(size_t nodeIndex, size_t rangeBegin, size_t rangeEnd) const
{
    // […] calculate Node Range, store in NodeBegin, NodeEnd  
 
    // node outside range, return empty MinMax()
    //               |------range----|
    //  |---node---|           or     |---node---|
    // this should never happen, but safe is safe
    if ((nodeEnd < rangeBegin) || (nodeBegin > rangeEnd))
        return MinMax
 
    // range spans node fully, return nodes' MinMax
    // |---------range-----------|
    //      |------node------|
    if ((rangeBegin <= nodeBegin) && (rangeEnd >= nodeEnd))
        return m_treeOntopOfBuckets[nodeIndex];
 
    // node cuts range left or right or both
    //       |------range----|
    //  |---node---| or |---node---| or
    //     |--------node-------|
    if ((nodeBegin <= rangeBegin ) || (nodeEnd >= rangeEnd))
    {
        const MinMax leftMinMax = queryNode(leftChild(nodeIndex), rangeBegin, rangeEnd);
        const MinMax rightMinMax = queryNode(rightChild(nodeIndex), rangeBegin, rangeEnd);
        MinMax resultMinMax;
        resultMinMax.min = std::min(rightMinMax.min, leftMinMax.min);
        resultMinMax.max = std::max(rightMinMax.max, leftMinMax.max);
        return resultMinMax;
    }
     
    // Node is leaf, re-scan its bucket for subrange
    //              |---------range-----------|
    //  |----node(leaf)----|   or   |----node(leaf)----|
    if( nodeIndex % 2 == 1)
    {
        MinMax scanResult;
        for(size_t i = std::max(nodeBegin, rangeBegin); i <= std::min(nodeEnd, rangeEnd); i++)
        {
            scanResult.min = std::min(m_dataInBuckets[i], scanResult.min);
            scanResult.max = std::max(m_dataInBuckets[i], scanResult.max);
        }
        return scanResult;
    }
}

Results

Visualizing 300.000 data points took more than 2 seconds with the naïve approach (drawing lines from point to point) on the embedded test hardware, while this approach brought down the time for queries to about 3 ms, such that the pure line rendering now even accounts for the bigger part of the cost (6 ms). We again have reached levels below the juicy target of 16 ms.

Further Optimization: Find Lowest Common Parent

In a similar setup, my colleague James Turner found that many of our node visits actually were searching down the tree for the lowest common node covering the full range. In these cases, we often ended up splitting the node in its two children, one of which containing the range fully and the other not containing the range at all.

This initial search down the tree took most of the node visits, while the actual desired min max scan (collecting all the nodes containing the range) was less than that.

So instead of searching from the top, beginning at the root node, I now calculate the left and right leaves, then determine the lowest common node spanning the full range, and perform the query from there (which is a simple loop on bit operations).

I hope, you found this useful. I am happy to hear your comments about this solution. At KDAB, we do these kind of optimizations specific to low-end hardware all the time, ask us if we can help you or check out our Profiling Workshop to learn how to find and tackle performance bottlenecks in your own projects.

The post A Speed-Up for Charting on Embedded appeared first on KDAB.

KDAB is presenting two great talks at Qt World Summit. At 13:30, you can get an in-depth look at a new concept for designers and developers with James Turner.

At 14:30, Milian Wolff will be presenting some of KDAB’s renowned opensource tools, some of which, like Hotspot, he created himself.

Streamlined integration of 3D content straight from 3DS Max and Blender into Qt3D

with James Turner

Qt 3D is now established as the ideal 3D rendering engine for cross platform applications needing to integrate 3D content with 2D interfaces in a seamless way. The scene graph lets developers import and control geometry and materials, while the frame graph can be used to build advanced rendering  effects with ease.

 However, just as in the case of 2D content, best results are achieved when designers and developer share a common experience and toolset. Yet the tools used to author 3D scenes, the geometry, the materials, the lighting, the animations, are all very different from what most developers are familiar with.

In this talk, we introduce techniques and principles which make the collaboration between designers and developers smoother. We identify the major pain points and highlight tools and patterns for addressing each of them. We will look at data formats, model conditioning, runtime integration and rendering effects in order to produce compelling applications integrating 3D content with a traditional 2D interface.

We will illustrate all these concepts with a running example.

KDAB’s Opensource Tools for Qt

with Milian Wolff

Richard Feynman is considered one of the greatest physicists, largely because he was able to build lots of practical mathematical tools that greatly simplify the analysis of complex phenomena, thus making them accessible and useful to many.

KDAB maintains its position as the leading Qt consulting firm, partly because of the huge effort KDAB’s top-class engineers put into R&D projects to build great tooling for Qt applications. This tooling has helped countless other engineers solve bugs in their code, fix performance issues and prevent more problems from slipping into codebases.

The cherry on top of all this Qt goodness is that KDAB publishes its tooling under Opensource licenses!

Come to this talk if you want to discover how KDAB’s awesome Opensource Tools for Qt can help you.

View the full agenda here. Sign up for Qt World Summit here.

The post KDAB Talks at Qt World Summit Berlin appeared first on KDAB.