Boosting Android Apps Performance

While mobile vendor giants struggle to produce the most intellectual device, it appears that a huge amount of users want to buy a device as cheap as possible but still prefer smartphones over old-school feature-phones. This trend is mostly driven by a huge demand from emerging markets such as China and other Asian countries. Sub-$100 Android devices become a significant part of the market. There are rumors that the next version of Android will be aimed at low-end phones market, which means that this trend will continue to emerge.
This trend is leading to a great opportunity for app developers. However, with every great change come new challenges. Low cost devices usually have limited memory, small screen, slow CPUs and other limitations. App developers should care about performance of their code like never before.
There are three basic rules of writing applications with a great performance:

1. Write efficient code.
2. If the code is not efficient - profile and fix it.
3. Continuously ensure that code is still efficient (ideally, after every single commit to your source control system). 

While items 1 and 2 are obvious, item 3 requires additional explanation. Our experience shows that most software developers start to care about performance of their app after users already posted angry comments on Google Play. It comes even harder to track app performance metrics when your app is big and complex and you ship new versions frequently. The problem is that it is much easier to fix performance issue right after it was introduced than in case it is here for a few weeks and tons of code dependent on that issue were written. Of course manual regression testing is inefficient in this case - you simply can’t do it frequently enough. Fortunately, there are things like continuous integration that make it possible to automate this routine and thus detect and fix performance issues as early as possible.
In this document we are going to focus on items 2 and 3, as the art of writing efficient code is bit complicated for a single publication.

Performance Optimization

Performance tuning of every Android application differs from application to application. However there is a general flow. It looks like this:

This is the so-called measure-evaluate-improve-learn cycle. This process can be applied to a variety of life activities. When tuning Formula 1 car, the same technique would be used. You just need different tools.
To start the optimization cycle we need to learn about the problem This can be done in the beginning of development, after manual testing with unsatisfactory performance results. This can be a report from CI tool, which was triggered after unsafe commit. This can also be a report from users on Google Play with complaints about low performance on specific device. Such an optimization may be needed before implementing new features, which is dependant on existing, not-fast-enough functionality.
After we learned about the problem we start the optimization. In order to improve performance, we need to identify the problem. And this is the place where measuring comes handy. Android SDK has built-in capabilities for profiling. With the help of traceview and DDMS one can profile and analyze performance of the application. Apart from that, measuring is helpful for automation of testing process, by giving determined data for comparison.
After the measurements are done, detecting the bottleneck is pretty straightforward. When the problem is found, it’s time to fix it by optimizing the code. That’s where you would need all your creativity and knowledge of the domain. If you know some hardcore techniques and cheats which still maintain the sustainability of code, try them! In case of Android, think about NDK and RenderScript usage. Good idea is to check how often GC is called inside your app. On low-cost devices heap size can be surprisingly small, which will lead to often garbage collection and therefore low responsiveness. Good practice is reusing memory, instead of allocating new blocks. Also in case of intensive IO operations, asynchronous reads/writes are a must, and probabilistic variations of algorithms together with intelligent caching techniques can save the day.
After each step of improvements we should again test the application, i.e. learn if the problem is fixed. If the results are not satisfactory, the process goes all over again until the problem is gone.
Performance optimizations usually involve a decent amount of refactorings. In order to make refactorings safe it’s a good practice to cover code with tests. In this case Continuous Integration tools can greatly simplify testing and maintenance job.

Continuous Application Support

Rome wasn’t build in a day. Even though people can make alpha mockups of applications in several days, a good application needs a lot more time to become mature.
When you own 1 application, you can choose couple of target devices and test your app on them. If you have 2 apps, you need twice as much time to test everything by hand. If you are an established mobile development company with a big mobile portfolio, costs for manual application testing can blow up your budget. Moreover, it’s hard to track how all the apps evolve over time.
Continuous Integration tools automate the process of application testing, deployment and maintenance. They help finding bugs early, save time and money during the lifespan of the project.
In case of Android projects, CI tools have proven their effectiveness, especially when there is a need to test applications on a variety of configurations. Currently, the most used CI for Android is Jenkins. It has an active and vibrant community of users, lots of useful plugins and is easily extensible.
Below is the proposed architecture for the private continuous integration testing environment, based on Jenkins CI:

This environment would allow:

• testing multiple device configurations
• tracking multiple projects
• measuring test execution performance over time
• analysing source code on every commit
• finding android lint issues
• automating deployment tasks

Actually, there’s almost no limit for the Jenkins CI improvements and customizations. With the help of custom plugins this system can be enriched with various kinds of actions, which automate daily DevOps routines.
Of course, there is no silver bullet and implementing CI environment also needs a lot of resources, intelligence and of course people. Tests have to be implemented by people who have programming skills, testing infrastructure has to be set up and maintained by an administrator, and whole environment needs additional hardware and software expenses.  The effectiveness of automation start to show off after certain amount of manual work redundancy, which is often the case with Android development.

Conclusion

Keeping up with fast-changing industry standards is far from trivial. Supporting thousands of different devices with different OS versions each is indeed not easy. Supporting several apps for this market is even harder. Keeping it all fast is close to impossible. Maintaining this with every new release or bugfix is a pure nightmare. Without a right tool.
Implementing functionality changes is very similar to production of goods on factory. Every good, as every update, is designed with accuracy and intelligence. But before reaching end users each one of them passes same strict quality control, keeping the production massive, continuous and reliable.

by Ostap Andrusiv, Victor Haydin and Markiyan Matsekh

NVIDIA Tesla K20 benchmark: facts, figures and some conclusions

Newest GPGPU flagman, Tesla K20 was announced by NVIDIA at Supercomputing conference in Salt Lake City yesterday (BTW, you can meet Roman Pavlyuk, ELEKS' CTO and Oleh Khoma, Head of HPC Unit there). Due to partnership with NVIDIA we got access to K20 couple of months ago and did lots of performance tests. Today we're going to tell you more about it's performance in comparison with several other NVIDIA accelerators that we have here at ELEKS. 

Test environment

We implemented set of synthetic micro-benchmarks that measure performance of following basic GPGPU operations:

• Host/Device kernel operations latency
• Reduction time (SUM)
• Dependent/Independent FLOPs
• Memory management
• Memory transfer speed
• Device memory access speed
• Pinned memory access speed

You can find more information and benchmark results below. Our set of tests is available on GitHub, so that you can run them on your hardware if you want. We ran these tests on seven different test configurations:

• GeForce GTX 580 (PCIe-2, OS Windows, physical box)
• GeForce GTX 680 (PCIe-2, OS Windows, physical box)
• GeForce GTX 680 (PCIe-3, OS Windows, physical box)
• Tesla K20Xm (PCIe-3, ECC ON, OS Linux, NVIDIA EAP server)
• Tesla K20Xm (PCIe-3, ECC OFF, OS Linux, NVIDIA EAP server)
• Tesla M2050 (PCIe-2, ECC ON, OS Linux, Amazon EC2)
• Tesla M2050 (PCIe-2, ECC ON, OS Linux, PEER1 HPC Cloud)

One of the goals was to determine the difference between K20 and older hardware configurations in terms of overall system performance. Another goal: to understand the difference between virtualized and non-virtualized environments. Here is what we got:

Host/Device kernel operations latency

One of the new features of K20 is Dynamic Parallelism that allows you to execute kernels from each other. We did a benchmark that measure latency of kernel schedule and execution with and without DP. Results without DP look like that:

Surprisingly, new Tesla is slower than old one and GTX 680, probably because of the driver which was in beta version at the moment we measured performance.  It is also obvious that AWS GPU instances are much slower than closer-to-hardware PEER1 ones, because of virtualization.

Then we tried to run similar benchmark with DP on:

Obviously we couldn't run these tests on older hardware because it doesn't support DP. Surprisingly, DP scheduling is slower than traditional one, but DP execution time is pretty much the same with ECC ON and traditional is faster with ECC OFF. We expected that DP latency would be less than traditional. It is hard to say what is the reason of such slowness. We suppose that it could be a driver, but it is just our assumption.

Reduction time (SUM)

Next thing we tried to measure was reduce execution time. Basically we calculated array sum. We did it with different arrays and grid sizes (Blocks x Threads x Array size):

Here we got expected results. New Tesla K20 is slower on small data sets, probably because of less clock frequency and not fully-fledged drivers. It becomes faster when we work with big arrays and use as many cores as possible. 

Regarding virtualization, we found that virtualized M2050 is comparable with non-virtualized one on small data sets, but much slower on large data sets. 

Dependent/Independent FLOPs

Peak theoretical performance is one of the most misunderstood properties of computing hardware. Some people says it means nothing, some says it is critical. The truth is always somewhere between these points. We tried to measure performance in FLOPs using several basic operations. We measured two types of operations, dependent and independent in order to determine if GPU does automatic parallelization of independent operations. Here's what we got:

Surprisingly, but we haven't got better results with independent operations. Probably we have some issue with our tests or misunderstood how does automatic parallelization work in GPU, but we couldn't implement the test where independent operations are automatically paralleled.
Regarding overall results, Teslas are much faster than GeForces when you work with double precision floating point numbers, which is expected: consumer accelerators are optimized for single precision because double is not required in computer games, primary software they were designed for. FLOPs are also highly dependent on clock speed and number of cores, so newer cards with more cores are usually faster, except of one case with GTX 580/680 and double precision: 580 is faster because of higher clock frequency.
Virtualization doesn't affect FLOPs performance at all.

Memory management

Another critical thing for HPC is basic memory management speed. As there are several memory models available in CUDA it is also critical to understand all the implications of using each of them. We wrote a test that allocate and release 16 b, 10 MB and 100 MB blocks of memory in different models. Please note: we got quite a different results in this benchmark, so it makes sense to show them on charts with logarithmic scale. Here they go:

Device memory is obviously the fastest option in case you allocate big chunk of memory. And GTX 680 with PCIe-3 is our champion in device memory management. Teslas are slower than GeForces in all the tests. Virtualization seriosly affects Host Write Combined memory management. PCIe-3 is better than PCIe-2 which is also obvious.

Memory transfer speed

Another important characteristics of an accelerator is speed of data transfer from one memory model to other. We measured it by copying 100 MB blocks of data between Host and GPU memory in both directions using regular, page locked and write combined memory access models. Here's what we got:

Obviously, PCIe-3 configurations are much faster than PCIe2. Kepler devices (GTX 680 and K20) are faster than other. If you use Page Locked and Write Combined models it makes your transfer speed faster. Virtualization slightly affects regular memory transfer speed, and doesn't affect others at all. We also tested internal memory transfer speed (please note, we haven't multiplied it by 2 as NVIDIA does usually in their tests):

Tesla K20s are faster than GeForce, but difference is not so big. M2050 are almost two times slower then their succesors.

Device memory access speed

We also measured device memory access speed for each configuration we have. Here they go:

Alligned memory access is way faster than non-aligned (almost 10 times difference). Newer accelerators are better than older. Double precicion read/write is faster than single for all the configurations. Virtualization doesn't affect memory access speed at all.

Pinned memory access speed

Last metric we measured was pinned memory access speed when device interacts with host memory. Unfortunately we weren't able to run these tests on GTX 680 with PCIe-3 due to issue with big memory blocks allocation in Windows. 

New Tesla is faster then old one. PCIe-3 is obviously faster. Aligned access is almost ten times faster and if you read double precision floats your memory access speed is two times bigger than if you work with single precision floats. Virtualized environment is slower than non-virtualized.

Conclusions

All-in-all new Tesla K20 performs slightly faster than their predecessors. There is no revolution. There is evolution - we got better performance, new tools that make programmer's life easier. There also are several things that are not mentioned in this benchmark, like better support of virtualization and as a result cloud-readyness of K20. Some results were surprising. We expect better results of K20 in several months when new, optimized version of drivers will be available (NVIDIA always has some issues with new drivers just after release, but usually fix them after several updates).

You can find spreadsheet with complete results at Google Docs. Benchmark sources are available at our GitHub.