Cloud computing: myths, fears and facts

Last time we were writing about Cloud Computing basics. Today we are going to discuss myths about cloud computing. Our main goal is to understand what is real and what is just a myth.

Lower cost 

Probably the most controversial topic about Cloud Computing is it's cost. On the one hand, vendors usually say cloud computing is cheaper then traditional IT. On the other hand, there are lots of examples when people count their cloud expenses, compare results with traditional infrastructure cost and get opposite experience. So, where is the truth?
Well, the truth is that Cloud is expensive but sometimes cheaper. How can it be? There are three basic principles of cost calculation when you are trying to determine cost of your cloud infrastructure:
1. Do correct TCO calculation.
2. Remember about workload patterns.
3. Do not forget about additional benefits of the Cloud.

Cloud cost is like an iceberg: you see only its top part and most its volume is hidden under water. People usually tend to calculate only hardware price when they are comparing cloud cost with on-premise infrastructure. However, total cost of ownership is much bigger than just a hardware price: you have to remember about staff salary, training, operations costs, risk management and lots of other things that cloud vendor do for you.
Another important thing you have to understand when you are considering cloud hosting for your application: workload patterns. There are several types of application workload that work best with the cloud:

Each of the patterns above utilize key property of the cloud: ability to scale up and down. Main difference with traditional IT here is in pay-per-use nature of the cloud: you don't have to pay for resources when you don't use them. With traditional IT you have to pay in advance for all resources you need and it is not so easy to get rid of them in case they are no longer needed.
And what if you have this:

Well, maybe it means cloud is not suitable for your case. Or you forgot about additional benefits of the cloud. What are these benefits?

Additional Benefits

First of all, cloud allows you to transfer your capital expenses to operational. Basically it means you don't have to pay in advance for your infrastructure. It is usually easier to pay money after you serviced your clients and got money from them than to pay in advance when you are still not sure if anyone will actually buy  your service.

Another great benefit of the cloud: faster time-to-value. With cloud you can shorten time, required for infrastructure setup from months to hours. It is very important for big enterprises, where IT services are often terribly slow. Sometimes it takes several months to buy and setup servers for some particular application. At the same time with the cloud it takes just several minutes to allocate resources and deliver them to development team. It means additional flexibility and agility for your business which automatically leads to competitive advantage.

And the last but not least: with cloud computing you can focus on your core business and don't care about IT infrastructure management. Let professionals do their part of job and focus on your part.

Fears

Cloud is still something new and unknown for lots of people. People are usually scared by new things and cloud is not an exception in this case. We are going to review most common fears about the cloud and determine whether they are real or not.

Security

Security is probably the most discussed thing about the cloud in the enterprise community. Businesses want to leverage cloud benefits, but they are worried about their data which sometimes is their biggest asset. How to make sure that data is secured in the cloud? Let's take a look at facts. Security risks in the cloud are pretty much the same as in your own data-center. Really. If you don't make some extraordinary efforts to secure your data center from unauthorized access, you have the same security as in the cloud. Moreover cloud providers continuously improve their security which probably means that your data center actually might be less secured. Simply because big vendors hire world's best security engineers and you usually don't.

Most vendors nowadays offer the customer different levels of security protection. You can setup virtual private clouds accessible only from your corporate network. You can use numerous encryption algorithms. You can have almost any type of security configuration you can imagine with modern cloud offerings. So, security is just a matter of right usage, not an actual issue of the cloud.

Data control lose

Some people are worried that they can lost access to their data in case of some catastrophic failure. It is true that it can happen, but let's look at the facts. In fact, the data in the cloud is replicated and geographically distributed, which means it is much safer than in most private data-centers. Just ask yourself, are you sure that the data in your data-center is replicated to several devices? And what about geographical distribution? What if there will be some natural disaster in the region where your data-center is located? Moreover, nowadays, if you lost the Internet connection it usually means lost data access is not your biggest problem. It is hard to underestimate importance of the Internet for modern international companies. In case they would lost the connection they could not operate at all. Data accessibility would not be biggest of their problems in such case.

Reliability

When Amazon data-center fails it looks like the whole Internet is down. Too many people host their sites on AWS and when it fails everybody notices that. Does it mean cloud is unreliable? Not at all. Important thing to remember: those guys who fail with Amazon don’t use geographical redundancy for some reasons. Amazon recommends to use it in case you want to deliver reliable service. It is not so hard to build reliable application hosted in the cloud and it is not a vendor problem if people don't do it. SLA is still 99.95% or so.

Additionally, imagine what would you do if such kind of failure happened in your data-center? With cloud you can have a mirror setup in another region or even in another vendor's cloud. It is hard to do have the same within your own data-center.

Performance

People often think that cloud computing is slower than traditional servers. There are various reasons of such opinion and some of them really make sense. People blame virtualization, but it is not the main reason of a slowness: vendors use hardware virtualization which means that for most operations they have the same performance as bare metal appliances. It is true that I/O latency is higher, but it matters only for high performance computing apps, not for most regular business software. It is also true that some of legacy applications could be slower after migration to the cloud. In fact, it is relatively easy to get good enough performance in the cloud if you think about it from the very beginning. It is just a matter of system architecture.

Legal regulations

There are some regulations that restricts some types of data to be hosted in a public cloud. Sometimes it simply means you need a mirroring to your private storage: just to have ability to provide government with physical access to your servers if it is required by some regulatory act. Some clouds are certified to be compliant with some regulations (e.g. there are plenty of HIPAA-compliant cloud storages). Sometimes there is no workaround and you have to use on-premise infrastructure for some part of your solution.

Conclusions

Cloud Computing is still new and evolving technology. It makes it interesting and attractive for engineers, but also there are plenty of missunderstanding and fears about it. Lots of people don't understand how it works, where it is applicable and when it makes sense to use them. High level of uncertainty leads to mistakes. Mistakes lead to disappointment. And disappointment produces fears. In fact, Cloud Computing is quite a nice technology which makes lots of things easier. You just have to understand how it works and what can you get with it. We hope this post brought you better understanding of Cloud Computing technologies.

We wish you Happy New Year and sweet winter holidays. See you in 2013!

GAME LOCALIZATION FOR WINDOWS PHONE 7

Have you ever noticed how serious gamers seem to have their own unique language and culture? From old-school Zelda and MMORPG World of Warcraft to Halo and Half-Life, virtual universes are where today’s hardcore gamers spend several hours a day; many have even grown up in these alternative worlds. So what does this mean for your effort to release your game in new language markets?

There’s a host of game localization success stories out there and your game can join their ranks. By sensitizing yourself to localization elements in your personal gaming endeavors and with little bit of our help, you’ll be ready to conquer the hearts of gamers around the world.

In order to re-create your in-game experience for players in other countries, you’ll want to work with linguists, translators, voice talent and testers who have extensive experience playing games (whether for console, computer, mobile device, etc). Ideally, they’ll also be familiar with your game genre and have an appreciation for the unique setting and terminology. When this isn’t possible, look for linguists who are experienced in transcreation (or copy re-creation) and cultural peculiarities specific to both your regional markets and gamer demographics.

To make things a little bit more clear for those who don’t have much experience in this area, take a look at a small demo game developed using Microsoft XNA framework and Visual C# for Windows Phone 7. 

Dive into the Cloud: brief technology introduction

Cloud Computing is one of the most speculated terms in IT industry last couple of years. We call "cloud" lots of things, sometimes absolutely different. We're starting the series of posts, dedicated to large set of technologies somehow related to Cloud Computing. We start our journey from the very beginning: let's define cloud and understand what does it mean.

Historical excursus

Cloud sounds like something very new. But history tends to be cyclic: if something is happening there is probability that it was already happening before. Before we start, let's get back in time and see if cloud is really something new, or it is just something very old and known. We will travel in time for more than hundred years, to 1882. This year Thomas Edison, one of the most famous inventors of the 19th century, launched Edison Electric Light Station - first ever public electric power plant. You might be wondering, how does it related to Cloud Computing? Well, let's take a look how people used electricity before first power plants were launched. 

Thomas Edison

It was quite a big challenge to have electric power at that time. If you need electricity in 19th century you have to buy steam-powered electricity generator. You also have to buy coal for it. And don't forget to hire someone to feed it with coal and repair the generator if it is broken. And yeah, if it is broken - you just don't have electricity at all. And what is really awkward: you have to find place for it and then suffer from it's noise and stink. Sounds like not a very nice perspective, right? 

And here comes Edison. His company offered people to pay for electricity, produced somewhere far away from their houses and delivered to them via wires. There is no coal. There is no workers. Finally, there is no stink and noise. We could only imagine how those victorian gentlemen were standing in queues shouting "Sir, could you please take my money?". Nowadays, we would say that it was massive win of Edison's startup. 

If you compare Cloud Computing with traditional in-house IT infrastructure you will probably find out that computing power is very similar to electricity. With traditional infrastructure you have to buy servers. You also have to pay for electricity. You have to hire system administrator. Nothing works if server is broken. Finally, server room is very noisy place. The only difference: servers don't stink. Usually. In case you use clouds you don't care about all of these problems. You just pay money and service provider does all the job for you. 

Definition

Cloud Computing is quite a fuzzy term. Lots of people mean very different things when they talk about cloud technologies. One might be wondering, why does it happen? It is like in the ancient story about blind men and an elephant:

Once an elephant came to a small town.  People had read and heard of elephants but no one in the town had ever seen one.  Thus, a huge crowd gathered around the elephant, and it was an occasion for great fun, especially for the children.  Five blind men also lived in that town, and consequently,  they also heard about the elephant.  They had never seen an elephant before, and were eager to find out about elephant.

Then, someone suggested that they could go and feel the elephant with their hands.  They could then get an idea of what an  elephant looked like. The five blind men went to the center of the town where all the people made room for them to touch  the elephant.

Later on, they sat down and began to discuss their experiences.  One blind man, who had touched the trunk of the elephant, said that the elephant must be like a thick tree branch.  Another who touched the tail said the elephant probably looked like a snake or rope.  The third man, who  touched the leg, said the shape of the elephant must be like a pillar.  The fourth man, who touched the ear, said that the elephant must be like a huge fan; while the fifth, who touched  the side, said it must be like a wall.

They sat for hours and  argued, each one was sure that his view was correct.  Obviously, they were all correct from their own point of view, but no one was quite willing to listen to the others.  Finally, they decided to go to the wise man of the village and ask him who was correct.  The wise man said, “Each one of you is correct; and each one of you is wrong.   Because each one of you had only touched a part of the elephant’s body.  Thus you only have a partial view of the animal.  If you put your partial views together, you will get an idea of what an elephant looks like.”

The moral of the story is that each one of us sees things exclusively within one’s point of view.  We should also try to understand other people’s points of view.  This will enable us to get a proper perspective on different  situations and events.

What can we learn from this ancient story? One's understanding of something is highly dependent on theirs perception of the thing. Those people who use SaaS say that Cloud is SaaS. Those who use VMware products say that Cloud is Virtualization. Those who use Apple products say that Cloud is synchronization service for their mobile devices. All of them are right. Cloud could be very different. But we need some kind of definition just to be on the same page when we talk about it. There are several of them, but here at ELEKS we usually use only one, called OSSM:

• On-demand: resource is already setup and ready to be deployed
• Self-service: customer chooses what they want, when they want it
• Scalable: customer can choose how much they want and ramp up if necessary
• Measurable: there’s metering/reporting so you know you are getting what you pay for

You can use mnemonic phrase "Cloud is OSSM" (read it like 'awesome') to remember.
Let's try to apply it to several popular "clouds":
1. Amazon EC2. When you ask about new virtual machine it is ready to be deployed. It is completely self-service. You can choose how many instances do you want. And of course there are lots of reporting that allows you to measure usage. Passed.
2. Dropbox. When you ask about additional storage it is ready to be allocated for you. You can do it by your own. You can buy more space in case you want it. And you can see how much space do you really use. Passed.
3. Virtual server farm operated by your IT staff. If they have enough servers they are ready to deploy your VM on-demand. Usually you can't allocate new VM by yourself, you have to ask your IT guys to do it. It could be scalable, but usually it is not measurable: you don't have access to list of all your VMs and don't understand how they are used. Failed.

Basic vocabulary

There are few buzzwords that are usually used within same context with the cloud. We need to clarify them before we proceed to next topics about the cloud. 

Service models

SaaS. PaaS. IaaS. [Software|Platform|Infrastructure]-as-a-Service. What does it mean? What is a cloud and what is not? Well, technically you can build any of these in way it wouldn't be OSSM. But I wouldn't recommend you to do it.

Let's take a look at IT infrastructure. We can imagine it as technology stack with network layer at the bottom and applications at the top. Difference between traditional IT, IaaS, PaaS and SaaS is in management responsibilities between you and your vendor:

In traditional IT you manage everything, starting with network and finishing with apps. When you use some IaaS cloud service (think Amazon EC2) vendor does all hardware management for you. But you are still responsible for all software layers: operating system, database, frameworks, runtimes etc. PaaS is higher level option where vendor provides you with fully configured platform that runs your applications (usually it means you have to adtopt your apps somehow, but cost of adoption usually is not so big). SaaS is top-level service option: vendor manages all components of your IT stack. People usually tend to compare all three models, but we won't do it. All three models are useful, they just have different goals and user audience. 

Basically, IaaS is an option for system administrators (or operations team) who don't want to deal with hardware. With IaaS they can just create virtual infrastructure, programmatically configure it and have fun. PaaS is an option for software engineers that don't want to deal with system administrators. PaaS allows them to deploy their apps to abstract platform that incapsulates all the stuff, related to hardware, configuration and other things. And, finally, SaaS is cloud for end-users: they just got their software working and ready to use, without any involvement of IT stuff or developers. 

Hosting models

Another kind of buzzword usually used with the cloud is hosting models. People talk about public, private and hybrid clouds. What does it mean? 

Public cloud is a cloud infrastructure made available to the general public or a large industry group and owned by an organization selling cloud services. Private cloud is cloud infrastructure operated solely for an organization. It may be managed by the organization or a third party and may exist on premise or off premise. And finally, hybrid cloud is any combination of private and public clouds working together. Very simple, right?

Conclusions

Cloud computing is one of the most interesting areas of IT nowadays. Cloud market is growing very fast and producing new opportunities for businesses one by one.
Electric power analogy is very nice for better understanding of the future of the cloud computing. Almost no one have their own power generators nowadays. Same thing will happen with computing once: people will have thin terminals (e.g. mobile devices) that will be used for accessing massive computing facilities in the cloud.
Using OSSM definition you can check whether service is really cloud or you are victim of aggressive marketing that attach cloud label to anything in order to sell it. You can better understand key properties of cloud services when you build them.
Different service models allow you to manage only those parts of your IT stack you want to manage. With public, private and hybrid clouds you can share infrastructure with other people, own it completely or use some combination of the above.
See you next time. Stay tuned!

Android vs. iOS: UI/UX Differences

Challenge

Create application that supports both platforms (iOS and Android) and has a similar functionality optimized for interaction design principles and users expectations unique to each platform.

Environment

The important thing to keep in mind is that iOS and Android environments are based on unique for each platform guidelines, user interaction architecture and design patterns. There is a number of iOS and Android similarities in look and behavior of various UI components like

• information structure; 
• basic UI elements (sliders, checkboxes, tabs, text boxes, fields etc.); 
• list-based navigation; 
• majority of gesture touch controls (excluding “tap and hold” gesture, which is commonly used to reveal contextual bar with options or enter a data selection mode).

However the number of differences is worth paying attention to. Below you may find a short overview of the core design peculiarities that should help better understand differences in the design approach for both Android and iOS.

1. “Back” navigation. 
In iOS applications “back” option is placed in the upper-left corner of the navigation bar. It is used to navigate backward within the defined screens in the application however it is not used to navigate backward across the entire device.
In Android devices there are two types of “back” actions: “up and “back”. “Up” is placed in the upper-left corner of the top bar and is used to navigate up the application’s information hierarchy. In contrast, “back” option is presented as a button on the physical device that allows navigating backward across the entire device.

Back navigation in GMail for Android and Dropbox for iOS

2. Top navigation. 
In iOS applications tab navigation is placed at the bottom of the screen. In addition, according to iOS guidelines there are no more than 5 tabs displayed at a time.
In Android applications tabs are recommended to be placed at the top of the screen. Besides scrollable tabs are allowed to be used in case there are more tabs than can fit in the viewable screen width.

Top navigation in Google Play and Dropbox for iOS

3. Switching between various data views.
In iOS applications switching between views of the single set of data is typically done through the bar divided into segments.  Each segment is responsible for one view.
In Android applications switching between views is done through the UI control “spinner”. This control is presented like a drop-down list of options. “Spinner” is usually placed at the top action bar.

Switching between data views in Google Calendar for Android and iOS Calendar

4. Search.
In iOS applications the searching UI control is placed at the top of the screen mainly.
In Android applications several searching options are available:

• “search bar” at the top of the screen that is similar to the iOS approach. However the bar is hidden until the user clicks on the search icon;
• “search widget” that can be placed anywhere within the application interface. Coomonly it is used within the application’s action bar at the top of the screen.

Search in Google Play and Foursquare for iOS

5. Actions. 
In iOS applications can be accessed through the toolbar that contains action buttons, through the action button that is in the upper-right corner hand side of the navigation bar or through the buttons within the interface screen.
In Android applications it is recommended to display actions in the action bar at the top of the screen. If there is any need in displaying more actions than can fit on the action bar, either an action overflow icon appears on the action bar for devices that don’t have a hardware “menu” button, or the user accesses additional actions by pressing a hardware “menu” button on devices where there is one. Android applications may also use contextual action bar. A contextual action bar is a temporary action bar that overlay the app's action bar for the duration of a particular sub-task.

Actions in GMail for Android and GMail for iOS

6. Screen sizes and resolutions. 
iOS phones, for instance, come in two screen sizes and three resolutions (including the latest iPhone 5).
Android devices are represented by a lager list of screen sizes and screen resolutions. This issue has a significant impact on the layout while designing the application.

Summary

It may appear as the straightforward idea to create one application for both platforms however important issue to consider is that interface elements of both platforms are not the same.
Though application’s core features and functionality may be the same on both platforms application’s interface should follow specific for each platform guidelines. Therefore to meet user expectations and ensure smooth user experience application’s design should be adapted to the unique platform design patterns and respect native UI standards.

by Iryna Pantel, UX Designer

Why use Exploratory Testing?

Why Do We Need Exploratory Testing?

• At times, it helps in revealing many unknown and un-detected bugs, which is very hard to find out through normal testing.
• As it covers almost all the normal types  of testing, it helps improving our productivity in terms of covering the scenarios in scripted testing and those which are not scripted as well.
• It is a learn and work type of testing activity where a tester can learn more and understand the software if he/she was not able to reveal any potential bug.
• Although disliked by many, helps testers in learning new methods, test strategies, and also think out of the box and attain more and more creativity.

Advantages of Exploratory Testing:

• It can uncover bugs, which are normally ignored (or hard to find) by other testing strategies.
• It helps testers learn new strategies, expand the horizon of their imagination that helps them in understanding and executing more and more test cases, and finally improve their productivity.
• It helps tester in confirming that he/she understands the application and its functionality properly and has no confusion about the working of even a smallest part of it, hence covering the most important part of requirement understanding.
• As in case of this testing, we write and execute the test cases simultaneously. It helps in collecting result-oriented test scripts and shading a load of unnecessary test cases which do not yield a result.
• Exploratory testing covers almost all types of testing, hence tester can be sure of covering various scenarios once the testing is performed at the highest level (i.e. if the exploratory testing performed, it can ensure that all the possible scenarios and test cases are covered). 
  
  
  by Andriy Skop, QA Project Lead

Exploratory Testing Styles

There are different Exploratory Testing Styles and variations that often yield similar results. What follows are some styles I've observed.

Intuit

This is the most common style. Testers who haven't learned specific exploratory testing techniques tend to do this naturally. When you ask them what they are doing when they are testing in the absence of pre-scripted test cases, they may say, "I don't know why I did that," or that they are using their intuition. Intuition is just a fancy way of saying, "I am doing this because of the insight I have based on my experience and knowledge." It can appear to be random or chaotic, but when the tester is pressed for an explanation of what he did, a structure and purpose emerge.

What is Exploratory Testing

"Exploratory software testing is a powerful approach, yet widely

misunderstood. In my experience, it can be orders of magnitude

more productive than scripted testing. All testers who create tests

at all practice some form of exploratory testing, yet many don't

even realize it. Few of us study this approach, and it doesn't get

much respect in our field. This attitude is beginning to change

as companies seek ever more agile and cost effective methods of

developing software."

James Bach

Exploratory testing (ET) is an approach to test software where the tester does not need to follow a specific test design. But rather, ET should facilitate the tester in testing the complete system comprehensively. ET is seen by some, as a way to conduct simultaneous learning, test design and execution of tests simultaneously. Today, ET is defined by most researchers as an activity where a developer/tester simultaneously learns, design and execute the tests. To summarise this, it means that the tester is exploring the software, learning its functionality and performing test execution on the basis of her intuition. No specific systematic approach is followed in terms of following a scripted test case document that leads the tester to execute the tests on a step by step basis. The tester himself controls the design of the tests while executing and learning more about the software. This helps her in building tests effectively while exploring the undiscovered parts of the software.

Open-source WPF Layout to Layout Transitions library

Modern users are cockered with well-designed fluid UIs. Based on our own experience we know what an wow effect can layout to layout transitions make for application. Unfortunately, by this time there were no free and open source solution to support them in WPF. Yuriy Zanichkovskyy has implemented an open-source library that for Layout-to-Layout transitions in WPF. You can find detailed description of the library on CodeProject and fork the code on GitHub.

A deep look into the Event Store

A deep look into the Event Store from Øredev Conference on Vimeo.

What if I told you that the new Event Store (OSS geteventstore.com) is an ACID compliant database with only 24 bytes of mutable data? This session will look deep inside the Event Store and architectural decisions and trade offs made in the development of it.

Greg Young
Greg Young is a loud mouth about many things including CQRS, Event Sourcing, and getting your tests to do something more than validating your code. He is currently involved with Event Store a functional database geteventstore.com

Penetration Testing vs Vulnerability Assessment

There seems to be a certain amount of confusion within the security industry about the difference between Penetration Testing and Vulnerability Assessment, they are often classified as the same thing when in fact they are not.

Penetration Testing may sound a lot more exciting, but most people actually want a VA not a pentest, many projects are labelled as pen tests when in fact they are 100% VA.

A Penetration Test mainly consists of a VA, but it goes one step further..

A penetration test is a method of evaluating the security of a computer system or network by simulating an attack by a malicious hacker. The process involves an active analysis of the system for any weaknesses, technical flaws or vulnerabilities. This analysis is carried out from the position of a potential attacker, and can involve active exploitation of security vulnerabilities. Any security issues that are found will be presented to the system owner together with an assessment of their impact and often with a proposal for mitigation or a technical solution.

A vulnerability assesment is what most companies generally do, as the systems they are testing are live production systems and can't afford to be disrupted by active exploits which might crash the system.

Vulnerability assessment is the process of identifying and quantifying vulnerabilities in a system. The system being studied could be a physical facility like a nuclear power plant, a computer system, or a larger system (for example the communications infrastructure or water infrastructure of a region).


Vulnerability assessment has many things in common with risk assessment. Assessments are typically performed according to the following steps:

1. Cataloging assets and capabilities (resources) in a system

2. Assigning quantifiable value and importance to the resources

3. Identifying the vulnerabilities or potential threats to each resource

4. Mitigating or eliminating the most serious vulnerabilities for the most valuable resources


This is generally what a security company is contracted to do, from a technical perspective, not to actually penetrate the systems, but to assess and document the possible vulnerabilities and recommend mitigation measures and improvements.

On the other hand, a pen test simulates the actions of an external and/or internal attacker that aims to breach the security of the organization. Using many tools and techniques, the penetration tester attempts to exploit critical systems and gain access to sensitive data. Depending on the scope, a pen test can expand beyond the network to include social engineering attacks or physical security tests. Also, there are two primary types of pen tests: "white box", which uses vulnerability assessment and other pre-disclosed information, and "black box", which is performed with very little knowledge of the target systems and it is left to the tester to perform their own reconnaissance. Typically, pen tests follow these steps:

1. Determination of scope
2. Targeted information gathering or reconnaissance
3. Exploit attempts for access and escalation
4. Sensitive data collection testing
5. Clean up and final reporting

by Andriy Skop  

NVIDIA Tesla K20 benchmark: facts, figures and some conclusions

Newest GPGPU flagman, Tesla K20 was announced by NVIDIA at Supercomputing conference in Salt Lake City yesterday (BTW, you can meet Roman Pavlyuk, ELEKS' CTO and Oleh Khoma, Head of HPC Unit there). Due to partnership with NVIDIA we got access to K20 couple of months ago and did lots of performance tests. Today we're going to tell you more about it's performance in comparison with several other NVIDIA accelerators that we have here at ELEKS. 

Test environment

We implemented set of synthetic micro-benchmarks that measure performance of following basic GPGPU operations:

• Host/Device kernel operations latency
• Reduction time (SUM)
• Dependent/Independent FLOPs
• Memory management
• Memory transfer speed
• Device memory access speed
• Pinned memory access speed

You can find more information and benchmark results below. Our set of tests is available on GitHub, so that you can run them on your hardware if you want. We ran these tests on seven different test configurations:

• GeForce GTX 580 (PCIe-2, OS Windows, physical box)
• GeForce GTX 680 (PCIe-2, OS Windows, physical box)
• GeForce GTX 680 (PCIe-3, OS Windows, physical box)
• Tesla K20Xm (PCIe-3, ECC ON, OS Linux, NVIDIA EAP server)
• Tesla K20Xm (PCIe-3, ECC OFF, OS Linux, NVIDIA EAP server)
• Tesla M2050 (PCIe-2, ECC ON, OS Linux, Amazon EC2)
• Tesla M2050 (PCIe-2, ECC ON, OS Linux, PEER1 HPC Cloud)

One of the goals was to determine the difference between K20 and older hardware configurations in terms of overall system performance. Another goal: to understand the difference between virtualized and non-virtualized environments. Here is what we got:

Host/Device kernel operations latency

One of the new features of K20 is Dynamic Parallelism that allows you to execute kernels from each other. We did a benchmark that measure latency of kernel schedule and execution with and without DP. Results without DP look like that:

Surprisingly, new Tesla is slower than old one and GTX 680, probably because of the driver which was in beta version at the moment we measured performance.  It is also obvious that AWS GPU instances are much slower than closer-to-hardware PEER1 ones, because of virtualization.

Then we tried to run similar benchmark with DP on:

Obviously we couldn't run these tests on older hardware because it doesn't support DP. Surprisingly, DP scheduling is slower than traditional one, but DP execution time is pretty much the same with ECC ON and traditional is faster with ECC OFF. We expected that DP latency would be less than traditional. It is hard to say what is the reason of such slowness. We suppose that it could be a driver, but it is just our assumption.

Reduction time (SUM)

Next thing we tried to measure was reduce execution time. Basically we calculated array sum. We did it with different arrays and grid sizes (Blocks x Threads x Array size):

Here we got expected results. New Tesla K20 is slower on small data sets, probably because of less clock frequency and not fully-fledged drivers. It becomes faster when we work with big arrays and use as many cores as possible. 

Regarding virtualization, we found that virtualized M2050 is comparable with non-virtualized one on small data sets, but much slower on large data sets. 

Dependent/Independent FLOPs

Peak theoretical performance is one of the most misunderstood properties of computing hardware. Some people says it means nothing, some says it is critical. The truth is always somewhere between these points. We tried to measure performance in FLOPs using several basic operations. We measured two types of operations, dependent and independent in order to determine if GPU does automatic parallelization of independent operations. Here's what we got:

Surprisingly, but we haven't got better results with independent operations. Probably we have some issue with our tests or misunderstood how does automatic parallelization work in GPU, but we couldn't implement the test where independent operations are automatically paralleled.
Regarding overall results, Teslas are much faster than GeForces when you work with double precision floating point numbers, which is expected: consumer accelerators are optimized for single precision because double is not required in computer games, primary software they were designed for. FLOPs are also highly dependent on clock speed and number of cores, so newer cards with more cores are usually faster, except of one case with GTX 580/680 and double precision: 580 is faster because of higher clock frequency.
Virtualization doesn't affect FLOPs performance at all.

Memory management

Another critical thing for HPC is basic memory management speed. As there are several memory models available in CUDA it is also critical to understand all the implications of using each of them. We wrote a test that allocate and release 16 b, 10 MB and 100 MB blocks of memory in different models. Please note: we got quite a different results in this benchmark, so it makes sense to show them on charts with logarithmic scale. Here they go:

Device memory is obviously the fastest option in case you allocate big chunk of memory. And GTX 680 with PCIe-3 is our champion in device memory management. Teslas are slower than GeForces in all the tests. Virtualization seriosly affects Host Write Combined memory management. PCIe-3 is better than PCIe-2 which is also obvious.

Memory transfer speed

Another important characteristics of an accelerator is speed of data transfer from one memory model to other. We measured it by copying 100 MB blocks of data between Host and GPU memory in both directions using regular, page locked and write combined memory access models. Here's what we got:

Obviously, PCIe-3 configurations are much faster than PCIe2. Kepler devices (GTX 680 and K20) are faster than other. If you use Page Locked and Write Combined models it makes your transfer speed faster. Virtualization slightly affects regular memory transfer speed, and doesn't affect others at all. We also tested internal memory transfer speed (please note, we haven't multiplied it by 2 as NVIDIA does usually in their tests):

Tesla K20s are faster than GeForce, but difference is not so big. M2050 are almost two times slower then their succesors.

Device memory access speed

We also measured device memory access speed for each configuration we have. Here they go:

Alligned memory access is way faster than non-aligned (almost 10 times difference). Newer accelerators are better than older. Double precicion read/write is faster than single for all the configurations. Virtualization doesn't affect memory access speed at all.

Pinned memory access speed

Last metric we measured was pinned memory access speed when device interacts with host memory. Unfortunately we weren't able to run these tests on GTX 680 with PCIe-3 due to issue with big memory blocks allocation in Windows. 

New Tesla is faster then old one. PCIe-3 is obviously faster. Aligned access is almost ten times faster and if you read double precision floats your memory access speed is two times bigger than if you work with single precision floats. Virtualized environment is slower than non-virtualized.

Conclusions

All-in-all new Tesla K20 performs slightly faster than their predecessors. There is no revolution. There is evolution - we got better performance, new tools that make programmer's life easier. There also are several things that are not mentioned in this benchmark, like better support of virtualization and as a result cloud-readyness of K20. Some results were surprising. We expect better results of K20 in several months when new, optimized version of drivers will be available (NVIDIA always has some issues with new drivers just after release, but usually fix them after several updates).

You can find spreadsheet with complete results at Google Docs. Benchmark sources are available at our GitHub.

Tesla K20 benchmark results

Recently we've developed a set of synthetic tests to measure NVIDIA GPU performance. We ran it on several test environments:

• GTX 580 (PCIe-2, OS Windows, physical box)
• GTX 680 (PCIe-2, OS Windows, physical box)
• GTX 680 (PCIe-3, OS Windows, physical box)
• Tesla K20Xm (PCIe-3, ECC ON, OS Linux, NVIDIA test data center)
• Tesla K20Xm (PCIe-3, ECC OFF, OS Linux, NVIDIA test data center)
• Tesla M2050 (PCIe-2, ECC ON, OS Linux, Amazon EC2)

Please note, that next generation Tesla K20 is also included into our results (many thanks to NVIDIA for their early access program).
You can find results at Google Docs. Benchmark sources are available at our GitHub account.
Stay tuned, we're going to make some updates on this.

UPD: Detailed results with charts and some conclusions: http://www.elekslabs.com/2012/11/nvidia-tesla-k20-benchmark-facts.html

Windows Azure Backup Tools available on GitHub

Migrating applications to the cloud involves a big step in the way we deploy and maintain our software. While leveraging all the tasty features provided by cloud platforms, such as high availability and seamless scalability, a good IT professional also wants to make sure that the cloud version of the application is every bit as reliable and secure as the on-premises version.

From the operations team's point of view, there are numerous aspects of running a cloud application properly, typical of which are:

• Application data must be regularly backed up, the time it takes to restore the data must be as little as possible. Quick restore means less downtime, less downtime means happier customers.
• It is preferable for the cloud application to be portable, which means it can be moved back and forth between a cloud-hosted datacenter and your on-premises environment without any modifications to the source code.
• Maintentance tasks should be automated and include as few steps as possible to reduce the probability of human error.

Nowadays, public cloud vendors offer quite different functionality as regards application maintenance. While some of them concentrate on rich web-based management UI, others invest their efforts in building a powerful API to automate these tasks. The more experienced and mature vendors do both. With this in mind, you have to weigh your typical operation tasks against the management features provided by concrete cloud vendor.

Having had some experience with migrating on-premises applications to Windows Azure, we must admit that while the new Metro-style management portal is quite pleasant and easy to use, it does not yet provide some features commonly required by our IT pros. For example, automatically backing up Windows Azure SQL Databases and restoring them locally is possible, but involves quite a lot of manual steps. Things become a little more difficult when you encounter such tasks as backing up data from on-premises applications to cloud storage as well as restoring such backups later: if you use private blob containers, managing such blobs is quite tedious because of the lack of UI tools.

In order to help the operations staff with common tasks, we have developed a few automated command-line tools that utilize various Windows Azure APIs behind the scenes. The source code is released under MIT License and is available on GitHub.

1. Backup Windows Azure SQL Database to Blob Storage.

This tool allows you to perform an automated backup of your SQL Database and store the backup to your Windows Azure Storage account as a blob (in BACPAC format). Later, this backup can be used to restore the database on another cloud SQL Database server as well as an on-premises Microsoft SQL Server instance. Internally, this tool utilizes DAC web service endpoints hosted on Windows Azure datacenters. Note that for every location the URL of the web service is different.

Usage example:

AzureSqlDbBlobBackup.exe --dac-service-url https://by1prod-dacsvc.azure.com/DACWebService.svc --db-server-name abcdef1234.database.windows.net --db-name Northwind --db-username db-admin --db-password db-admin-secret --storage-account northwindstorage --storage-account-key s0e1c2r3e4t== --blob-container backups --blob-name NorthwindBackup.bacpac --append-timestamp

2. Archive local files or folders to Blob Storage.

This tool allows you to upload zip-compressed copies of your local data to Windows Azure Storage, which can be helpful if you frequently use cloud as a reliable off-site storage for your digital assets.

Usage example:

ZipToAzureBlob.exe --source-path E:\MyData --compression-level=9 --storage-account northwindstorage --storage-account-key s0e1c2r3e4t== --blob-container backup --blob-name MyDataBackup.zip --append-timestamp

3. Download files from (private) Windows Azure Blobs.

The purpose of this tool is quite straightforward: it enables you to download a blob from Windows Azure Blob Storage to your local filesystem, which works especially good when the blobs are stored in a private container, thus not so easily downloadable from the management portal. This tool, combined with the zip-archiving tool above, provides a pretty quick and easy solution for automating the process of data backup/restore that utilizes reliable cloud storage.

Usage example:
DownloadAzureBlob.exe --storage-account northwindstorage --storage-account-key s0e1c2r3e4t== --blob-container backup --blob-name MyDataBackup.zip

Storage Emulator notice

Since these tools are primarily intended to be used in a production environment, we did not currently add support for Windows Azure Storage emulator (UseDevelopmentStorage=true), although stay tuned for upcoming updates to our GitHub repository.

HTML5 Canvas: performance and optimization. Part 2: going deeper

Last time we were talking about JavaScript optimization, we used some basic optimization techniques to achieve better performance of Flood Fill algorithm on HTML5 Canvas. Today we’re going to go deeper.

We discussed results internally and came out with several low-level fixes. Here they go:

Minimize created objects count

First of all we thought about enormous number of objects we created during execution of our algorithm. As you probably remember, we’d applied fix named ‘Temp object creation’, when we tried to minimize number of performed arithmetic operations. It had negative effect on performance because of increased memory allocation and garbage collector overhead. So, the less objects you created – the better performance you get. It is not hard to notice that most of objects in our code are created here:
What if we don’t create new objects here at all? Let’s store in stack individual coordinates instead of creating wrapper objects. Sure, it makes code more complicated and unreadable, but performance is our main goal today. So, we came out with this:
Results follows:

Please note, we removed two bad fixes from the previous article. We’ve got nice results in all browsers, but in Safari results were really amazing: about 45% performance boost. If we remember bad “Temp object” fix from the previous article we notice that Safari was dramatically slower than other browsers after that fix, so such result is logical consequence of some issues Safari has with objects allocation and/or garbage collection.

Inline functions

Let’s go deeper. Most modern compilers do function inlining automatically or provide you with ability to mark function with some kind of inline attribute (think C’s inline keyword or AggressiveInliningAttribute from .NET 4.5). JavaScript don’t allow you to do that. Although function inlining might have dramatic performance effect in case you call function very often. We call isSameColor about 2 million times and setPixelColor about 700K times. Let’s try to inline them manually:
Again, it makes our code less readable and understandable, but we really want better performance, so we don’t care about code readability here.

Isn’t it amazing? Absolutely incredible results: we’ve got 70% boost on Firefox, 56% on IE and 36% on Safari. And what about Chrome? Surprisingly it is 9% slower. It seems that Google already implemented automatic function inlining in V8 optimizer and our manual fix is worse than theirs. Another interesting thing: with that fix Firefox is almost two times faster than Chrome, previous leader.

Optimize CPU cache utilization

There are one thing we never thought before in context of such a high-level language as JavaScript: CPU cache utilization. It is quite an interesting topic and it deserves separate article. As for now you can read more about it here, for example.
ImageData array has two dimensions that are packed into one dimensional array line by line. So, 3x3 pixel matrix with coordinates (x;y), is basically stored in memory like that:

Let’s look at the following lines in our code:
Let’s think about the order we access neighbor pixels if we are in (1;1):

So, we’re going left, then once again left, then right, then again right. According to best practices of cache utilization optimization it is better to access memory sequentially, because it minimizes a chance to have cache miss. So, what we need is something like that:

Let’s rewrite our dx,dy arrays:

Here is what we’ve got:

Well, the only browser reacted significantly was Chrome: we’ve got about 10% better performance with it. Other browsers were up to 3% faster which is in the area of statistical error. Anyway, this is interesting result that means we should pay attention even to such low-level optimizations when we write code on JavaScript – they are still important.

Fixing own bug – reorder if statements

Those of you who read inlined code carefully might already noticed that we actually did a mistake there:

We check pixel color and only then make sure that we don’t go outside array bounds. In statically typed language we’d got some kind of IndexOutOfBoundsException or Access Violation error here. But in JavaScript arrays are basically hash tables, so noting prevents you from accessing negative index here. But due to the cost of array element access operation it makes sense to check array bounds before checking colors:

Results are surprising:

Most of the browsers results were in the area of statistical error, but Chrome was more than two times faster and got its crown of fastest browser back in our benchmark! It is hard to tell what the reason of such a dramatic difference is. Maybe other browsers use instruction reordering and already applied that optimization by themselves, but it looks strange Chrome don’t do it. There also may be some hidden optimization heuristics in Chrome that help it understand that stack is used as simple array, not hash-table and it makes some significant optimization based on that fact.

Conclusions

Low-level optimizations matter even if you write your code in high-level language such as JavaScript. Code is still executed on same hardware as if you write it on C++.
Each JavaScript engine is different. You can have significant performance boost in one browser, but at the same time your fix may make your code slower in another browser. Test your code in all browsers your application must work with.
Keep the balance between code readability and performance considerations. Sometimes it makes sense to inline function even if it makes your code less readable. But always make sure that it brings you desired value: it doesn’t make sense to sacrifice code readability for 2 ms performance boost for code that is already fast enough.
Think about object allocation, memory usage and cache utilization, especially if you work with memory intensive algorithms such as Flood Fill.

You can find all the results with exact numbers at Google Spreadsheets: https://docs.google.com/open?id=0B1Umejl6sE1raW9iRkpDSXNyckU
You can check our demo code at GitHub: https://github.com/eleks/canvasPaint
You can play with app, deployed on S3: https://s3.amazonaws.com/rnd-demo/canvasPaint/index.html
Thanks to Yuriy Guts for proposed low-level fixes.
Stay tuned!