Google Glass Development without Glass

Here are the slides of my "Google Glass Development without Glass" presentation. As I promised, here is the follow-up blogpost with some code samples. If you haven't seen my presentation, I strongly suggest clicking through the slides!

0.Intro

Technologies emerge. Extremely. Wearable trend takes over the world. It seems like every week we start with articles about brand new devices, which would change our life! We've heard about lots of glasses, watches, wristbands, you name it. The most popular one is definitely Google Glass. Actually, it is Glass, who started this wearable hype!
You can just imagine, how all the wearbles can impact our daily and extreme activities. Check out, how we imagined that at eleks: https://glass.eleks.com/

From the other point of view, each new device brings new UX, SDKs and APIs. That's why we, developers, have to quickly adopt to the new techs. In this sense, Google Glass is a rather good device for developers. It runs the world-known Android OS. It just has an extremely strange UI and UX.

Google created a set of "best practices" of awesome Glassware. Actually, it can be applied to any wearble software:

• Design for device -- know & test your device and it's abilities. 
• Don't get in the way -- show the best data when users want it and be out of the way when they don't.
• Keep it timely -- platform is the most effective when in-the-moment and up-to-date.
• Avoid the unexpected -- surprising users with unexpected functionality is bad. On any platform.

Glass is different from the regular phone or tablet. The usual way of creating Android apps is through the Android SDK. Because of specific Glass's usage principles, Google introduced Mirror API. We'll dive into both of them later in the post.

1. Obtaining Glass

If you are among those 10000 people, who have Glass, you can skip this chapter. Unfortunately, that's just 0.000002% of the world's population. So, welcome Stranger to this amazing world of Glass hacking without Glass.

Hopefully, you have some Android device. Then, you can install Google Glass UI there (disclaimer: I've tried this with Nexus 7 tablet with 4.2.2 rom and it worked perfectly, but I can't guarantee it will work on other device). In short, Google Glass UI is just another Home screen provider for your Android. In order to set it up, follow these steps:

1. Go to: https://github.com/zhuowei/Xenologer
2. Download Glass APKs: https://github.com/zhuowei/Xenologer#install
3. Install them as you usually install APKs from third-parties
4. Follow steps at http://imgur.com/a/IBqFf to set up your "Nexus Glass"
5. Say "ok glass, take a picture" and share your "woohoo-reaction" with the world #throughglass

You've obtained Glass. Now its time to hack!
Let's dive into Glass development with the help of our friend Mr. Bond. James Bond.

Once upon a time, he was in London. Somehow, he's got his Google Glass and enjoys the experience on a daily basis.

2. Example #1: Mirror API

If you are eager to try Mirror API out and don't want to mess with quick-start projects, I have a shortcut for you!
Let's try inserting a simple card into Glass Timeline:

1. Go to: Google API Explorer --> Mirror API --> mirror.timeline.insert 
2. Authorise via OAuth 2.0. Use Glass scopes:
• https://www.googleapis.com/auth/glass.timeline
• https://www.googleapis.com/auth/glass.location
3. Populate request fields with values from the API.
4. Execute request!
5. Here's a link with some pre-populated data.

Now it's time to have even more fun! Let's try setting up demo-project.

1. Set up a new project in https://code.google.com/apis/console
2. Go to API Access, update project settings:
1. Add to Redirect URIs: 
• http://localhost:8080/oauth2callback
2. Add to JavaScript origins: 
• https://plusone.google.com
• http://localhost:8080
• https://mirror-api-playground.appspot.com (will be useful later) 
3. Clone https://github.com/googleglass/mirror-quickstart-java
4. Update file src/main/resources/oauth.properties with the values provided in Google API Access Console.
5. mvn jetty:run it.

If everything goes smooth, you'll be able to authorize and have fun with your Glass Timeline! If you want to have even more fun with Mirror API, especially with Subscriptions to Location and Notifications, you should deploy your war to the real webserver. Google sends these updates only to https secured web-sites. For development purposes you can use provided ssl-proxy. Just modify MirrorClient#insertSubscription method.

3. Example #2: GDK

Update: Google recently released GDK Sneak Peak. Right now, when you try creating apps with it and running it on Nexus Glass, you'll get [INSTALL_FAILED_MISSING_SHARED_LIBRARY]. We're trying to find the way, to avoid that. As of now, there's no support for the official GDK Sneak Peak (auth, live cards, etc).

Google Glass is just another Android device with 640x360 screen (hello, screen fragmentation), Android 4.0.4, API level 15. Maybe, in the next releases of Glass, this will change. Now, when you want to port your existing Android app to Google Glass, you should just:

1. Build your APK.
2. Install it.
3. Run via adb.
4. Understand, that you need to update touch gestures support.
5. Understand, that your UI is hardly usable.
6. Understand, that your awesome Google Maps stopped working.
7. So, double-check your application on real device! Everything can change in the near future!

It's a good idea to check out Google Glass Quick Start by Google. It explains a lot about Timeline, UI, Gestures and current limitations of the GDK.

In order to support touch gestures of Google Glass, have a look at this article: Touch Gestures. In short, here's the table with events corresponding to keycodes:

Tap	KEYCODE_DPAD_CENTER
Swipe right	KEYCODE_TAB
Swipe left	KEYCODE_TAB + isShiftPressed()
Swipe down	KEYCODE_BACK
Camera	KEYCODE_CAMERA

4. Example #3: GDK + Timeline API

Update: Google released GDKvXE12 update. It has Static Card support, but it is still limited and lacks menus, html templates, etc. The code mentioned below follows hacky way and may not work on the latest version of Glass.

So far, we've tried the official Mirror API way and the hacky GDK road. Now its time to combine both approaches. Let's insert a card into Timeline from withing your app!

I've  set up a sample project at github for you: https://github.com/pif/ukrbash-for-glass. After you fork/clone it, you can see glasslib.jar inside libs folder. This library would probably become the aforementioned GDK. It provides everything you need to know about Timeline UI.

All the magic is done via these steps:

1. Initialise Timeline, get Timeline contentresolver. Look at:
   com.andrusiv.glass.bash.GlassService#onStartCommand(Intent, int, int).
2. Create MenuItems for your card.
3. Create TimelineItem.
4. Insert it into ContentResolver.

Look through the classes & methods provided inside glasslib.jar. I hope, you'll find loads of interesting information!

5. Outro

Thanks everyone, for reading all the way down here. I hope now you have your pseudo-Glass and know how to develop for it. Share your experience in the comments and on github! Ok guys, have fun #throughglass!

https://plus.google.com/+GoogleGlass/photos

Writing a ReSharper plugin. Quick fixes.

Introduction.

It is almost impossible to find a .NET developer who does not use ReSharper. The reason is obvious - ReSharper is always step ahead of Visual Studio in its refactoring, auto-completion and code-generation features. But not many know that since the 5th version ReSharper has a set of extensions points that allow developers to create their own productivity tools. 

In this series of articles we will deal with the creation of useful ReSharper plugin for code that uses .NET Reflection API. 

Starting up.

The first step you need to do to start ReSharper plugin development is to install ReSharper SDK. Installer adds project templates to Visual Studio and samples that cover most extensions points available. Samples may be found in “Program Files (x86)\JetBrains\ReSharper\v8.0\SDK\Samples\” folder. Although ReSharper extensibility points, base classes and interfaces lack of documentation, samples source code is well written and easy to understand so it will definitely give you a clue.

Implementing a ReSharper QuickFix.

QuickFix is a set of executable actions that modify a part of code where cursor is located. It is always associated with some kind of highlighting (custom highlightings will be described in the subsequent article). There are pretty a lot of quick fixes available in ReSharper and they are frequently used by developers (e.g. Make property public or internal, check for null some reference, optimize imports, etc.).

IQuickFix interface is quite simple and self-descriptive.

public interface IQuickFix

{

    IEnumerable<IntentionAction> CreateBulbItems();

    bool IsAvailable([NotNull] IUserDataHolder cache);

}

CreateBulbItems is responsible for reporting menu items that will be available for execution. IsAvailable used to report whether quick fix is available in current context. In most cases it is enough to inherit from QuickFixBase class. QuickFix needs to have a public constructor that accepts any type that implements IHighlighting interface as an argument. The last thing that needs to be done to make things working is to mark implemented class with QuickFixAttribute. 

[QuickFix]

public class SampleQuickFix : QuickFixBase

{

        public SampleQuickFix(AccessRightsError error)

        {

        }

        protected override Action<ITextControl> ExecutePsiTransaction(ISolution solution, IProgressIndicator progress)

        {

            return null;

        }

        public override string Text

        {

            get

            {

                return "Sample quick fix";

            }

        }

        public override bool IsAvailable(IUserDataHolder cache)

        {

            return true;

        }

}

Quick fixes that will be described in current article use highlightings that are already reported by ReSharper problem analyzers:

• AccessRightsError – is reported when you are trying to access internal or private property in scope where it is not allowed.
• NotResolvedError – for cases where reference expression couldn’t be resolved.

"Use Reflection" QuickFix.

It is a known fact that you may violate encapsulation principle by using Reflection to access private/internal fields, properties and methods. When using Reflection in such case you should be aware that you may leave an object in inconsistent state and it may behave as it was not designed to. So be aware!

As for me I can hardly remember any project where we didn’t need to access some internal methods and properties. Just as a quick example from previous project – we needed to modify default WPF DataGrid behavior on column header click so that instead of sorting it selects all cells belonging to that column. First approach with using public APIs failed as it was too slow. After that we have found that DataGrid uses internal methods for selection of large regions. And here Reflection came to rescue.

The quick fix implemented helps to generate Reflection code to access specific class member. You might remember that it is quite easy to miss some required BindingFlags, and you will see that your code doesn’t work only during program execution. 

Before diving into implementation details you may watch the video how “Use Reflection” quick fix works.

Although implementation code is a bit verbose for first example, it is easy to understand what happens there, especially if you have basic understanding of what AST is (ReSharper works with code representation in form of PSI, changes to the tree are mirrored in the editor at once).

public UseReflectionQuickFix(AccessRightsError error)

{

     _error = error;

     _declaredElement = error.Reference.CurrentResolveResult.DeclaredElement;

     _languageForPresentation = error.Reference.GetTreeNode().Language;

}

Public constructor accepts AccessRightsError highlighting with reference node, representing access to member that violates access rights. 

The main implementation resides in ExecutePsiTransaction method. The implementation handles different corner cases: 

• Reflection runtime invocation returned value is always object. I wanted the quick fix to generate correct C# code thus I have added casting support to property type value.
• Assignment operation treated separately.
• When invoking a method - array of arguments needs to passed to InvokeMember method.

protected override Action<ITextControl> ExecutePsiTransaction(ISolution solution, IProgressIndicator progress)

{

    var accessExpression = _error.Reference.GetTreeNode() as IExpression;

    var replacementNode = accessExpression;           

    if (replacementNode == null)

        return null;

    var modifiers = _declaredElement as IModifiersOwner;

    if (modifiers == null)

        return null;

    bool isAssign = replacementNode.Parent is IAssignmentExpression;

    bool needsCasting = !isAssign && !(replacementNode.Parent is IExpressionStatement)

        && !_declaredElement.Type().IsVoid() && !_declaredElement.Type().IsObject();

    if (replacementNode.Parent is IInvocationExpression || replacementNode.Parent is IAssignmentExpression)

    {

        replacementNode = (IExpression)replacementNode.Parent;

    }

    CSharpElementFactory factory = CSharpElementFactory.GetInstance(replacementNode, applyCodeFormatter:true);

    AddSystemReflectionNamespace(factory);

           

    string flags = "BindingFlags.NonPublic";

    if (modifiers.IsStatic)

    {

        flags += "| BindingFlags.Static";

    }

    else

    {

        flags += "| BindingFlags.Instance";

    }

    flags += "| " + GetInvokeMemberBindingFlag(_declaredElement, isAssign);

    IExpression instanceExpression = modifiers.IsStatic ? factory.CreateExpression("null") : ((IReferenceExpression)accessExpression).QualifierExpression;

    IExpression argsExpression = factory.CreateExpression("null");

    if (isAssign)

    {

        argsExpression = factory.CreateExpression("new object[] { $0 }",

            ((IAssignmentExpression) replacementNode).Source);

    }

    if (replacementNode is IInvocationExpression)

    {

        var invocationExpression = (IInvocationExpression)replacementNode;

        if (invocationExpression.Arguments.Count != 0)

        {

                   

            argsExpression = CreateArrayCreationExpression(

                TypeFactory.CreateTypeByCLRName(

                "System.Object",

                accessExpression.GetPsiModule(),

                accessExpression.GetResolveContext()), factory);

            var arrayCreationExpression = argsExpression as IArrayCreationExpression;

            foreach (var arg in invocationExpression.ArgumentsEnumerable)

            {

                var initiallizer = factory.CreateVariableInitializer((ICSharpExpression) arg.Expression);

                arrayCreationExpression.ArrayInitializer.AddElementInitializerBefore(initiallizer, null);

            }

        }

    }

    var reflectionExpression = factory.CreateExpression("typeof($0).InvokeMember(\"$1\", $2, null, $3, $4)",

        ((IClrDeclaredElement)_declaredElement).GetContainingType(),

        _declaredElement.ShortName,

        flags,

        instanceExpression,

        argsExpression);

    if (needsCasting)

    {

        reflectionExpression = factory.CreateExpression("($0)$1",

            _declaredElement.Type(),

            reflectionExpression);

    }

    replacementNode.ReplaceBy(reflectionExpression);

    return null;

}

The following code is used to import “System.Reflection” namespace if it is not present in using directives.

private void AddSystemReflectionNamespace(CSharpElementFactory factory)

{

    var importScope = CSharpReferenceBindingUtil.GetImportScope(_error.Reference);

    var reflectionNamespace = GetReflectionNamespace(factory);

    if (!UsingUtil.CheckAlreadyImported(importScope, reflectionNamespace))

    {

        UsingUtil.AddImportTo(importScope, reflectionNamespace);

    }

}

private static INamespace GetReflectionNamespace(CSharpElementFactory factory)

{

    var usingDirective = factory.CreateUsingDirective("System.Reflection");

    var reference = usingDirective.ImportedSymbolName;

    var reflectionNamespace = reference.Reference.Resolve().DeclaredElement as INamespace;

    return reflectionNamespace;

}

For PSI tree creation CSharpElementFactory class is used. It provides method for creating different types of  AST nodes and immediate corresponding code formatting capabilities. Notice that for formatting it uses ‘$0’ placeholder instead of usual to .NET developers ‘{0}’.Element formatting works with ability to pass other PSI tree nodes.

"Did you mean?" QuickFix.

I will not dive deep into implementation details as I’m quite sure that you have got enough information already. This quick fix works with NotResolvedError highlighting and allows selecting type members that have most similar name to not resolved reference. As it was needed to provide multiple menu items implemented quick fix inherited directly from IQuickFix interface. Implementation uses Levenshtein distance to get most similar names and a part of ReSharper auto-completion API to get available symbols for specified reference. 

Debugging and testing.

If you have created plugin project from available template it will start another Visual Studio instance in debug mode without any actions required. Basically the project has set “Start Action” set to “Start External Program” and command line arguments “/ReSharper.Plugin ReReflection.dll /ReSharper.Internal”. Specified command line arguments makes ReSharper to load created .dll as a plugin. “/ReSharper.Internal” switch enables access to many internal menus that helps to debug your plugin, analyze PSI tree, etc.

It is hard to make your plugin stable without continuous integration. Luckily ReSharper provides base classes for most kinds of extension points available. For reference you may see tests that were implemented for described quick fixes.

Conclusion.

The first article in the series describes basics of ReSharper plugin creation. Of course the code is far from production quality and during debug mode you may encounter unhandled exceptions reported by ReSharper. In the next article I will describe how custom highlighting are implemented and what ElementProblemAnalyzer is.

The code of plugin is available on GitHub.

Useful links.

1. http://blogs.jetbrains.com/dotnet/tag/plugins/ - Official JetBrains blog
2. http://hadihariri.com/2010/01/12/writing-plug-ins-for-resharper-part-1-of-undefined/
3. http://mediocresoft.com/things/what-i-learned-about-writing-resharper-plugins
4. https://code.google.com/p/agentjohnsonplugin/source/browse/#svn%2Ftrunk%2FAgentJohnson%2FCodeCleanup%2FRules – open source plugin