Release: Raiku 0.6.0 by Gideon de Kok

I just updated Raiku to the 0.6.0 release, build on Akka 2.2 final and supporting the new functionality available through Riak 1.4+. This means that Raiku 0.6.0 and later won't directly support older versions than 1.4.0 of Riak, futher instructions can be found in the README.

The new version can be downloaded through SBT , is available through the GitHub repository and sports the following bugfixes / functionality

New functionality 

- Support for pagination in ranged 2i requests is added;

- All 2i queries can now be streamed through Play powered Enumerators;

- Ranged query support for binary indexes is added;

- Full support for Riak powered Counters is added with a easy-to-use API;

Bugfixes 

- A issue was fixed which occurred when multiple MR queries where run on the same actors;

- MR queries aren't restricted anymore to 2 or more phases (single phased queries should work now) 

Architectural choices in mobile development by Gideon de Kok

This is a repost (from Tumblr) of a blogpost I wrote 23th May 2011

Last friday at PyGrunn (a local Python conference here in Groningen), I did a short presentation about the architectural choices you should and could make during development of mobile applications.

Some of the people in the crowd asked me if I could summarize them in a post for future reference, so here we go:

Should we treat mobile clients different then normal “wired” clients?

Yes… With mobile clients we have five key points which are different opposed to a “normal” web client:

  1. Usability;
  2. Performance;
  3. Connectivity;
  4. Data Access;
  5. Security. 

Usability

When we look at mobile clients, we have two kinds:

  • Native applications
  • Mobile Web applications

Users of the second kind of applications (mobile web) will expect the same functionality as on traditional web clients: you load, you post, you wait and you load. If the connectivity fails, you get an error and you reload and send again. All in all a “traditional” synchronous approach of sending and receiving.

The first kind of applications, is completely different; the native applications.

While using native applications, users generally just want to get things done. They want to send a mail, tweet or photo without any pop-ups nagging about lesser connectivity. They just want to press send, lock the phone and go on with their normal flow. 

Because multiple tasks can be done on modern phones, the user generally doesn’t wait until the message is send and doesn’t want any error messages of a mail app to pop-up while playing Angry Birds.

This creates a more asynchronous approach in the behavior of the application: the users doesn’t wait to receive status messages and often queues multiple messages at a time for the server to handle.

Performance

In overall, in mobile architectures, performance best practices are generally the same as opposed to traditional web(service) architectures. Your architecture should scale vertically and horizontally when needed to create optimal response times.

There is one big difference between a normal web-service and one suited for mobile applications:

Make your server work.

Don’t do number crunching or large data calculations on your clients. As opposed to normal clients, mobile clients have limited CPU power and memory. Next to that, heavy calculations will drain the battery quick and will leave your users without power. 

Because network activity is costly on battery and CPU, when you can:

Push from your servers

Most mobile devices already have a service running, which enables you to push information or tasks to the client. This saves battery and resources because the device only has to manage one outgoing “socket”.

Connectivity

For usability, connectivity and performance sake:

Cache

Store data, retrieved from you servers, locally for offline access or slow access.

Requests are costly, so it is better to pull as much as possible (text) data in to serve out of the cache then to pull every item from the server when the user needs it.

Refresh that data periodically or on user requests. Try to postpone periodical refreshes until the user has a better connection.

Expire and delete not-used or old information both on the client as from the stream a client pulls in… Even if the user only follows 4 users, tweets from 4 months ago generally aren’t that relevant anymore.

Queue

Queue and combine requests as much as possible, send less important non-realtime requests (like a “like” on Facebook”) together with high priority stuff. 

Concatenating messages will reduce the requests you send to a server and will limit the problems you have with latency.

If a connection fails when sending, wait… Reconnecting on a bad wire is costly (and will generally fail anyway). Storing the queue and waiting for a better connection reduces data and battery usage. 

Minimize connectivity

If you don’t use it or need it, don’t push or pull it… 

Only send information to the client which he actually uses. If you don’t use the tags of a news post in your application, strip them from the serialized objects.

Make you payloads tiny, use JSON over XML and GZIP all possible content! 

Do payload adaption

Adapt the content you send to the connectivity of the user. When loading news items for instance, only send the full-fleged article with 4 photos and video to users with a WiFi connection. Send items with 1 image to 3G users and send text-only articles to users with GRPS connections.

Doing this, don’t get usability in the way… Support it: let the content load as fast as possible and let the users decide to wait on rich content.

Data Access

Feels like 2005 saying this, but: No more direct database access

Do what everyone should do (and hopefully is doing right now), create a API!

Having a API gives you the possibility to:

  • Switch server architecture (OS, Database etc.) without updating the app;
  • To do number crunching and caching on you servers;
  • To distrust your users.

Never trust API input

Your application isn’t the only one which can talk with your servers. You should perform simple validation on your client for usability and performance. But should always check server side, for validation, authorization and logic.

Security

When it comes to man in the middle attack, clients have never been as vulnerable as mobile clients. With user hopping from WiFi spot to WiFi spot, data is easily sniffed and misused.

So, encrypt your data streams… Use HTTPS when possible and encrypt secret stuff with techniques like private => public key encryption.

Store your secret stuff well, client-side caching makes your data reside on both servers as clients. But don’t safe too secret stuff on clients, reverse engineering of your app probably already finished yesterday!

Buri: A easy to use LevelDB powered key-value storage with 2i for Objective-C by Gideon de Kok

I'm simply in love with the LevelDB implementation in Riak. 

On iOS, I've used CoreData to many times, hacked around with Tokyo Cabinet in several weekends. But none of the solutions delivered a pragmatic and easy approach to object storage in iOS.

So with some help from some other projects, I started to implement my own storage framework for iOS: Buri

The current code should be treated as a very pre-pre-release, but will use it as my main storage client for multiple applications. 

Clone or fork at: https://github.com/gideondk/Buri.git

AngularJS Tutorial implemented in LiveScript by Gideon de Kok

Writing AngularJS code in LiveScript can be a bit daunting, most examples are written in Javascript or CoffeeScript at best. 

To help the LiveScript curious, I pushed the implemented tutorial in LiveScript at: https://github.com/gideondk/phonecat-livescript

Simply was to lazy to implement the tests using testacular, but will be adding those soon :-) 

Choosing a 'new' web framework by Gideon de Kok

I've been a Cappuccino enthousiast for a very long time, created three "serious" applications with the framework and did my own share of forking, updating and customisation on the framework. As a Objective-C programmer, the framework really lives up to it's hype and delivers a easy-to-use web framework sporting a impressive set of UI elements in a environment every iOS / Cocoa developer should be familiar with. 

Transient

As a all-round developer / architect, it's eventually not that good to use on large projects. The chances of hiring (good) iOS developers are still pretty thin, at least a lot more difficult then finding equally good front-end (as in Javascript) developers.

Besides the (common) recruitment argument, the platform has some real problems which discourage further development with the framework (at least for me):

Performance

Where Objective-C and Cocoa make sense in a compiled environment and enable you to write high-performant, low-level code. The Objective-J and Cappuccino stack simply take away a lot of performance being interpreted on run-time. This doesn't only render the framework useless on even the most beefy tablets or phones, but also stagnates on performance if you try to build something big.

Most Objective-J classes are wrappers around simple Javascript types. Using those classes in combination with large datasets will result in more actions being called and more memory being used in the process. In the end I ended up using the normal Javascript types, which render most parts of the framework obsolete in the scenarios I encountered.

Theming

The framework really shines when you don't want to worry about style, layout or even the term CSS. Because the framework is still focusing on support on older (IE) browsers, most of the theming is done using images. This makes customisation in terms of look & feel more than uncomfortable.

We live in 2012 and things like text-shadows, rounded-borders, box-shadows and animations should be done using hardware-accelerated methods. If you want to use CSS3 functionality, you basically have to subclass most parts of the UI framework and add the styles using DOM mutations (which is notoriously, horribly slow when you have a lot of elements to update).

Or you simply have to write your own lay-out rendering engine, as I did... Canvas helps, but also doesn't give the performance you need when you want to animate large amounts of objects on screen. 

Backing

The Objective-J and Cappuccino stack was originally build by the 280 North team, which was focussing on creating a XCode / Interface Builder like experience for the web using Atlas. After Motorola took over 280 North, development on Atlas stopped and eventually only fueled the development of Motorola's Montage

Some of the frameworks largest advocates ended up in a startup called Stripe. Who chose not to use Cappuccino, but a combination of Backbone.JS, CoffeeScript and Eco to develop their dashboard application (which would be a very good fit for Cappuccino). The community is still very alive and harbours a lot of talent, but is lacking a good (funded) roadmap.

Now what?

I'm still much in love with Cappuccino, Objective-J and all the Cocoa-like-goodness it brings. And  will still be developing smaller applications with my custom Ristretto Framework (which will eventually be open-sourced). 

I will, because of the performance issues, won't use the platform on larger projects anymore. I've already started to rewrite portions of existing code from Objective-J to LiveScript and am currently experimenting with a whole bunch of MVC frameworks to handle the rest. 

At this moment, I've fallen in love with both Ember.js as AngularJS. And trying to build something more complex then a to-do application with both ;-) I will soon post a follow-up on this post with the pros and cons of both frameworks and which is going to shape my future of front-end web development.

Additions to the Scaliak Driver by Gideon de Kok

Last april, I wrote a post about some additions I made to the Scaliak Driver originally created by StackMob: https://github.com/gideondk/scaliak

In the past months the fork has grown over time and I’ve added some new features:

Code should still be considered draft and will be refactored in the upcoming month, nevertheless most additions are already being used in production.

Protocol Buffers

For extra performance, I’ve added a support for Protocol Buffers next to the provided HTTP client. Because the Protocol Buffer implementation in Riak is still missing out on functionality compared to the HTTP client. The HTTP client is still being used to get and set properties on buckets. 

Therefor, when initialising this client, you have to pass both the PB as HTTP ports:

https://gist.github.com/3113188

Client Pools

Client pools have been added to the client for further performance and stability, they are created in a similar fashion as the Protocol Buffer implementation and are used like the “normal” client implementation:

https://gist.github.com/3113203

The ScaliakBucket is modified to use the connection pool when possible, which means all  operations are automatically done within a pool when using this implementation.

FileObjectHelper

Has been added, which implements a simple interface for storing and fetching “binary” data in and out Riak.