Thursday, April 5, 2012

Node.js on Mac OS X - 30 minute guide

Today I decided to give node.js a try on Mac OS X Lion. Having no real hands-on experience in node.js, and wanting to get something end-to-end up and running fast, so I went off in search for a good hands on tutorial.

For those of you who don't know what node.js is : "Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices."

I stumbled upon a great nodejs intro on github by indexzero. that gives you a nice end-to-end introduction to nodejs and CouchDB. The sample features a REST based api to perform CRUD operations on bookmarks. It uses CouchDB as a datastore.

The goal of this post is to do the required setup in order to get the sample up and running under 30 minutes. I decided to give it a try and started by installing both node.js and CouchDB on my Mac OS X Lion. By the end of this tutorial, you'll have node.js and CouchDB installed, and will be able to run the REST based API on your OS X installation.

This article assumes you have Git installed on your machine. If you don't have Git installed, checkout the Set Up Git on Mac guide.

Installing node.js and npm

The easiest way to download and install node.js from the the nodejs download section and pickup the Macintosh installer. It will install both node and the node package manager (npm).

After the install, you'll have access to both the node and npm command.

Installing CouchDB

As the node js tutorial uses CouchDB to store objects, we'll need to install CouchDB.

Installing CouchDB is a bit more effort in the sense that we'll download the sources and compile them. Before we can do that, we first need to install Homebrew by executing the following commands :

git clone https://github.com/mxcl/homebrew.git
cd homebrew/bin
brew install autoconf automake libtool
brew install couchdb
Important note ! : There seems to be an issue with the CouchDB recipe introduced a couple of days ago that will prevent you from installing CouchDB. In order to fix this, you'll need to manually edit ~/couch/homebrew/Library/Formula/couchdb.rb

Change

require 'formula'

class Couchdb < Formula
  url 'http://www.apache.org/dyn/closer.cgi?path=couchdb/source/1.1.1/apache-couchdb-1.1.1.tar.gz'
  homepage "http://couchdb.apache.org/"
  md5 'cd126219b9cb69a4c521abd6960807a6'

into this (notice how the "source folder" needs to be removed.

require 'formula'

class Couchdb < Formula
  url 'http://www.apache.org/dyn/closer.cgi?path=couchdb/1.1.1/apache-couchdb-1.1.1.tar.gz'
  homepage "http://couchdb.apache.org/"
  md5 'cd126219b9cb69a4c521abd6960807a6'
If the installation hangs, you can continue CTRL-C the install and try it again by executing
./brew install -v couchdb
More info on the process can be found in "Installing CouchDB on OSX".

After the CouchDB compile is complete, you can start it by executing "./couchdb". You can verify that CouchDB is running by opening a browser and navigating to http://127.0.0.1:5984/_utils.

Downloading the tutorial

Now that we have everything setup, we can continue with the node js intro sample.

We start by checking out the source.

git clone https://github.com/indexzero/nodejs-intro.git 

Creating the CouchDB database

Before we can start the tutorial, we're going to have to create a CouchDB database/ We're going to be using the command line (make sure CouchDB is started) :
$ curl -X PUT http://127.0.0.1:5984/pinpoint-dev10
{"ok":true}

You can visit the CouchDB Futon http based console on http://127.0.0.1:5984/_utils. You should see your newly created database in the console.

There's an excellent CouchDB guide here.

Starting the tutorial

The node js sample is constructed in a modular way. The lib folder contains several modules that can be bootstrapped using the server script in the bin folder.

For example, to start the CouchDB tutorial, all you need to do is execute the following command from the bin folder

./server -t 02couchdb -s

The -t flag allows you to specify a module from the lib folder, the -s flag will setup the pinpoint-dev database we created earlier.

The sys - util change

Depending on the version of Node that you're using you might see the following error or warning :

$ node -v
v0.7.7-pre

$ ./server -t 02couchdb -s

node.js:247
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: The "sys" module is now called "util".
    at sys.js:1:69
    at NativeModule.compile (node.js:572:5)
    at Function.require (node.js:540:18)
    at Function._load (module.js:297:25)
    at Module.require (module.js:357:17)
    at require (module.js:373:17)
    at Object. (/home/ubuntu/nodejs-intro/bin/server:3:11)
    at Module._compile (module.js:444:26)
    at Object..js (module.js:462:10)
    at Module.load (module.js:351:32)
In order to get rid of the error, you'll need to replace all calls to `require("sys")` with `require("util")` in all the .js files you're using in your project (and its dependent modules).

Node v0.6.14 doesn't throw an error, but provides a warning that the package has been changed.

$ node -v
v0.6.14

$ ./server -t 02couchdb -s
The "sys" module is now called "util". It should have a similar interface.
Pinpoint demo server listening for 02couchdb on http://127.0.0.1:8000

Running the tutorial

When you run the tutorial, you're going to get some erros

$ ./server 02couchdb
The "sys" module is now called "util". It should have a similar interface.

node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: Cannot find module 'optimist'
    at Function._resolveFilename (module.js:332:11)
    at Function._load (module.js:279:25)
    at Module.require (module.js:354:17)
    at require (module.js:370:17)
    at Object. (/Users/ddewaele/Projects/Node/nodejs-intro/bin/server:5:12)
    at Module._compile (module.js:441:26)
    at Object..js (module.js:459:10)
    at Module.load (module.js:348:31)
    at Function._load (module.js:308:12)
    at Array.0 (module.js:479:10)
The tutorial has several dependencies that we'll need to download using the Node Package Manager (npm). The npm command comes with the nodejs installation on OS X.

Installing node packages

Node packages (dependencies) are installed using the npm like this :

$ npm install optimist
npm http GET https://registry.npmjs.org/optimist
npm http 200 https://registry.npmjs.org/optimist
npm http GET https://registry.npmjs.org/optimist/-/optimist-0.2.8.tgz
npm http 200 https://registry.npmjs.org/optimist/-/optimist-0.2.8.tgz
npm http GET https://registry.npmjs.org/wordwrap
npm http 200 https://registry.npmjs.org/wordwrap
npm http GET https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.2.tgz
npm http 200 https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.2.tgz
optimist@0.2.8 ../node_modules/optimist 
└── wordwrap@0.0.2

The packages will be installed in a node_modules folder in the root folder of our tutorial :

$ ls -l ../node_modules/
total 0
drwxr-xr-x  10 ddewaele  staff  340 Apr  1 18:54 optimist

the node js intro requires the following modules to be installed :

npm install winston
npm install cradle
npm install journey
npm install optimist

Running the tutorial

From the bin folder, start the tutorial by executing the following command :

$ ./server -t 02couchdb -s
The "sys" module is now called "util". It should have a similar interface.
Pinpoint demo server listening for 02couchdb on http://127.0.0.1:8000

Going to http://127.0.0.1:8000/bookmarks in a browser should return the following response :

{"bookmarks":[]}
This means our service is up and running. In order to add some data in CouchDB, we're going to be using http-console to access our REST service and send some data over the wire.

Installing http-console

A great tool to help you debug your services is something called http-console. You can install http-console through the node package manager. This time, we'll install the http-console package globally, making it available system-wide. We do this by using the -g (global) switch.

sudo npm install -g http-console

You'l notice that the http-console executable is now available on the command-line. Unfortunately, when starting http-console you'll get the following error.

$ http-console 


node.js:201
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: require.paths is removed. Use node_modules folders, or the NODE_PATH environment variable instead.
    at Function. (module.js:378:11)
    at Object. (/usr/local/lib/node_modules/http-console/bin/http-console:6:8)
    at Module._compile (module.js:441:26)
    at Object..js (module.js:459:10)
    at Module.load (module.js:348:31)
    at Function._load (module.js:308:12)
    at Array.0 (module.js:479:10)
    at EventEmitter._tickCallback (node.js:192:40)

You can fix this by editing the /usr/local/lib/node_modules/http-console/bin/http-console file, and removing the following line :

require.paths.unshift(path.join(__dirname, '..', 'lib'));

After having removed the line above, http console should start. When providing no arguments, it will connect to http://localhost:8080 by default. If you want to connect to another server/port, just specify it as the first argument.

In order to connect to the server provided by the tutorial, we'll start the http-console like this (notice how we use the \json command to set the correct content-type):

$ http-console http://127.0.0.1:8000
The "sys" module is now called "util". It should have a similar interface.
> http-console 0.6.1
> Welcome, enter .help if you're lost.
> Connecting to 127.0.0.1 on port 8000.

http://127.0.0.1:8000/> \json
http://127.0.0.1:8000/> 

Accessing the REST service

Inside the http-console, executing a GET request is as simple as typing GET /bookmarks. You should get the same response as the one you saw in your browser earlier :

http://127.0.0.1:8000/> GET /bookmarks
HTTP/1.1 200 OK
Date: Sun, 01 Apr 2012 17:23:27 GMT
Server: journey/0.4.0
Content-Type: application/json;charset=utf-8
Content-Length: 16
Connection: keep-alive

{
    bookmarks: []
}

You can also execute POST commands like this by providing a JSON snippet.

http://127.0.0.1:8000/> POST /bookmarks
... { "url": "http://nodejs.org" }
HTTP/1.1 200 OK
Date: Thu, 05 Apr 2012 11:45:55 GMT
Server: journey/0.4.0
Content-Type: application/json;charset=utf-8
Content-Length: 91
Connection: keep-alive

{
    bookmark: {
        _id: 'WD-G-1',
        resource: 'Bookmark',
        url: 'http://nodejs.org'
    }
}

When executing the GET request again, as you can see, our server responds with the bookmark that has now been inserted in CouchDB.

http://127.0.0.1:8000/> GET /bookmarks
HTTP/1.1 200 OK
Date: Sun, 01 Apr 2012 17:23:27 GMT
Server: journey/0.4.0
Content-Type: application/json;charset=utf-8
Content-Length: 16
Connection: keep-alive

{
    bookmarks: [
        {
            _rev: '1-cfced13a45a068e95daa04beff562360',
            _id: 'WD-G-1',
            resource: 'Bookmark',
            url: 'http://nodejs.org'
        }
    ]
}

References

Monday, September 5, 2011

Android Market Horror Story

As a registered Android developer with a published application on the Android Market, I'm becoming more amazed each day by how bad the Android Market team is supporting their developers.

The relationship between developers and the Android Market has never been good, caused to a large degree by the many bugs and issues developers are facing on that Android Market.

The reason for me writing this post was something that happened a couple of days ago on the Android Market. Have a look at the video below to see what happens when users try to search for my application :


A couple of days before, a fellow developer contacted me that his sales dropped to almost zero since the new Android Market was released. When I went on the Market to search for his app, I noticed the following:



It's not only end users who are facing serious issues in finding, downloading and purchasing applications, but developers are having a very difficult time in getting any kind of support from Google for the many issues they are facing.

One should not forget that Google is taking 30% of all sales. Something that they are entitled to, and something I don't have any issue with whatsoever, but on the other hand, developers should expect some level of quality and support coming from the Android Market when working in such a model.

A couple of weeks ago, Google has decided to close the Market Technical Forum for developers, making it an end-user forum only. To quote Google, "Because of the nature of developer issues, we feel that one-on-one support is best for the types of threads that have historically been posted to this forum by app developers."

The Android Market forum was a place where developers could interact with each other on market specific issues, however, it seemed Google noticed that the forum was full of frustrated developers, making the Market look bad. Every once in a while, external sites like The Register started picking up stories from the forum, like when developers were not getting paid for a part of their app sales.

Every once in awhile a Google employee posted something on the forum, mostly to state that they are looking into the issue.No due dates, no updates, effectively leaving developers all alone to rant about the various issues they are having.

A forum might indeed not be the best way to deal with issues, but obviously a one-on-one support model also does not work in its current form. Either their support team is heavily under-staffed, or issues are given very low priority. I have the impression that Google is trying to hide these issues from the public instead of actually put effort in resolving them properly.

Like me, many developers have logged tickets via email and don't get any response beyond "look at the known issues page", or worse, no response at all. This seems very strange for a service that is taking 30% of your profits. I wonder what kind of support big companies are getting when it comes to getting their issues resolved.

Clearly, there is a need for support giving the many (sometimes serious) issues users and developers are facing with the Android Market. Just have a look at the known issues pages for both users and developers.
The issues that stand out for me, that have been open for a long time, and are hurting independent developers are listed below.

It's about time Google starts taking the Android Market seriously, but putting in place a decent support model, staffing the team appropriately and really start looking into resolving these issues. If not, I'm convinced that many developers will steer away from Android as a development platform.

Saturday, August 6, 2011

Google Latitude API and OAuth 2.0

Introduction

In this article, I'm going to discuss OAuth 2.0 and the Latitude API. A couple of days ago, OAuth 2.0 support was announced for the Latitude API, and I figured it might be a good idea to take a more in-depth look on how OAuth 2.0 can be used with the Latitude API and how the
Google APIs Client Library for Java implemented the Oauth 2.0 spec.

The article is accompanied by a sample application on Github. It's heavily inspired by existing samples for Google APIs Client Library for Java. In order to run the sample, clone the Git repo, import the project in Eclipse, fill in the Oauth constants (see subsequent section) and run the LatitudeSample class. In order to resolve the project dependencies, it's advised to use the Maven2 plugin for Eclipse.

The goal here is to start executing requests on the Latitude API, secured via OAuth 2.0. For this, we'll use a simple command line application. For the same flow using the Latitude API on the Android platform, checkout Oauth 2.0 flow in Android

Monday, July 18, 2011

Accessing the Google Public Location Badge programmatically

Introduction

The Google Public Location Badge provides a convenient way to share your Latitude location with others. The Google Latitude apps page allows you to enable your Public Location Badge, and provides you with an HTML snippet that you can add to your blog or website, where it will render a Google Map.


JSON responses

Another way of accessing your location through the Public Location Badge is by using JSON. This allows for a more programmatic access of your location that you can embed in your applications.

This method is very handy if you want to access the current location of someone who has exposed his location through the Public badge.

Each public location badge is associated with a userid. This numeric value is associated with each google account.

If you want to access the JSON string, you can type the following URL in your browser.

http://www.google.com/latitude/apps/badge/api?user=3356651955207924992&type=json

It should return a string like this :

{"type": "FeatureCollection", "features": [
{ "type": "Feature",
"geometry": {"type": "Point", "coordinates": [4.47658, 51.02511]},
"properties": {
"id": "3356651955207924992",
"accuracyInMeters": 0,
"timeStamp": 1311050105,
"reverseGeocode": "Arrondissement of Mechelen, Belgium",
"photoUrl": "http://www.google.com/latitude/apps/badge/api?type=photo&photo=gJsAQTEBAAA.rGdvZsJpNd7LFS7MT5g_Bg.YtjvkPGuqXl4iqhY7QGOyg",
"photoWidth": 96,
"photoHeight": 96,
"placardUrl": "http://www.google.com/latitude/apps/badge/api?type=photo_placard&photo=gJsAQTEBAAA.rGdvZsJpNd7LFS7MT5g_Bg.YtjvkPGuqXl4iqhY7QGOyg&moving=true&stale=true&lod=4&format=png",
"placardWidth": 56,
"placardHeight": 59
}
}
]
}

Java access

We can use the Google APIs Client Library for Java to access your public location badge from a java application. It provides a simple an elegant way to make the REST call, and transform the JSON string into a java based model.

The sample application I'll be showing here is very simple, and is made up of 1 java class that contains a main method.

We start by creating our HttpTransport and requestfactory, required to perform the REST call.

private static final HttpTransport transport = new ApacheHttpTransport();
 private static final String PUBLIC_BADGE_URL = "http://www.google.com/latitude/apps/badge/api";

 public static HttpRequestFactory createRequestFactory(final HttpTransport transport) {
     
    return transport.createRequestFactory(new HttpRequestInitializer() {
     public void initialize(HttpRequest request) {
      GoogleHeaders headers = new GoogleHeaders();
      headers.setApplicationName("Google-Latitue-Public-Badge-Sample");
      request.headers=headers;
      JsonHttpParser parser = new JsonHttpParser();
      parser.jsonFactory = new JacksonFactory();
      request.addParser(parser);
     }
  });
 } 

The Java model

We also need to model the JSON string into java. As we're interested in the users location, the simplest model I could come up with looks like this :

public static class PublicBadge {

  @Key
  public Feature[] features;
  
  @Key
  public String type;
  
  public static class Feature {
   @Key
   public Geometry geometry;
  }

  public static class Geometry {
   @Key
   public float[] coordinates;
  }

 }

Performing the REST call

What's left now is to perform the actual call, and parse the response into our java model.

public static void main(String[] args) throws Exception {

  HttpRequestFactory httpRequestFactory = createRequestFactory(transport);
  HttpRequest request = httpRequestFactory.buildGetRequest(new GenericUrl(PUBLIC_BADGE_URL));
  request.url.put("user", "INSERT USER ID HERE");
  request.url.put("type", "json");
  //System.out.println(request.execute().parseAsString());
  PublicBadge publicBadge = request.execute().parseAs(PublicBadge.class);
  System.out.println("Longitude " + publicBadge.features[0].geometry.coordinates[0]);
  System.out.println("Latitude " + publicBadge.features[0].geometry.coordinates[1]);
 }

Project setup

You can copy paste these bits of code into a single java class. In order to use the
Google APIs Client Library for Java, you need to have the following dependencies in place :



The easiest way to do that is to define the Google APIs Client Library for Java dependencies in a pom.xml, and use maven to resolve the dependencies for you. The pom.xml to have this sample up and running looks like this :


  4.0.0
  
    com.google
    google
    5
  
  com.ecs.latitude
  publicbadge
  1.0-SNAPSHOT
  
    
      
        maven-compiler-plugin
        
          1.6
          1.6
        
      
    
  
  
   
      com.google.api.client
      google-api-client
      1.4.1-beta
    
   
      com.google.api.client
      google-api-client-googleapis
      1.4.1-beta
        
  
  
   
  google-api-services
  http://mavenrepo.google-api-java-client.googlecode.com/hg
 
  
  
    UTF-8
  



The sample app will simply output the latitude / longitude associated with the Public Badge.

Latitude 4.47658
Longitude 51.02511

References

Monday, July 11, 2011

Samsung Galaxy S GPS issues after Gingerbread upgrade

After upgrading my Samsung Galaxy S to Gingerbread, I was having some serious GPS issues.
The unability to get a location fix in a timely fashion, and the many drops during location requests were driving me crazy.
Screenshots below show my daily commute. Location fixes were requested every 30 seconds. You can clearly see the difference before and after the fix.

Before the fix
After the fix


How it got fixed ? By "patching" the GPS receiver with a simple piece of paper....
It seems to do the trick, but I'll need to do some more field testing to be sure. 










I found the possible solution (that I didn't really believe at first) at the Dutch AndroidWorld forum.
For more details on the solution, check the XDA Developers forum.







Monday, May 23, 2011

Introducing the Google Places API

A couple of weeks ago, at the Google I/O developer conference in San Francisco, Google announced the opening up and general availability of the Google Places API. The Google Places API is an API that returns information about Places. It is part of the Google Maps API Web Services, a collection of HTTP interfaces to Google services providing geographic data for your maps applications. The goal of this article is to go over the API by using a sample application that actually does these API calls. We'll cover each API request in detail, look at the raw JSON response coming back from the Google Places API, see how to process the response using the Google APIs Client Library for Java. We'll also convert the JSON response into a java based model, and use that model to display the results.


Monday, May 16, 2011

Introducing the Google Fusion Tables API


In this article, I'll be providing you with a quick overview of the Fusion Tables API.
Google Fusion Tables is a modern data management and publishing web application that makes it easy to host, manage, collaborate on, visualize, and publish data tables online. We'll go over the API by creating a sample application that interacts with our Fusion Tables. The sample app will use the Google APIs Client Library for Java and the complete source code is available on GitHub.