Tuesday, August 26, 2014

Playing with real-time content delivery (WebSocket)

This time I'd like to share some results of experimenting with Firebase and Socket.IO while playing with real-time content delivery with WebSockets a bit.

FireBase - NoSQL database, has a powerful API to store and sync data in real-time using WebSocket. It also has perfect Security Rules system, which allows to restrict read and write access to data in the manner I was dreaming about when playing with CouchDB back in 2011. The goal was to let everyone access only his or her data right from the browser. So, here is an example of security rules that suites this need:
{
  "rules": {
    ".read": false,
    ".write": false,
    "users": {
      "$user": {
        ".read": "auth.uid === $user",
        ".write": "auth.uid === $user"
      }
    },
    "data": {
      "$data": {
        ".read": "auth.uid === data.child('uid').val()",
        ".write": "auth.uid === data.child('uid').val()"
      }
    }
  }
}

Then there are many ways to authenticate. With Simple Login, it could be static web site hosted on FireBase, similarly to CouchApp, fantastic! But I decided to try custom authentication, and integrate Firebase with Atlassian Connect Add-On in Node.js using atlassian-connect-express.

Here is relevant part of client javascript. That simple to get own data:

addon.js

  var rootRef = new Firebase("https://example-multi-tenant.firebaseio.com/data");
  rootRef.auth(firebaseToken, function(error) {
    rootRef.on("value", function(snapshot) {
      var data = snapshot.val();

And firebaseToken is generated serverside basing on authentication information provided by Atlassian Connect. So I map clientKey to uid for Firebase security rules:

routes/index.js:

    var FirebaseTokenGenerator = require("firebase-token-generator");
    var tokenGenerator = new FirebaseTokenGenerator(process.env.FIREBASE_SECRET);

    app.get('/whoslooking', addon.authenticate(), function (req, res) {

        var clientKey = req.session.clientKey;
        var firebaseToken = tokenGenerator.createToken({ "uid": clientKey });

I did not come to particular useful use-case yet, and have tried to clone existing Who's Looking add-on for JIRA, but there must be better ideas to try out with this approach.

For such simple case, it looked overcomplicated to me, so I've given Socket.IO a try, and ended-up actually within Engine.IO, which is the WebSocket semantics transport layer Socket.IO uses.

The main interesting part for me is authenticating connections, so that only add-on could connect. Here is the trick to leverage atlassian-connect-express authentication within allowRequest function that can decide whether to allow WebSocket request or not in app.js:
var authentication = require('atlassian-connect-express/lib/middleware/authentication.js');
var authenticationHandler = authentication.authenticate(addon, true);

var server = http.createServer(app);
var io = require('engine.io').attach(server, {allowRequest: function(req, next) {
  req.query = req._query;
  req.header = function(name) {
    return req.headers[name];
  };
  var res = {
    send: function(code, msg) {
      next(msg); // error
    },
    setHeader: function() {}
  };
  authenticationHandler(req, res, function(err) {
    next(err, !err); // success
  });
}});

Then Who's Looking logic itself is very simple. State is just kept in memory serverside, and is updated when connection is open or closed, and every time simple message is sent to client with all users currently looking at JIRA issue, no heart-beat (or long polling) and database. Nice and easy, isn't it?

Such add-on can be deployed to Heroku, by the way, it has support for WebSockets already!

Monday, March 10, 2014

Building website with Punch

Finally, I've got something, hopefully useful, to share. Today I would like to describe fully automated and completely free way of publishing and hosting static website with Punch and Jenkins.

Punch


Punch is a simple, intuitive web publishing framework, like, probably better known, Jekyll. I've found it better for myself because of possibility to write custom content handlers. Overall,  I think, Punch can be viewed as more generic website publishing framework than Jekyll. Also it's Node.js application.

Though I was missing automatic publishing like gh-pages. But I have recently come across nice post about Building a private gh-pages clone with dexy and gitlab, which mainly inspired me to investigate this drawback.

DEV@Cloud (Jenkins)


Jenkins is the #1 open source continuous integration tool. And it's offered by Cloudbees as a Service, also well known as DEV@Cloud. Luckily, among great bunch of possibilities, it allows to build Node.js projects, which is the core part of my solution, though very simple.

My manual build shell script is as follows:
curl -s -o use-node https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/node/use-node
NODE_VERSION=0.10.4 . ./use-node
npm install -g punch
npm install
punch p
The last command generates and publishes site. Punch supports publishing to Amazon S3 or to a custom server with SFTP out of the box, but with GitHub or Bitbucket you can host your static website for free, see GitHub Pages again and Publishing a Website on Bitbucket. So, we'll need to define custom publishing strategy.

Punch git publisher


I have not found available punch-git-publisher useful - at the moment of writing it is unfinished. So I've created own git publishing strategy: punch-git-publisher.js. It needs to be configured in punch config.json as follows:
"plugins": {
    "publishers": {
        "git_strategy": "./lib/punch-git-publisher.js" 
    }
},

"publish": {
    "strategy": "git_strategy",
    "options": {
        "repo_url": "git@bitbucket.org:azhdanov/azhdanov.bitbucket.org.git"
    }
}

Brining it all together


The most interesting part, automating :) I host my website project on Bitbucket, similarly to website itself. Although I believe it will work with GitHub in very similar way for User or Organization and Project pages, however for the latter you will need to modify punch-git-publisher.js to commit in gh-pages branch.

Below is few simple steps to make your changes in punch project trigger automatic build and publishing:
  1. Obtain Jenkins API key. For DEV@Cloud see https://<account>.ci.cloudbees.com/user/<the-user-portion-of-your-email>/configure, e.g. https://avalez.ci.cloudbees.com/user/azhdanov/configure.
  2. Enable 'Trigger builds remotely' in your Jenkins project configuration.
  3. Add commit hook, e.g. see Jenkins hook. For DEV@Cloud EndPoint must be similarly to above, i.e. https://<the-user-portion-of-your-email>:<apitoken>@<account>.ci.cloudbees.com, e.g. https://azhdanov:***@avalez.ci.cloudbees.com
  4. Add Jenkins ssh public key to your Bitbucket account to let build push generated website content to the repository. Here is good writeup for Github: How to use private github repositories with cloudbees, and Bitbucket: Add an SSH key to an account.
Now, just do your changes, commit, push, and if you're lucky, you get your website updated in a minute, all like Using Jekyll with GitHub Pages. Here is the overall picture:



Let me know how it works for you ;)

References

  1.  punch-git-publisher.js