Fork me on GitHub

Integrating node.js

for real-time awesomesauce

i.e. 4k ♥'s node.js

Michal Minecki

What we'll go over.

  • Node.js Basics
  • Three Quick Case studies.
    • Pubsub Stuff
    • Drupo Poetry
    • Live Video Commenting
  • Other integration options

What is Node.js?

What is Node.js?

From Nodejs.org

Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

WAT?

Node.js is a platform built on Chrome's JavaScript runtime [...]

 

  • Chrome's JavaScript runtime is called V8, it's one of the reasons Chrome is considered so fast.
  • node.js leverages V8's speed on the server!
[...] easily building fast, scalable network applications.

 

  • Node wants to act as a message passer (more on this later).
Node.js uses an event-driven, non-blocking I/O model [...]

 

  • Only one thing gets executed at a time, but libraries allow I/O or other intensive tasks to be executed asynchronously so the script doesn't need to wait for them to finish.

What Node like to work with?

  • Javascript without a browser.
  • Runs as a server.
  • Ships with basic libraries.
  • Has a built in package manager.

What makes it awesome to work with?

  • Packages, lots of them very easy to manage.
  • Same language as in browser.
  • Large number of PAAS hosting options.
  • Large and fast growing community
  • It's Javascript.

What could I do without?

  • Async programing can be difficult to organize. (Callback Hell)
  • Fast moving project lots of updates
  • Error handling is really important, server crash means whole app stops responding.
  • It's Javascript.

If it's so great why don't you marry it?

Drupal Kicks Ass

  • Fully backed features. - Node has libraries, drupal has solutions.
  • Fully baked systems - Users, content, taxonomies.
  • Fully bakced admin interface.
  • Allows a great deal of flexibility for site builders.

Why does Drupal well with node?

  • Feeds is pretty awesome at injesting data
  • Lots of options for authenticaiton
  • Easy to connect to using services module
  • Easy to create custom endpoints in services

Case Studies

PubSubHubbub

Hub

PubSub

SRSLY, what's up with that crazy name?

  • RSS feeds don't necessarily get updated very frequently.
  • As a provider it's undesirable to have consumers generating load when there's no new content.
  • The solution: PubSub!
  • Notifies subscribers when new content comes in, so they don't need to keep checking.

The need

  • About 100+ Drupal sites.
  • Need to consume about 300+ data feeds that didn't natively support PubSub.

Wait, Drupal supports polling feeds, right?

  • Yes! And in fact it's pretty good at it.
  • BUT, these feeds were time sensitive.
  • It was unrealistic to have a Drupal process continuously running to do the polling.

Ok, so what'd you do?

  • Augmented the RSS feeds with node.js to support PubSub.
  • Node took care of the polling and implemented the PubSub protocol so the feeds module would "just work".
  • Deployed the node code in the joyent cloud, separate from the Drupal installations.

The result

  • Almost indiscernible load on the node server.
  • Load is only generated on the Drupal servers when there's actually work to do.
  • Drupal is able to transparently subscribe to the node hub with the feeds module (i.e. Drupal devs don't need to know or care how the node code is working as long as it's up).

PubSub questions?

Drupal Poetry

Responsive Application Development with Drupal and Backbone

Drupal Poetry

One codebase to rule them all.

video

Desktop

Tablet

Mobile

Drupal Admin

Why We Built it

  • Have fun. (Built on 20% time)
  • Work with hipster technolgies.
  • See if Responsive works for Applications.

Technologies

Front-end JS:

Backbone.js and Underscore

  • Backbone
    • Lightweight framework for creating web applications
    • Almost a MVC
  • Underscore
    • Set of utility functions that make working with JavaScript less insane.

require.js

  • requirejs.org
  • JavaScript file and module loader.
  • Simplifies code reuse on client and server.
  • Includes build/optimization tools.

Node.js

  • nodejs.org
  • Server side JavaScript platform.
  • Built on Google Chrome's V8 JavaScript engine.
  • Event-driven, non-blocking.

MongoDB

  • mongodb.org
  • NoSQL database/document store.
  • Objects stored in BSON (very similar to JSON).
  • Schemaless design.

Architecture

Poem life cycle

Fresh Page

  • Nginx serves the HTML.
  • Backbone.js requests the cached taxonomy terms to populate the word libraries.
  • User starts to put words in the work area.
  • All changes are stored in the browser memory.
  • User clicks share.

Saving a Poem

  • Poem is saved.
  • If there is no Session: Session is lazy created by Node.js and stored in Browser Local storage.
  • Poem is saved as a json object directly in MongoDB by Node.js.
  • Permalink is created by Node.js from the MongoDB hash and sent to browser.
  • Reference to Poem is asynchronously saved in Drupal using services. (Send some other data with it but we feel bad about that.)
  • Drupal responds with NID.
  • Node.js Updates the MongoDB document with the the NID.

Fun stuff.

  • User is encouraged to login with Twitter OAuth and/or tweet it.
  • If they don't login and don't clear cache they can come back and edit.
  • Now it gets interesting.
  • Moving the words saves the new poem in MongoDB. Autosave!
  • Same anon user can edit poems without login.
  • Users can "fork" poems. Use someone else's poem as a starting point.

Autosave Workflow

Actual

Better

questions

Drupo Questions?

EEERRRRR MMERRRRE GERD VERDERO KERMENTING

What we did

The Need

  • A live event with viewers across the globe
  • Needed live-feed of all participants thoughts to be shared
  • As we were using a live video, we could not have any page refreshing


Why not Drupal's AJAX?
It's slow, and resource intensive

The Solution

  1. Had a second server that was running Node.js with socket.io.
  2. Each user had a Drupal form that would submit a discussion node with their thoughts, thus saving the new node in Drupal.
  3. The node.js server pinged the Drupal server every 10-15 seconds and grabbed a JSON object with all new discussions
  4. Node.js would then push out all the new discussions to all users with socket.io, and update any formatting
  5. The client would receive the new nodes in a JSON object, display it on their feed.

The Effect

  • Real-time data was easily shared for every user.
  • Minimal Drupal interface, while still keeping all data within Drupal.
  • Single page for all video, discussions, etc. to allow all users to fully interact.
  • All pages work fully on a variety of devices.

The Nerdy Stuff

The Future

  • Link the video timeline with comments, so that the comments are displayed when they were made during the video when it's replayed.
  • Include some sort of voting API work, so that we can have live voting an analytics with the discussions
  • Move the handing of new comments and discussion to Node.js, within reason.

Drupal Module

  • Looks like a winner.
  • Appears well written and useful.
  • Opinionated and attempts to be comprehnsive
  • Didn't use it because we wanted low code debt.

Fork us on GitHub!


Thanks!

Questions?