How NoteHub is built

I would lie if I would say that NoteHub is an app that solves an extremely important life problem. No, it was just one of my rather modest needs and — to be honest — it wouldn’t exists today if I wouldn’t stumble upon a website implemented entirely in Clojure. At that point I knew already what Clojure is, I knew even about ClojureScript as an alternative to JavaScript, but both were out of scope of my interest. However, when I saw that you also can render HTML & CSS directly from Clojure, without any templates, I knew immediately: I have to try this out. An experiment with the code name NoteHub was born.

No HTML, No CSS, Just Clojure!

I have developed several web apps in my life. I have written them from scratch and using web frameworks. And the most annoying thing was always the necessity to keep in mind a lot of nuances which have nothing to do with main programming language, which was used for the server logic (and in the past, the server site was the main part). These nuances comprised things like JavaScript’s special dangerous “features” and pitfalls. One often even has to learn some aspects & syntax of a new programming language to write templates (e.g., the templating engine of Play! bases on Groovy). Many problems of CSS (no constants, no arithmetic, no mixins, no nested rules, etc.) were addressed by solutions like LESS/SASS. But the problem, that it is yet another language with its own nuances, always remained.

Clojure and ClojureScript have promised to hide all these nuances behind one simple and clean language (approx., it holds: ClojureScript ⊂ Clojure). I was really curios and gave it a chance.

Let’s Start

The code described here can be found on GitHub (state: June 2012). I won’t talk about Clojure itself, I’ll concentrate on the interesting parts, which are relevant for web development.

HTML (Templating)

Due to the fact, that Clojure is a dialect of Lisp, it has almost no syntax. The entire program is written using s-expressions which are nothing more but nested lists. Combine this with the fact, that lists are ordered collections and you get trees. This tree-like code structure allows us to represent any hierarchical data (like XML) at no cost. Hence, HTML can be represented as follows (file pages.clj, lines 71-77):

[:div#hero
  [:h1 (get-message :name)]
  [:h2 (get-message :title)]
  [:br]
  [:a.landing-button {:href "/new" 
                      :style "color: white"} 
                     (get-message :new-page)]]

I think, it’s quite easy to see how this code corresponds to its rendered counterpart. Every element of the form [:tag#panel.shady {:key value} content] will be transformed to:

<tag id="panel" class="shady" key="value">
    content
</tag>

Obviously, we can have any amount of key-value pairs and all these elements can be nested. Are you scared of the famous Lisp-parentheses? So was I. Here is the other side of them: in Lisp code, there is no need for closing tags or error-prone whitespace, because everything is inside of enclosing parentheses. There is still a chance to close a parenthesis at the wrong place, but any decent editor will visually assist here by default.

What’s also cool about this HTML generation, is that we can arbitrarily inject native Clojure code into the HTML “template”, because it is not real templating. Instead, we express the HTML code with the native Clojure data structures, which are converted to HTML later by a special function html (provided by the hiccup library). In the next example I use a map function to produce a table with three totally equal (wrt. formating & structure) table columns iterating over three keywords, which are used to extract the column contents.

[:table.central-element.helvetica-neue
  [:tr
   (map
     #(html  
        [:td.one-third-column
         [:h2 (get-message %)]
         (md-to-html (get-message (keyword (str (name %) "-long"))))])
     [:column-why :column-how :column-geeks])]]

The hiccup library provides the most elegant and natural templating I’ve ever seen.

CSS

The file css_generator.clj contains a Clojure function global-css, which generates the global styles. A typical CSS rule is represented like this:

  (rule ".button"
        :border-radius :3px
        helvetica-neue
        :cursor :pointer
        :border [:1px :solid foreground]
        :opacity 0.8
        :font-size :1em
        :background background)

Here, helvetica-neue, is another CSS rule (a mixin) which is used in several other rules as well. foreground and background are just constants, containing a color value. As in the HTML part, I make here full use of Clojure. So, the most interesting part about the CSS generation, is that the function global-css is parameterizable and generates slightly different rules depending on the specified parameters. The parameters are provided by the server logic according to the client’s request. Namely, it gets a parameter defining a colorscheme name, which is then resolved to concrete colors for the background, foreground, link colors and so on. The entire colorscheme itself is encoded as nested Clojure maps:

(defn- color [& keys]
  (get-in {:dark {:background :#333
                  :foreground :#ccc
                  :background-halftone :#444
                  :foreground-halftone :#bbb
                  :link {:fresh :#6b8
                         :visited :#496
                         :hover :#7c9 }}
           :default {:background :#fff
                     :foreground :#333
                     :background-halftone :#efefef
                     :foreground-halftone :#888
                     :link {:fresh :#097
                            :visited :#054
                            :hover :#0a8 }}} keys))

That is, the keyword :dark points to a map containing 4 colors and another map, which in turn describes the color of a link in different states. To get the color of a hovered link of the dark colorscheme we can call:

(color :dark :link :hover)

And get the value :#7c9. Cool huh?

JavaScript ClojureScript

I always considered writing JavaScript as a kind of punishment. People say that JS has some good parts. Well, I guess I never faced them. If I’d judge from my personal experience, than I’d say that JS has probably all bad things from all languages I know. (UPDATE 2016: Today, with ES6, I’d never say this.) To alleviate the pain, during my last web-app development I used the Underscore.js library, which mimics kind of functional programming approach. A nice idea, but the result is not really smooth, due to the JS’ boilerplate: there is no syntax sugar for creating abstract functions. But now, with ClojureScript, all this headache might just disappear.

Officially, “ClojureScript is a dialect of Clojure that targets JavaScript as a deployment platform.” ClojureScript is a subset of Clojure and has several significant differences from Clojure, which arise due to fundamentally different platforms. For example, JavaScript has no Character data type (honestly, I wasn’t even aware of this fact before).

Thanks to some great libraries, the client-side code of NoteHub (file main.cljs) is extremely short and beautiful. I do use JQuery (through jayq lib)! First, let’s bind the most frequently used DOM-elements to a ClojureScript values:

(def $draft ($ :#draft))
(def $preview ($ :#preview))
(def $session-key ($ :#session-key))
(def $preview-start-line ($ :#preview-start-line))

Now, let’s consider one example: the functionality of the preview button from the new note creation. When a user clicks it, the entered Markdown code is sent to the server via an AJAX call, and as soon, as the server returns the formatted text, we insert this text into a special area and automatically scroll the browser window to the preview section.

First, we register a “click” event on the preview button and provide an anonymous callback function:

(.click ($ :#preview-button)
        (fn [e]
          (xhr [:post "/preview"]
               {:session-key (val $session-key) 
                :draft (val $draft)}
               (fn [json-map]
                 (let [m (js->clj (JSON/parse json-map))]
                   (do
                     (inner $preview (m "preview"))
                     (val $session-key (m "session-key"))
                     (show $preview-start-line)
                     (scroll-to $preview-start-line)))))))

The callback function calls on the server the route /preview and provides a session key (the session keys are used against flooding) and the content of the text area with the entered markdown text. Once the server response has arrived, it gets converted from JSON map to a Clojure map. Then, we unhide a dashed line splitting the edit and the preview areas, insert the formatted text to the corresponding html-element, insert the new session key into a hidden input field and scroll to the dashed line.

On the server, the function responding to this AJAX call is defined in pages.clj and looks as follows:

(defpage [:post "/preview"] {:keys [session-key draft]}
         (when (flash-get session-key)
           (generate-string
             {:session-key (get-flash-key)
              :preview (md-to-html draft)})))

The server checks the session key validity, converts the provided Markdown to HTML, and returns a map consisting a new session key and the generated HTML. This map gets converted to JSON by the generate-string function from the cheshire library.

So, there are no surprises on the client side: it is still Lisp, there are nice libs and a layer abstracting JQuery. What’s interesting, is how about to use literally the same code on the server and on the client side? Duke Nukem would say “Peace Of Cake!” And I tried this too.

Crossover Code

As a simple spambot protection (I hope, that the majority of bots won’t evaluate heavy JS) (UPDATE 2016: LOL), I wanted to implement a hash function, which should be computed on the session-key and the entered markdown text. This hash would be then sent to the server together with the user input and the server should be able to validate the hash, making the same computations as the client’s browser. Obviously, it should be the same algorithm. The hash function doesn’t need to be very hash-resistant and I tried to use the built-in hash function. Unfortunately, due to platform differences this function provides different hash code for the same string on JVM and in a browser. Ok, then let’s implement one.

I spent on this task more time then I would like to admit. The problem was the before mentioned absence of chars in JavaScript. How should I map a string (a sequence of letters) to some integers, equally on JVM and JS? Well, first I translated a string to a list of strings, where each string contained one char, e.g.:

("H" "e" "l" "l" "o" "," " " "w" "o" "r" "l" "d")

This list is the minimal decomposition possible on JS, and would be the same thing on both platforms, because we don’t use chars per se (they’re packed into strings). But now, I should map these strings to integers, such that this code would work equally both on JVM and JS! In Clojure I could simply get the char code of such a string like this:

user=> (.codePointAt "a" 0)
97

The .codePointAt function is a function from the Java’s String class. Obviously, this function is missing in JS. Instead, in JS I could use:

(.charCodeAt "a" 0)

How should I combine both things in one code? With Clojure[Script] we have all the power of higher order functions, so let’s just abstract the runtime-specific differences away. The new simple hash (file lib.clj):

(defn hash [f s]
  (let [short-mod #(mod % 32767)
        char-codes (map f (remove #(contains? #{"\n" "\r"} %) (map str s)))
        zip-with-index (map list char-codes (range))]
    (reduce
      #(short-mod (+ % 
                     (short-mod (* (first %2) 
                                   ((if (odd? %)
                                      bit-xor
                                      bit-and) 16381 (second %2))))))
      0 zip-with-index)))

This function takes a function f and a string s. Then it produces a list of strings, containing all chars of s, applies f to them and makes some trivial arithmetic modulo 216 (in contrast to JVM, JS doesn’t produce overflow if the integer range is violated).

Bingo! This code can be and is executed both on JVM and in the browser now. From Clojure, this hash function is called like this:

(hash #(.codePointAt % 0) some-string)

…and from JavaScript:

(hash #(.charCodeAt % 0) some-string)

Hence, up to the injected dependencies, it is really possible and is very easy to share the same code between client and server.

Java Interoperability

One of the most important advantages of a JVM language is the possibility to use countless, high performant Java libs. What I needed for this project was a markdown compiler. I checked out some existing implementation in Clojure, but since it was somehow half-finished, I googled for a Java solution and in five minutes I had this code (file pages.clj, lines 25-30):

(def md-processor
  (PegDownProcessor.))

(defn md-to-html [md-text]
  (.markdownToHtml md-processor md-text))

Ok, the calls to Java objects are not really eye-candy, but they can be hidden behind a comfortable layer: first I create a new Java object, then I define a simple function, which calls the Java function .markdownToHtml on this object, when some markdown should be processed.

That’s it.

Leiningen

Another point in the Clojure-based development I’d like to emphasize is the tool I used to maintain the project: leiningen. Basically, it is a tool, which runs your project, your tests, compiles it, exports a JAR and so on. I didn’t download a single library manually, I just googled, found solutions, looked up their package names and defined them as dependencies in the project file. The rest (like downloading and publishing on the Classpath) has happened fully automagically! No pain, no hassle whatsoever.

Conclusion

NoteHub is a tiny project. I just can’t draw any serious conclusions from it. And yet I have a strong feeling, that Clojure is something huge. I didn’t struggle yet with a heavy refactoring of Clojure code or with the absence of the static type system, but I hope that my usual approach (abstract and modularize) will handle this.

Published by
Christian Müller

UPDATE 2016: Almost 4 years after the publication, I would like to add my long-time experience maintaining NoteHub, which ironically led to ditching Clojure in favor of JavaScript on the client and server side! Main reasons for switching to JavaScript: