jQueryUI 1,000,000 custom downloads every quarter

I’ve been helping jQueryUI to port its DownloadBuilder into node.js. The rewrite is actually live and serving downloads since Oct/2012.

Today, I wrote my very first post in the jQueryUI Blog with insights into what we’ve built, and the trends we’ve noticed so far.

Good reading… (original at http://blog.jqueryui.com/2013/04/1000000-custom-downloads-in-four-months/)

We surpassed the millionth download of jQuery UI using our recent DownloadBuilder and ThemeRoller rewrite back on February. As of today, we have served 1,730,000 downloads and counting. Read on for some insights into what we’ve built, and the trends we’ve noticed so far.

The former server-side code was written in PHP. It’s been rewritten in JavaScript and runs on node.js, and it is much more integrated with jQuery UI release process overall. The client-side has been rewritten as well, although we didn’t make any big changes to the UI/UX.

On the client side, despite the few visual changes, we have some interesting updates. The DownloadBuilder now remembers what user selects and makes it linkable, so it’s easy to share or go back and modify a custom theme. We’re also shortening links automatically if they get too big, by zipping parts of the query string.

The backend in-memory-caches the source files and theme images to speed up the downloads. Since it serves custom downloads, the parts are not simply assembled, but rather modified on build-time and then assembled. The average build & package time is 1.3s.

The download traffic is pretty uniform and constant; we hit an average of 66,000 downloads per week, having more traffic during the weekdays and less traffic during the weekends. When we publish a new release, we see a 10% increase on traffic. Adoption of the new release is really fast, legacy downloads drop virtually immediately. Although, we still have a significant amount of 1.9.x downloads after the 1.10.x release, splitting the total as chart shows below.

Downloads per version

29% of users download the default components with the default theme. Other than that, we have all sorts of custom combinations happening. They choose different components, different themes, or a mix of both.

Among the component customizations (which represent 26% of all total downloads), 15% are only Datepicker (the winner by far), followed by No Components (8.5%), which packages the theme only, Autocomplete (4.5%), Dialog (4.25%), and Tabs (3.75%).

Custom component selection

- Datepicker and its dependencies

- Datepicker, mouse and position

14.95% (4.34% of all downloads)

- 12.72% (3.69% of all downloads)

- 2.23% (0.65% of all downloads)

No components (theme only) 8.55% (2.48% of all downloads)
Autocomplete and its dependencies 4.53% (1.31% of all downloads)
Dialog only 4.25% (1.23% of all downloads)
Tabs only 3.77% (1.09% of all downloads)
Accordion only 2.91% (0.84% of all downloads)
Slider only 2.58% (0.75% of all downloads)
All, but effects 1.87% (0.54% of all downloads)
Core components (no widgets or interactions) 1.60% (0.46% of all downloads)
Sort interaction only 1.37% (0.40% of all downloads)
Interaction and core (no widgets) 1.33% (0.38% of all downloads)
Draggable interaction only 1.22% (0.35% of all downloads)
Effects only 1.05% (0.30% of all downloads)
Tooltip only 1.04% (0.30% of all downloads)
The core component (solely) 1.02% (0.30% of all downloads)
total 100.00% (26.37% of all downloads)


Theme customizations (choosing something other than the default UI Lightness theme) represent 57.5% of all downloads. If we skip the base theme Smoothness too, theme customizations are actually 42.35% of all total downloads. 16% of all downloads are user created themes (Custom Themes), followed by the Redmond (4.86%), UI darkness (2.73%), and Start (2.38%) themes.

Within the users that create a Custom Theme, the majority of users (77%) download the full “all components” bundle, 5.5% download it with no components (theme only), and 17.5% do it with a custom component selection.

(top 11)
Default Component
Custom Component
UI lightness (default theme) 38.76% (28.53% of all DLs) 53.20% (14.03% of all DLs)
Custom Theme 17.10% (12.59% of all DLs) 14.11% (3.72% of all DLs)
Smoothness (base theme) 16.34% (12.03% of all DLs) 11.64% (3.07% of all DLs)
Redmond 5.13% (3.77% of all DLs) 4.13% (1.09% of all DLs)
UI darkness 2.80% (2.06% of all DLs) 2.54% (0.67% of all DLs)
Start 2.60% (1.91% of all DLs) 1.78% (0.47% of all DLs)
Cupertino 2.42% (1.78% of all DLs) 1.97% (0.52% of all DLs)
Blitzer 1.58% (1.16% of all DLs) 1.29% (0.34% of all DLs)
Flick 1.44% (1.06% of all DLs) 1.52% (0.40% of all DLs)
Sunny 1.41% (1.04% of all DLs) 0.91% (0.24% of all DLs)
Dark Hive 1.17% (0.86% of all DLs) 0.72% (0.19% of all DLs)
total 100% (73.63% of all DLs) 100% (26.37% of all DLs)


Thanks to clark and Splunk for helping us make sense of all this data!

As usual, if you find any bugs or if you have any ideas on how to make the DownloadBuilder or ThemeRoller even more amazing, we’d love to hear from you! But please, don’t use the comments, rather please file an issue here.

Quick script to fetch diff history [of a file] using svn

If you wanna see the whole history differences of some file in svn repository, you could use this:
$ svndiffhist file
Retrieving releases...
OK (found 17 steps)
## 908:895
## 895:890
## 890:880
... etc

Where svndiffhist is the following script:

function svndiffhist {

    echo -n "Retrieving releases..." >&2
    releases=$( svn log $FILE | grep '^r[^a-z]' | cut -d " " -f1 |
                cut -dr -f2- )
    num_steps=$( echo $releases | wc -w )
    echo -e "tOK (found "$num_steps" steps)" >&2

    # Sort them in ascrending order
    #releases=$( echo $releases | tr " " "n" | sort -n )
    # Group them as A:B B:C C:D
    releases=$( echo $releases | sed 's/ ([0-9]+)/:1 1/g' )

    for EACH_STEP in $releases; do
        #svn diff -r $EACH_STEP $FILE | gvim -
        echo "## $EACH_STEP" >&2
        svn diff -r $EACH_STEP $FILE

A REST client interface for Python

The py-restlib: [http://code.google.com/p/py-restlib] is a GNU GPL library that implements a REST client interface for Python.

Here it goes its Getting Started section:


Py-restlib is supposed to be a simple REST client interface for python. But, it also claims that writing a python client to communicate with RESTful applications on the Web should be an easy task.

Getting Started


Check out the latest release using svn:
svn checkout http://py-restlib.googlecode.com/svn/trunk/ py-restlib
svn checkout http://py-restlib.googlecode.com/svn/trunk/ py-restlib

If you want support for translating JSON format into python structures, download json-py package, and unzip it to json-py subdir.


The REST client

from restlib import *

users = Resource("http://hostname/api/users")

# List all users (GET /api/users)

# List user where id==1 (GET /api/users/1)

# List user where id==1 && foo=='bar' (GET /api/users/1?foo=bar)
users.get(1, foo='bar', ...)

# Create a new user (POST /api/users)
users.create(foo='bar', ...)


# You are also able to use it that way:
api = BaseResource("http://hostname/api")

The REST server (in this case a Ruby on Rails application)

class Api::UsersController < ApplicationController
    protect_from_forgery :except => :create
    # GET /api/users
    def index
        @users = User.find(:all) # FIXME: if param[id] - under construction

        request.format=:json if request.format=:html
        respond_to do |format|
            format.xml {render :x ml => @users.to_xml}
            format.json {render :json => @users.to_json}

    # POST /api/users
    def create    # FIXME: under construction
        request.format=:json if request.format=:html
        respond_to do |format|
            format.xml {render :x ml => params.to_xml}
            format.json {render :json => params.to_json}

Python POST request

If you’re trying to send a python POST request with httplib, but it’s not working like example tells. Try this minor fix:

Here is an example session that shows how to “POST” requests:

>>> import httplib, urllib
>>> params = urllib.urlencode({‘spam’: 1, ‘eggs’: 2, ‘bacon’: 0})
>>> headers = {“Content-Type”: “application/x-www-form-urlencoded”,
… “Accept”: “text/plain”}
>>> conn = httplib.HTTPConnection(“musi-cal.mojam.com:80″)
>>> conn.request(“POST”, “/cgi-bin/query”, params, headers)
>>> response = conn.getresponse()
>>> print response.status, response.reason
200 OK
>>> data = response.read()
>>> conn.close()

Thanks to wireshark and the working command below, so example above could be compared.

curl -d foo=bar http://localhost

Criptografia em Linux, utilizando EncFS

Introdução (o que?)

Este tópico descreve o procedimento para criptografia de dados no GNU/Linux, utilizando o sistema de arquivos EncFS.

Público Alvo (pra quem?)

Este tópico deve ser útil para usuários de laptop.

Vantagens e Desvantagens de utilizar o EncFS

Modos de criptografia


Como utilizar


Instalar os pacotes: (utilizar o gerenciador de pacotes de sua distribuição)

  • FUSE (Filesystem in Userspace)
  • EncFS

Criação do diretório criptografado

Na 1a tentativa de montar seu volume criptografado, ele é criado.

Existem várias opções de criptografia. Caso opte-se pela praticidade, siga os valores default.
$ mkdir ~/.pessoal.enc # dados criptografados (raw data)
$ mkdir ~/pessoal # ponto de montagem (dados legíveis)
$ encfs ~/.pessoal.enc ~/pessoal # na 1a tentativa de montagem, o volume criptografado é criado
Volume key not found, creating new encrypted volume.
Password: [password entered here]
Verify: [password entered here]

O diretório ~/pessoal, agora, é um ponto de montagem de ~/.pessoal.enc e todo o conteúdo que estiver dentro dele (~/pessoal) estará seguro quando offline.



$ encfs ~/.pessoal.enc ~/pessoal
Password: [password entered here]

Diretório legível: (montado)

$ cd ~/pessoal
$ echo "hello foo" > foo
$ echo "hello bar" > bar
$ ln -s foo foo2
$ ls -l
total 8
-rw-r--r-- 1 vgough users 10 2003-11-03 21:44 bar
-rw-r--r-- 1 vgough users 6 2003-11-03 21:44 foo
lrwxrwxrwx 1 vgough users 7 2003-11-03 21:44 foo2 -> foo

Diretório criptografado: (de fato, armazenado)

$ cd ~/.pessoal.enc
$ ls -l
total 8
-rw-r--r-- 1 vgough users 6 2003-11-03 21:44 eEM4YfA
-rw-r--r-- 1 vgough users 10 2003-11-03 21:44 gKP4xn8
lrwxrwxrwx 1 vgough users 7 2003-11-03 21:44 i7t9-m,I -> eEM4YfA


$ fusermount -u ~/pessoal

Criptografando os dados de seus aplicativos: Firefox, Thunderbird, etc.

Deseja ter criptografados seus emails, informações de calendário, conversas do instant messenger e informações de seu browser? Mova o diretório de configuração de cada aplicativo para dentro do volume criptografado e crie um link de volta. Como no exemplo abaixo:

Antes de iniciar os aplicativos (óbvio), mova seu diretório de configuração para dentro do volume criptografado e link-o de volta. Por exemplo:
cd ~
mv .mozilla ~/pessoal
ln -s ~/pessoal/.mozilla .

Faça o mesmo para mozilla-thunderbird, gaim, evolution, entre outros.

Ambiente Gráfico

Em ambiente gráfico, pode ser mais prático criar starers de aplicativos que, antes de inicializar o aplicativo em si, cheque se a unidade criptografada está montada. Caso não esteja, o próprio starter poderá montá-la, utilizando gtk2-ssh-askpass, x11-ssh-askpass ou zenity. Exemplo:
encfs --extpass=/usr/bin/zenity-encfs $ENC $MNT



Uma vez que todos os dados estão concentrados no diretório criptografado ~/.pessoal.enc, torna-se fácil a realização de backup. Basta copiá-lo para uma mídia removível e, sim, o backup também estará criptografado.

Exemplos de como realizar o backup:

  1. Compactar o diretório tar -czf ~/pessoal.enc.tgz ~/.pessoal.enc e copiá-lo para um DVD (sugestão: k3b).
  2. Utilizar alguma forma de backup diferencial/incremental (é copiado apenas o que alterou), sem sugestões…

Links externos