Posts from the “Backend Development” Category

Pitfalls of the Coffeescript Syntax

I’ve been using Coffeescript extensively over the last few weeks. Overall, I love it. The flexibility of the language and the clean, minimalist syntax occasionally causes problems, though. In this blog post, I’ve documented some of the pitfalls that have cost me hours of time. Each of these problems is the result of user error, but are also largely the result of language design choices. Hopefully reading this will save you from these pitfalls!

1. Optional Function Parenthesis and Spacing
The low down: a - 1 is math, a -1 is a function call.
Advice: Always place spaces on both sides of mathematical operators, and think about whether your code could be interpreted as an argument list.

Example Coffeescript:

1
2
3
4
5
6
7
a = 1
if (a - 1 == 0)
    b = 2
if (a-1 == 0)
    b = 2
if (a -1 == 0)
    b = 2

Resulting Javascript:

1
2
3
4
5
6
7
8
9
10
a = 1;
if (a - 1 === 0) {
    b = 2;
}
if (a - 1 === 0) {
    b = 2;
}
if (a(-1 === 0)) {
    b = 2;
}

2. Safely Iterating Backwards
The low down: Converting a C-style for loop to for i in [array.length - 1 .. 0] produces unexpected behavior when array.length = 0.
Advice:When you intend to iterate by -1, add by -1 to the end of your for statement to ensure that you never iterate from [-1..0]

Example Coffeescript:

1
2
3
4
5
6
7
# Bad - calls array[-1] if array.length == 0
for x in [array.length - 1..0]
    array.splice(x, 1) if array[x] == true
 
# Good - no iteration occurs if array.length == 0
for x in [array.length - 1..0] by -1
    array.splice(x, 1) if array[x] == true

Resulting Javascript:

1
2
3
4
5
6
7
8
9
10
11
12
13
var x, _i, _j, _ref, _ref1;
 
for (x = _i = _ref = array.length - 1; _ref <= 0 ? _i <= 0 : _i >= 0; x = _ref <= 0 ? ++_i : --_i) {
    if (array[x] === true) {
        array.splice(x, 1);
    }
}
 
for (x = _j = _ref1 = array.length - 1; _j >= 0; x = _j += -1) {
    if (array[x] === true) {
        array.splice(x, 1);
    }
}

Even with these pitfalls, Coffeescript is a great language. It’s incredibly readable and provides class structures that make it easy to write object-oriented javascript. Loose rules governing parenthesis and brackets mean that Coffeescript is sometimes ambiguous – “a -1″ could be a function call, or basic algebra. However, the ability to leave off parenthesis makes for beautiful function declarations.

Know of more Coffeescript syntax pitfalls? Help me make this blog post a great resource—let me know what you think in the comments and I’ll cite you!

Explorations of Varnish

For the last couple days, I’ve been working on a Sinatra-based web service for resizing images. You can hit the service with an image URL and a desired size, and it uses the high-speed image library VIPS to convert it to the appropriate size. To cache images, the Populr team decided to use Varnish. Varnish makes it painless to cache HTTP responses by proxying requests to your web service and returning cached data when it’s available.

EngineYard provides a Chef script for configuring Varnish that automatically configures it based on the size of your instance. However, when I activated the Chef script and started a deployment, the instance wouldn’t spin up. The /var/log/syslog on the machine showed that Varnish was failing to spawn a child process:

1
2
3
4
Jun 28 12:21:28  varnishd[12518]: Pushing vcls failed: CLI communication error (hdr)
Jun 28 12:21:28 varnishd[12518]: Child (12521) died signal=16
Jun 28 12:21:28 varnishd[12518]: Child (-1) said
Jun 28 12:21:28 varnishd[12518]: Child (-1) said Child starts

It turns out, Varnish is built with 64bit architectures in mind and the default settings assume a 64bit stack. On his blog, Kristian Lyngstol noted:

Varnish works on 32-bit, but was designed for 64bit. It’s all about virtual memory: Things like stack size suddenly matter on 32bit. If you must use Varnish on 32-bit, you’re somewhat on your own. However, try to fit it within 2GB. I wouldn’t recommend a cache larger than 1GB, and no more than a few hundred threads… (Why are you on 32bit again?)

Antonio Carpentieri wrote a great blog post on trying to start Varnish on a 32-bit Amazon instance, and found that the problem is the sess_workspace configuration parameter. It’s default value 262144 (256k) is too large to fit on a 32-bit stack. He suggests setting it to only 19264 (16k) and also changing the thread_pool_stack to 64k to prevent problems with Varnish trying to start child processes.
When using the Varnish Chef recipe, these changes need to be made in varnishd.monitrc.erb, which contains the commands monit uses to start and stop Varnish. The configuration parameters are conveniently passed as arguments to the start command, so it’s easy to edit them.

1
2
3
4
check process varnish_80
  with pidfile /var/run/varnish.80.pid
  start program = "/usr/sbin/varnishd -a :<%= @varnish_port %> -T 127.0.0.1:6082 -s <%= @cache %> -f /etc/varnish/app.vcl -P /var/run/varnish.80.pid -u nobody -g nobody -p obj_workspace=4096 -p sess_workspace=262144 -p listen_depth=2048 -p overflow_max=<%= @overflow_max %> -p ping_interval=2 -p log_hashstring=off -h classic,5000009 -p thread_pool_max=<%= @thread_pool_max %> -p lru_interval=60 -p esi_syntax=0x00000003 -p sess_timeout=10 -p thread_pools=<%= @thread_pools %> -p thread_pool_min=100 -p shm_workspace=32768 -p thread_pool_add_delay=1"
  stop program = "/usr/bin/pkill -KILL varnish"

Rather than pursue these changes and handicap Varnish on a 32-bit server, we decided to create a custom build of the VIPS static library for EngineYard’s 64-bit high CPU instances. This turned out to be a bit of a hassle, but EngineYard’s default Varnish Chef recipe ran without a hitch on the 64 bit machine.