Posts from the “Uncategorized” Category

Looking Outwards – Final Project

I’m still tossing around ideas for my final project, but I’d like to do more experimentation with the kinect. Specifically, I think it’d be fun to do some high-quality background subtraction and separate the user from the rest of the scene. I’d like to create a hack in which the users body is distorted by a fun house mirror, while the background in the scene remains entirely unaffected. Other tricks, such as pixelating the users body or blurring it while keeping everything else intact could also be fun. The basic idea seems manageable, and I think I’d have some time left over to polish it and add a number of features. I’d like to draw on the auto calibration code I wrote for my previous kinect hack so that it’s easy to walk up and interact with the “circus mirror.”

I’ve been searching for about an hour, and it doesn’t look like anyone has done selective distortion of the RGB camera image off the kinect. I’m thinking something like this:

Imagine how much fun those Koreans would be having if the entire scene looked normal except for their stretched friend. It’s crazy mirror 2.0.

I think background subtraction (and then subsequent filling) would be important for this sort of hack, and it looks like progress has been made to do this in OpenFrameworks. The video below shows someone cutting themselves out of the kinect depth image and then hiding everything else in the scene.

To achieve the distortion of the user’s body, I’m hoping to do some low-level work in OpenGL. I’ve done some research in this area and it looks like using a framebuffer and some bump mapping might be a good approach. This article suggests using the camera image as a texture and then mapping it onto a bump mapped “mirror” plane:

Circus mirror and lens effects. Using a texture surface as the rendering target, render a scene (or a subset thereof) from the point of view of a mirror in your scene. Then use this rendered scene as the mirror’s texture, and use bump mapping to perturb the reflection/refraction according to the values in your bump map. This way the mirror could be bent and warped, like a funhouse mirror, to distort the view of the scene.

At any rate, we’ll see how it goes! I’d love to get some feedback on the idea. It seems like something I could get going pretty quickly, so I’m definitely looking for possible extensions / features that might make it more interesting!

Explorations of Varnish

For the last couple days, I’ve been working on a Sinatra-based web service for resizing images. You can hit the service with an image URL and a desired size, and it uses the high-speed image library VIPS to convert it to the appropriate size. To cache images, the Populr team decided to use Varnish. Varnish makes it painless to cache HTTP responses by proxying requests to your web service and returning cached data when it’s available.

EngineYard provides a Chef script for configuring Varnish that automatically configures it based on the size of your instance. However, when I activated the Chef script and started a deployment, the instance wouldn’t spin up. The /var/log/syslog on the machine showed that Varnish was failing to spawn a child process:

1
2
3
4
Jun 28 12:21:28  varnishd[12518]: Pushing vcls failed: CLI communication error (hdr)
Jun 28 12:21:28 varnishd[12518]: Child (12521) died signal=16
Jun 28 12:21:28 varnishd[12518]: Child (-1) said
Jun 28 12:21:28 varnishd[12518]: Child (-1) said Child starts

It turns out, Varnish is built with 64bit architectures in mind and the default settings assume a 64bit stack. On his blog, Kristian Lyngstol noted:

Varnish works on 32-bit, but was designed for 64bit. It’s all about virtual memory: Things like stack size suddenly matter on 32bit. If you must use Varnish on 32-bit, you’re somewhat on your own. However, try to fit it within 2GB. I wouldn’t recommend a cache larger than 1GB, and no more than a few hundred threads… (Why are you on 32bit again?)

Antonio Carpentieri wrote a great blog post on trying to start Varnish on a 32-bit Amazon instance, and found that the problem is the sess_workspace configuration parameter. It’s default value 262144 (256k) is too large to fit on a 32-bit stack. He suggests setting it to only 19264 (16k) and also changing the thread_pool_stack to 64k to prevent problems with Varnish trying to start child processes.
When using the Varnish Chef recipe, these changes need to be made in varnishd.monitrc.erb, which contains the commands monit uses to start and stop Varnish. The configuration parameters are conveniently passed as arguments to the start command, so it’s easy to edit them.

1
2
3
4
check process varnish_80
  with pidfile /var/run/varnish.80.pid
  start program = "/usr/sbin/varnishd -a :<%= @varnish_port %> -T 127.0.0.1:6082 -s <%= @cache %> -f /etc/varnish/app.vcl -P /var/run/varnish.80.pid -u nobody -g nobody -p obj_workspace=4096 -p sess_workspace=262144 -p listen_depth=2048 -p overflow_max=<%= @overflow_max %> -p ping_interval=2 -p log_hashstring=off -h classic,5000009 -p thread_pool_max=<%= @thread_pool_max %> -p lru_interval=60 -p esi_syntax=0x00000003 -p sess_timeout=10 -p thread_pools=<%= @thread_pools %> -p thread_pool_min=100 -p shm_workspace=32768 -p thread_pool_add_delay=1"
  stop program = "/usr/bin/pkill -KILL varnish"

Rather than pursue these changes and handicap Varnish on a 32-bit server, we decided to create a custom build of the VIPS static library for EngineYard’s 64-bit high CPU instances. This turned out to be a bit of a hassle, but EngineYard’s default Varnish Chef recipe ran without a hitch on the 64 bit machine.

Drill Down WebView Navigation

The next version of NetSketch will include a community browser, allowing you to view uploaded drawings, watch replays, and leave comments without leaving the app. When I started working on the community interface, I looked to other apps for inspiration. Almost every app I’ve used on the iPhone use a sliding navigation scheme, giving you the feeling that you’re drilling down into content as you use the application. This interface is intuitive in a lot of contexts, and dates back to the original iPod. The Facebook app allows you to browse other people’s facebook pages and uses a drill down navigation bar. This works well for the social-network space because you can drill down to look at information and then return to the first page quickly.

I decided to use a UINavigationBar and implement a similar drill-down interface for NetSketch. However, I didn’t want to create custom controllers for each page in the community. I wanted to be able to improve the community without updating the app, and didn’t want to write a communication layer to download and parse images and custom XML from the server.

Using a UIWebView seemed like the obvious choice. It could make retrieving content more efficient, and pages could be changed on the fly. With WebKit’s support for custom CSS, I could make the interface look realistic and comprable to a pile of custom-written views.

I quickly realized that it wasn’t all that easy to implement “drill down” behavior with a UIWebView. Early on, I ruled out the possibility of creating a mock navigation bar in HTML. Since Safari on the iPhone doesn’t support “position:static” or “position:fixed” CSS tags, there was no good way to make the bar sit at the top of the screen. I decided that a native UINavigationBar would be more practical and provide a better user experience. However, UINavigationController was built to use separate controllers for each layer, and doesn’t worry about freeing up memory when the stack of controllers gets big. I thought it was important that a maximum of eight UIWebViews were in memory at once, since Mobile Safari obeys that limitation and because pages could potentially be very large.

I tried several solutions, and finally created a custom DrillDownWebController class with a manually managed UINavigationBar to handle the interface. The class maintains a “stack” of DrillDownPages, with each page representing a single layer in the drill-down hierarchy. It can be a root level controller, or it can be loaded into an existing UINavigationController. When it appears, it silently swaps its parent’s navigation bar with it’s own.

The DrillDownPage is a wrapper for a UIWebView that acts as its delegate and provides higher-level access to important properties of the page, such as it’s title. When the user clicks a link in a web view, a new DrillDownPage object is created and it begins loading the requested page in an invisible UIWebView. The controller displays an activity indicator in the top right corner of the navigation bar, and slides in the new page when it finishes loading. All the other pages in the page “stack” are notified that their position in the stack has changed.

The notification step is important, because it allows the Page objects to

The Best WordPress Site Ever?

So I accidentally clicked an ad this afternoon and stumbled across Ecoki.com, an online community for eco-friendly folks. I hadn’t even scrolled half way down their home page when I found myself thinking: “What was this built in?” Ecoki is quite possibly the best designed wordpress site I’ve ever seen. I had to look at the page source to figure it out.

http://ecoki.com/

It looks like it’s a completely custom template. Must have cost a fortune… Great look though!

4 1/2 stars! But you can’t compete with sex…

Layers finally dropped off the list of top paid entertainment apps on the App Store this week. It’s not all that surprising – for such an expensive app, I was happy to see it on there at all! I think this screenshot pretty much sums up my feelings, though. Of the top 100 entertainment apps, 13 of them are some variant of “hot babe” apps. Looks like you just can’t compete with sex.

The Best WordPress Site Ever?

So I accidentally clicked an ad this afternoon and stumbled across Ecoki.com, an online community for eco-friendly folks. I hadn’t even scrolled half way down their home page when I found myself thinking: “What was this built in?” Ecoki is quite possibly the best designed wordpress site I’ve ever seen. I had to look at the page source to figure it out.

http://www.ecoki.com/

It looks like it’s a completely custom template. Must have cost a fortune… It seems like there are only four staff authors on the site – so they’ve been pretty busy!

The NDA is gone!

Apple has finally removed the NDA from non-beta iPhone software, so the community can finally come out of hiding. I can’t wait to actually – you know – talk to people! I’ve got a few things in the pipes for NetSketch, and hopefully I can post some tutorials and info here. Then this blog might actually be useful…

- Ben