Friday, April 11, 2014

Tachikomas Twerking

I made this in Cinema 4D.

The soundtrack's Night Is On My Mind.

Tuesday, April 8, 2014

The Internet Will Never Be Secure, Ever

Because it's written in C.

To quote yaakov34 on Hacker News:

There was a discussion here a few years ago about memory vulnerabilities in C. Some people tried to argue back then that various protections offered by modern OSs and runtimes, such as address space randomization, and the availability of tools like Valgrind for finding memory access bugs, mitigates this...

My opinion, then and now, is that C and other languages without memory checks are unsuitable for writing secure code...

This vulnerability is the result of yet another missing bound check. It wasn't discovered by Valgrind or some such tool, since it is not normally triggered - it needs to be triggered maliciously or by a testing protocol which is smart enough to look for it (a very difficult thing to do, as I explained on the original thread).

The fact is that no programmer is good enough to write code which is free from such vulnerabilities. Programmers are, after all, trained and skilled in following the logic of their program. But in languages without bounds checks, that logic can fall away as the computer starts reading or executing raw memory, which is no longer connected to specific variables or lines of code in your program. All non-bounds-checked languages expose multiple levels of the computer to the program...

We can't end all bugs in software, but we can plug this seemingly endless source of bugs which has been affecting the Internet since the Morris worm. It has now cost us a two-year window in which 70% of our internet traffic was potentially exposed. It will cost us more before we manage to end it.

To clarify, if you're not familiar with C, you might not realize that it basically makes all of its memory available to any code at any time. It's kind of a counter-intuitive design choice.

So the comment is saying the problem with Internet security is that the entire Internet, and the entirety of every Unix-style operating system, is based on the foundation of a programming language which allows you to read arbitrary data out of memory.

This is indeed a very big problem. If you want security, you probably don't want an entire online economy built on C.

The only thing I don't like about the comment is that the last few sentences:

we can plug this seemingly endless source of bugs which has been affecting the Internet since the Morris worm. It has now cost us a two-year window in which 70% of our internet traffic was potentially exposed. It will cost us more before we manage to end it.

Can we, really? How? Who the fuck is going to pay for that?

Here's what we would have to do first: rewrite every Unix-style operating system from the ground up in a new language. And that's the easy part. Afterwards, we would have to make this new operating system, or group of operating systems, so easy to use and install that everybody who already has stuff installed and running out there in the big wide Web would find it cost-effective to start over with a new operating system. We would then have to repeat the process for every programming language written in C.

What are the fucking odds? We live in a world where COBOL and Fortran became hot languages in 1999 because of the Y2K bug.

It's theoretically possible that a really, really savvy government could fund such a project, but that's not the world we live in. This is the government we actually have:

Unixes are old, and deep, and complicated. It is much easier to understand Unix (and friends) through the lens of an archaeological dig than it is to build a mental model of it as a coherent, well-designed piece of engineering. It has layers and layers and layers of systems which all interact successfully, without being designed to be aware of each other in any sense at all. The whole reason Unix works is because it has this very simple "everything is a file" paradigm, which enables lots and lots of small, highly specific programs at the system level to interact with much larger and more complex software at the application level, using very simple and manageable interfaces.

When it comes to security, though, because everything's built on C, it's less like "everything is a file" and more like "everything is a fail." It is utterly inevitable that more problems like Heartbleed will occur. Our entire online economy is built on an inherently unsafe foundation.

Long story short, the only way to get a secure Internet is to completely rebuild every part of it from scratch. That's just not how technological advances occur. People build on the stuff that is already there. The team behind Go probably want to do this kind of total Internet rebuild, and replace HTTP with SPDY while they're at it, but not even Google has enough Imperial stormtroopers to pull off that kind of coup.

Sunday, March 23, 2014

Mysterious Shenanigans

Friday, March 21, 2014

Remote Mentoring For Fun And Profit

I've done paid and semi-paid programmer mentoring for a while, and I'm going to tell you what it's like.

In 2010 I ran a coaching service, basically mentoring other programmers for an hourly rate. It made good money, but I shut it down for two fairly capricious reasons. First, I had several people who never showed up for their sessions, but continued paying me, and I didn't have any policy in place to manage things if these folks suddenly said, "OK, I've paid you for ten hours, you have to deliver now." I think the odds of that happening were actually very, very low, but I hate overlooking things like that. (My projects are never quite as haphazard as they seem.)

More importantly, I had planned to do a new mini-business every month in 2010, and the success of the coaching service was distracting me from that exploratory goal. This was actually part of a larger system. In 2009, I set out to create a new site every month, the goal being to a) get better at deploying stuff, b) get better at finishing stuff, c) explore a bunch of random languages and frameworks, and d) get some skill at product design.

That last part definitely worked out; I'm glad to say that one of my 2009 mini-sites got a mention on Mashable, as well as receiving what might be the ultimate hacker compliment, a copycat in the iOS App Store. Anyway, my goal in 2010 wasn't actually to make a living off my entrepreneurial experiments, although this side effect was certainly very nice, but rather to explore a large space of possible options, and learn about them. The coaching program made things too easy, so I killed it.

More recently, I've taken on two students. One of them was an Internet friend who successfully pestered me to re-open the coaching program, just for them; the other was a friend from actual physical space with at least 15 years of experience as a drummer (including a spot on Conan). I offered him Rails lessons in exchange for drum lessons, and he took me up on it. This actually started out as an in-person thing, but switched to remote when I moved to New Mexico.


I did virtually zero marketing to launch or perpetuate this business. I blogged about it when I launched it in 2010. I'm blogging about it now. I think that's it.


In terms of money, I have people pay ahead of time with PayPal. With my drum teacher/Rails student, we do a drum lesson and then a code lesson. I tried doing the same thing with a machine learning expert — machine learning lessons in exchange for electronic music production lessons — but it didn't really take off.

When I had a whole group of students, I had a virtual assistant set up the scheduling. I didn't go with some company in India or whatever, as Tim Ferriss and others might recommend. I knew a smart operations person from a previous employer, I knew they were smart and capable, and so I had them handle it for an hourly rate. They did great, and it was very simple and painless. These days, since I only have two students, I just set up scheduling myself.

I don't use tmux or screen, which may surprise people. I'm interested in doing that, but in practice, they never seem worth the effort. I used to use Skype; although there's no doubt Skype sucks in some ways, virtually everybody has it, and it got the job done just fine. These days, I still use Skype for video chat, but I use ScreenHero for screen sharing. It's not perfect, but it's a lot less imperfect than Skype, and as a bonus, the NSA probably isn't listening in quite as much. I also use GitHub a lot, for obvious reasons.


In 2010 I'd mostly just have the student show me whatever they wanted to show me, and/or ask me whatever questions they had. These days, since I have a much smaller group of students, I prepare material ahead of time more often. In practice, I'm either pairing on code, or giving advice about career moves and/or product design. (A couple of my students in 2010 had startups and wanted feedback on usability and features.)

Hearing people say "pairing" when they mean "mentoring" or "explaining" is one of my Silicon Valley pet peeves — I've found that people in the Valley are often reluctant to learn new ideas, but eager to use the names of those new ideas as synonyms for things that they already do — so I want to point out that it's usually more lopsided than pairing's meant to be in the traditional sense. Also, to be fair to my students, the degree of lopsidedness varies from case to case. And one of the most enjoyable things about doing this is that if you have a student for a long period of time, you get to see the pairing become less lopsided as time goes on. In some cases, I've seen it become much less lopsided, and that's been great.

Anyway, I've also just demo-ed the process of writing Clojure, for a person with a C background who'd done extensive reading in modern web dev, but had no actual experience with it. And for my drum teacher/Rails student, I had him start by writing unit tests for essential pieces of the Ruby language like Array and String.


I'd say any programmer who can teach something, should. I think that's actually a large number of programmers. My guess is that most programmers reading this will underestimate the degree to which they know things that they can teach. I'm not saying that teaching is a universal skill, but rather that the culture of open source tends to over-valorize a small community of "experts."

The minute you get beyond rank newbie status, you have something you can teach. You may have to learn the skill of teaching, but giving talks at conferences and user groups is a great way to build that ability, and it's actually very important to do that anyway. It's no good if you can write great code but you can't explain it to anyone.

Anyway, I say it's worth doing because typically our work is somewhat dry and impersonal, but teaching generates a lot of warm fuzzies. It's fantastic when you see a student succeed (especially when they succeed by following your advice). Also, the more you help somebody when they get started, the more logical it is to assume that they're going to help other people too, once they get a bit more senior.

Any time I see a bit of open source drama or similar group dysfunction — which, let's face it, is whenever I open Twitter — I feel glad that I've done a little bit to make the world of hacking a bit more helpful and a bit less bickery. After all, I only know a few ways to deal with open source drama: ignore it, make fun of it, or find some way to make it at least a little less commonplace. The last method seems the best.

Go And Do Likewise

I think most people who read this will have the ability to do the same thing, and I definitely think you should. Do it for money, do it for free, do it for the warm fuzzies or for political reasons, but one way or another, you should take time to mentor other programmers. It's good shit.

Wednesday, March 19, 2014

Idea: Drum-Based Dance Music Performance Context

I've got a design I like, and I want to explain its motivations.

I'm from Chicago, and in the 90s I went to a lot of raves in San Francisco. I was more of a nerdy hermit in Chicago (and am more of a nerdy hermit today), but I remember being surprised in San Francisco by the way everybody danced facing the DJ. In clubs in Chicago, everybody danced facing each other, which is to say, pointed in a thousand different random directions.

The dance music history Last Night A DJ Saved My Life notes this distinction, and points out that in the early days of DJing, DJs were not even visible from the dance floor. Dance music venues began putting the spotlight on the DJ (literally) once DJs became names capable of selling tickets and bringing people in the door.

So what I saw in the 90s in San Francisco was part of a historical continuum, and I think it's becoming more and more the norm. I remember it being a major issue in some corners of the underground dance music scene, because people felt that "superstar DJs" were making themselves the focus of events whose real (and more noble) purpose was to create a feeling of unity and communion throughout a large crowd.

From that point of view, things have gotten worse:

It's hard to think of a more unexpected turn of musical events than EDM's commercial triumph. For decades, the US remained impervious to the charms of the house music and techno that had been invented under their noses in the 80s. Then suddenly, nearly a quarter of a century after the rest of the world cottoned on, dance music has become very big business indeed.

From the outside, it's inexplicable. Perhaps examining the work of Joel Zimmerman can shed some light. As Deadmau5, he is not only arguably EDM's biggest star – as evidenced by a recent Rolling Stone cover – but also the scene's self-appointed spokesman. He took Madonna to task for the scarcely imaginable crime of mentioning drugs at a rave, suggesting it was akin to "mentioning slavery at a blues concert". It was redolent, he said, of the days when "a dark veil" hung over dance music, before he and others had "taken EDM so goddamn far". By this "dark veil" period, he presumably meant the 35 years when dance music had to content itself with merely providing a glorious, euphoric voice for disenfranchised minorities, being a genuine countercultural phenomenon, repeatedly revolutionising music and changing the face of popular culture. This, of course, was before it found its true, noble calling: soundtracking Las Vegas pool parties and providing music for gurning frat boys to mosh to.

I love this, because I think Deadmau5 is basically insane to consider himself an improvement over Chicago house. However, I don't think it's just commercialization and egotism which focuses people's attention on the DJ, as opposed to each other. From a commercial standpoint, DJs fill venues, and from an artistic standpoint, the performer supplies the music, and with it, a lot of the overall vibe of an event.

There's an inherent conflict here, between an event which is all about some superstar DJ, and an event which is all about communion, about uniting the dance floor as one. So I set myself a design goal: imagine what it would look like to create a space where a dance music event could balance these competing dynamics with grace and harmony.

At the same time, dance music's grown tremendously in sophistication, both as an art form and in technological terms. Back in the days of vinyl turntables, the core technical skills of DJing were beatmatching and handling the mixer levels. Even then, these skills were trivial, and nothing compared to the much more important (and more ephemeral) skills of track selection and reading a crowd. But today, computers can easily handle both beatmatching and mixer levels automatically, leaving many DJs with nothing to do but stand around and pump their fists in the air. This is especially problematic in the case of DJs whose track selection is predictable, and who don't bother to read a crowd.

Software like Traktor and Ableton Live makes it possible to combine live instrumental music, in the classic sense, with sample playback, looping, and all the postmodern benefits of performing music by playing recordings, rather than individual notes and sounds. One very interesting way to do this is by playing electronic drums which send MIDI signals instead of, or as well as, making sounds.

Here's a demo by the drum technician for Netsky's recent tour:

Say for the sake of argument you have a performer like that delivering dance music. The risk of an event which places all of its emphasis on the superstar performer, and none at all on creating a sense of unity on the dance floor, is actually enormous.

I think I have a design which reconciles these competing interests.

The secret sauce: user-generated visuals and software-enabled audience participation.

By user-generated visuals, I mean that in addition to using the MIDI signals from an electronic drum pad or drum kit to create music, you can also use those same signals to drive visualization software as well or instead. Although this is just a thought experiment, I've built a very simple prototype of drum-triggered visuals. This is very much a work in progress, but you can see a rough demo video here:

(The code is on GitHub.)

What I envision is a raised drum platform with video screens and concert lighting attached. The video screens plug into software which accepts MIDI input from the electronic drums, so that the musician also acts as a VJ, or in collaboration with a VJ (who could choose to process the MIDI input through any visualization software which accepts MIDI).

Throughout the room, you also have a drum circle of MIDI-enabled drums, which do not create any sound, but which do drive visualizations on screens nearby. People in drum circles always look cooler than they sound; this system leverages that fact. These drums would go on smaller raised platforms, probably enclosed in pods for security and to simplify managing them during the event.

These are audience participation pods. They enable audience members to interact with the music and the VJ software, and thereby to also interact with each other.

Here are some 3D mockups. Please pardon their primitive nature, and see them as diagrams to help the imagination.

Drummer in the foreground, audience participation in the background. This diagram represents a drummer seated at an electronic drum kit (specifically a Roland Octapad), on a raised platform with video screens attached to the platform, and strobelights attached above the screens. (The video screens are displaying screenshots of generative art made in Clojure with Quil, which is essentially Processing.)

The drummer's platform is actually a turntable with a very slow rotation speed; something like one rotation per hour. This enables the drummer/DJ to see all of the crowd around them, rather than just the people in the "front row." (Since this design sits at the center of a room, rather than one end, there isn't a front row, more a set of concentric rings.)

Audience participation close up. An audience member stands before a single electronic drum. Above their head, there's a pair of video screens. One screen faces the audience member and shows them the visuals that their drumming creates; the other screen faces the opposite direction so everybody else in the crowd can see.

Audience participation in the foreground, star performer in the background. Additional audience participation pods in the far background.

Overhead view. The empty space in between the drummers would be filled with dancers.

This design achieves a nice hybrid between a rave and a gigantic Guitar Hero party, while showcasing a more sophisticated and interesting level of musicianship which can nonetheless include sampled beats, loops, and all the up-close-and-personal crowd-reading that can make for a truly amazing DJ set.

The audience participation pods could also be a new revenue source; just charge people $X to jump in for an hour. The benefit here is it'd make it easy to organize who goes in them, although the risk is that it further commercializes a fundamentally spiritual experience. Another option could be to put ancillary performers in the drum circle pods, i.e., backup percussionists, kind of like go-go dancers but less sexist and more friendly.

Learning C & Clojure In 2014

A tweet got me started on the idea.

(I started with BASIC when I was 11, and began doing all kinds of random web stuff long before I ever learned Lisp, but better late than never.)

To be fair, I basically already learned Clojure last year, in the sense that I took a workshop on algorithmic music in Common Lisp and translated what I was learning into Clojure so I could use Overtone and Quil to make MIDI-controlled generative art and Markov chain basslines. But translating entry-level Common Lisp into entry-level Clojure is pretty damn far from really learning Clojure, so that's still happening in 2014.

I built this with Clojure last year.

I normally scoff a little at the "learn a new language every year" thing, for a few reasons. First, there's a risk there of creating language hipsters, and that theoretical danger is quite obviously often realized. (You know who you are.) Second, the whole idea came from the Pragmatic Programmers, who are book publishers after all. Can we say "conflict of interest?" It doesn't completely invalidate the idea, but it's still pretty sensible to take "learn a new language every year" with a grain of salt when it's coming from a book publisher.

Third, in every programming language that I know well, I can very quickly spot the difference between somebody who's been using the language for years vs. somebody who's been using it for a year or less. It might take more than a year to learn any given language. You can write a lot of things in JavaScript before you ever have to give any serious thought to bind, call, apply, or this.

I first built something with Clojure in 2008 or 2009, so it's safe to say I've spent more than a year learning Clojure already, and I'm pretty damn sure it'll take me more than a year to get anywhere serious with C, but that's fine. I'm learning C because it's the foundation for Objective-C (along with Smalltalk), and because I ultimately would like to be able to build synthesizers for iOS with Objective-C. I'm pretty certain that if/when I do that, I'll want to drop down to C for some of the heavy lifting. Most of the serious literature on writing synths uses C or C++, and although I haven't been able to find a link for this post, I once saw John Carmack say that in developing Infinity Blade for iOS, they saw massive performance gains (something like 20x) just by abandoning NSString in favor of traditional C strings.

Another major reason I'm learning C and Clojure: the evolution of JavaScript. I mainly work in JavaScript and Ruby, and while Ruby's not changed very much in the 8 or 9 years I've been working with it, JavaScript's morphed into a completely different beast. Today, if you want to use JavaScript to its fullest potential, you're getting into WebGL, and that requires a very C-like mentality. But if you want to write clean, readable JavaScript, you're also getting into functional programming. So I think this work with C and Clojure will make me better at JavaScript.

There's yet another reason:

This excellent piece of work was put together by friends of friends, who are looking for a CTO. This was the first "we're hiring!" email I've seen in a long time where I just wasn't qualified at all -- background required in electrical engineering and embedded systems -- and it was also the most appealing. This prompted me to look into similar job postings, and all of them require Processing, which is easy, and openFrameworks, which requires C++, which is of course built on C.

(In fact, pretty much everything is built on C. If you're interested in hacking on Ruby's internals, or Python's, you need C. If you want to hack on the V8 JavaScript engine, or WebKit, you need C++. Come to think of it, I saw another interesting job ad which required a C background - Mozilla's hiring a research engineer to help them build Rust.)

I've also seen some very inspiring work in Clojure, and everything Arduino runs on C.

Anyway, just to end with some useful tips, I haven't gotten into the classic text yet, but I've found these books useful for C, and I personally like The Joy of Clojure over either Clojure Programming or Programming Clojure, although I've gotten some mileage out of both of those, despite their repetitive naming scheme.

(Actually, when it comes to books, Pat Shaughnessy's Ruby Under A Microscope is another major reason I got curious about C.)

Tuesday, March 11, 2014

I'm Learning 3D Graphics with Cinema 4D

I've made a few things in the process.

Here's a TIE fighter:

Here's a bunch of them, plus an Imperial Star Destroyer:

Here's a couple Tachikomas with a cheeseball 1990s cyberpunk backdrop:

Here's some ravey visuals, plus soundtrack:

And here's some quick spirals:

Thursday, February 20, 2014

Jim Weirich: Rest In Peace

I met Jim at several conferences, indeed in 2008 I think we did three of the same conferences in a row. It was like we were travel neighbors. I think I first met him at a Pragmatic workshop where he taught me and a room full of other programmers how to do TDD. He was warm, generous, and positive, and a great example of the old-school Ruby attitude. He will definitely be missed.