Quantcast
Channel: Archagon Was Here
Viewing all 44 articles
Browse latest View live

Gathering the Goods, Part 2

$
0
0

Caption goes here.

My first run at a packing list was suprisingly solid: everything worked pretty much as intended and there wasn’t anything I desparately missed or needed during my US trip. However, I misjudged the needs of my travel bag:

  • I very rarely used my Silver Streak bag as a backpack, and when I did, it was just too heavy for prolonged use. (The total weight of all my equipment was almost 50 pounds.) As a result, I mostly ended up detatching the backpack and carrying the main bag on a shoulder strap, which was very uncomfortable.
  • All my important equipment rarely left my backpack. I hardly ever used the expanded compartment in the main bag for anything other than a few spare parts, souvenirs, and my Wacom tablet.
  • I gathered a lot of food ingredients during my travels, and to avoid a mess, I carried them in a separate cloth bag. There was never any need to store perishables in the main bag.

As a result, I have replaced my Silver Streak with an Eagle Creek Switchback 22 (on sale). The Switchback does almost everything the Silver Streak does, but adds one vital feature for city use: wheels! Contrary to most of the advice I found on travel blogs, I desparately wished for wheels many times during my trip. (Maybe this would be different if I were spending long miles walking on unpaved roads, but the fact is that most of my traveling happens in modern cities and not the countryside.) The backpack half of the bag now serves as the store for all my tech equipment, while the main bag keeps all my clothes and accessories. Among other things, this consolidation means that I no longer have to move things from bag to bag when I’m leaving the main bag in a locker or on a bus/train/plane. Space is more tight than in the Silver Streak, but everything still fits. (At the moment, I’ve decided to roll up my jacket and clip it onto the handle, since cramming it in takes a bit too much effort.) The Switchback offers great flexibility in regards to transport: backpack zipped onto the main bag, backpack threaded onto the carrying handle, or backpack and main bag separate. (So far, I’ve found the most convenient option to be zipping the backpack onto the main bag and using the wheels. You can walk many miles like this without a problem.) There are many other details that make the Switchback feel like a high quality product, from the multipurpose outside straps to the secondary handle position all the way down to the stitching. It’s clear that a lot of thought was put into the design. One minor annoynace is that the backpack straps have to be taken off the backpack to be used with the main bag, but since I’m mostly set on using the wheels, this isn’t a big deal for me. Before I settled on the Switchback, I also tried the Osprey Meridian, but I was simply not able cram all my stuff in!

Caption goes here.

I also made a few other changes:

  • An Eider Roc de Chere jacket in blue (also on sale). I made a bad call on my previous jacket: it was cold, scratchy, and most definitely not waterproof. I briefly considered applying more wax, which is what you’re supposed to do to make a waxed cotton jacket more water resistant, but I decided it wasn’t worth my time. Instead, I set out to find a no-compromise jacket this time around: waterproof, warm, comfortable, usable in all cold weather conditions, featuring a detatchable hood, and not horrendously ugly. (Most of the widely-recommended technical jackets and rain shells fell into that category for me.) I tried on a large number of different jackets from REI, Nau, Carhartt, Patagonia, Marmot, Eider, and Helly Hahn. Close contenders were the REI Montour (green is not my favorite color), the Carhartt Grayling (fit perfectly, but material felt plasticky), and the Nau Temp (super comfy, but too warm and the fit wasn’t right around the waist). The Roc de Chere’s material is light enough for use even in warm-ish weather, and the color and material are fairly unique. Still, I wish it had buttons down the front! Is jacket envy a thing?
  • A pair of Adidas Duramo slides. The Nike Benassis I had earlier were very comfortable, but the mesh lining took forever to dry out, to the point where, one, I simply couldn’t shower in them if I had to leave the next day, and two, they actually had a propensity to mold! Their everlasting wetness also ensured that they could never double as slippers. Despite being made of a plastic-like material, my new Duramos are shockingly comfortable and dry out pretty much immediately. Somebody on the internet described them as “prison sandals”, but no worries: all you have to do is lower your standards and you’ll be going out in public in no time!
  • Two cloth bags: one for food (as mentioned earlier) and one for carrying dirty laundry around. They barely take up any space.
  • A very small tea cup from DAVID’s Tea. I keep it in the same bag as my socks and glasses. Purchased in Toronto!
  • An Anker Astro E5 battery charger. Way too often, I would leave my house without having charged my phone and then end up desparately looking for a power outlet by the end of the day. With this device, I always have some extra charge on me. (Quite a bit of extra charge, actually.) It’s also useful for charging all my small electronics when I’m too far from an outlet, or when I want to keep my belongings close to me in a hostel. An essential gadget.
  • A tiny Lightning cable for my keychain. It bends in a way that allows me to hook up my iPhone to the aformentioned battery charger in my jacket pocket without having to deal with cable spaghetti.
  • A small multitool, consisting mostly of pliers and screwdrivers. No knives, so it’s (nominally) TSA-safe.
  • A ShedRain Windjammer umbrella, based on last year’s recommendation on The Wirecutter. London Fog’s umbrella was awful and broke after about a week of use.
  • A long stainless steel mesh tea infuser. Aside from a little metal tag which I bent out of the way, this thing fits perfectly into my Klean Kanteen and allows me to brew loose-leaf tea with plenty of room for leaf expansion. It’s the only one of its kind that I was able to find! Every other teaball is either tiny or doesn’t fit into the Kleen Kanteen.
  • A Logitech G602 wireless mouse. It works well enough and eliminates an annoying stretch of cable. Sensitivity, tracking, and latency are great for gaming use. (In fact, I saw some measurements that put the latency on par with wired mice!) Battery lasts a month. Range is really iffy, but I’m only planning to use it next to my computer, so that’s not really an issue.
  • A set of collapsible compact chopsticks. I occasionally find myself in situations where I could really use some sort of utensil, and these chopsticks get the job done in most situations. (Plus, they’re great for munching on small, greasy foods like popcorn or chips!)
  • A Pilot Metropolitan fountain pen— just because it’s sweet. :)

Already, HoboPack Mark II feels so much more useful and refined. Onward to more adventures!


Old iPhone Tricks: GPS Tracker

$
0
0

This is the time of year when a lot of people are upgrading their phones. If you are, don’t chuck your old phone just yet!

Here’s a thing you might not know about your iPhone1: the GPS unit works even without a data connection. I was confused for a long time about what “assisted GPS” actually meant. My understanding was that it was impossible for an AGPS phone (which the iPhone is) to acquire a GPS signal without a data connection. Turns out it works fine — you just have to wait a bit longer for the phone to find a satellite.

So here’s what I do with my old iPhone 4. Whenever I set out for a walk, I load up an app called myTracks and hit the Record button. Then I chuck the phone in my bag and forget about it. At the end of the day, I retrieve the phone — more often than not with plenty of charge remaining — and stop the recording. Finally, I can connect my phone to my computer and export this data as a kml file (among several others) to use with Google Maps:

(Incidentally, setting this viewer up was surprisingly easy. First, I exported the kml data from the app — in this case, EasyTrailsLT, which I’m also trying out. Next, I went and got a Google Maps Javascript API key. Finally, I wrote a quick Jekyll plugin to replace kmlasset tags with a bit of boilerplate Javascript code that calls Google Maps. End result: I can drop my kml in a folder, add a tag, and have the map show up magically in my blog post. This is just a small example of how Jekyll makes doing data-heavy blogging a lot more simple, which I’ll delve into in later posts!)

Most of the GPS apps I’ve looked at still support iOS5, so your phone doesn’t even have to be a recent model. I’d still use my 3GS for this purpose if the battery was up to snuff.

People always panic when Google or Apple slips up and caches a bit of your location data, but I feel the opposite. With an old iPhone and just a bit of dilligence, I can create a map of everywhere I’ve walked in the world!


Jekyll assets created over the course of this exercise

  • kmlasset_tag.rb— A Liquid tag that feeds a kml file url into a Google Maps applet.
  1. Since I only have Apple hardware at the moment, I have no idea if this also applies to Android. Sorry!

The Trials and Tribulations of Writing a 3rd Party iOS Keyboard

$
0
0

I recently released my first commercial project: Translit Keyboard, a 3rd party keyboard for iPhone that lets you transliterate Latin characters into Cyrillic and some other alphabets. When I was first investigating this problem, around the time that iOS 7 came out, I discovered that I could implement an elegant solution in OSX using the lesser-known Input Method Kit. My program sat between the keyboard and text field; the framework provided me with each raw character as it was entered, and I could either pass it along with the default behavior, or instead take control of the output and send off whatever I pleased1. Sadly, iOS was off-limits: since 3rd party extensions weren’t even on the radar at the time, there was nothing anyone could do to enable this sort of behavior system-wide. The only solution was to make a nice app with a special text field that you could copy and paste from — too clunky for rapid-pace tasks like messaging.

When iOS 8 was announced and unexpectedly blew open the doors on extensions, my interest was immediately piqued. Although, on initial glance, the keyboard APIs were rather crude — you were given a gray box at the bottom of the screen and practically nothing else — this gave me a roundabout “in” to making input methods work on iOS. What if I were to recreate the system keyboard from scratch? I could make it behave any way I wanted!

And so, Tasty Imitation Keyboard was born. If I may take a moment of vanity, the keyboard is very solid: on most phones, it looks and types very much like the system keyboard, minus the landscape side buttons on the 6 and 6+. (iPad support isn’t too high on my priority list, but it is functional.) This was a satisfying project to work on, giving me great insight into Swift, autolayout (since removed), Core Graphics, and a bit of Interface Builder. But it also gave me experience with something that I had yet to encounter in my sheltered iOS development existence: the horror of undercooked Apple frameworks and tools.

Below are as many of the problems I encountered during development as I can remember. I have high hopes that these will be fixed over the next few years, but until then, perhaps this article will help some poor programmer in dire straits!

Behavioral Bugs

Prevalent in Apple land is the expectation that software should adhere to the conventions of its host OS. We like our apps to use common widgets, behave consistently, react smoothly, and acknowledge the current design trends; very much unlike the “anything goes” philosophy that Windows and Android have often adopted. So it surprises me that 3rd party keyboards are even a thing on iOS. To take such an essential, constantly used, frequently visible UI element and put it at the mercy of developers — many with their own twisted ideas of how a keyboard should look — seems like a very un-Apple-y thing to do.

But it gets worse. 3rd party keyboards on iOS don’t just have the opportunity to look alien; they act alien, and this is something that programmers can’t guard against. Consequently, all the 3rd party keyboards currently available on iOS feel like some sort of jailbreak hack.

That Pop-In

This is the first thing you notice. Unlike the system keyboards, which smoothly slide in from the bottom of the screen, 3rd party keyboards just… pop into existence. (Most of the time. Sometimes they slide in, though it’s hard to predict when this will happen. There are also some scary bugs associated with this particular entrance. More below.) Worse, if you’re switching from a default keyboard to a 3rd party keyboard, the entire keyboard disappears for a second while the 3rd party keyboard loads up.

Invalid State

And guess what? Sometimes you’ll have to wait forever!

On occasion — frighteningly frequent occasion — I’ve seen the keyboard simply fail to appear, even after waiting for many seconds. When this happens, it’s hard to tell when the keyboard is going to come back. Will it happen when you hit the home button and re-open the app? (Sometimes it doesn’t.) When you switch to another app? (Sometimes it doesn’t.) One of the few sure-fire ways to fix this problem is to force-close and re-open your current app, which isn’t always possible. (Spotlight.) And until then, have fun not being able to type!

There are other fun states that your keyboard can get into. For example, sometimes the keyboard view will show up, but never actually load the keyboard:

And the fun is not just limited to 3rd party keyboards! If you hit the globe icon on a system keyboard, sometimes the button will run away from under your finger — and stay that way for a long time:

What Is the Text Even Doing?

The text entry behavior of 3rd party keyboards is a bit erratic. For example, try using SwiftKey (or any other 3rd party keyboard) to type out part of a saved URL in Safari — to the point where it starts to autocomplete the rest — and then press delete. You’d expect the highlighted part to disappear, right? No: what actually happens is that the cursor simply moves to the left, leaving the autocompleted part intact and still highlighted. This does not happen with the system keyboard.

Settings? What Settings?

With 3rd party keyboards, you can’t access any of the keyboard preferences that the user has set, including auto-capitalization, automatic spaces after periods, disabling the shift key, and playing clicks when typing. As a result, you either have to assume a default, or re-implement the settings entirely.

(Fortunately, accessibility settings like “reduced transparency” are still available.)

Broken Apps

Some apps are downright broken with 3rd party keyboards.

Editor’s note: None of the problems below seem to happen anymore with Google’s recent Maps redesign. Still, the fact that they were happening at all is troubling, since there’s nothing an app should theoretically be able to do to influence keyboard behavior.

In Google Maps, a number of strange behaviors happen with the search box. First, it’s one of the few text fields I’ve seen that have custom keyboards slide back in after they’ve been dismissed, rather than just popping in as described above. Why this particular text field and not others? I have no idea. Furthermore, if you track the UIInputViewController lifecycle during this process, you’ll see that even though viewWillAppear is called correctly when this happens, viewDidAppear is not. This is especially odd given that the keyboard still appears to go through init.

Next, if you tap on the search field in portrait and then rotate to landscape, your keyboard will suddenly expand almost to the top of the screen.

How about another app? Hilariously, in the App Store app, if you try to write a review using a 3rd party keyboard, you’ll find that the page turns entirely blank!

Reloading the Extension

If you’ve recently updated a 3rd party keyboard, your apps might still have the old keyboard bundle loaded in. This is fixed by force-closing and reopening the app.

Mystery Crashes

I have seen numerous mystery crashes affecting their 3rd party keyboards. Sometimes they just… close for no reason. And you don’t even necessarily see it in your crash logs!

Mystery Glitches

3rd party keyboards can sometimes exhibit bizarre graphical glitches. For example, this one is easy to consistently reproduce.

Missing Features

There are some features that customers are used to — or even find essential! — in their keyboards, but that we cannot currently provide. One is autocorrect, including the red-squiggle variety. Another is physical keyboard support.

Full Access Is Almost Necessary

There are two ways that 3rd party keyboards can run. By default, the keyboard cannot do much of anything outside its own sandbox. But if the user goes into their settings and manually flicks a carefully hidden switch, they can enable “full access” for that particular keyboard. In the process, they are greeted with this terrifying popup:

This totally makes sense if there’s a chance that your keyboard could send your precious keystrokes over the net. Unfortunately, there are a number of other vital features that require getting past the full access barrier. Sharing data between your keyboard and your container app. Sharing data between multiple keyboards in a single app. Having a settings bundle. Using in-app purchases with your keyboard. Playing sounds, for crying out loud. Some of these are reasonable since they could allow private data to escape out of the sandbox in roundabout ways, but it’s causing many developers to strongly encourage their users to enable full access.

I wanted to be as secure as possible in my app, so I worked around these problems in various ways. To compensate for the demo aspects of IAP, I released my app for a fixed price, and then released a Lite version with a useless transliterator (Old Slavonic) for trial purposes. Instead of having a single keyboard with a language selector accessible from the container app, I opted to have multiple keyboards instead. The one feature in my keyboard that still requires full access is sound; this defaults to “off” in the in-keyboard settings and offers users an explanation on how to enable it.

The Reviews; Oh, the Reviews!

As you would expect, customers have no idea that any of these behaviors are caused by the OS. Just look at these reviews for various 3rd party keyboards:

You’ll find it difficult to find a 3rd party keyboard with more than 3 stars. (And can you blame the customers? After seeing these behaviors, I certainly wouldn’t switch to a 3rd party keyboard full-time!)

What can we do about this? Nothing! We have to pay for these bugs with our precious developer cachet.

Programming Problems

In addition to the above hassles, us programmers have other things to worry about. The framework that Apple has given us is… rough around the edges.

Changing the Height

Unlike the system keyboard, your custom keyboard is confined to the rect it was created in. There’s no drawing outside the box and no adding transparency. This becomes a problem if you want to add popups to your keys like the ones in default keyboard. How do you handle the top row when you can’t draw over the top?

In the early betas, you couldn’t do much at all. Your best option was to either draw sideways popups or just do nothing. Closer to release, we got the ability to expand the height of the keyboard view. Sadly, the implementation feels like an afterthought.

Apple specifies in the docs that you can change the height of your keyboard by adding an autolayout constraint with a constant value to the keyboard view. However, if you actually follow these instructions, you’ll find something like this in your log:

2014-11-08 17:17:09.741 Keyboard[8079:3155417] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want. Try this: (1) look at each constraint and try to figure out which you don't expect; (2) find the code that added the unwanted constraint or constraints and fix it. (Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSLayoutConstraint:0x618000096350 'UIView-Encapsulated-Layout-Height' V:[UIInputView:0x7fe0be909690(216)]>",
    "<NSLayoutConstraint:0x608000093b50 V:[UIInputView:0x7fe0be909690(246)]>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x608000093b50 V:[UIInputView:0x7fe0be909690(246)]>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKit/UIView.h> may also be helpful.

Adding a constraint to the keyboard view doesn’t set the height; it rams against the existing autolayout height constraint created by the autoresizing mask, and overrides it in most cases. This is important: if you add the constraint before the view appears on the screen (this is specified in the docs), or in certain cases where your keyboard view doesn’t contain any subviews with autolayout constraints (this is not), your height constraint will lose and the keyboard will revert to the default height. That second case is particularly mystifying: at one point, I had to create an invisible kludge view with autolayout constraints and add it to my keyboard view in order for the height to set correctly!

But even with the extra height, you can’t actually make any part of your keyboard transparent. This means that if you want to implement popups for the top row, you either have to add an ugly strip to the top of your keyboard (my solution — it’s a good place to put extra UI anyway), or alternatively try increasing the height whenever you tap a key in the top row and then shrink it back down afterwards.

I ran into another problem with this constraint technique. For my keyboard, I didn’t really want to provide an entirely unique keyboard height; all I needed was an extra 30 points or so on top of the native keyboard height. The first idea I had was to change my height constraint from just a constant, to “equal to the keyboard height” plus 30. This did not work. Next, I tried to capture the keyboard height before I added my constraint, as well as on rotations; unfortunately, after my constraint was added, I found that this technique stopped working. (It warrants further investigation, however.) Finally, I decided to just hardcode the keyboard heights for each device and each orientation.

Speaking of which: with a custom height constraint, on rotation, everything goes nuts. I don’t think I’m doing anything too crazy. This is what my rotation code looks like:

override func willRotateToInterfaceOrientation(toInterfaceOrientation: UIInterfaceOrientation, duration: NSTimeInterval) {
    self.keyboardHeight = self.heightForOrientation(toInterfaceOrientation, withTopBanner: true)
}

…which simply sets the height constraint’s constant to the correct height.

But if you perform the rotation in the Simulator, you see things like this:

And this:

And maybe this:

In the process of trying to figure out what was causing this, I discovered that during certain points in this transition, the bounds of the keyboard view (or its superview, or its layers) were incorrect. For example, in the second screenshot, the bounds would be 320×216, even though they’re clearly 568×162 at that point.

I couldn’t find a way to work around this issue, and so I was freaking out for a few days until I discovered that it didn’t happen on device unless your performance was really slow. With that said, if you don’t add the Apple-sanctioned height constraint, none of this occurs.

Straight-Up API Errors

The UIInputViewController class, as an adherent to UITextInputDelegate, receives several useful callbacks from the text field. You’ll notice that they’re named pretty clearly: selectionWillChange, selectionDidChange, textWillChange, and textDidChange. In fact, none of them actually do those things. The selection methods never gets called at all, and the text methods only get called — get this — when the selection changes or the cursor is moved!

(Incidentally, the UITextInput object that you get back with these methods is crippled. Many of its methods don’t seem to do anything.)

viewWillTransitionToSize is another offender. As of iOS 8, if you use the old willRotateToInterfaceOrientation method, Xcode will tell you that it’s deprecated and that you should be using viewWillTransitionToSize. However, viewWillTransitionToSize never actually gets called! You’re forced to use a deprecated method in one of Apple’s newest UIKit additions.

UITextInputTraits Transgressions

Your UIInputViewController has a mysterious NSObject called textDocumentProxy that serves as the interface between the keyboard and the current text field via the UITextDocumentProxy protocol. (Why is it a nebulous NSObject instead of a UITextDocumentProxy?) Among other things, this protocol lets you access certain attributes of the text field.

Perhaps the most important of these is keyboardAppearance, which tells you if your keyboard should be using dark mode or not. One would expect this value to be correct on keyboard initialization, but no: you actually have to wait for the keyboard to appear on screen before this field populates with the correct value. I have not tested, but I have a hunch this applies to the other input traits as well. To my knowledge, this is not described in the documentation.

The properties of UITextInputTraits may change while the keyboard is still open, if the user switches from one text field to another. This is pretty sensible, if uncommon. What’s not sensible is that the textDocumentProxy object is not KVO compliant, meaning that there’s no easy way for you to observe these changes! The only thing you can really do is poll the object, 60 times a second. (I’m not suggesting that there’s a performance penalty for this, but still… yuck.)

Audio

As mentioned above, you need to enable full access to get any audio going. You would think, at least, that you could call the standard AudioServicesPlaySystemSound function to play the keyboard tock; but instead, you have to do this:

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), {
    AudioServicesPlaySystemSound(1104)
})

And yes, that dispatch_async is necessary; otherwise, your keyboard will simply stop working if full access is disabled.

iPad Woes

This is more sensible than the other issues, but I still think it’s worth mentioning. If you want to sell an iPhone-only keyboard… well, you kind of can’t. True, the app won’t show up in the iPad App Store; but if you install the container app on your iPad, you’ll still be able to install the keyboard in the settings! This is especially a problem if users find your app via your product website rather than through the App Store, since they probably won’t check whether your app is “optimized” for iPad before buying.

The Cruel Mockery of Size Classes

Apple strongly encourages the use of size classes in iOS 8. Seems like such an elegant solution: why worry about device dimensions or even portrait vs. landscape when you can just read the size class and show the appropriate UI? Alas, in the case of keyboards, they’re useless. Aside from the 6+ in landscape, keyboards are all strictly Compact, even though the layout constants for landscape are very different from those of portrait. Even worse: like the 6+, the 6 has those handy side buttons for editing in landscape — but its size class is still Compact, while the 6+ is Regular! And what about the iPad? Its keyboard is sized Regular in both orientations, even though it has exactly nothing in common with the 6+ keyboard in landscape.

Autolayout? Yeah, Right

This is kind of a digression, but I think it fits in here. When I was first implementing my keyboard, I tried to be a good platform citizen. I decided to use autolayout: Apple was strongly recommending it, and multiple screen sizes were coming in quick, so why not? Well… if you’re implementing a keyboard or anything remotely like it, don’t use autolayout!

In learning this technology, I found several places where autolayout was very appropriate. For example, if you’re making a single-screen view with a limited number of static elements — preferably in Interface Builder — it’s a great idea to use autolayout. But if you’re creating something where lots of views are generated dynamically and/or need to be laid out in an orderly fashion, just forget about it. Aside from the fact that you’ll spend your days knee-deep in constraintWithItem:attribute:relatedBy:toItem:attribute:multiplier:constant: initializers and spacer views2, you’ll also have to battle bizarre autolayout warnings and auto-resolutions when you decide to make even the tiniest change to your layout. It gives you zero flexibility.

Autolayout can optionally use something called the “visual format language” to create multiple constraints at once. A VFL string might look something like this: |-(2)-[myView(30)]-(2)-|. But every view in such a string must be named. For my implementation, this meant that all those anonymous keys and spacer views suddenly needed to have unique identifiers. It also meant that I couldn’t just write out these strings by hand, but instead had to use Swift’s string interpolation to put the correct names in place. I also had to dynamically combine multiple format substrings into one because I had no way of knowing how many keys my keyboard had ahead of time. In the end, it turned into a completely illegible, unmaintainable mess. Instead of making my life easier, all it did was increase my workload and pollute my codebase. (Again: if you have a single-screen view with just a handful of subviews, this works great. But it doesn’t scale.)

On top of it all, autolayout incurs a significant performance penalty. With my setup, featuring on the order of a hundred views, autolayout took a few seconds longer than just laying everything out in layoutSubviews.

In retrospect, settling on manual layout was certainly the right decision, and I’m not sure why I even spent so long trying to get autolayout working right. With layoutSubviews, there’s no worrying about priorities or inequalities, no seeking out conflicting constraints, no implicit calculations, no linear algebra. It was as if a fog had been lifted from my mind. My layout code shrunk to only a third of what it was before. And as extra validation, soon after I released my keyboard into alpha, somebody asked me if they could lay out their keys in an arc. This was the simplest of changes with layoutSubviews, but made me shudder to think of the horrible contortions that would have been required had I still been using autolayout!

As an aside, I recently encountered a project called Masonry that intends to be a simplified autolayout syntax. Here is an example from their docs:

[view1 mas_makeConstraints:^(MASConstraintMaker *make) {
    make.edges.equalTo(superview).with.insets(padding);
}];

To me, that looks marvelous, and I wish Apple would adopt something similar. Right now, even though autolayout is very powerful, it is completely unsuited for many applications, including most programmatic interface designs.

The What…? Under the Hood

If you examine at the view hierarchy of your keyboard, you’ll find it completely baffling. A view, which is the same as your inputView… added on top of a view… on top of another view… on top of another view… on top of the window, maybe? What are all these views even doing?

User visput on Russian site Habrahabr illustrates this with the following screenshot from his excellent article on iOS keyboard development:

View Hierarchy

In short, I give this framework a rating of NC-17.

Tooling Troubles

On a recent episode of ATP (was it ATP?), the gang talked about how the condition of Apple’s tools is a troubling sign that they might be barging ahead too fast. Believe me: it’s worse than you could ever imagine.

Xcode

Xcode quitting “unexpectedly”, stuck builds, stuck indexing, unresponsive buttons, and the never-ceasing SourceKit crash log blinking boxes of doom have become an hourly ritual for me. It’s gotten to the point where I instinctively reach for the “force quit” button if a task hasn’t completed within a few seconds. (I’ve also force-closed SourceKitService in Activity Monitor out of spite.)

Code completion frequenty stops working for me. I have to perform byzantine tricks to get it working again. (Sometimes, it never really does.) This is infuriating when you Command-Click on some UIKit class to see the header and Xcode tells you that it doesn’t exist, or when you try to call a particularly long method and Xcode fails to find a match.

Offline documentation has been in a state of disrepair for the longest time. Apple may have fixed it in 6.1, but I am loathe to check.

Slow. Scrolling is slow. Code completion is slow. Documentation is slow. Search is slow. Typing is slow. Everything is dog, dog slow. I am not exaggerating when I say that it sometimes takes half a second for my text to show up on the screen! (Albeit, this is with far, far too many browser tabs open, but Sublime keeps chugging on without a problem. And… it’s freaking text entry!)

It’s become a frequent occurrence that I load up an updated version of my keyboard, tap on my keyboard in the globe menu… and crash out back to the default keyboard. I have to either open the container app, force close the current app, press “stop” in Xcode, or do some other weird trick to get it to show up correctly.

A lot of the time, when I try to run the Keyboard target directly, it’ll open up the keyboard and then just sit on “waiting to attach” for eternity. Sometimes I can’t fix this.

Sometimes my dynamic frameworks won’t compile, and I have to press the run button twice in a row for everything to link up properly. Remember the old quip about how insanity is doing the same thing twice and expecting different results? Doesn’t apply to Xcode!

It’s such a shame because I actually, unironically like Xcode. Visual Studio disturbs me on a visceral level. How can I tell people that Xcode is a better-designed IDE when this kind of stuff is still going on? It’s unacceptable.

The Simulator

That smiley… that smug, mocking smiley.

As of Xcode 6.1, you can no longer open 3rd party keyboards in most apps in the Simulator, including your own. (And not just 3rd party keyboards: most system keyboards refuse to show up, too.) Not only does this make debugging a whole lot harder, but it also prevents you from easily making screenshots of your keyboard if you don’t own some of the newer devices. This issue is marked as a bug in Radar, and seeing as how it first appeared in the Xcode 6.1 betas, I cannot fathom how it got through to release.

Fin

There’s no real conclusion. As of iOS 8.1, 3rd party keyboards are functional, but look and feel janky. Programming them is also a horrible pain. I hope Apple fixes this stuff soon. The end.

Thanks for reading! If you’ve found this article or my keyboard project useful, please consider buying something for yourself on Amazon via my affiliate link.

Addendum 2014-11-9: Added paragraphs on visual format language and performance to autolayout section.

  1. There were a number of other interesting ways to accomplish the same goal, including Quartz event taps and .keylayout files. You can check out the code, in various states of completion, in my Github repo. (Warning: gnarly code ahead!)

  2. If you want to have equal spacing between a set of views, you need to add an equally-sized set of “spacer” views between them. This is the technique recommended in the official documentation.

What's Up with Google Maps These Days?

$
0
0

When Apple replaced its Google-powered maps app with a complete in-house rewrite for iOS 6, most users had little good to say about it. It was inaccurate. It lacked public transit directions. The search was paltry, as was the iconography. At best, it could be described as slick, with the pixellated pop-in of previous generations replaced by crisp, swiftly scaling vector lines. But this was not enough to offset the lack of functionality compared to Google’s definitive mapping solution, and so Tim Cook apologized and we all went on with our lives.

It’s now a couple of iOS releases later, and over the past year — especially with OSX support in tow — I’ve been frequently finding myself reaching for Apple Maps before going anywhere near a browser. Gradually, Apple has been tending to its garden: improving its data; getting better business listings; making its icons more contextually meaningful. At the same time, in lockstep, Google Maps has been driving me progressively up the wall. And as someone who has had maps.google.com burned into his fingertips since its inception, I can’t help but see this as building towards a major shift in the balance of power in the mobile sphere.

One of the most frustrating issues with Google Maps is seemingly minor, but hits me every time I use it. When you go to open a map, it initially — silently — loads as a static image. After a few seconds, the UI appears, followed by the actual interactive map. But nothing is done to distinguish the fake image-map from the interactive Javascript map, which means that when I first go to drag the map with my mouse, I am inevitably left with a useless thumbnail dangling from my cursor. Every time. Even worse, the UI widgets appear to be clickable during this image phase, but have absolutely no function! The only thing you can do to avoid driving yourself crazy is wait for the tell-tale pop-in that heralds the interactive map — assuming you didn’t miss it on initial load.

The UI just doesn’t seem to work half the time. Widgets always take a long time to load. Popups frequently overlap each other. Controls often stop working, as does dragging the map, forcing me to use the arrow keys to move around. Zoom takes too long to work and behaves erratically, especially if you use the Ctrl+double-click shortcut. Embedded maps only exacerbate all these issues, breaking more stuff and confining you to a tiny, immobile box. Just trying to use the map feels like a Mexican standoff between yourself, the browser, and Google’s byzantine Javascript code.

Google’s own native iOS app never goes above 15 to 30 frames per second. When I first tried the app on release day on my iPhone 4, I figured the performance was a forgivable issue with the 1.0, or that my phone’s hardware simply wasn’t cutting it anymore. But no: even today, on the flagship redesigned version, with an iPhone 5S on iOS 8, everything still runs with this lag. For a while, I figured, OK, Google didn’t want to give us the full, fluid maps experience on their primary competitor’s platform; whatever. Imagine my shock when I picked up a recent Android device, opened Maps, and found that it had the same 15-30fps bottleneck! On their home turf!

Apple Maps, from the start, has been running at a solid 60 frames per second. It’s no joke: you can feel the smoothness in every scroll, every pinch, every rotation. There’s no longer any pop-in, aside from the initial load: when you zoom in, street lines get more detailed and smooth out instead of reloading, indicative of the updated vector graphics. It was widely rumored that one of Apple’s reasons for moving to their own app was that Google wasn’t giving them access to their full vector data, which they were supposedly already using on Android. If true, why is Google’s native app performance so atrocious, even to this day? Why do we still see tiles? Why do streets still have to unblur as you zoom in? If Google is actually using vectors, how are they using them in such a way that still causes these issues?

Speaking of smoothness: gestures in Apple Maps, both on iOS and via the trackpad on OSX, are a dream to use. Pinch-zooming, rotating, and panning have become so ingrained in my muscle memory that I keep trying to do it to Google Maps in my browser window — only to watch my fonts grow a size too big.

When I learned that Google Maps on iOS supported caching maps for offline use, I got excited. Now here was a great feature that Apple Maps would probably never offer! Well, how’s this for irony? Although Apple doesn’t explicitly allow offline caching, it does, in practice, cache large swaths of map that you’ve recently viewed. I’ve been using my phone offline for the past few months, and in every city I’ve been to, I’ve gotten beautiful offline maps of the area after peeking at the surroundings in Wi-Fi mode. There’s no blur: I can zoom in with no apparent loss of street names or any other detail. Perhaps it’s a bad idea to rely on a cache that may theoretically invalidate at any time, but in practice, it works great.

In Google Maps — an app, I must remind you, that explicitly supports offline mapping— saved maps don’t actually seem to work much of the time. I was unfortunate enough to discover this while walking around in a foreign city: upon opening my saved Google map, I found a blurry mess of pixels with no labels whatsoever. (Fortunately, Apple’s cache saved me from having to ask for directions.) To be fair, sometimes the maps do load, but I can’t seem to trigger this behavior deterministically. Even better, offline maps sometimes “expire” at arbitrary times, rendering them unusable. (The official expiration time — why even have one? — is about a month, but I’ve seen maps expire earlier than that for no reason.) Finally, for reasons I don’t understand, not all areas are cacheable. That’s right: if you try to save a map in Slovenia, you’ll be hit with an “area not available” error, even though you can save maps in Croatia by stepping over the border! What a mess.

So what about the elephant in the room: public transit? The lack of public transit directions on iOS is definitely a deal-breaker for me, and Apple’s kludgy third party solution just doesn’t cut it. (To remind readers: if you select public transit directions in Apple Maps, you are given the option of sending your request to any app that supports the transit directions API, or alternatively searching the App Store for one.) However, with the release of third party extensions, I think Apple has a tremendous opportunity on their hands. Although it’s convenient to have all your public transit directions curated by a central party, it’s hard for one organization to keep tabs on every public transit system in the world. How many bus lines are out there? Thousands? Google Maps mostly works in this regard, but I have on more than one occasion run into situations where the directions were inaccurate, misleading, or missing. Even Google can’t extend its feelers to every little city around the globe.

On the other hand, local transit apps are not only accurate and up-to-date, but can even offer cool features like real-time arrival times and GPS tracking. It’s also pretty easy to find them: Apple already lets you see what apps are most frequently used in an area. But these apps aren’t able to integrate with system-wide services: you have to either open them separately or load them via Apple’s kludgy Maps hook, making them inconvenient for everyday use.

So here’s an idea. What if Apple kept its third party transit direction support, but instead of having public transit directions open up in a separate app, brought that information directly into Maps itself?

Just a year ago, this would have been inconceivable. But today, with third party extension support, it would be a perfect fit with Apple’s strategy. Extensions go through specific, resource-constrained “extension points” to avoid hogging system resources, and Apple explicitly put future extension points on the table in their keynote reveal. How about this? With transit extensions, you could have an app transmit local, up-to-the-minute timetables, GPS markers, and other metadata for public transit services right into any native Maps view!

This isn’t something that Google could implement with its centralized, web-centric approach. But it’s almost obvious with Apple’s app-based infrastructure. All the pieces are already in place; all they need to do is give developers an API for it. I don’t think this would necessarily obviate the need for centralized, curated public transit directions, but it would be a great way to augment and future-proof such an offering.

There are still many problems with Apple Maps. Bookmark management is a nightmare. Getting directions between arbitrary points is nearly impossible. Things like subway stations are poorly delineated. But these are not high level problems; they are easy problems to fix, and Apple has progressively been fixing them. And so, I’m crossing my fingers for an amazing future where I can open Maps, switch over to satellite view, and have a private little “supervillain” moment as I watch the bus and train pins moving around the city, slowly etching out their hard-worn routes, in real-time.

All at a buttery 60fps.

State of the Union

$
0
0

I’m sure bad at this blogging thing. Since I first set out in — technically, July 2013, but let’s say August — I’ve only gotten one week into my journey before hanging up. I’d like to see if I can try again, perhaps this time with a little more success.

Right now, I’m writing this from Dublin, Ireland. In the interim, since my last entry from an actual city, I’ve visited Vancouver, Toronto, Montreal, Quebec City, New York, Boston, Philadelphia, Chicago, New Orleans, Austin, Los Angeles, Barcelona, Paris, London, Cornwall, Bath, Oxford, Cambridge, York, Newcastle, Edinburgh, Amsterdam, Luxembourg, Prague, Berlin, Vienna, Ljubljana, Zagreb, Split, Bologna, Modena, Milan, Florence, Rome, and Turin — plus a number of smaller towns along the way. That’s more than a dozen different countries over the course of the past two years. To the person first setting out on this trip — the person who expected to arrive in London and cautiously venture to other countries only on rare occasion — this list would have been unimaginable. I was nervous. I’d never been alone in a place where the primary language wasn’t English. I wasn’t sure about the logistics. I didn’t even know if I’d be able to tinker with my projects in peace.

But it’s all worked out. I’ve cracked the code. With some trial and error, I’ve figured out how to live affordably and comfortably in different cities around the world. I’ve gotten a lot more bold about venturing to places where I have to resort to broken apologetic phrases in a new language. I’ve discovered many breathtaking sights, incredible architectural wonders, and fascinating local tastes. I’ve pared my belongings down to a bare minimum appropriate for most climates. I’ve worked in many lovely cafés, bars, and apartments. I feel like I understand so much more about the world than when I started.

On the other hand, I’ve also had to face certain facets of myself head-on, with uncertain results. I’ve learned that my impression of different cities around the world was heavily painted by my childhood imagination. The places I’d hoped would bring my soul some peace — a magical sort of quality that I faintly recall in cities like London, Barcelona, and Rome from when I was little — were found to be fairly ordinary in contrast. I’ve found it disheartening to overwrite so much impressionistic imagery in my head with reality — because I know those feelings will never be back. I’ve learned that no matter how lonely I might get at times, there is still no part of me that will seek out others for company. I’ve learned that I can swallow up entire months at a time and wake up on the other end a good chunk older, but with nothing to show for it. All the attributes I hated about myself as a young teen are still there, entrenched, and at this point as much a part of my personality as anything else. I’ve learned that there are parts of me that I constantly have to fight, lest I end up old, unfulfilled, and alone.

I’ve also learned that time moves quickly, and that it’s easy to get stuck in a simple routine. I’ve developed a great, burning fear that prods me almost every waking minute. The fear of aging. The fear of death. The fear of it “being too late”. Every day, I count the fractions of my age. 26 point 0. Point 25. Point 3, repeating. Every day that I’m not doing something productive — which is most days, to be honest — I feel like I’m actively destroying my potential. Other dates have been creeping into my head, too. How old is too old to live in a house with other people? How old is too old to get married? To have kids, if I decide to have them? To have a nice house, friendly neighbors, warm dinner parties? When I was in college, I half-jokingly feared that my study habits — put it off, put it off, then half-ass it on the last day — would translate in broader strokes to the rest of my life. But now I see that this was no joke. This is exactly what happens when you let a negative aspect of your personality fester for years and years.

But these dark thoughts are not a reflection on travel. My journey has neither been a negative experience nor an entirely positive one. I see it as just another page in the book of my life: many new and interesting inputs from the environment feeding into the black box of my mind, allowing me to discover new truths about the world and myself.

So that’s what’s going on in my head on this day, January the 31st, 2015, in the city of Dublin. And now, let’s step back into the past, to Vancouver, as it was during the last few weeks of August of 2013

(Technical note: from this point on, I will be posting all my blog entries on the beta-blog subdomain of my website. They will be linked from the main site, but I highly suggest reading on beta-blog instead. Eventually, the main site — hosted on Squarespace — will be removed, and the beta-blog page — hosted on Github Pages using a static site generator — will replace it. I will backport all the existing entries before I do that, and all the links should still hopefully work.)

Vancouver

$
0
0

On August 23rd, 2013, my Seattle friends and I boarded the Amtrak Cascades and headed up north in the direction of Vancouver. I couldn’t resist bringing along a small bag of produce and dairy that I’d been using to cook in the hostels, even though it added a few pounds of heft to my already-painful load. (This pattern will be shown to repeat itself throughout my travels.)

The ride was clean and beautiful. We planted ourselves in the observation car for most of the 4-hour trip. Dangerous electric tea was made. I was introduced to a devious card game called Bartok, in which each player adds a new, arbitrary rule during their turn. Every time a player breaks a rule or asks a question, they draw a card. As you might expect, it gets a little bit crazy.

Half an hour from the border, I realized that I was woefully underprepared for my customs interview. What was I doing? Where was I going? Should I have brought some printouts of my itinerary? Did I need to hide my butter?!

The border agent gave my slightly disheveled self a stern look before letting me through. Little did he know of my contraband!

I booked into my hostel, while my friends checked into a hotel in a… slightly seedy part of town. (Given how high Vancouver is rated in terms of life satisfaction, I didn’t even know it had seedy parts!) We had a good laugh observing some friendly neighborhood drug dealers from the hotel room’s 3rd story window.

So… Vancouver! What’s it all about?

Well, it’s a peaceful little city. Stanley Park is lovely to walk through, and it seems like there’s a lot of interesting restaurants and other businesses. (Once place I want to give a shout-out to is Fritz European Fry House. I don’t think the poutine there was “real” — the cheese curds were kind of melty — but it was still some of the best poutine-like stuff that I tasted during my Canadian trip.)

However, much like the last time I visited, it didn’t strike me as a city I wanted to spend too much time in. The architecture was kind of drab, and the whole place had an overbearing “new development” feel. Not cozy. Didn’t warm the heart. I don’t know how else to put it.

However, I’m very thankful for the company I had. The trip wouldn’t have been the same without it!

Onward we go… aboard the VIA Rail.

Who cares about the Apple Watch Edition?

$
0
0

It’s disappointing to see many of my favorite Apple bloggers devote so much of their time to Apple Watch Edition gossip. How much of it is going to be solid gold? Will it be $5000, $10000, $15000? How much of them will Apple sell in their first quarter? Will they be a hit with the rich and powerful in China?

My question is: why in the least bit should us techies care? Sure — catering to the unfathomably well-off will make an impact on Apple’s fortunes in the long run, and spinning out into a fashion brand might make the company grow at an even more breakneck pace. But I was under the impression that we were Apple fans in the first place because they made best-in-breed products for everyone, not just the elite.

We loved our iPods and iPhones for their sleek design and smooth UI, even when people dismissed them as “expensive toys”. We knew our $2000 laptops were incredible for the price, even while people mocked us for not buying cheap, creaky Windows machines. When Android and Windows users poked fun at our platforms for lacking in free tools, we lauded the benefits of carefully crafted, paid-up-front software. We let our Apple logos shine bright because we were proud to be affiliated with one of the few companies that seemed philosophically bent on setting a new standard for mass-market products.

There’s a good reason why the tech community might have a “shit-fit” if Apple enters the realm of $10000 watches and $2000 bands. Gruber’s word choice makes it sound like a juvenile reaction by the fashion-unconscious. But that’s just the thing. We’re geeks. We believe in the intrinsic value of things. We’ve always made fun of name-dropping headphones and $1000 shirts when the quality couldn’t match the hype. Apple taught many of us that there’s more to the price of an item than just the cost of materials; design can be worth a lot, too. But we’ve been adamant for years that we bought Apple products because they were good, not because they were fashionable. And now, with the Beats acquisition and these gold-encased watches, Apple is showing signs that they no longer want to be bound by intrinsic value.

Conspicuous consumption? Veblen goods? Roped-off areas in Apple stores? Is this is what the indie Mac community is going to talk about now?

An Apple that sells “bling” to billionaires is not a company that many of us could muster any passion about. Maybe Woz had the right idea all along.

What's Up with Google Maps These Days?

$
0
0

When Apple replaced its Google-powered maps app with a complete in-house rewrite for iOS 6, most users had little good to say about it. It was inaccurate. It lacked public transit directions. The search was paltry, as was the iconography. At best, it could be described as slick, with the pixellated pop-in of previous generations replaced by crisp, swiftly scaling vector lines. But this was not enough to offset the lack of functionality compared to Google’s definitive mapping solution, and so Tim Cook apologized and we all went on with our lives.

It’s now a couple of iOS releases later, and over the past year — especially with OSX support in tow — I’ve been frequently finding myself reaching for Apple Maps before going anywhere near a browser. Gradually, Apple has been tending to its garden: improving its data; getting better business listings; making its icons more contextually meaningful. At the same time, in lockstep, Google Maps has been driving me progressively up the wall. And as someone who has had maps.google.com burned into his fingertips since its inception, I can’t help but see this as building towards a major shift in the balance of power in the mobile sphere.

One of the most frustrating issues with Google Maps is seemingly minor, but hits me every time I use it. When you go to open a map, it initially — silently — loads as a static image. After a few seconds, the UI appears, followed by the actual interactive map. But nothing is done to distinguish the fake image-map from the interactive Javascript map, which means that when I first go to drag the map with my mouse, I am inevitably left with a useless thumbnail dangling from my cursor. Every time. Even worse, the UI widgets appear to be clickable during this image phase, but have absolutely no function! The only thing you can do to avoid driving yourself crazy is wait for the tell-tale pop-in that heralds the interactive map — assuming you didn’t miss it on initial load.

The UI just doesn’t seem to work half the time. Widgets always take a long time to load. Popups frequently overlap each other. Controls often stop working, as does dragging the map, forcing me to use the arrow keys to move around. Zoom takes too long to work and behaves erratically, especially if you use the Ctrl+double-click shortcut. Embedded maps only exacerbate all these issues, breaking more stuff and confining you to a tiny, immobile box. Just trying to use the map feels like a Mexican standoff between yourself, the browser, and Google’s byzantine Javascript code.

Google’s own native iOS app never goes above 15 to 30 frames per second. When I first tried the app on release day on my iPhone 4, I figured the performance was a forgivable issue with the 1.0, or that my phone’s hardware simply wasn’t cutting it anymore. But no: even today, on the flagship redesigned version, with an iPhone 5S on iOS 8, everything still runs with this lag. For a while, I figured, OK, Google didn’t want to give us the full, fluid maps experience on their primary competitor’s platform; whatever. Imagine my shock when I picked up a recent Android device, opened Maps, and found that it had the same 15-30fps bottleneck! On their home turf!

Apple Maps, from the start, has been running at a solid 60 frames per second. It’s no joke: you can feel the smoothness in every scroll, every pinch, every rotation. There’s no longer any pop-in, aside from the initial load: when you zoom in, street lines get more detailed and smooth out instead of reloading, indicative of the updated vector graphics. It was widely rumored that one of Apple’s reasons for moving to their own app was that Google wasn’t giving them access to their full vector data, which they were supposedly already using on Android. If true, why is Google’s native app performance so atrocious, even to this day? Why do we still see tiles? Why do streets still have to unblur as you zoom in? If Google is actually using vectors, how are they using them in such a way that still causes these issues?

Speaking of smoothness: gestures in Apple Maps, both on iOS and via the trackpad on OSX, are a dream to use. Pinch-zooming, rotating, and panning have become so ingrained in my muscle memory that I keep trying to do it to Google Maps in my browser window — only to watch my fonts grow a size too big.

When I learned that Google Maps on iOS supported caching maps for offline use, I got excited. Now here was a great feature that Apple Maps would probably never offer! Well, how’s this for irony? Although Apple doesn’t explicitly allow offline caching, it does, in practice, cache large swaths of map that you’ve recently viewed. I’ve been using my phone offline for the past few months, and in every city I’ve been to, I’ve gotten beautiful offline maps of the area after peeking at the surroundings in Wi-Fi mode. There’s no blur: I can zoom in with no apparent loss of street names or any other detail. Perhaps it’s a bad idea to rely on a cache that may theoretically invalidate at any time, but in practice, it works great.

In Google Maps — an app, I must remind you, that explicitly supports offline mapping— saved maps don’t actually seem to work much of the time. I was unfortunate enough to discover this while walking around in a foreign city: upon opening my saved Google map, I found a blurry mess of pixels with no labels whatsoever. (Fortunately, Apple’s cache saved me from having to ask for directions.) To be fair, sometimes the maps do load, but I can’t seem to trigger this behavior deterministically. Even better, offline maps sometimes “expire” at arbitrary times, rendering them unusable. (The official expiration time — why even have one? — is about a month, but I’ve seen maps expire earlier than that for no reason.) Finally, for reasons I don’t understand, not all areas are cacheable. That’s right: if you try to save a map in Slovenia, you’ll be hit with an “area not available” error, even though you can save maps in Croatia by stepping over the border! What a mess.

So what about the elephant in the room: public transit? The lack of public transit directions on iOS is definitely a deal-breaker for me, and Apple’s kludgy third party solution just doesn’t cut it. (To remind readers: if you select public transit directions in Apple Maps, you are given the option of sending your request to any app that supports the transit directions API, or alternatively searching the App Store for one.) However, with the release of third party extensions, I think Apple has a tremendous opportunity on their hands. Although it’s convenient to have all your public transit directions curated by a central party, it’s hard for one organization to keep tabs on every public transit system in the world. How many bus lines are out there? Thousands? Google Maps mostly works in this regard, but I have on more than one occasion run into situations where the directions were inaccurate, misleading, or missing. Even Google can’t extend its feelers to every little city around the globe.

On the other hand, local transit apps are not only accurate and up-to-date, but can even offer cool features like real-time arrival times and GPS tracking. It’s also pretty easy to find them: Apple already lets you see what apps are most frequently used in an area. But these apps aren’t able to integrate with system-wide services: you have to either open them separately or load them via Apple’s kludgy Maps hook, making them inconvenient for everyday use.

So here’s an idea. What if Apple kept its third party transit direction support, but instead of having public transit directions open up in a separate app, brought that information directly into Maps itself?

Just a year ago, this would have been inconceivable. But today, with third party extension support, it would be a perfect fit with Apple’s strategy. Extensions go through specific, resource-constrained “extension points” to avoid hogging system resources, and Apple explicitly put future extension points on the table in their keynote reveal. How about this? With transit extensions, you could have an app transmit local, up-to-the-minute timetables, GPS markers, and other metadata for public transit services right into any native Maps view!

This isn’t something that Google could implement with its centralized, web-centric approach. But it’s almost obvious with Apple’s app-based infrastructure. All the pieces are already in place; all they need to do is give developers an API for it. I don’t think this would necessarily obviate the need for centralized, curated public transit directions, but it would be a great way to augment and future-proof such an offering.

There are still many problems with Apple Maps. Bookmark management is a nightmare. Getting directions between arbitrary points is nearly impossible. Things like subway stations are poorly delineated. But these are not high level problems; they are easy problems to fix, and Apple has progressively been fixing them. And so, I’m crossing my fingers for an amazing future where I can open Maps, switch over to satellite view, and have a private little “supervillain” moment as I watch the bus and train pins moving around the city, slowly etching out their hard-worn routes, in real-time.

All at a buttery 60fps.


State of the Union

$
0
0

I’m sure bad at this blogging thing. Since I first set out in — technically, July 2013, but let’s say August — I’ve only gotten one week into my journey before hanging up. I’d like to see if I can try again, perhaps this time with a little more success.

Right now, I’m writing this from Dublin, Ireland. In the interim, since my last entry from an actual city, I’ve visited Vancouver, Toronto, Montreal, Quebec City, New York, Boston, Philadelphia, Chicago, New Orleans, Austin, Los Angeles, Barcelona, Paris, London, Cornwall, Bath, Oxford, Cambridge, York, Newcastle, Edinburgh, Amsterdam, Luxembourg, Prague, Berlin, Vienna, Ljubljana, Zagreb, Split, Bologna, Modena, Milan, Florence, Rome, and Turin — plus a number of smaller towns along the way. That’s more than a dozen different countries over the course of the past two years. To the person first setting out on this trip — the person who expected to arrive in London and cautiously venture to other countries only on rare occasion — this list would have been unimaginable. I was nervous. I’d never been alone in a place where the primary language wasn’t English. I wasn’t sure about the logistics. I didn’t even know if I’d be able to tinker with my projects in peace.

But it’s all worked out. I’ve cracked the code. With some trial and error, I’ve figured out how to live affordably and comfortably in different cities around the world. I’ve gotten a lot more bold about venturing to places where I have to resort to broken apologetic phrases in a new language. I’ve discovered many breathtaking sights, incredible architectural wonders, and fascinating local tastes. I’ve pared my belongings down to a bare minimum appropriate for most climates. I’ve worked in many lovely cafés, bars, and apartments. I feel like I understand so much more about the world than when I started.

On the other hand, I’ve also had to face certain facets of myself head-on, with uncertain results. I’ve learned that my impression of different cities around the world was heavily painted by my childhood imagination. The places I’d hoped would bring my soul some peace — a magical sort of quality that I faintly recall in cities like London, Barcelona, and Rome from when I was little — were found to be fairly ordinary in contrast. I’ve found it disheartening to overwrite so much impressionistic imagery in my head with reality — because I know those feelings will never be back. I’ve learned that no matter how lonely I might get at times, there is still no part of me that will seek out others for company. I’ve learned that I can swallow up entire months at a time and wake up on the other end a good chunk older, but with nothing to show for it. All the attributes I hated about myself as a young teen are still there, entrenched, and at this point as much a part of my personality as anything else. I’ve learned that there are parts of me that I constantly have to fight, lest I end up old, unfulfilled, and alone.

I’ve also learned that time moves quickly, and that it’s easy to get stuck in a simple routine. I’ve developed a great, burning fear that prods me almost every waking minute. The fear of aging. The fear of death. The fear of it “being too late”. Every day, I count the fractions of my age. 26 point 0. Point 25. Point 3, repeating. Every day that I’m not doing something productive — which is most days, to be honest — I feel like I’m actively destroying my potential. Other dates have been creeping into my head, too. How old is too old to live in a house with other people? How old is too old to get married? To have kids, if I decide to have them? To have a nice house, friendly neighbors, warm dinner parties? When I was in college, I half-jokingly feared that my study habits — put it off, put it off, then half-ass it on the last day — would translate in broader strokes to the rest of my life. But now I see that this was no joke. This is exactly what happens when you let a negative aspect of your personality fester for years and years.

But these dark thoughts are not a reflection on travel. My journey has neither been a negative experience nor an entirely positive one. I see it as just another page in the book of my life: many new and interesting inputs from the environment feeding into the black box of my mind, allowing me to discover new truths about the world and myself.

So that’s what’s going on in my head on this day, January the 31st, 2015, in the city of Dublin. And now, let’s step back into the past, to Vancouver, as it was during the last few weeks of August of 2013

(Technical note: from this point on, I will be posting all my blog entries on the beta-blog subdomain of my website. They will be linked from the main site, but I highly suggest reading on beta-blog instead. Eventually, the main site — hosted on Squarespace — will be removed, and the beta-blog page — hosted on Github Pages using a static site generator — will replace it. I will backport all the existing entries before I do that, and all the links should still hopefully work.)

Vancouver

$
0
0

On August 23rd, 2013, my Seattle friends and I boarded the Amtrak Cascades and headed up north in the direction of Vancouver. I couldn’t resist bringing along a small bag of produce and dairy that I’d been using to cook in the hostels, even though it added a few pounds of heft to my already-painful load. (This pattern will be shown to repeat itself throughout my travels.)

The ride was clean and beautiful. We planted ourselves in the observation car for most of the 4-hour trip. Dangerous electric tea was made. I was introduced to a devious card game called Bartok, in which each player adds a new, arbitrary rule during their turn. Every time a player breaks a rule or asks a question, they draw a card. As you might expect, it gets a little bit crazy.

Half an hour from the border, I realized that I was woefully underprepared for my customs interview. What was I doing? Where was I going? Should I have brought some printouts of my itinerary? Did I need to hide my butter?!

The border agent gave my slightly disheveled self a stern look before letting me through. Little did he know of my contraband!

I booked into my hostel, while my friends checked into a hotel in a… slightly seedy part of town. (Given how high Vancouver is rated in terms of life satisfaction, I didn’t even know it had seedy parts!) We had a good laugh observing some friendly neighborhood drug dealers from the hotel room’s 3rd story window.

So… Vancouver! What’s it all about?

Well, it’s a peaceful little city. Stanley Park is lovely to walk through, and it seems like there’s a lot of interesting restaurants and other businesses. (Once place I want to give a shout-out to is Fritz European Fry House. I don’t think the poutine there was “real” — the cheese curds were kind of melty — but it was still some of the best poutine-like stuff that I tasted during my Canadian trip.)

However, much like the last time I visited, it didn’t strike me as a city I wanted to spend too much time in. The architecture was kind of drab, and the whole place had an overbearing “new development” feel. Not cozy. Didn’t warm the heart. I don’t know how else to put it.

However, I’m very thankful for the company I had. The trip wouldn’t have been the same without it!

Onward we go… aboard the VIA Rail.

The Klean Kanteen 32oz Insulated Stainless Steel Growler (With Swing Lok Cap)

$
0
0

Great on the train! Now with pizza!

One vital lesson I’ve been recently learning in regards to the pleasures of rampant materialism is that it’s incredibly important to surround yourself with objects that not only work well, but feel good. Quality finishes; smooth hinges; great materials and top-shelf workmanship. It’s not about pretension or an abstract desire for “good design”. Rather, the aim is to get the objects in your life to radiate just a little warmth your way by virtue of their excellence. Here’s an example of what I mean. A few months ago, I needed to buy a new paper notebook. I was all set to pick up the cheapest one at the grocery like I always do, but this time I decided to peek into a stationary store just to see if something could tempt me away from the budget option. There, I discovered a stunningly beautiful notebook with a textured, gently undulating cover and perfect pearly pages. I immediately fell in love with it. The sticker price was a shocking $15, and while I berated myself at the time for spending so much money on basically paper, I’ve noticed over the past year that I always feel a little burst of joy whenever I leaf through its pages or run my hands over its cover. Point being, if you surround yourself with enough of these kinds of things, you’ll find yourself smiling when doing something as simple as picking up your favorite pen or putting a pot of water on the stove.

Unfortunately, this therapeutic attribute is hard to quantify and review. What exactly makes an object feel good? You can have things that are incredibly sturdy but feel poor to use; but it’s also easy to manufacture something that looks great at first but exudes cheapness. No video or photo will do the trick here, and lists of stats and tech specs only confuse the matter. The only reliable way to tell is to actually handle the thing in person.

Which is why I’ve been spending the past few days obsessively looking for the perfect stainless steel beer growler.

Some background first. I recently went on a yearlong trip around Europe, and one of my most prized and reliable accessories was a Klean Kanteen insulated 20oz bottle. I happily used it almost every day and I have hard time imagining travel without it. Unfortunately, it’s gotten pretty banged up to the point where it doesn’t even stand straight, so I’ve been casually keeping an eye out for a new bottle to replace it with. I also discovered the wonders of craft beer during my trip, and I found that many stores and breweries were starting to install growler fill stations: places where you could bring your so-called “growler” (basically a large jug designed for holding beer, usually in 32 or 64 ounce sizes) and get it filled up from one of their kegs — often for lower-than-retail prices. After coming back to California, I learned that even though California law prevents grocery stores from filling growlers like they do in other states, it’s perfectly legal for breweries to do so. And we just happen to have several world-class breweries in the Bay Area that have very limited bottle distribution but happily fill outside growlers, including Cellarmaker in San Francisco, Sante Adairius Rustic Ales in Santa Cruz, and Fieldwork in Berkeley. I also discovered that thermally insulated 32oz stainless steel growlers, including a brand new one by Klean Kanteen with the same construction as my trusty 20oz, were quickly becoming a thing. All these factors, combined with the fact that 20oz of water was often not enough to get me through the day, made me start looking into making a new water bottle (slash covert beer growler) purchase.

I only had a few initial requirements. The growler had to be stainless steel. It had to be thermally insulated. It had to have the volume printed somewhere on the bottle. And it had to be leak-proof. There were a few REIs around me with bottles by Hydro Flask, Klean Kanteen, Stanley, and Miir, so off I went to get my hands on them.

Hydro Flask

My first impression of the Hydro Flask was that it was a cheaper-built version of Klean Kanteen. The paint had a bumpy finish. There was a hollow sort of ring and vibration when I put the bottle down, as if the inner metal layer was thinner and less firmly attached than it should be. The cap was made of cheap plastic — particularly evident on the inside — and I was worried that I would damage the thread whenever I screwed it on tightly. (There was no obvious stopping point as with my 20oz Klean Kanteen.) But the more I handled this bottle, the more I liked it. In particular, the top-to-bottom finish looks sleek, especially in black. I got over the bumpy finish very quickly; Klean Kanteen’s glossy and matte finishes still feel much better, but this one has its own charm. The wide 2” mouth is a point in its favor, and even though the volume designation is painted on the side (where it might wear off more easily), customers on Amazon say that the paint remains perfectly intact even after a year of use. Finally, this is easily the most compact 32oz form factor out of all the bottles I handled. It’s only a little wider and actually shorter than my 20oz bottle!

Stanley

Stanley’s bottle felt the best-constructed overall. I loved the solid, textured outer finish, the incredibly smooth hinges, and the the bells and whistles like the insulated cup attachment and the cap that doubles as a spout. This is a beautiful workhorse bottle. Unfortunately, the screw cap is incompatible with using the bottle as a growler. Since the cap screws into the bottle and not onto and outside of it like on the Hydro Flask, you can’t fill it to the very top without wasting lots of beer. Many breweries will refuse to fill bottles designed this way. The Stanley also felt bigger and heavier than the other bottles I tried. But what sealed this bottle’s fate were all the negative reviews of the cap, claiming that recent redesigns made them far less leakproof and reliable than before. Oh well.

Miir

I wanted to love the Miir bottle— it was the only 32oz swing top bottle available with a black matte finish, and the company had a great mission — but it had too many problems. (Note that I examined the 64oz bottle at REI, but it looks to have the same cap design as the 32oz.) First, the swing top. You’ll see in reviews that people have a lot of trouble opening and closing the cap. After undoing the latch, you have to kind of rotate the cap backwards over the latch at a certain angle. Is this a difficult procedure? No, of course not. But to my surprise, the extra 1-2 seconds required to figure it out went a long way towards making the bottle unenjoyable and frustrating to use. The second issue is with the way the swing top attaches to the bottle. It appears that the outer part of the neck is somehow removed and a metal ring added in its place, with holes drilled in the sides to attach the swing top mechanism. Unfortunately, this is done rather clumsily: there are jagged edges all around the lower part of the ring, and if you look down into the mouth of the bottle, you’ll see a small gap between the ring and the opening — surely an incubator of mold and errant smells. This design also makes the mouth of the bottle really thick and ugly. My final problem with this bottle is the actual seal. While the bottle does seal quite tightly and smoothly (once you work out the puzzle latch), the rubber/silicone that the seal is made from looks really cheap. Especially given the rough neck design, I’m concerned that it will deteriorate over time. On the plus side, this is the only bottle that had the volume and government warning solidly engraved on the bottom.

Klean Kanteen

Now, for the winner.

The Klean Kanteen felt the best in my hands out of all the bottles I tried. Geometry-wise, it’s practically perfect. There are no hard edges or jagged seams to be found. You can run your hand over the outside and feel one smooth curve, save for a small gap where the body meets the base. This is in line with my experience with Klean Kanteen as a whole: you can tell that their design and manufacturing is a step above the other players.

The mouth of the bottle is about 1.75” wide. My 20oz bottle had a wide 2” mouth which (barely) fit a tea infuser, so I was reluctant to consider a smaller mouth at first. However, I quickly learned that the the narrow mouth has benefits of its own. First, the smooth curvature of the lip is very pleasant to drink from, unlike the hard edge of the 2” bottle. (A subtle but important point! I never enjoyed drinking out of the 20oz wide mouth. Add all those negative moments up over a year and what do you get?) And second, the smaller mouth makes the bottle perfect for pouring, even when filled to the brim. Whenever I tried pouring out my wide mouth bottle, I inevitably ended up spilling liquid all over the place (or myself). These two issues factor far more into my life than the ability to brew tea inside the bottle, so the decision wasn’t too agonizing. (And in any case, there are thin+long as well as flexible silicone infusers that would surely fit through the smaller opening.)

The insulation of this bottle is very good. Like most thermally insulated stainless steel bottles, the Klean Kanteen features a double-walled design, harboring a layer of vacuum between the outer and inner steel layers. Some Amazon reviews claimed that this bottle wasn’t as good as other Klean Kanteens at keeping liquids hot or cold, so I put it to the test. I filled both the 32oz and 20oz bottles with boiling hot water and measured their temperature over the course of four hours — sometimes open, sometimes closed. Surprisingly, the bottles rarely varied by more than a degree or two, losing a few degrees every hour when closed (with a sharper decline at the start) and about half a degree per minute when open. One thing I did notice with the 32oz was that the neck and cap of the bottle felt warm, while the gasket actually felt hot. When filled with very cold liquid, the neck also sweat a little. But this did not seem to affect the bottle’s thermal properties.

Klean Kanteen Growler (32oz)Old Klean Kanteen (20oz)
0 minutes95.7 ℃95.7 ℃
8 minutes95.0 ℃94.0 ℃
Open
21 minutes85.0 ℃86.5 ℃
Closed
35 minutes83.7 ℃84.2 ℃
Open
56 minutes73.6 ℃(Mismeasured)
Closed
71 minutes72.3 ℃73.5 ℃
98 minutes68.7 ℃70.6 ℃
220 minutes65.0 ℃63.8 ℃

Perhaps the biggest draw of this bottle is the swing cap. At first glance, it might seem like a simple aesthetic choice. (And don’t get me wrong, the retro milk jug appearance is quite attractive!) But in fact, swing caps and screw caps have several important functional and usability differences. First, leak-proofness. Practically all modern water bottles have a rubber or silicone gasket that prevents liquids and gases from escaping. In screw top bottles, the seal is held down by the friction of the plastic or metal screw; its effectiveness is determined by how tightly you screw the cap on. Swing caps are relatively simple: the only thing pushing down the gasket is the latch. The common advice for keeping carbonated beverages fresh is to screw on the cap as tightly as possible; with a swing cap bottle, you never have to worry about this.

At first, I was a little concerned that the simple design — just a loose silicone seal clamped down by metal — would be less reliable than the multiple layers of a screw cap’s thread. But the ease with which the bottle can be closed conceals the fact that there’s a lot of pressure bearing down on that cap. When I tried to push the cap down to its closed position without using the latch, I couldn’t compress it anywhere near its final position! Furthermore, a Klean Kanteen representative told me that the cap was measured to be waterproof at 35psi. You might notice that the Klean Kanteen website lists the Swing Lok cap as “non leakproof”, whereas many of the screw caps are. I don’t think this is an accurate assessment. I’ve had plenty of leaks in my screw cap Klean Kanteen when filled with hot liquid, and you can even find videos of screw caps popping off completely when the bottle is filled with hot liquid and shaken. A swing cap, by virtue of its sturdy construction, does not have this outlet. The only way that pressure can escape from the bottle is through its seal. In that sense, it has to be “non leakproof”.

(It should be noted that the description of the Swing Lok cap on the Klean Kanteen website indicates that it’s not designed for hot liquids. I asked the rep about it and they said that this was because they “do not want a hot beverage to spill out and leak on anyone”. My understanding is that the silicone seal will not be damaged by boiling water; as with all Klean Kanteens, you just have to make sure not to agitate the bottle too much when filled with hot water to prevent pressure from building up.)

Just to make sure that 35psi was enough for my needs, I battle-tested the bottle in several different ways. I filled it with carbonated water and tilted it downwards overnight. I filled it with hot water and let it sit out for a while. I got it filled with beer and left it on its side in the fridge for 5 days. At no point could I detect any leaks. Fortunately, the (delicious) beer was still carbonated at the end of the test. And according to my research, even bottled beer — presumably under much higher pressure than growler fill beer — rarely goes past 35psi. When asked whether the screw cap or the swing cap would be better for keeping carbonated beverages fresh, the Klean Kanteen rep told me that the bottle’s Swing Lok cap would be the best. (Many people in the beer community share this opinion about swing tops in general, and a Miir representative went as far as to say that their swing top bottles were guaranteed to be better than screw top bottles at keeping beer fresh.)

Swing caps have another benefit when it comes to growler fills. Although relatively few in number, there are breweries out there that will refuse to fill screw top bottles! Several reasons exist for this, including freshness concerns, being able to fill to the very top without wasting beer, and ease of use for the bartenders. Another point in the swing cap’s favor.

One final benefit of this particular cap is that the mechanism isn’t built into the neck, but instead attaches around it. This means that you can take it off entirely and use a screw cap if you want a change of scenery. (The threads for the screw cap are still there.) There are also no finicky holes that can retain moisture and odor. It’s an elegant, simple construction.

The Klean Kanteen has some downsides that are worth mentioning.

First, the volume text. The logo on the side of my old 20oz Klean Kanteen has long ago worn off, but the text on the bottom — including the volume designation — has remained perfectly intact. As expected, the 32oz Kanteen has its volume text printed in the exact same place. (Side note: not the case for the non-insulated growlers!!) Unfortunately, it seems that the company has changed the ink it uses for this purpose sometime over the past year. Whereas the writing on the bottom of my 20oz bottle is slightly elevated and subtly textured, the writing on the 32oz bottle is perfectly flat and “inky”. I already see some of the letters showing signs of wear. For growler fills, the volume designation is very important — several breweries have told me that they need to see it on the bottle for them to fill it — and I’m worried that this new ink will wear off over time like the logo on the side of my 20oz. If so, I’ll have to figure out a way to get the text back on. (Laser etching?)

Next, the cap. It’s not perfect. Several models of the growler I tried in-store at REI felt very clicky and squeaky when opened, to the point where I immediately pushed the Klean Kanteen to the bottom of my priority list. The model I got on Amazon did not have this problem — it’s a pleasure to lock and unlock and even requires far less force to close — but it’s something to look out for. A second, minor issue is that when closed, the cap has a very slight tilt towards the clasp side of the bottle. Again, this was more pronounced on the REI bottles than the bottle I ordered online, but it’s still noticeable if you look for it. Finally, the silicone seal does touch the liquid inside the bottle, so you’ll have to clean it carefully to avoid residual smells.

Perhaps my biggest gripe with this bottle is the paint job. It looks lovely in photos, but it’s not as great in person. Unlike the beautiful glossy and matte coatings on other Klean Kanteen bottles, this one looks and feels more like anodized or tinted steel. There’s also no paint on the neck, despite what the marketing photos may show: the body paint stops in an abrupt and somewhat blurry line towards the end of its curve. It’s clear that these are essential compromises to prevent the swing cap from damaging the paint; if you attach the Swing Lok cap to a typical colored Klean Kanteen (which — again — look and feel stunning), you will get scratches and void your warranty. Still, I wish it was a little better done. On the bright side, I do like the actual tint quite a bit. It’s a very unique coppery color with a nice sheen to it.

All that said, these are merely minor quibbles. This bottle is great. I love holding it. I love opening it. I love handing it off to get a growler fill, and I love the feel of its weight when I get it back. It does every job I throw at it admirably. Really, it makes me super happy — happy enough to write a 3000 word review.

And really, what more could you ask for from a water bottle (slash covert beer growler)?

Vancouver

$
0
0

On August 23rd, 2013, my Seattle friends and I boarded the Amtrak Cascades and headed up north in the direction of Vancouver. I couldn’t resist bringing along a small bag of produce and dairy that I’d been using to cook in the hostels, even though it added a few pounds of heft to my already-painful load. (This pattern will be shown to repeat itself throughout my travels.)

The ride was clean and beautiful. We planted ourselves in the observation car for most of the 4-hour trip. Dangerous electric tea was made. I was introduced to a devious card game called Bartok, in which each player adds a new, arbitrary rule during their turn. Every time a player breaks a rule or asks a question, they draw a card. As you might expect, it gets a little bit crazy.

Half an hour from the border, I realized that I was woefully underprepared for my customs interview. What was I doing? Where was I going? Should I have brought some printouts of my itinerary? Did I need to hide my butter?!

The border agent gave my slightly disheveled self a stern look before letting me through. Little did he know of my contraband!

I booked into my hostel, while my friends checked into a hotel in a… slightly seedy part of town. (Given how high Vancouver is rated in terms of life satisfaction, I didn’t even know it had seedy parts!) We had a good laugh observing some friendly neighborhood drug dealers from the hotel room’s 3rd story window.

So… Vancouver! What’s it all about?

Well, it’s a peaceful little city. Stanley Park is lovely to walk through, and it seems like there’s a lot of interesting restaurants and other businesses. (Once place I want to give a shout-out to is Fritz European Fry House. I don’t think the poutine there was “real” — the cheese curds were kind of melty — but it was still some of the best poutine-like stuff that I tasted during my Canadian trip.)

However, much like the last time I visited, it didn’t strike me as a city I wanted to spend too much time in. The architecture was kind of drab, and the whole place had an overbearing “new development” feel. Not cozy. Didn’t warm the heart. I don’t know how else to put it.

However, I’m very thankful for the company I had. The trip wouldn’t have been the same without it!

Onward we go… aboard the VIA Rail.

Backgroundifier

$
0
0

I made a Mac app! It’s called Backgroundifier, and it turns any image into a desktop background. (But it’s better for fine art and illustration.)

You can pass an image like this…

…through the droplet…

…to get something like this:

I collect lots of art from websites like /r/imaginarycityscapes and artist blogs. Unfortunately, there never seems to be enough time in the day to actually sit down and look through it all. As a result, it mostly sits and gathers dust in a directory in my Dropbox — not a great place for art to be.

So I’ve been thinking of ways to get it in front of my eyes. On a Mac, the desktop background seemed like the perfect place to put it, especially since OSX natively supports randomly rotating your desktop background from a directory of images. Unfortunately, since all my art was in different sizes and aspect ratios, it looked ugly with the default letterbox color that OSX added to compensate.

After seeing the visual design of tomkinstinch’s Artful app, I realized that images could be framed more cleverly. By processing the image instead of using a solid color, you could create a background that hinted at contents of the image while still being subdued enough to serve as a backdrop. But Artful didn’t support local files; it pulled its art from the web. Furthermore, like many Mac users, I’m a proponent of keeping things close to the defaults. What I wanted was a basic utility that could simply input my images and output the backgrounds, allowing the OS do the background rotation for me. No need to keep any apps open; no compatibility issues; nothing but a basic command line utility with a bit of GUI attached.

So that’s what I made. In addition to the GUI, the app actually supports command line mode. If you Terminal into the MacOS directory inside the package, you can run the Backgroundifier executable straight from the command line. (On my machine, I’ve even set up an Automator script to watch my input image directory and automatically convert any new arrivals.) Unfortunately, due to sandboxing restrictions, you can only read and write to the ~/Pictures directory unless you use the GUI… but check in the Resources directory in the bundle and you might find something a bit more robust!

This was mostly a small side project for me, but I couldn’t help but implement a few bits of UI bling. One is the animation of the droplet:

(That shadow looks better when it’s not in a gif!)

Unfortunately, doing this on OSX is a bit more tricky than on iOS. Whereas in UIKit, you can access (and transform!) each view’s layer without any issues, this is disabled by default in AppKit. And even if you do enable layer-backed views, transforming them is not officially allowed. One of the reasons UIKit feels so good is because layers are supported on the most fundamental level; I hope that we get a similar framework update for OSX sometime in the near future. Visually, the current tech stack feels like it’s stuck in the 90’s.

The app is mostly open source. I’ve decided to not release my one user interface nib file for now, but everything else is up for grabs. It’s written in Swift 2. (The repo is a bit out of date, but I hope to commit my latest changes in the near future.)

iPad Pro + Pencil Slow Motion Bug

$
0
0

I noticed an interesting problem with the Apple Pencil while developing my app. It seems that if you’re using the Pencil while simultaneously using a gesture recognizer (as, for instance, in a scroll view), touch processing goes into slow motion. (Approximately half-speed, according to some quick measurements.) Seems there’s some sort of interference between Pencil and gesture event processing. Notably, the framerate remains stable while this is happening.

I noticed that the Paper app also has this problem. Other drawing apps seem to avoid it (accidentally or intentionally) by disabling canvas navigation while drawing. In Procreate, you can adjust the brush sliders while drawing without any slow motion, but I think this has to do with the fact that Procreate uses a custom OpenGL-based implementation for their widgets, not UIGestureRecognizer.

I can reproduce this bug in Apple’s TouchCanvas demo by sticking a scroll view to the left of the screen and continuously scrolling it while drawing. At first, it behaves correctly. But when the CPU usage hits a high enough level, you get the behavior described above. If you do the drawing with your finger, the problem disappears. The framerate does drop, but the touches don’t continue when you lift your finger; they simply get delivered with less frequency, and the scroll view stops scrolling immediately once you lift your finger.

My hunch is that the sampling frequency of the Pencil messes up the usual touch handling behavior when under load. That would explain the 2x factor: the Pencil has a 240Hz refresh rate while touches normally get sampled at 120Hz.

Regardless of whether this is an iOS bug or something I messed up on my end, I’d love to know if there’s a way to fix this! Simultaneously scrolling with your hand while drawing with the Pencil should be a given.

The Stanley 64oz Classic Vacuum Growler

$
0
0

Shortly after I posted my Klean Kanteen review, Stanely offered to send me one of their own bottles to play around with: the 64oz Classic Vacuum Growler. I was quite happy with the build quality of the Stanleys I saw at REI, so at risk of becoming a professional growler reviewer, I figured I’d give this bottle an equally thorough look.

Construction

There’s a lot of good things to say about the construction of this growler. The outer shell, painted in the usual Stanley enamel-matte-green, is very solid and attractive in both look and feel. There’s certainly no concern about the coating fading or chipping with use. The plastic is also very smooth and feels higher quality than the typical materials used in these kinds of bottles.

The latching mechanism that encircles the neck and lid of the bottle is made entirely from plastic. This is very concerning to me. The way it works is very simple: a metal loop, attached to a plastic tab, snags around a plastic hook in the lid and then depresses with a plastic lever. Looking closely at the lower plastic hinge, there are white stress marks where the hinge attaches to the lid, right out of the box. Similarly, on the upper plastic hook, you can see the plastic starting to wear down and whiten from the shape of the metal loop. This wasn’t just my bottle, either: every Stanley growler I’ve seen at REI has had these marks. I’d love to be proven wrong, but my feeling is that these parts will be the first to fail.

On the other hand, the rubber/silicone material used in the gasket feels very sturdy. I expect it to last for a long time.

The Stanley logo on the side of the bottle looks like a sticker and I’m concerned it will eventually fall off. Compared to the rest of the body it looks a bit cheap.

Design

The lid opens very, very smoothly — almost surprisingly so. There’s just enough friction in the joint that it stops almost parallel to the neck of the bottle without ever touching the body. It feels great.

The latch is slightly problematic in that the lever is very easy to smack against the side of the growler — loud and a bit troubling due to the plastic construction. (But you learn to avoid this fairly quickly.) However, the design is certainly preferable to the Miir-style puzzle-latch. It’s simple and effective.

This is a wide-mouth bottle, and you can read about the trade-offs in my previous article. (To summarize: I think you will find the small, rounded lip of the Klean Kanteen better for pouring and drinking, but the wide mouth of the Stanley will allow for easier fills as well as use with tea infusers, ice, etc.) This is clearly not meant to be a bottle that you drink straight out of, especially given the wide ring of plastic around the lip (though I’ve been doing it a little bit and it’s been fine). While similar in overall design, Stanley’s neck ring looks a lot better than Miir’s. The tolerances are much tighter, though there are still some very small gaps that I fear water might get into.

Pouring this bottle is very easy. The lid opens perpendicular to the handle, and due to the fact that the plastic latch ring is inset from the actual, metal lip of the bottle by a centimeter or so, you don’t get the usual problem of liquid trickling down the side of the bottle when you start pouring. Even if you pour slowly, the plastic edge acts as a spout and directs the stream right into your glass! I specifically wanted to avoid a handle when picking my previous growler, but after using this one for a while, I’ve come around to it: at 64oz, it’s really the best way to pour out the contents of your growler when full. One minor issue is that, due to the fact that the rubber/silicone on the underside of the lid has a central ring that juts out, liquid can gather and drip from this outcropping when you pour.

64oz is a lot of fizzy water, so I have not yet tested how well the growler keeps liquids carbonated. However, there was a lot of suction in the seal after a few hours keeping hot tea, so I think it should perform well in this regard.

The bottom has the volume and government warning engraved. Hallelujah! No chance of these rubbing off.

Volume-wise, the space in the neck gives you an extra 4oz or so over the base 64oz.

This bottle is listed as dishwasher-safe — unusual for thermally-insulated bottles.

Temperature

Due to the fact that this growler holds twice as much water as my Klean Kanteen, I performed two tests: one with 64oz of water and one with 32oz. Since 64oz of water takes a lot longer to change temperature than 32oz, I figured it would be more fair to do it this way for comparison’s sake with previous results. Do note, though, that since there’s a lot more space for the heat to dissipate when filled with 32oz, performance at that level is going to be worse than in a bottle designed to hold 32oz.

Stanley Growler (Filled 64oz)Stanley Growler (Filled 32oz)
0 minutes95.7 ℃94.3 ℃
8 minutes95.1 ℃92.7 ℃
Open
21 minutes91.3 ℃82.4 ℃
Closed
35 minutes90.7 ℃80.3 ℃
Open
56 minutes85.5 ℃68.9 ℃
Closed
71 minutes84.7 ℃67.6 ℃
98 minutes83.6 ℃66.3 ℃
220 minutes79.7 ℃61.3 ℃

Results with 64oz are fantastic and great with 32oz as well, keeping in mind the temperature/volume considerations above.

Conclusion

I’m very concerned about the durability of the plastic latch, but overall, this growler feels great. It’s also one of the most attractive 64oz growlers I’ve seen, especially in the flip-top arena. In terms of 64oz, I’d be torn between the Stanley and the Klean Kanteen, but I’d definitely pick the Stanley over the Miir and probably over the Hydro Flask. Having continuously used this growler for a few months with water, tea, and beer and receiving admirable performance in every regard, I give it a thumbs up!

(Better yet: at the time I wrote this review, Stanley didn’t offer a growler in 32oz size, but it looks like they do now! Definitely worth checking out.)


The Perfect Travel Gaiwan

$
0
0

Loose-leaf tea is a bit of a finicky hobby. Unlike the boring old teabag, you can’t just dump a bunch of tea leaves in a cup of boiling water and expect good results. At the very least, you need a way to strain your tea leaves quickly and without burning yourself. Many tea geeks enjoy the use of gaiwans for this purpose — small, lidded cups whose shape and thermal properties make them useful as single-serving teapots.

But what do you do when you travel?

Gaiwans tend to be made of porcelain or clay, so they’re very fragile. Until recently, I assumed I’d just have to make do with makeshift mug-plus-strainer brews when on the road. But a few months ago, I discovered a wonderful product that effectively fixed this problem, as well as several I didn’t even know I had: the Asobu Imperial Beverage Insulated Cup.

Although not marketed as a travel gaiwan, this product is suited almost perfectly to this role. The lid attaches loosely with a rubber seal, allowing for quick removal and keeping the temperature stable. There’s a hook on the back of the lid that can be used as a handy stopper when tilting it back for straining. The mug is vacuum insulated, allowing you to hold it normally even with boiling hot water inside; there’s no need to perform the crazy hand gymnastics you have to use with an ordinary gaiwan. The insulation also ensures that the temperature inside the mug remains very hot throughout the steeping process — hotter, I would imagine, than any other material, as barely any heat radiates out! Because the mug is made from steel, you can throw it in your backpack without worrying about it breaking. And brewing aside, it’s great for public transit: if you have a thermos full of hot tea or coffee, you can slowly sip from this mug without burning your hands.

Sure, it’s a bit ugly, while my porcelain gaiwan is beautiful. But in all honesty, I’ve been reaching for this mug every time I brew my loose-leaf tea. The practical benefits far outweigh the aesthetic concerns.

There are a few issues worth mentioning. The shape is far blobbier than it looks in the marketing photos. The paint is of poor quality and has chipped away from the bottom even with fairly light use; I’d recommend getting the stainless steel model and avoiding the colors. But overall, this mug is an incredibly functional and unique product that, to my knowledge, has no rival on Amazon.

Rethinking Musical Notation with Composer's Sketchpad

$
0
0

This blog post is part of a series on the development of Composer's Sketchpad, a new iPad app for making musical rough drafts and doodles.

Just last month, I released my first major project for the iPad: Composer’s Sketchpad!

Composer’s Sketchpad is an interactive, “doodle-y” take on music sequencing and notation. When you launch the app, you’re presented with a giant canvas that can be panned around with your finger. The canvas is covered with a grid, indicating time on the horizontal axis and pitch on the vertical. To draw musical notes, you hold down the canvas with one finger and draw with another. (You can also zoom using a similar gesture.) Unlike most sequencers, the app lets you start your notes at any time and bend them to any pitch, giving you the ability to sketch out twisted solos and complex rhythms with no extra effort. You can also snap to the gridlines if you wish.

In the coming months, I’m going to (try to) post a series of articles concerning the technical, design, and marketing aspects of the project. But for now, I’d like to write about the genesis of the idea.

I made Composer’s Sketchpad for two reasons.

The first was my inability to compose music using the tools I had at hand. Today, with the help of computers, creativity in practically all artistic mediums is blooming. Everything is digital, iterable, undoable: instead of having to buy messy paints or develop your film after every 30 shots, you can open your favorite graphics editor, grab your tablet, and tear through hundreds of sketches at a time. This reduction in creative friction must be an astounding source of growth for the arts!

Unfortunately, I feel composition has not quite made the same magnitude of leap forward. While the studios of the past can now be entirely replaced by powerful tools running on our computers, most of them are dreadnoughts aimed at heavy production or performance use. The rhetorical art of composition — the process of taking musical notes and putting them in an order that sounds good and meaningful to our ears — has yet to see the equivalent of a Word or a Photoshop. To put it another way, there’s very little music software out there with a tight creative feedback loop specifically tuned to plonking down a few notes, playing them back, and repeating the process until you get something that sounds good. You could certainly use a DAW sequencer, Finale, or even a tracker for that purpose — many composers successfully do! — but I’ve found that the delay between editing and playback is still too high in those applications, to say nothing of the often immense UX hurdles. Worse yet, barely any of these tools are optimized for touch or stylus input — surely an ideal interface for composing at the piano (or under a tree)!

DAW interfaces tend to be aimed at production, not freehand composition.

Tooling aside, many musicians write music by simply improvising on their instrument and recording the results. Sadly, this is not a tenable approach for amateurs. Whenever I’m sitting at the piano and get the flicker of an interesting idea, I always lose the thread by the time I get around to actually playing or writing it down. I’d need many more years of practice to actually be able to compose anything interesting using this approach.

It was clear that my musical life was missing a tool that allowed for the rapid entry and immediate playback of notes — a kind of interactive, audible sheet music. Whenever inspiration struck, I wanted the ability to pull out my tablet or phone and jot down my musical thoughts in a matter of seconds.

Composer’s Sketchpad fulfills this demand by heavily prioritizing navigation and note entry above all else. There’s no mode for moving around the canvas: you simply swipe like in a mapping app. To make a note, you hold down the canvas with one finger and draw with another, as if applying pressure to a particularly slippery piece of paper. Undo/redo and erase are right there in the corners, and the playback controls are within easy reach at the bottom of the screen. (Your current viewport is also your playback position.) A piece is divided into several layers of notes — each with their own instrument or percussion set — and the active layer can be changed with a simple swipe of the instrument box at the top of the screen. Doing this brightens the active layer and dims any background layers, allowing you to edit the current layer exclusively while still having a sense of what all the other instruments are doing. In short, there’s barely any cognitive overhead: every tool you need to rapidly sketch out music is right there in front of you.

The UI in Composer's Sketchpad is carefully tuned to enable a tight creative feedback loop.

The other problem I wanted to tackle was the antiquated nature of sheet music.

I love classical and popular music alike. Unfortunately, popular music differs enough from classical to make traditional notation simply the wrong tool for the job. For one, practically every piece of popular music uses syncopated rhythm. Most classical music is fairly on-the-beat, and notation is designed with that in mind. With syncopated music, you usually end up with a mess of rests, ties, and dotted notes that is hard to read and write. Modern music is also irreverent when it comes to duration and pitch. Solos are a great example: they’re fluid and expressive, and each note only lasts and remains on pitch as long as the performer wishes. Once again, notating them is an incredible pain.

What about music from other cultures? The scales for many musical traditions differ vastly from our own. Simply put, their notes fit between our notes! Composers shouldn’t have to switch their system of notation just to play around with different kinds of music.

What about our own (Western) myriad of different scales and tunings? We’re so used to Equal Temperament that we’ve completely forgotten the incredible polyculture of tunings in early music, to say nothing of the microtonal experimentation of the 20th century. Indeed, the Scala microtonal tuning archive has over 4000 scales! All impossible to convey with traditional notation.

An excerpt from "Comfortably Numb". Solos are cumbersome to write out using traditional notation.

Instead of trying to shoehorn all music into a Western style of notation invented several centuries ago, it occurred to me that maybe an entirely new approach was needed. In fact, why encode the music at all? The barest, most obvious form of notation is a graph of time and pitch. This system would have been too difficult for musicians to read from and write to back in the day, but that’s not really an issue when the music can play itself and your musical canvas is tactile and effectively infinite. It seemed like the best shot at covering all bases.

In Composer’s Sketchpad, each “note” is a simple array of times and pitches. Note names, measures, and time signatures still exist, but only as a grid or stencil over the absolute time/pitch graph — a feature of the tooling, not an intrinsic part of the piece. You use the current scale and meter grid to align and snap your notes, but you can also change them up for use in later sections without having to worry about your existing music. Under the hood, it’s all the same: scattered little point clouds of absolute time and pitch.

As a result, writing out complicated and expressive sections of music in Composer’s Section no longer feels like caging a canary. The grid is your friend, not an oppressive bureaucrat. If you want to write more conventional music, snapping to time or pitch works just as it did with traditional notation. But turn snapping off and you can bend your notes or extend them out to the exact length of time needed. Because the shape of your notes corresponds exactly to their time and pitch, a quick glance at your music tells you exactly how it’s going to sound without having to decode a mess of symbols in your head. And you can edit any part of any section or overlap as many notes as you like without having to worry about fulfilling the “note quota” of a measure, as you constantly have to do in sheet music apps like Finale.

"Comfortably Numb" again in Composer's Sketchpad, transcribed by ear. This style of notation is great for conveying expressive melodic lines.

And now, a bit of a pretentious digression, as well as a few words about the future of the project!

Most programmers today are very excited about building for the web. Each day, the front page of Hacker News is filled with stories of up-and-coming web-based startups as well as new Javascript frameworks and libraries. There’s also a lot of related talk about how pay-to-own software is going away, and about how subscriptions, in-app purchases, and advertising are going to replace them.

Speaking strictly for myself, I can’t stand the idea of my software residing permanently on other people’s servers or relying on money from other people’s products. For every new and exciting startup that I read about, there’s a related story of yet another acquisition or shutdown flushing years of work down the drain. What were all those man-hours, well-intentioned ideas, and midnight flashes of inspiration good for in the end? No, that style of development is not for me. I don’t want to create services, networks, or support organizations; I want to create beautiful objects, little bits of clockwork that are both lovely and useful in their whole. Once they’re made, I want to be able to take my hands off them and let them live without my further involvement. I’m not interested in running servers or providing exciting new content patches week after week. You pay once — you get the object. There is certainly room for updates, but only in the interest of making the object better.

I’m still far from that ideal — for example, in-app purchases might be tempting down the line — but that’s the direction I want to head with my current and future projects.

Composer’s Sketchpad isn’t a profit-driven venture looking for a market or buyer. It’s one of those objects that didn’t exist in the world before I made it, and its reason for being is to help me be creative. I plan to add many new features over the years as my compositional needs develop, and I hope that eventually I’ll be able to port it to other platforms and release the source code for everyone to use. It’s a tool from my own personal toolbox that I’m happy to put out into the world.

Some of the many design sketches made during development.

Using cloc, the project comes in at around 20,000 lines of code. This is a massive leap for me, and I’m incredibly excited to get started on the next 20,000!

Path Rendering in Composer's Sketchpad

$
0
0

This blog post is part of a series on the development of Composer's Sketchpad, a new iPad app for making musical rough drafts and doodles.

Before starting any work on Composer’s Sketchpad, I had to ask myself: was the app as I envisioned it even possible to make? My initial goals were as follows:

  1. Have an (effectively) infinite canvas with a large number of notes visible at once.
  2. Allow the canvas to zoom and pan with without any lag.
  3. Allow the canvas to zoom without the notes losing any sharpness.
  4. Have notes belonging to the current layer blur or fade when the layer switches.
  5. Allow the notes to stretch and skew when the grid scale changes without distorting their appearance. (I ended up dropping this for the release version.)
  6. Have the whole thing run at 60fps on my iPad 3.

I had barely done any graphics programming up to this point, so I had to feel out the limits of OpenGL as I blindly barged ahead — always a painful way to develop a project.

At first, I briefly considered implementing drawing using Photoshop-like brushes: that is to say, directly toggling pixels on a bitmap as the user moved their fingers around the screen. This seemed appropriate for something that behaved so much like a drawing application. However, I quickly realized that the representation of my notes should really be separate from their rendering (as with any good MVC design), which meant that I couldn’t just save the bitmap and be done with it. Furthermore, the finite resolution of the bitmap presented a number of problems. How could I implement high-quality zoom without having to re-render everything? How would it be possible to have an infinite canvas without having to implement a complex tiled rendering system? (I was aiming for maximum possible simplicity in my architecture at this point.) How could I switch layers seamlessly without storing a bitmap for each layer? It just wasn’t a good fit given the dynamic features of my app.

So I decided try for a path rendering approach instead. My musical notes would be stored as points in time/pitch space, and then the renderer would convert them into pleasing, smooth curves to draw on the screen.

There were two obvious techniques that came to mind. The first was to use the CoreGraphics path renderer in the form of UIBezierCurve. This approach was compelling because CoreGraphics’ path drawing support was very robust, to the point of being the foundation of heavyweight applications like Pixelmator. Strokes, fills, and complex shapes were incredibly simple to define and draw, and it was all built-in and battle-tested by thousands of developers.

The second approach that I immediately considered was to convert each curve into polygons and draw them using the GPU. My experience with hardware-accelerated vector graphics, though minor, was very positive: performance seemed smooth and stutterless compared to software rendering in the apps that used it, and the artifacts that occurred while waiting for data to load (jagged lines turning into smooth lines) felt a lot more pleasing than those in software renderers (blurry lines turning into crisp lines). Intuitively, even though I didn’t know any of the details at this point, the idea of manipulating groups of polygons on the GPU rather than plotting out curves pixel-by-pixel seemed like a very efficient approach. In addition, I knew that with polygons, scales and other translations would be effectively free.

Unfortunately, there weren’t any built-in iOS frameworks that allowed me to do this. (Or — so I thought. I later learned that SceneKit had support for path rendering, but it turned out to be fairly ugly, slow, and not particularly configurable.) I didn’t really feel up to the task of rolling my own solution, so I decided to hunt around for a framework. With high high-DPI resolutions ruling the roost and minimalist UIs being all the rage, surely somebody had a stable framework for hardware-accelerated path rendering on iOS?

During my search, I learned of OpenVG, the Khronos Group’s standard for hardware-accelerated path rendering. It seemed to be exactly what I was looking for! Unfortunately, the standard appeared to be all but abandoned1, with Nvidia’s NV_path_rendering being the only other standard trying to take up the mantle. (Naturally, this was not an extension that iOS OpenGL ES supported.) However, I did manage to find an OpenGL ES framework called MonkVG that incorporated a subset of OpenVG suitable for my project. Looking at the terrifying, twisted, OpenGL-ES-2-wrapping-OpenGL-ES-1-code, I feared that I might have great difficulty using the framework on iOS. Fortunately, it turned out that MonkVG only handled shaders and polygon drawing, not setting up the context or any other platform-specific technical details. After creating my GL UIView and fixing a couple of minor errors, I was good to go.

(Quick aside: over the course of my research, I learned that there were several ways to hardware-accelerate path rendering. MonkVG’s tessellation approach worked fairly efficiently, but it was also imprecise on account of the fact that you ended up with polygon strips in the end. It also required the actual tessellation step to be done on the CPU. In contrast, there are newer approaches that can render the paths directly using shaders. These tend to have high accuracy and detail, and they only require a single bounding-box polygon for the entire shape. Unfortunately, in my limited testing, I found the performance of this approach to be lacking on my iPad 3. Additionally, as I further discuss below, the polygon strip approach turned out to be ideal in the case where shapes needed to be generated once and then cached for future reuse and transformation. From what I can tell, it’s far more difficult to cache rendered curves using the shader approach.)

Having figured out how to use both frameworks, I made a quick performance prototype: a simple draw loop at 60fps with a set of full-screen, randomly generated Bézier curves in each frame. (I considered this my worst-case scenario, as in the case of changing the width or height scales of my grid.) There were two rendering paths: one for CoreGraphics and one for MonkVG.

A randomly-generated scene from the stress test. CoreGraphics couldn't handle it while MonkVG passed with flying colors.

Sadly, as much as I wanted to stick with the battle-hardened CoreGraphics approach, it wasn’t even able to draw a single animating curve at a solid 60fps on my iPad 3. MonkVG, on the other hand, tore through 10+ curves without breaking a sweat. Graphically, the results of both approaches looked quite similar to me — or at least good enough for the app I was trying to make.

After closely examining the technical underpinnings of both technologies, I learned that CoreGraphics was doing all of its work on the CPU, while MonkVG was tessellating its curves into polygon strips CPU-side and then sending them to VBOs on the GPU — one per curve. Interestingly, the performance difference still applied even when accounting for the tessellation step. Presumably, this is because drawing multiple polygons is a very fast and parallelizable operation on the GPU, even if they number in the thousands, while generating a Retina-sized Bézier curve on the CPU requires you to touch tens of thousands of pixels by “hand”. (But don’t quote me on that — I am definitely still a novice in this area.) It also helps that MonkVG is much less precise than CoreGraphics in its tessellation.

Despite these finds, I really wanted to work with CoreGraphics, and so I attempted to return to the technology a number of times during my project. (Perhaps I missed something that would account for the massive performance difference?) Notably, I tried using it together with CATiledLayer, hoping that this particular optimization would offset the inefficiencies of the software render. But even though I could now pan and zoom at 60fps, the chunks loaded far, far too slowly for realtime use — I’m talking on the order of several seconds whenever you zoomed in. So that was that.

For much of the project, owing to my inexperience, I was burdened with the question of the framerate cap. If everything was done perfectly, how high could I go? After getting caching and draw call batching working correctly, my MonkVG implementation yielded an acceptable framerate of 30-60fps in the general case, but I still wondered if I was an order of magnitude off on account of my incompetence. How did Apple Maps manage to work so smoothly with so many shapes on screen? How did web browsers manage to display thousands of character paths — possibly with transforms! — and still attain smooth performance? In truth, the geometry I was showing on screen was fairly complex: each quarter-measure note had about 300 triangles once you included the outline and endcaps, leading to an upper bound of almost 400,000 triangles on screen for a dense piece (12 notes, or 4 chords, per measure per layer, with 10 full layers and 10 measures on screen). Surely a breeze for modern machines, but quite a lot to handle for an old iPad! It’s always important to be able to answer the question, “where is all that performance going?”, and in my case it was going towards the multitude of dynamic features in my app.

The final mesh structure of the note curves. Each blue dot is a recorded time/pitch sample.

In retrospect, it was quite fortunate that I just happened to fall into the optimal rendering implementation for my project. My path rendering requirements were a little different from the norm: after eliminating point #5 from my initial goal list, I no longer needed most of my curves to be dynamic. The only curves that actually changed their structure from frame to frame were those belonging to the notes currently being drawn or erased — and even those were localized to the point of editing, since my notes were comprised of multiple tiny Bézier segments chained together. (I’ll talk details in a later article.) Instead, my highest priority was to translate and scale my notes at a constant 60fps while still preserving their sharpness, and polygons happened to be uniquely qualified for this task. At the expense of jaggedness on close zoom — fine by me — polygon tessellation effectively gave me infinite resolution along with free transforms. They were also perfectly suited for caching: once tessellated, I could store the vertices of the generated polygons for all my note objects without wasting too much space, wheareas doing the same with textures would have quickly filled up my memory. (To say nothing of being terribly ugly when scaled.) Better yet, I eventually learned from profiling that it was the tessellation step — not drawing — that was the biggest roadblock in my path rendering implementation, so caching was absolutely critical to getting the performance I needed! Had I used a shader-based approach, I would have had to write a ton of additional code to enable caching, and it still probably wouldn’t have gotten close to my 60fps benchmark for large scenes.

For convenience, I decided to use Cocos2d as the graphical core of my project, so I had to figure out a way to wedge MonkVG in there somehow. Fortunately, it was fairly simple to write a Cocos2d CCNode wrapper around MonkVG’s rendering functions, since each CCNode in Cocos2d had a lovely overridable draw method that allowed you to call OpenGL functions directly. (One reason why you might want to use it over SpriteKit!) Still, especially as the project moved forward, the framework was becoming a massive cognitive burden. MonkVG had a fairly old and creaky codebase, and as the hacks I made to enable unsupported features such as real-time color changes piled on, I realized that it would take me forever to actually fix everything up and integrate it into my project in a sensible and modern way. Additionally, the one-VBO-per-curve approach was quickly becoming unwieldily. Already I had to synchronize a ton of state between MonkVG and the rest of my app in order to cache previously-tessellated curves for performance reasons; far more daunting was the idea of cycling VBOs in and out of GPU memory while retaining the underlying geometry client-side, which I wasn’t doing yet. Unfortunately, this was going to be necessary to enable my potentially-infinite canvas, and MonkVG didn’t natively support it.

At this point, I realized that even though writing my own hardware-accelerated path renderer was probably too much work, my path rendering needs were actually very simple. I didn’t need to create unions of shapes, arbitrary curves, or even strokes! All my note shapes were simple Bézier “tubes” with round endcaps and outlines, and so creating a super-simple note renderer for direct use in Cocos2d would require minimal effort. Better yet, most of the work was done already: for the purpose of creating better outlines, I was already interpolating all of my Bézier curves myself and only using MonkVG to draw and tessellate the final polygon tubes from the perimeter vertices. The only thing I needed to do was to create polygon strips from my points and send them through Cocos2d’s native rendering system, as well as to write a bit of code for the round endcaps. Sure, it bummed me out that this would preclude me from creating more complex and dynamic shapes in the future, but I was on a deadline and the project needed to be shipped. Despite my initial trepidation, the process was extremely quick and only took a couple of days of work.

As I dove deeper into Cocos2d’s architecture, I was struck by the beauty of its rendering pipeline. Unlike MonkVG, there was no VBO juggling here at all: the geometry for each object in the entire scene graph was sent to the GPU anew in every frame. (I soon learned that this was called “geometry streaming”.) This approach completely eliminated the need to track the mapping between tessellated curves and their corresponding VBOs, eliminating hundreds, if not thousands, of lines of brittle complexity in my app. What’s more, Cocos2d batched draw calls automatically, meaning that all your geometry would automatically coalesce into just a couple of draw calls without having to do any extra work, even if it resided in completely separate CCNodes. This was a massive benefit I was not expecting!

There was a new problem, though. With the new path rendering system (now using geometry streaming), my performance hovered at around 50%-70% of what I was getting in MonkVG. Intuitively, I feared that a fix was impossible: wouldn’t uploading thousands of polygons in every frame be naturally more resource-intensive than storing polygons in VBOs for later reuse? But after some digging, I learned something very interesting: OpenGL ES actually allowed you to map GPU memory directly into a shared application-space buffer, eliminating the need to “upload” your geometry at all! I did a sanity check and realized that there was no way that copying even hundreds of thousands of polygons to a buffer could be the roadblock. But Cocos2d was already using memory mapping in its renderer! So what was the problem?

There was another issue with the Cocos2d renderer. If I opened a project with lots of notes, my performance was usually 60fps at the default zoom level. But if I zoomed out (getting lots of notes on screen) and then zoomed back in, the performance dropped to 30fps or lower and then never actually recovered, no matter how closely I zoomed! This didn’t make any sense since my notes were always culled to the current viewport, meaning that performance should have always been identical for any given view. This also never happened in MonkVG.

I decided to dig even deeper. After several hours of Googling keywords related to geometry streaming versus VBO performance, I zeroed in on a fantastic thread in the OpenGL forums. In this thread, Rob Barris provided a very clear rundown of exactly how OpenGL managed memory in streaming situations. As it turned out, there was a certain art to managing your geometry buffers in a way that avoided stalls in the pipeline.

Digging around in Cocos2d’s renderer, it seemed that the streaming architecture had some inefficiencies and bugs. There was a single memory-mapped buffer that was used for all the geometry, but it got “orphaned” and subsequently re-allocated in every frame. This caused a continuous allocation of up to several megabytes per frame — a significant performance consideration. What’s more, if a given scene was big enough to fill up the buffer, it expanded in place, but then never actually shrank back to its original size once the extra space was no longer needed. This meant that if you happened to render a giant scene — say, requiring a 9MB buffer — you would get an allocation of 9MB in every single frame after that, regardless of its complexity!

At this point, I had a solid intuition for my next course of action. Rob Barris’s advice, as well as some other references, indicated that it was possible to use the same buffer over the course of several frames without having to reallocate it. You could simply keep a pointer to the last filled slot in the array and start from there for your next frame. Once you ran out of memory, you could orphan your buffer and have OpenGL allocate you a new one. If you set your sizes correctly, you’d have one large allocation every few frames instead of an allocation in every single frame. In the end, I opted for a slightly simpler (architecturally-speaking) double-buffer solution: two large buffers side-by-side, with one buffer always serving as the main buffer. Once the main buffer ran out of space (over the course of several frames), the other would become the main buffer and allow the first buffer to orphan and reallocate.

Fixing the renderer required a number of very low-level changes, but it was somewhat easier than expected, largely on account of Cocos2d’s excellent code organization. After making my changes and fixing a couple of Cocos2d bugs along the way, I fired up the app and was delighted to see my old MonkVG framerate — maybe even a little better! Not only did I have my original performance back, but I also gained an immense amount of architectural simplicity along the way.

In the end, all the goals I set for myself were successfully met. The app does run at 60fps much of the time on my iPad 3, though it’s naturally far more prone to dropping frames than newer devices. (An iPad Air 1 and my iPhone 5s barely dipped below 60fps even in complex scenes.) Real-time tessellation is still something I want to implement for grid scale changes; I doubt I’d be able to truly do it in real-time for a full screen of notes, but I can think of several ways to fake-transform existing tessellated notes while waiting for the correct tessellation to load.

Finally, I do hope that somebody releases a stable, hardware-accelerated path rendering framework for iOS. It’s sorely needed for vector-based apps with dynamic content!

  1. Curious, I asked programmers.stackexchange.com about this issue and got some interesting answers.

Liberating Pitch and Taming MIDI in Composer's Sketchpad

$
0
0

This blog post is part of a series on the development of Composer's Sketchpad, a new iPad app for making musical rough drafts and doodles.

I wanted Composer’s Sketchpad to have the ability to represent musical notes at any pitch. In order to do this, I needed to solve two problems: representing arbitrary pitches internally and making them compatible with MIDI.

Human perception of pitch follows a logarithmic curve, meaning that a frequency will sound an octave higher when multiplied by two. However, we tend to think of notes in a linear fashion: C4 is a fixed distance from C3 on the piano, just as C3 is from C2.

The naive approach to representing pitch would be to simply store the frequency in hertz and be done with it. But this didn’t sit right with me: since my canvas depicted pitches linearly like on a piano keyboard, I’d have to be constantly taking the logarithm of my points and subsequently introducing possible floating point errors as we went up the ladder. The pitches would also have to be stored as irrational floating point numbers, making it impossible to tell whether a point is sitting precisely on a pitch gridline.

So I decided to represent my pitches as cents. Cents are the linear unit counterpart to frequency: C4 is 1200 cents from C3, and C3 is 1200 cents from C2. (Per equal temperament tuning, each piano key is 100 cents apart from the next.) This means that cents aren’t an absolute unit like pitch, but rather the function of two frequencies: in order to get the expected 1200 cents from C4 (261.6Hz) to C3 (130.8Hz) we take the base-2 logarithm of C4 divided by C3 and multiply by 1200. As convenient as these units were, I still needed to represent my points in an absolute way, and so I created an internal unit of “absolute cents”: simply the number of cents a pitch is from A440. If you peek inside a Composer’s Sketchpad JSON file, you’ll see that C4 has a value of -900, B4 a value of -1000, etc. Mathemtacially convenient and human-readable!

Different representations of pitch for the inflection points on a single note.

The second problem was a little trickier. Internally, the app was using the built-in MIDI functionality found on iOS, in the form of MusicPlayer and AUMIDISynth. Unfortunately, traditional MIDI — having been designed in the stone age of computing — didn’t support arbitrary pitch values. Instead, you were given a measly 128 MIDI notes, each corresponding to a note on a standard, equally-tempered (and slightly extended) piano. This was great for interfacing with hardware MIDI keyboards, but hardly appropriate for playing back arbitrary pitches.

(To be clear: MIDI is simply a standard for sending instructions to a synthesizer. While the standard is very limited and fiddly, it does have the advantage of being supported ubiquitously. You can also save your MIDI packets to a file and use it with a wide variety of software. However, synthesizers themselves are usually much more robust. When interfacing with them directly, you may well be able to play arbitrary pitches and use other custom functionality. The thing you’d lose by going this route is compatibility with existing technology, which is frankly a very big hurdle.)

There are several ways to alter the pitch of a MIDI note, some more widely supported than others. The most common is using the pitch-bend wheel. Another is using the MIDI Tuning “Standard” (which is in fact hardly supported anywhere). Yet another is using polyphonic aftertouch, but only after setting up your synthesizer to correctly parse the signals. For its ubiquity and semantic correctness, I decided to go with the pitch-bending approach. To play back an arbitrary pitch, I’d simply play the closest MIDI note and then bend it up or down to the desired frequency. However, there were two issues with this approach. First, the pitch-bend wheel applied bending to the entire keyboard range, not just individual notes. This meant that with the naive implementation, you could only play a single arbitrary pitch at a time! Second, the default range for the entire pitch-bend wheel was a measly whole tone up or down, which was simply insufficient for arbitrary bends. (For wider bends, one might consider getting around this problem by bending to a note, stopping the first note, playing the second note, and continuing the bend. However, this sounds pretty poor due to the fact that most instruments have a distinctive-sounding “attack” that appears as soon as you play a note. This makes the bend sound discontinuous at MIDI note boundaries.)

I’ll get into the specifics of my MIDI architecture in a later article, but in brief, I solved the first problem using MIDI channels and multiple MIDI instruments. A MIDI instrument can often have 16 so-called channels, which are sort of like presets. Each channel has its own setting for instrument, volume, vibrato, and — conveniently — pitch bend, among many other properties. Whenever you play a MIDI note, you assign it to a channel and it plays with the corresponding properties for that channel. For my use case, this meant that if I used each MIDI channel for playing just a single note at a time (as opposed to the usual approach of playing multiple notes per channel and assigning each channel to a unique instrument), I could have 16 notes simultaneously pitch-bending at once! I wanted more polyphonic notes than that, however, so I decided to simply create a new virtual MIDI synth for each instrumental layer in my app: 16 channels per instrument, with 10 maximum instruments at once (for now). Surprisingly, even 10 maxed-out MIDI synths playing simultaneously didn’t peg my iPad 3’s CPU too hard. Kudos to a great audio architecture!

The second problem — limited pitch-bend range — was solved using a so-called MIDI RPN, or registered parameter number. These are special, widely-supported MIDI commands that let you configure certain properties of your synth, with one of the primary ones being the range of your pitch-bend wheel. (Note that I say widely supported, not universally. Only about half the software I’ve tried seems to understand the pitch-bend range RPN. Fortunately, Apple’s built-in synth does just fine.) Rather than having each tick on my virtual pitch-bend wheel correspond to 0.024 cents (as is the default), I sent an RPN command at the start of playback to make each tick equal to one cent. Completely impractical for a physical weel, but quite conveinent for our use case! (Incidentally, this makes the new pitch-bend range +/- almost 7 octaves. Except for the most esoteric use cases, it’s totally unnecessary to go any further than that, since even a pitch-bend of a single octave sounds pretty terrible on most synths.)

All in all, it’s a messy, imperfect system, but it gets the job done. I can take a bunch of pitches stored as “absolute cents” in my JSON file, push them through a few conversion functions, retrieve a set of MIDI packets on the other end, send them to a bunch of virtual MIDI synths, and have them sound as the correct, precise audio frequencies through my speakers. Maybe someday a more modern standard like OSC will reign supreme and allow this sort of architecture to be radically simplified, but for now, we’re unfortunately a bit stuck in the 80’s.

Headphone Jacks Are the New Power Plugs

$
0
0

Blog comments are out, blog responses are in — and so I thought I’d respond to John Gruber’s recent article titled “Headphone Jacks Are the New Floppy Drives”. Here’s why I think removing the headphone jack would be a bad idea at this moment in time:

  • Poor wireless options and standards. I use Bluetooth headphones and I love them, but they’re a world of compromises. Audio quality is far from lossless, and not just because of the codecs: with the sound off, you are likely to hear noise and static from the radio right next to your ear. (This does not bother me, but would drive many people crazy.) Switching between devices is a pain. Pairing is a pain. You have to remember to charge them. There is unbearable latency for games and occasionally even movies. Few audiophile-level headphone makers bother with Bluetooth headphones, leaving us with just the consumer brands. They can only be as powerful as the battery-powered driver. Might Apple introduce a new wireless codec that tackles all of these pain points? Sure. But then we get:
  • Vendor lock-in. Apple Wireless or Lighning headphones wouldn’t be compatible with much else. Not a problem for cheap earbuds, but definitely a big deal for high-quality, $400+ headphones. After years of freedom, audio would be siloed. As Gruber mentions, this is in Apple’s best interests; but among all our gadgets, headphones have always been among the most universal and independent. They are a true analog path between our disparate electronics — an intuitive and surprisingly error-free technology in a world where devices routinely refuse to talk to each other. You wouldn’t find yourself spending an hour helping your mom troubleshoot the headphone jack. This change would be a major pain point, especially when it comes to:
  • Loss of plug-and-play. I constantly plug my headphones from my phone to my laptop and back. Bluetooth can sort of do this, but it always takes me about a minute with my wireless headphones. With Lightning headphones, it wouldn’t even be a possibility. (Barring Lightning-endowed Macbooks, which would be utterly bizzarre. What else would that port be used for? How would it be differentiated from USB-C?) A once-flexible workflow would be completely subverted.
  • Needless complication. Headphones are a very simple thing: just a wire leading to drivers. Very few things can go wrong in this arrangement, as evidenced by the proven durability and versatility of headphones over the past few decades. Headphone makers have gotten really good at working with these few parameters to create truly world-class audio devices. Indeed, some of the most esteemed headphones in the low-end audiophile space (I’m thinking of Grados) are basically glued together by hand in a workshop. If we start shoving more electronics — Lightning circuitry or a DAC, most obviously — into headphones, we make this proven system far more brittle than it needs to be. Headphones will malfunction in frustrating ways. Noise will be introduced. Designs will become more bloated to accommodate the extra circuitry. Every headphone having its own DAC is like every monitor having its own video card: clearly putting technology on the wrong side of the divide.

What is all this for? What do we gain in return?

In the past, every time a prominent piece of technology was removed from my Apple hardware — most recently the CD drive and the Ethernet port — my response was ambivalent because I had already been happily using the alternative for a while. Wi-Fi, despite its flaws, offered countless advantages over Ethernet, leading to rapid adoption. Steam, iTunes, and Netflix had made me almost forget that CDs were still a thing by the time I got my Retina Macbook Pro. It almost goes without saying that these technologies were standard and universal — nobody would have accepted them otherwise. But there’s no Next Best Thing in headphones. This is an entirely artificial change.

Were there an existing high-quality wireless standard for headphones, I’d be somewhat on board, especially if the phone could be waterproofed in exchange. But we’re not there yet, and I fear that in this instance, Apple is looking out for their corporate interests instead of their users. When Apple removes features, I can usually envision the “better tomorrow” they’re striving for. Here, what future can we look forward to if we’re all using bloated, proprietary, and fragile headphones that sound like garbage?

I can already hear the cry that “the average consumer won’t care”. Sure, maybe not. But their listening experience wouldn’t really be improved by the change, their options for audio hardware would become a lot more limited, and their lives would become riddled with new minor frustrations. The “average consumer” doesn’t care about typography, True Tone displays, or Retina graphics, either. But it all adds up. I respect Apple because they’re internally motivated to strive for quality, and a move towards pointless proprietary standards — towards profit-driven mediocrity with the “average consumer” as a scapegoat — would be a sad blow to that image.

There’s a good chance I’ll keep buying iPhones without a headphone jack, but also a 100% chance I’ll end up carrying a 3.5mm adaptor wherever I go. One more thing to lose. A permanent ugly tail sticking out of Ive’s immaculately-designed round rect.

Good work, team?

Viewing all 44 articles
Browse latest View live