Always Test on Hardware... Why Simulators Suck.

Twice in the past week I’ve run into situations where code I was working on was running flawlessly in a simulator but not on a real hardware device.

The first was with the BlackBerry SDK. I was recently involved with the release of an app to the BlackBerry App World. We tested the app extensively on a several devices but due to costs we couldn’t afford to purchase and run every different device. In some cases so we relied on the simulator but that proved fatal. Unfortunately, we discovered that although the app worked flawlessly in the Bold simulator, it didn’t work as well on a fresh, brand new, Bold out of the box.

The second instance was working with the iPhone SDK. I was creating a game and decided to be all cool and build it using PDF files for resolution independence–just incase Apple releases some fancy new device in the future. The app ran perfectly in the simulator but on the device it was frozen at start-up.

These examples illustrate the problem with simulators–they’re simulators. I’m still not sure what was wrong with the BlackBerry app but in the case of the iPhone app it was just that my computer was so much faster and the simulator. If I let the device sit there for a minute it would eventually kick in and start running because it took a lot longer to render all the PDFs.

I would like it much better if simulators were more like the device. Throttle the simulator down to the speed of the device. Disable keyboard input and make us click the keys. Make us work to enter information just like we would on the device. The simulator should simulate the device 100%–including all it’s annoyances. If you’re annoyed at using some aspect of your app in the simulator chances are you’d be annoyed at the same thing on when you ran it on your device.