Archive for the 'debugging' Category

A little trick

Thursday, July 8th, 2010

Most subclasses add instance variables. Objects initialized using initializer methods inherited from superclasses will only be partially initialized. Therefore you should override all initializers to avoid accidentally creating a partially initialized object. However unless there is a reasonable default for all instance variables this may not be possible. In such cases, one usually overrides the initializer by a method returning nil. However it is difficult to debug nil-errors. A better solution is to override the method to invoke doesNotRecognizeSelector: Since _cmd is always the current method’s selector one can use the same method body. For instance:

- (id) init
{ [self doesNotRecognizeSelector:_cmd]; return nil; }

What to do if XCode won’t add a source code file to a Project

Monday, November 30th, 2009

Sometimes XCode won’t let you add a source code file to a project when you click “Add, Existing Files…” from the “Groups & Files” context menu: you find that the filename is greyed out and cannot be selected.

To work around this, you can drag the file from the Finder to the “Groups & Files” pane.

Teach yourself Cocoa Touch Programming in 24 hours

Monday, October 19th, 2009

A few months ago, Pearson Education contacted me about writing a book about Cocoa Touch. I agreed because although I had a lot of experience on other platforms, I found Cocoa somewhat bewildering when I first encountered it. Most books either assumed a Mac background or really wanted to teach me Object-Oriented programming from the ground up. Often after reading one of them, I had the impression I had learnt something until I tried to apply whatever it was I learned: The problem was that they glossed over how Cocoa really works, and why it was architected in that way.

Cocoa and Objective-C are very powerful tools to build Applications and Graphical Interfaces quickly. This power comes from splitting GUI tasks into a few well-chosen abstractions that reduce the amount of code we have to write. Once understood, these abstractions seem obvious, so most books simply introduce them as the way things are done in Cocoa, leaving beginners bewildered.

Instead I show readers how these abstractions work and how they simplify their code. Because they understand the mechanics of each task, readers will be able to resolve most difficulties on their own quickly, rather than exploring new levels of frustration. To further help, the book is interlaced with debugging techniques. My goal in writing the book was to provide a sound foundation that programmers can use to build functional applications, and feel confident that they can solve the problems they will encounter. Instead of covering the latest API fashions, I concentrate on the APIs and tools you’ll actually use and will need to understand.

The book was published last Thursday, Oct 15 2009, and is available now at the publisher’s. Amazon shows they have yet to receive it (and therefore discount it). It’s in full color!

Understanding the Objective-C Language

In the first five chapters, you’ll build a basic application: a Calculator. You’ll learn how to use the key tools used when developing iPhone applications: Xcode, the debugger and Interface Builder. It also teaches you Objective-C: Objective-C is a thin layer on top of C which adds language support for object-oriented programming. Cocoa uses reference counting for memory management, but the judicious use of auto release pools makes this much easier than you might expect. You’ll also learn about Cocoa Touch’s Foundation classes with provide basic functionality such as Unicode strings, arrays, dictionaries. Practically speaking, by the end of chapter 4 you’ll have written a functioning calculator.

When I first learned Objective-C, I found many bugs intractable because I did not understanding messaging. To help you avoid this pitfall, I explain messaging and how it is implemented in chapter 4. Similarly, it took a while for me to understand auto release pools. They simplify your code, but are often presented as a form of magic you just use. My book is a magic-free zone, clarifying how they work and their limitations up-front. Other Objective-C particularities you’ll learn about are class objects, updating classes on the fly, class clusters, key-value coding and key-value observing. By the end of chapter 5, for most intents and purposes you’ll be an Objective-C expert. Unlike competing books, and Apple’s own documentation, I detail the computational complexity of Objective-C arrays and dictionaries.

User Interface Foundations

The building blocks of user interfaces are views. To build a user interface from views, one simply builds a tree of views (a view hierarchy) which specifies the location and order in which views are drawn. Most interfaces can be built using Interface Builder, a graphical design tool. Unlike competing solutions, Interface Builder does not generate code, but a NIB file that states how to build the view hierarchy. Most books only cover building user interfaces in Interface Builder. As a programmer this left me with a bad taste, as I was relying on some unknown mechanism working behind my back. Debugging was difficult because I had no idea how NIB files were loaded. Instead I show you how to build view hierarchies in code, how NIB files are loaded, and the pitfalls you may encounter.

Views draw themselves using a 2D renderer called Core Graphics (or Quartz). They respond to user interaction dispatched to them by Cocoa. Implementing your own User Interface element (a button) will give you a clear understanding of the ins and outs of user interface elements, and will help you polish your applications with custom user interface elements.

Unlike most user-interface solutions, Cocoa is not architected as a library. That is to say, your application does not just call library functions at will. Cocoa is architected as a framework: it calls your application code when it needs to. This turns out to be a powerful solution, but is disconcerting for programmers coming from other platforms. Because misunderstanding the run loop causes seemingly unrelated bugs I spend entire chapter discussing how run loop is used and how it interacts with other Cocoa sub-systems such as the auto release pool, and the responder chain.

A distinguishing feature of the iPhone’s user interface is how much animation it uses. These animations are created with Core Animation. Core Animation uses the iPhone’s 3D graphics chip and a separate thread to create smooth animations. Understanding Core Animation’s architecture will help you use it and remember its limitations. To further help you I share tips learned developing commercial software. By the end of this chapter, you’ll have implemented your own Cover Flow clone shown above! (animal images courtesy of

Advanced User-Interface Elements

View Controllers reduce the amount of code you must write. Unlike many beginning programmers, you will not confuse them with views because the book clearly differentiates them. An example will solidify your understanding: displaying a Scientific Mode in the calculator when the iPod is rotated.

Tables are an essential component of most user interfaces. The iPhone’s table support is particularly flexible, and I’ll show you how you can customize it to create fast scrolling distinctive tables. In this hour, you’ll build a table based application that lazily loads data from an Internet source: a Twitter application.

Many applications present information on multiple screens. Navigation bars and Tab bars provide a standard way of navigating between the screens. I show you how to build applications using these user interface elements, and show you how the views and view controllers interact.

iPhone OS 3.0 introduces undo/redo functionality. Cocoa Touch solves this problem in a particularly elegant manner. However, it differs substantially from other solutions you may have used. To help you understand it fully, I dedicate an entire chapter to understanding the problems it solves, how it works, and why it was architected in this way.

Accessing the Internet

By providing full internet connectivity and the ability to render most web pages, the iPhone enables whole new classes of applications. To help you build robust applications, chapter 14 discusses how networks work in detail. Working through a real example (adding error handling to the Twitter application) shows you how this is done in practice.

Chapter 15 introduces UIWebView, a versatile User Interface component able to render documents in HTML, PDF, RTF, RTFD, Microsoft Office or iWork formats. You’ll learn how to use Javascript to update HTML formatted pages or parse JSON strings, and how to use new HTML5 features supported by the iPhone.

Saving and Retrieving Data

Cocoa Touch provides four means of saving and retrieving data: application preferences, files, a small sql database, and Core Data. You’ll learn how to add application preferences to the Twitter application, that users can customize in the Settings application. In chapter 17, I discuss when and how to use files or the sqlite database. iPhone OS 3.0 added Core Data which lets you load and save objects to storage transparently by providing a single object definition instead of writing your own classes and your own serialization code. To help you decide whether or not to use Core Data, I discuss its performance characteristics.

Interacting with the World

Only a few years ago, the idea of a tiny device containing your music and video library, finding your location anywhere on the planet, and using accelerometers as a user interface would have seemed like Science Fiction. The iPhone is a remarkable convergence device. Chapters 19 and 20 explains how to use these capabilities in detail.

Chapter 21 shows you how to share data with other applications. Custom URLs let you start other applications. Pasteboards provide cross-application copy and paste functionality. To export data from your device, I show you how to send emails from your application, or run a tiny webserver.

Completing Your Application

Completing your application involves four tasks: debugging it, optimizing it, localizing it and shipping it.

I devote an entire chapter to debugging, showing you not only how to use the debugger gdb, but also dtrace, valgrind, and nib2obj. Sometimes however, bugs are caused by misunderstanding how the frameworks work. Usually other programmers can help you, but sometimes the only solution is to reverse engineer the framework. To help you do this, the chapter ends with a short tutorial teaching you how to do this.

Optimizing your application is another key topic and has its own chapter. You’ll learn how to write your own profiling code, and learn how to use the two iPhone profilers: Shark and Instruments. Because of the iPhone’s memory limitations, it is particularly important to minimize your application’s memory consumption. To further help you, I introduce the Clang static analysis tool which helps you detect memory leaks at compilation time.

The final chapter covers localization and shipping your application. Because the App Store is the only authorized means of distributing applications, and because Apple can reject your application once you submit it for publication, developing for the iPhone is somewhat risky. To help you mitigate this risk, I analyze the types of applications Apple has refused to distribute on the AppStore, and provide some simple guidelines to follow.

Additional materials

If you haven’t programmed C in a while, I provide Appendix A to quickly refresh your memory of the salient points.

While developing for the iPhone you may encounter a number of issues due to Apple’s development tools. To help you resolve these issues quickly, Appendix B devotes 18 pages to debugging these issues.

Appendix C lists resources I have found useful when developing iPhone applications.

Appendix D includes a number of advanced topics of interest to expert programmers: how Cocoa starts your application, a deeper discussion of exceptions, and how and when to use threads.

Article published in Mac Tech

Saturday, June 6th, 2009

Mac Tech just published my article on call stack logging which lets you see which function calls led to my NSLog replacement being invoked. The nice thing is that it works for C, C++ and Objective-C unlike most of the solutions you see elsewhere. The source code should show up on their ftp site some time soon. With few developer magazines surviving (Dr Dobbs webified, BYTE gone), I’m glad Mac developers still have a print magazine to read.

setStringValue not clearing the screen ???

Tuesday, January 20th, 2009

I had this strange bug: Setting the string value of a NSTextField wasn’t removing its previous value before drawing the new one. The net result was a mess on the screen, and it was happening randomly.

Of course, I started by blaming Cocoa… I tried using setNeedsDisplay to no avail. Even marking the whole window as needing a refresh didn’t work.

So I stopped, and thought about the randomness. Seems a bit like a race condition… And then I realised my code was running in a sub-thread. Using performSelectorOnMainThread to update the UI once the thread had completed fixed the bug.

Solving the evolving File Format problem

Tuesday, August 14th, 2007

Andy Matuschak brought up a really good obstacle to providing an easier upgrade and downgrade path for applications: changing file formats.

Changing file-formats is far from free. It fractures your user base as your users find they cannot share their documents with their friends. It makes it difficult for you to regression test your code. And it dissuades third parties from supporting your file-format, because you’re giving them more work each time you change. Overall changing file-formats should be avoided where possible. How then can you extend your application?

Future proof your file-formats!

I try to keep my file-format forwards compatible: the same file should work with older versions of my software. This makes it easier to find bugs that I may inadvertently have added: I can regression test against an older version of the software.

I use a file-format that lets the Application store but not use the fields it does not understand. There are many such solutions: SQLITE database, Trees, XML, python pickles… The reason to store fields one does not understand is that one can save them back out later.

For example consider an HTML document. Even if your HTML editor does not know what the “blink” tag means, it can remember the fact that that there is a “blink” tag around the text over there. Even if you edit the text within the blink tag, it can make reasonable guesses about grouping. Of course if you delete the text and type it back in, the blink tag will be gone. But most of the time you don’t just lose your formatting because a colleague used a different tool to edit your file. The key idea here is that you can use an extensible hierarchical data format to keep things associated with their attributes.

Keep the conversion code separate!

Sometimes, to simplify the main application’s code, it does makes sense to change file-formats. In that case, write a bidirectional converter. A version field in the file-format specifies (by naming convention) the converter to use so that the App when encountering a file of a different file-format can figure out the chain of conversions it must apply to get something it understands.

Now if you downgrade, you’re in almost the same position as your friend who never upgraded: your application can’t read the new format files. There is however a difference: as part of the upgrade process a bidirectional converter was added. It does not need to be removed on a downgrade. Because the the older version of the application knows how to figure out which converters to call, it will just find them and call them.

Note that because the original file-format is extensible, it’s possible for the bi-directional converter not to throw away any data. Similarly because the oldest version of the app stores the stuff it does not understand and saves it back out again, changes made by a co-worker with an older version of the software should cause little data loss.

Additional benefits of independent converters

Because you have independent converters you can earn bonus points with people who have not yet been able to upgrade. You can give or sell them the tool to convert the files. The tool can even advertise the features they’d get by upgrading.

You benefit because your code is simpler. The application code is simpler because it only knows its native file-format, the converter’s code is simpler because it’s focussed on one task. You also benefit because it’s easier to require a new base OS for your next version without fracturing your user-base.

Of course writing writing a converter is a pain, and a bidirectional one even more of a pain, but as I said in the beginning changing file formats should be avoided. Writing a bidirectional converter forces you to consider more cases, leading to a better upgrade converter. Furthermore you can reduce the pain for third parties by giving or licensing your converter to them.

What about standard file formats?

I think the same idea of independent converters can be applied, although it may not be possible to keep the additional information known to later versions of the file-format in earlier versions of the file format.

Nasty printing bug fixed

Thursday, January 4th, 2007

Sometimes printing would crash Find It! Keep It!… The problem was the “sometimes”. It happened extremely rarely on my Mac Mini, but often on my Intel Macbook. Clearly Rosetta was to blame!

Well, no. Finally someone other than me experienced it so I decided to dig in deeply.

The first problem is that debugging a PPC process under Rosetta is an order of magnitude slower. Having written a debugger and having been fascinated by emulators for a long time (ZX Spectrum emulator on the Atari ST, UAE’s 68000 emulator, Bochs, and Awesim), I know why it’s slow: when debugging you have to keep track of state such as the instruction pointer you can simplify away when running at full speed. So the first task was to find a webpage that would crash regularly on PPC.

As luck would have it, displaying a big database with my new theme would cause the crash every second time you tried to print it. My first guess was memory being freed that shouldn’t be…

WebLibrarian(216,0x1b37e00) malloc: *** Deallocation of a pointer not malloced: 0x3927e70; This could be a double free(), or free() called with the middle of an allocated block; Try setting environment variable MallocHelp to see tools to help debug

Look at that! Clearly memory allocation failing!

Well, no. I had noticed that a separate thread was being started by the printing process, but the pieces only fell into place when I saw:

(gdb) info threads
* 12 process 216 thread 0x802f 0x95eb4e88 in khtml::main_thread_malloc ()
10 process 216 thread 0x6807 0x9001f08c in select ()
8 process 216 thread 0x7103 0x9000ab48 in mach_msg_trap ()
7 process 216 thread 0x6903 0x9000ab48 in mach_msg_trap ()
6 process 216 thread 0x450f 0x90049ea8 in syscall_thread_switch ()
4 process 216 thread 0x3603 0x9000ab48 in mach_msg_trap ()
2 process 216 thread 0x2303 0x9000ab48 in mach_msg_trap ()
1 process 216 local thread 0xf03 0x95eb4e88 in khtml::main_thread_malloc ()

Main thread malloc probably should not be called in two threads. Indeed, googling on it found Added assertions to ensure that main_thread_malloc and friends are only called on the main thread.

Let’s look at the two threads. The printing thread looks like this:

#0 0x95eb4e88 in khtml::main_thread_malloc ()
#1 0x95c8000c in KWQListImpl::insert ()
#2 0x95cfbb5c in khtml::RenderBlock::insertFloatingObject ()
#3 0x95cfb520 in khtml::RenderBlock::skipWhitespace ()
#4 0x95cf9d4c in khtml::RenderBlock::findNextLineBreak ()
#5 0x95cf8cec in khtml::RenderBlock::layoutInlineChildren ()
#6 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#7 0x95cf886c in khtml::RenderBlock::layoutInlineChildren ()
#8 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#9 0x95cf886c in khtml::RenderBlock::layoutInlineChildren ()
#10 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#11 0x95cf886c in khtml::RenderBlock::layoutInlineChildren ()
#12 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#13 0x95cf886c in khtml::RenderBlock::layoutInlineChildren ()
#14 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#15 0x95cf886c in khtml::RenderBlock::layoutInlineChildren ()
#16 0x95cf54d4 in khtml::RenderBlock::layoutBlock ()
#17 0x95cf5fac in khtml::RenderBlock::layoutBlockChildren ()
#18 0x95cf54ec in khtml::RenderBlock::layoutBlock ()
#19 0x95cf5fac in khtml::RenderBlock::layoutBlockChildren ()
#20 0x95cf54ec in khtml::RenderBlock::layoutBlock ()
#21 0x95cf224c in khtml::RenderCanvas::layout ()
#22 0x95cf1b04 in KHTMLView::layout ()
#23 0x95dd3750 in KWQKHTMLPart::forceLayoutWithPageWidthRange ()
#24 0x95de84a4 in -[WebCoreBridge forceLayoutWithMinimumPageWidth:maximumPageWidth:adjustingViewSize:] ()
#25 0x95adc92c in -[WebHTMLView layoutToMinimumPageWidth:maximumPageWidth:adjustingViewSize:] ()
#26 0x95b103dc in -[WebHTMLView _setPrinting:minimumPageWidth:maximumPageWidth:adjustViewSize:] ()
#27 0x95b107ec in -[WebHTMLView knowsPageRange:] ()
#28 0x9392df4c in -[NSView(NSPrinting) _knowsPagesFirst:last:] ()
#29 0x9392dc68 in -[NSView(NSPrinting) _setUpOperation:helpedBy:] ()
#30 0x9392d7b8 in -[NSView(NSPrinting) _realPrintPSCode:helpedBy:] ()
#31 0x9392d6f4 in -[NSConcretePrintOperation _doActualViewPrinting] ()
#32 0x9392d520 in -[NSConcretePrintOperation _continueModalOperationToTheEnd:] ()
#33 0x92961194 in forkThreadForFunction ()
#34 0x9002b508 in _pthread_body ()

There are other variations, but basically the printing thread renders the window differently to print it. Clearly you have to do that if CSS has different settings for display and print media. In fact you always do it.

So what’s happening in the main thread?

#0 0x95eb4e88 in khtml::main_thread_malloc ()
#1 0x95c8000c in KWQListImpl::insert ()
#2 0x95d21f1c in DOM::NodeImpl::dispatchGenericEvent ()
#3 0x95d21d0c in DOM::NodeImpl::dispatchEvent ()
#4 0x95d2682c in KHTMLView::dispatchMouseEvent ()
#5 0x95d24294 in KHTMLView::viewportMouseMoveEvent ()
#6 0x95d23994 in KWQKHTMLPart::mouseMoved ()
#7 0x95adb2dc in -[WebHTMLView(WebPrivate) _updateMouseoverWithEvent:] ()
#8 0x95adb040 in -[WebHTMLView(WebPrivate) _updateMouseoverWithFakeEvent] ()
#9 0x9296bbf8 in __NSFireDelayedPerform ()
#10 0x907f0550 in __CFRunLoopDoTimer ()
#11 0x907dcec8 in __CFRunLoopRun ()
#12 0x907dc47c in CFRunLoopRunSpecific ()
#13 0x93208740 in RunCurrentEventLoopInMode ()
#14 0x93207d4c in ReceiveNextEventCommon ()
#15 0x93207c40 in BlockUntilNextEventMatchingListInMode ()
#16 0x93730ae4 in _DPSNextEvent ()
#17 0x937307a8 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#18 0x9372ccec in -[NSApplication run] ()
#19 0x9381d87c in NSApplicationMain ()
#20 0x0060910c in init_AppKit ()
#45 0x00006e00 in py2app_main (argc=-1870946304, argv=0x7204, envp=0x3986548) at src/main.c:952
#46 0x00007964 in main (argc=1, argv=0xbffffaec, envp=0xbffffaf4) at src/main.c:1007

Each time the mouse moves over the WebView, WebKit tells javascript about it (to enable popups, etc). So my main thread was pottering along as normal, blindly oblivious of the other thread.

Telling the printOperation NOT to be threaded should fix the bug, preventing non reentrant code from being reentered. So I changed




and suddenly printing no longer crashes. Not only that, but some other bugs went away too: sometimes printing would make the browser redisplay the website incorrectly, sometimes the printout would have massive gaps in it. All these suddenly were fixed.

Now another piece fell into place. I had noticed that the crashes happened more often on large complex pages than simple ones. The bigger a page, the longer it takes the printing thread to render it, and the more likely the two threads would interfere with each other. Furthermore, it happened more often on the Intel Mac… which is a dual core processor. If the threads were running concurrently on different cores, they would interfere much more often.

So where did I get that crazy idea of threading the print? From the apple developer mailing lists. It runs out that the code is also in Shiira and Adium.

For kicks, I tried Shiira on a large complex document which crashed my program regularly… and BOOM! This is apparently a known problem which is worse on dual core systems… The bug’s still in Shiira 2.0, so I’m trying to email Shiira’s author to tell him.

Argh… Beta 3

Sunday, December 3rd, 2006

The first bug I got back from Beta 2 showed that the new Crash Reporter functionality wasn’t always started up correctly… despite my testing for that case. Turns out that what I thought was memory given to me by Python was in fact garbage collected (and therefore … disappears after an indeterminate amount of time). Since the Crash Reporter is the means for users to signal bugs, I’ve had to rush the third beta.

Awfully quiet around here…

Wednesday, October 4th, 2006

Sorry about that! Usually I’m quiet when I’m working too hard. The good news is that I will be entering beta soon.

The first alpha testers received their copies a week ago, and the first lesson is that Find It! Keep It! doesn’t work on Intel Macs, even with Rosetta emulation. In the mean time, I’ve been busy writing documentation, building the website’s content, and fixing bugs and rough edges. There’s still a lot to do, but the end is in sight.

While I’m blogging, I want to mention this post by bbum. On Linux I used Valgrind. It was invaluable, and I would love to have it on Mac OS X. I’ve tried various Mac tools (OmniObjectMeter, ObjectAlloc) and while they’re pretty they slow Find It! Keep It! too much to be useable. As usual, with Apple, all along I had a tool called “leaks” on my harddrive… Would have been nice to know about it! Quoting bbum:

To summarize:

In a Terminal window….

setenv MallocStackLogging /path/to/
… you should see a diagnostic message like …

malloc[PID]: recording stacks using standard recorder
… then, in another Terminal window…

leaks PID
… do whatever it is in the app that you suspect causes leaks. The leaks process will print detailed information about the leak, including the backtrace of the appropriate thread within the application at the time the memory was allocated.

Bbum’s link doesn’t work right now, so this is the Google Cache, and the post he references is still on the Wayback machine.

It turns out the Webkit crew use it too

Gdb Tip: Associating a command with a breakpoint

Saturday, August 26th, 2006

Gdb can associate commands with a breakpoint!

For instance (lifted wholesale from Louis’ page):

(gdb) b myFunc
Breakpoint 1 at 0×900107a8
(gdb) commands 1
Type commands for when breakpoint 1 is hit, one per line.
End with a line saying just “end”.

The biggest problem with the web is, as Arthur C Clarke said: “Why should I go to the Niagara Falls when all I want is a glass of water?”. The same can be true of overly feature rich tools. Having spend a few fruitless hours googling for this, I have been resorting to piping python scripts into gdb.