The Mysteries of Bookmarks

[david1951]

In developing what I hope will be a useful bit of software ("MarksMan") to manage bookmarks on the iPod, I'm discovering many of the complexities of what the iPod does with bookmarks.

Because my software peeks directly at the iTunes database on the iPod I can see exactly when, and under what circumstances, the iPod writes out the bookmark in a permanent form onto its hard disk. Alas for my proposed software, this doesn't happen as and when I thought.

Here's what I've discovered:

1) The iPod does NOT write a permanent bookmark to disk when you pause the book.

2) The iPod does NOT write a bookmark when you pause and let it time out (shutdown to 'light sleep').

We already knew the above. What was a real surprise to me was the following:

3) The iPod does NOT write a bookmark when you connect the iPod to the computer.

4) The iPod does NOT write a bookmark when you pause the book and start another song/book. It does SAVE a bookmark, but it must save it in memory - it doesn't get written to the disk. You can start and pause several books (I tried it with up to four books), and it retains these bookmarks only in memory.

5) When you allow the iPod go into Deep Sleep, or manually do a reset, it loses ALL of the bookmarks in memory.

The obvious question then is, when DOES the iPod write a permanent bookmark to disc? Well, hardly ever, it seems. There is only one trigger which forces the iPod to write bookmarks to disc:

6) When you start up iTunes with the iPod connected, after it detects the iPod, it automatically writes all memory resident bookmarks to disc. I assume that this is so that the PC can read them.

So it seems that if you want to be REALLY sure that your iPod won't lose your current bookmark, you have to connect it to your PC or Mac, and start iTunes - it's probably worth setting up the iTunes options so that iTunes starts automatically when the iPod is first connected.

If any of my readers can contradict the above, I would love to know, with details.

 

[arsolot]

A couple of questions:

Does the iPod write a bookmark if you manually shut it down, or is this the same as letting the iPod time out?

I've yet to get through any book I've acquired from Audible.com without losing a bookmark at least once. And, when I've listened to books acquired from the library (and transferred to my iPod through iTunes, and made "bookmarkable" by making the file extension ".m4b") I've NEVER lost a bookmark. Any explanation for this (to me) interesting contrast?

 

[david1951]

arsolot:

The answer to your first question is NO, it doesn't save the bookmark(s).

I think a LOT of the confusion we've had about bookmarks is that most of us pretty regularly start up iTunes when we have the iPod attached - after all, you need to do that to transfer content. The fact is, starting up iTunes forces the iPod to write its bookmarks to disk. Other than that, I actually can't find any event where the iPod writes the bookmark to disk, it's all (I think!) just held in memory.

As for a difference between Audible content and home-brew .m4b files, I can't see any difference in behaviour when it comes to bookmarks - I have both types of file on my iPod at present, and I've tested both in regard to the bookmark writing issue.

A more interesting question is WHY the iPod behaves this way.

I'm starting to think maybe the iPod keeps the entire iTunesDB in memory when it's not connected to a PC (the whole file on my iPod is only half a megabyte in size, nothing really in memory terms). This means it only needs to write out changed data (such as bookmarks, playcounts, ratings, etc) at one time - when it is connected to a PC or Mac and talks to iTunes, rather than having to constantly write while turned on. I'm assuming this saves battery power, or something.

What I'd LOVE to know, but can't imagine Apple telling us, is exactly how and what iTunes does to the iPod to make it write its database out. If I knew that, my MarksMan software (coming soon to a web site near you!) could trigger the same thing. As it is, I'll have to tell users that it is compulsory to run iTunes before using my software.

 

[david1951]

The obvious further question is - if the iPod only WRITES the iTunesDB to its hard drive when it connects to iTunes, when does it READ the database back into memory?

Surprisingly, this doesn't seem to happen when iTunes closes, but at the point at which the iPod is ejected / disconnected from the computer. Any changes made to the iTunesDB file (such as through iTunes or through the software I'm writing) seem to take effect at that point.

The same, of course, would have to happen when the iPod is woken up from Deep Sleep.

It's starting to all make sense for me, and it clarifies quite a lot the problems we've all been having with the impermanence of bookmarks. They ARE impermanent, until you connect to the computer and start iTunes. Entering Deep Sleep, or doing a manual reset before that occurs will lose the current bookmarks. On waking up from Deep Sleep, the iPod will read the iTunesDB from the hard drive, thus loading the 'prior' bookmarks (that is to say, the ones written to disk the last time you started iTunes with the iPod attached).

 

[kirkmc]

In my experience, this article is hopelessly out of date. These problems don't happen any more, if they ever did. While I did see the "light sleep" problem, I haven't since the iPod video. If you have a very old iPod, these problems may occur, but otherwise I'd ignore this information.

Kirk

 

[david1951]

I can't agree that my article is "hopelessly out of date". Certainly many reasonably modern iPods, including the original Nano and my 4G color iPod, behave this way with bookmarks. I still lose my bookmark on Deep Sleep; and my experience of when the bookmark is written to 'permanent' memory (that is, the iPod hard disk) still remains true.

If Apple have finally fixed these problems with the 5G iPods, then great. I don't have one to experiment with; but I would frankly be surprised.

 

[kirkmc]

As I said, I haven't seen any of these problems, with the exception of the sleep one, and that seems to have been resolved with the video iPods. I have some problems with bookmarks syncing from a 2G iPod shuffle, but that's only with homemade files, and not even all of them.

But some of the things you say are just wrong (at least with today's iPods): 3 and 4, for example, are things I've never seen, with a variety of iPods over several years. As to writing the bookmark to disk, it clearly does this; it doesn't require you to sync to a computer to record the bookmark.

Kirk

 

[david1951]

Well, if you are correct, then that's a good thing.

But the "Deep Sleep" losing of memory-stored bookmarks on the iPod still happens, for sure, on my 4G colour iPod. The fact that it is lost clearly indicates that the bookmark is stored in memory and not stored on the iPod disk.

The rest of what I said is based on using my software tool to examine what is in the iPod hard drive copy of the iTunesDB file after various actions (connecting to PC, running iTunes, etc), and matches my experience and that of several others with how the iPod behaves in various circumstances and when it loses bookmarks.

If, however, all this is now obsolete because Apple has finally fixed it all, that's just great and I'm happy to have this article removed. But it would be good to hear from others about the "Deep Sleep" issue. Personally, if there was an iPod firmware fix for my 4G iPod, I would be extremely pleased!

 

[orev]

I have a 4G here, and while it is a little older, it's still perfectly functional. This information, while maybe not applicable to new ipods, still applies to ipods still in "service".

david1951:
Thanks for posting this research. The bookmarking thing has been very annoying for me, and now at least I can see what the ipod is doing. It doesn't fix the problem, but at least I know I'm not the only one .

 

[kirkmc]

I'm having skipping problems with a new iPod video - not bookmarks, since they are always stored, no matter what I do - but when I go to play a homemade M4B audiobook, it plays a couple of minutes then either stops and goes to the main menu, or goes to the next audiobook. In another thread, it was suggested that this occured with books over 5 hours; I'm seeing it (as is Robert) in shorter books as well.

So I'm wondering - since the iPod has the ability to bookmark any files (remember playback position) has anyone tested this with non-M4B files? Aside from M4B files showing in the Audiobooks menu, there is no other advantage to converting them to this format. I don't mind having a playlist for audiobooks. I haven't had time yet, but will convert some stuff to M4A and see if the remember playback position setting is more reliable than bookmarks with M4B files. If anyone else wants to try as well...?

Personally, I'm finding it much easier - for ripped audiobooks - to keep the original files and dump them in a smart playlist which only shows those file that have a play count of 0. That way, I can see when I get to a place where I can pause, and I'll never lose my place (unless somehow the play counts wig out).

Kirk

 

[robert]

Faster Play advantage of m4b and Play Count Increments +1 on skip out

I'm having skipping problems with a new iPod video - not bookmarks, since they are always stored, no matter what I do - but when I go to play a homemade M4B audiobook, it plays a couple of minutes then either stops and goes to the main menu, or goes to the next audiobook. In another thread, it was suggested that this occured with books over 5 hours; I'm seeing it (as is Robert) in shorter books as well.

... has anyone tested this with non-M4B files?

Robert comment: My recent listens include under five hour homemade files which skipped out and an audible file which did not.

Aside from M4B files showing in the Audiobooks menu, there is no other advantage to converting them to this format....

Robert Comment: An m4b, unlike an m4a file, will play at faster speed. I find faster listening a considerable advantage on most audio books.

Personally, I'm finding it much easier - for ripped audiobooks - to keep the original files and dump them in a smart playlist which only shows those file that have a play count of 0. ...(unless somehow the play counts wig out).

Robert Comment: My skip outs increment play count by 1, so I have to find the book in the audiobook folder since it is no longer in the smart playlist with play count = 0 condition.



Kirk

Robert comment: There is a new v 1.3.1 iPod nano and 1.2.1 video ipod update. Dunno if this solves the
problem.

 

[kirkmc]

Not for the iPod video... (No update, I mean.)

Kirk

 

[robert]

Not for the iPod video... (No update, I mean.)

Kirk

I just connected my 30GB video iPod to my computer and it updated to v 1.2.1 from v 1.2. Older nano updated to v1.3.1.

Kirk, difference may be mac vs pc or maybe later video models. I can't keep up with generational designations anymore. I plan to buy a mac laptop when Leopard becomes available and run pc programs like SoundTaxi, YAMB, TuneBite, etc, with either Parallels or Boot Camp, depending on how integrated Boot Camp becomes in Leopard. Then I will have to decide if I want to re-format my iPods to mac or run on the mac laptop in pc mode.

 

[kirkmc]

I got the 1.2.1 update in December (I just checked the date on the update file on my Mac). I'd be surprised if there's a difference between Windows and Mac; more likely, it's iTunes 7.1 that checks automatically for new versions of the iPod software, and you probably hadn't done the update before then...

As for moving to a Mac, it would probably make sense to use Parallels (more flexible) but format the iPod for Mac - it's easy; you just connect, and iTunes tells you it's not happy and reformats.

Kirk

 

[Podunk]

Robert, if my experience is anything to go by, I'd say run your iPod on the Mac.

I run mine on my Windows machine, but my iPod responds much more quickly on my wife's pokey old iBook.

 

[astridman]

Kirk - Smart Playlist Solution?

Kirk, I am having the exact same issue with M4B files on my 5g ipod...playing from a bookmark only goes for a minute or two then stops and dumps back to the menu (I only have one book loaded, so I assume that if I had another book it would move to that one)

Can you describe your M4A/Smart Playlist solution in a little more detail?

 

[kirkmc]

Rip your book as M4A files, select all the files, and set them to Remember Playback Position. (File > Get Info, then check the box at the bottom of the window.)

Tag all the files as you want, with author as artist, for example, and book name as album.

Create a smart playlist where Album is [book name] and Play Count is 0. Name the playlist after the book.

Sync to iPod.

Select the playlist on the iPod, and start listening. It will go through each unplayed file in order, and if you don't finish a file, the playback position will be bookmarked. When the book is finished, the playlist is empty, and you can delete it.

Kirk

 

[astridman]

so the saving of position becomes a function of the smart playlist instead of the individual files?

(I guess I shouldn't dissect this too far, if it fixed the problem for you I'll give it a go)

 

[kirkmc]

Well, the saving of position is indirectly done by the smart playlist, since files you've listened to won't show in the playlist.

Kirk

 

[robert]

Ripped Audio CD book losing Bookmark

I recently ripped an audio book CD set directly into AAC/mp4 with iTunes, joined the resulting m4a files into one 5:30 file (usually got away with a small over 5 hour file) with YAMB. Renamed the file from m4a to m4b.

When iPod goes into deep sleep (apple logo appears in display upon start up), restarts at proper book mark, plays for a couple of minutes and skips out of book, adds one to play count -- losing bookmark entirely.

I had thought the problem might be using non iTunes AAC codec, but using iTunes to convert to AAC eliminates that as the problem. This is a repeat problem of one that had gone away with intervening update versions.

I don't see that smart playlists solve the problem. I have used the smart playlist with playcount = 0 for a long time.

I want an m4b file so that I can listen at faster speed.

The only solution I can see is fall back on the old one: When beginning play after deep sleep, play for a few seconds, leave the audiobook for another selection, and then return to the audiobook, thereby forcing the bookmark to set. This is annoying to have this bug reappear. Hope Apple will fix this re-appeared bug again.

I am thinking that MarkAble created books will have the same problem as will any homemade m4b file.