Quick First Report on Chapter Tool Used on a Mac

[moriond]

I made an "enhanced podcast" out of an audiobook that will play with chapter navigation and display under iTunes, but I haven't been able to navigate this file by chapters on my 3G iPod. I'll have to see whether running the iPod updater software will fix this. My configuration: Mac OS 10.3.9, iTunes v. 4.9, Quicktime v. 6.5.2, current ChapterTool beta (v. 2.0b8 (4)) -- note this is not upgraded to Tiger, but satisfies the system requirements for using ChapterTool.

Added 15 minutes after posting: See this Voxmedia Wiki link for a description of differences in the performance on clickwheel and non-clickwheel iPods.
Read the M4B versus M4A section.

A quick list of the results of some experiments:

• Running ChapterTool on any of my previously encoded audiobooks and even recent podcasts prepared in aac format, such as the one prepared by Doug Adams on AppleScripts for iTunes fails: ERROR: Couldn't open input movie 'Doug.m4a' -- and returns error code -43; all these files are identified as "movies".
• No problems running ChapterTool on their sample .m4a file, which is 64 kbps stereo, encoded at 22.050 kHz.
• There is a new AAC encoding option named "podcast" under iTunes 4.9 that uses 32 kbps (mono)/ 64 kbps (stereo), 22.050 kHz, and voice filtering. Once files were converted with this setting, ChapterTool worked.
• ChapterTool creates a .m4a file which has to be made bookmarkable (on the Mac, by running the Make Bookmarkable AppleScript) in order to hold bookmarks.
• When you play back an "enhanced podcast" under iTunes a small icon of an open book with a ribbon bookmark and a short sidebar with up arrow and down arrow appears just to the left of the title display window. Clicking on this expands a "popup window" that shows a list of the Chapters with the titles you've set, along with thumbnails of any images or web links (browser icon) you may have inserted at these points on the left. You can use the arrows to scroll up and down the list of Chapters. This provides a way to navigate through the chapters by selecting (highlight and return) one of these entries. A small diamond at the left indicates your current chapter. The normal keyboard shortcuts that work with Audible files (on the Mac CMD-Shift-right arrow and CMD-Shift-left arrow -- PCs may have to substitute the CNTRL key for CMD) also allow you to navigate chapter marks in your "enhanced podcast".
• As the "enhanced podcast" plays, any images or links you've inserted show up in the
  Now Playing window (click above it to make sure it is set to display what is now playing instead of Selected Song, the other possible setting).
• After a manual transfer to my 3G iPod I couldn't see the chapter breaks on the display (or navigate them) the way these appear for Audible files with Chapter marks. It's possible this could get fixed by running the iPod software update, but the software description claims no new functionality is added for the 3G iPod over previous versions of
  the iPod software update.

The ChapterTool basically uses a .xml file that you create to mark chapters by time (mm:ss). This is something that could be scriptable on either Mac or PC platforms, with a series of entries that might look like (insertion of picture or web links optional):

• <chapter starttime="00:25">
  <title>Introduction</title>
  <picture>pictures/mypic.jpg</picture>
  </chapter>

As mentioned in other threads, the demo clearly implies that a version will be released for Windows. (They supply an "enhanced podcast" with instructions on how to use ChapterTool, and there are comments like, "you can use any simple text editor, such as TextEdit on the Mac or Notepad on Windows. . . ").

Originally posted by Comrade Cruz in another thread
The "Enhanced Audio Guide" it says that "Chapter Tool is a command line application, that must be run from Terminal on the Mac or the command shell on Windows." If this beta version doesn't support Windows, I'm sure a future version will. I'm also guessing that Apple already has a newer version that they are using, because on the podcast FAQ it reads "When you play an enhanced podcast on an iPod, your iPod displays the name of the chapter you're listening to above the playback position in the episode." That feature is not there with the enhanced podcasts created with this beta Chapter Tool.

It may be that the chapter "displayed" while you're listening requires one of the newer generation photo iPods -- I'd guess that they are designing this so that the images show up on photo iPod screens the same way that Album art now gets displayed.

PC iPod users who have upgraded to iTunes 4.9 can check out the ChapterTool features by subscribing to the iTunes New Music Tuesday podcast, which uses these features. (Sorry, you can't download individual copies of this, you have to subscribe. But you can always end your subscription later). You can try making this bookmarkable and seeing whether you can navigate chapters on your own iPods. Maybe someone with a photo iPod can tell us whether he/she can navigate through chapters.

The other point of interest is the "Podcast" AAC import setting; the 22.050 kHz setting is the same one Audible uses, and may be a preferred setting for encoding homemade audiobooks. David, this might help extend the maximum safe file size for MarkAble. I didn't see any way to toggle mono encoding on this setting, although it was implied that you could do either mono or stereo recording. My test case was about 2 hours long, so this wouldn't check maximum length. The command tool does print out a duration counter in its stats, which look like this:

• ChapterTool 2.0b8 (4)
  Copyright (C) 2005, Apple Computer, Inc., all rights reserved.
  WARNING: XML validation disabled b/c XML library on the system is too old.
  format: aac
  bitDepth: 16
  sampleRate: 22050
  numChannels: 2
  numChapters: 5
  avgBitRate: 64000
  duration: 6132.993164
  qtVersion: 6.5.2
  status: ok

People with 4G and photo iPods should try experimenting with a podcast which has Chapter markings created with this tool, to see whether they can get it working.

Edited to remove the preamble message to choose the appropriate forum for posting

[Comrade Cruz]

That wiki post summed up what I was going to say. Except that they didn't mention the fact that the Apple created "enhanced podcasts" like Podfinder and iTunes New Music Tuesday do show the chapter names above the playback position when in the progress slider mode, but none of the ones created by people using the Chapter Tool are able to do that. That is why I believe Apple has a newer tool that they are using.

[EDIT] I am using a 4G iPod (with monochrome display)

[david1951]

MY big question about all of this (even though it would take away much of the need for MarkAble) is why the new Podcast encode setting doesn't create bookmarkable files!!!! This, I would have thought, would be a no-brainer for Apple.

Even the example for the new Chapter Tool doesn't indicate that (on a PC, at least) you could change the extension of the output file to .m4b to get a bookmarkable file.

Don't Apple want to support bookmarking? Or are they paranoid about our homebrew books?

I'm very confused about Apple's approach. Downloading a standard podcast through iTunes and transferring it to my iPod leaves me with an .mp3 file, completely un-bookmarkable. Yet the publicity for iTunes 4.9 seems to indicate that podcasts will be bookmarkable.

[david1951]

Ah, now I get it - the older iPods are restricted to bookmarks only in .m4b files, but the newer ones will bookmark anything (after installing the latest updater). Am I right? And will Apple release an updater for older iPods which will do the same thing?

And an interesting issue - now that bookmarks are becoming 'mainstream', will Apple FINALLY fix the 'lose all your bookmarks on Deep Sleep' bug?

[moriond]

Originally posted by david1951
Ah, now I get it - the older iPods are restricted to bookmarks only in .m4b files, but the newer ones will bookmark anything (after installing the latest updater). Am I right? And will Apple release an updater for older iPods which will do the same thing?

And an interesting issue - now that bookmarks are becoming 'mainstream', will Apple FINALLY fix the 'lose all your bookmarks on Deep Sleep' bug?

Yes, if you have the 4G iPods, run the firmware update, and use iTunes 4.9 for the downloaded podcasts you should be able to bookmark mp3s as well, and not worry about changing m4a to m4b in order to get bookmarkable files.

However, the new firmware upgrade affects the live updating of smart playlists, so people might not want to do this without forethought. The most useful discussion I've read on this and how to recover from this is given in the links of the disussion at smartplaylists.com, although it has also been discussed in this iPodlounge forum thread (Mac OS X users who have iPod Updater 2005-03-23 in their Utilities File (may be under iPod Software Updater) can apparently recover by running this to reset their iPods.

This would affect people who use smart playlists to mange playback of multi-file audiobooks on their iPods, since the playlist counters would no longer use live updating.

Apple has published an article describing this loss of dynamic updating behavior for clickwheel iPods, and will update the document as new information becomes available.

Edited to add link to Apple document

[david1951]

Robert has been doing some tests for me (I don't have a 4G ipod), and he reports that .mp3 and .m4a files STILL DO NOT BOOKMARK after the latest updater is applied.

He's also sent me some info from Apple which seems to indicate that all they are actually adding is the ability for podcasts (through the Apple Chapter Tool) to have section marks so make them easier to navigate.

Which poses a very good question - WHY don't Apple make podcasts bookmarkable? Is it maybe just not possible for mp3 files?

[Comrade Cruz]

To: david1951

I just tried it an I can confirm on my 4G that MP3 podcasts do keep bookmarks. I tried doing this both through the automatic updating feature, and also through just dragging the podcast MP3 file into my iPod.

[moriond]

Originally posted by Comrade Cruz
To: david1951

I just tried it an I can confirm on my 4G that MP3 podcasts do keep bookmarks. I tried doing this both through the automatic updating feature, and also through just dragging the podcast MP3 file into my iPod.

Is it only podcasts?(i.e., ones loaded through the iTunes Music Store). Can you get bookmarks working on other MP3 podcasts or music files transferred to your 4G iPod?

[Comrade Cruz]

I was not able to keep bookmarks with any other MP3 file, iTunes might put some tag on the MP3's as they are downloaded from the Music Store to make them markable. I tried a regular MP3 file and changed the genre to podcast, but no bookmarking.

[robert]

mp3 imported to iTunes, even with Podcast import AAC preferences will not bookmark for me.

[Comrade Cruz]

I did some researching through a few .mp3 files, and saw that iTunes does add several tags to .mp3 podcasts to make them bookmarkable. If you already have an mp3 file imported through iTunes, it should have most of the tags and you can simply add one tag and it will be bookmarkable.

How-To: (Adding a tag to iTunes imported[encoded] mp3s to enable bookmarking)

Keep in mind that this only works for mp3 files encoded through iTunes, I actually tried with an mp3 encoded using dbPowerAmp on a PC and this method didn't work. Also keep in mind that the mp3 files I tried this with were encoded using the iTunes 4.9 and 4.8. You might not have the same results with an mp3 file encoded with an older version of iTunes. mp3 bookmarking is not currently supported by iPods older than 4G

1. Get a HexEditor
A couple free ones are HexEdit for OS X and Hex Editor XVI32 for Windows

2. Open your iTunes imported(encoded) mp3 in your hex editor.

3. Paste this hex string before the TEN tag (see image)

You might try doing a search for

00 54 45 4E 00

first to find the TEN tag then paste this string right before it

50 43 53 00 00 04 00 00 00 00

4. Save file, and test

Please post your results!

Also if anyone knows of a way that we could just inject that code into the mp3 files, then we wouldn't have to go through the hex editing programs.

[david1951]

Thanks for this excellent research. I can see I'm going to have to buy a 4G ipod just to try all this out!

Here's my take on it:

1) What in fact you are searching for is the TEN or TENC frame in the ID3 v2 tag - this indicates who encoded it - presumably Apple. You probably won't find this in every MP3 file.

2) What you are inserting is a new 3-character ID3 Frame tag : "PCS", which I think Apple must have invented (not a standard frame tag though in fact the current standard uses 4 characters, not 3). PCS = "podcast", of course!.

3). The 04 just looks like a length value, so there is no additional data.

if anyone knows of a way that we could just inject that code into the mp3 files, then we wouldn't have to go through the hex editing programs.

You could probably add it with a decent ID3 tag editor (I use ID3-TagIt).

However, it also looks like a good feature for me to add to MarkAble, I'll do that this weekend.

How silly of Apple to do it this way - why not just make ALL audio files bookmarkable and be done with it?

[moriond]

Originally posted by Comrade Cruz
I did some researching through a few .mp3 files, and saw that iTunes does add several tags to .mp3 podcasts to make them bookmarkable. If you already have an mp3 file imported through iTunes, it should have most of the tags and you can simply add one tag and it will be bookmarkable.

How-To: (Adding a tag to iTunes imported[encoded] mp3s to enable bookmarking)

Keep in mind that this only works for mp3 files encoded through iTunes, I actually tried with an mp3 encoded using dbPowerAmp on a PC and this method didn't work. Also keep in mind that the mp3 files I tried this with were encoded using the iTunes 4.9 and 4.8. You might not have the same results with an mp3 file encoded with an older version of iTunes. mp3 bookmarking is not currently supported by iPods older than 4G

1. Get a HexEditor
A couple free ones are HexEdit for OS X and Hex Editor XVI32 for Windows

2. Open your iTunes imported(encoded) mp3 in your hex editor.

3. Paste this hex string before the TEN tag (see image)



You might try doing a search for

00 54 45 4E 00

first to find the TEN tag then paste this string right before it

50 43 53 00 00 04 00 00 00 00

4. Save file, and test

Please post your results!

Also if anyone knows of a way that we could just inject that code into the mp3 files, then we wouldn't have to go through the hex editing programs.

This checks out with what I found doing a simple diff on podcasts loaded directly and from the iTMS (not using a hex editor), and is in the same location as the m4a or m4b tag which, if changed in the file (from m4a to m4b), can also turn bookmarking on.

David1951, Apple's solution makes sense; there are two parts of the file -- the encoded portion (mp3 or aac) and the container. And incidentally, you can see from the diagnostics in the earlier post that the aac files are average bit rate -- not constant bit rate, so there isn't a simple merge tool. The mp3 files that pass through the Music store have the outside container modified.

The behavior is also consistent with the fact that mp3 files formats are really not as standard as one would like to think, and some of the problems people have had with playing conversions on the iPod have to do with files originating from 3rd party encoders that produce slightly different results.

A file change is fairly easy to handle as a shell script on a Mac, though I have no way of testing this without a 4G. Doug Adams can probably do this with an AppleScript.

One further comment:

Originally posted by Comrade Cruz That wiki post summed up what I was going to say. Except that they didn't mention the fact that the Apple created "enhanced podcasts" like Podfinder and iTunes New Music Tuesday do show the chapter names above the playback position when in the progress slider mode, but none of the ones created by people using the Chapter Tool are able to do that. That is why I believe Apple has a newer tool that they are using.[EDIT] I am using a 4G iPod (with monochrome display)

My "enhanced podcasts" created with the Chapter Tool do show the chapter names above the playback position.

I meant to add that the Voxmedia Wiki post links have been updated a number of times. If you follow some of the reference links back, there are a number of related articles. One of these (the link to the blogger who reported that "Enhanced Podcasts" created with ChapterTools also play as QuickTime movies -- something I've confirmed) also shows the chapter names above the playback position. According to this user's blog, he can run the ChapterTool directly on pre-existing m4a files (while I could only run this on files converted with the new Podcast aac setting). The difference in our setups is that he is using Tiger (10.4.1) with the same QuickTime library (6.5.2) (This guy also has a 3G iPod, so he also can't use the Chapter navigation).

I'd be curious about differences in how well bookmarks are held in the different 4G models: iPod mini (iPod Software 1.4); Click Wheel (iPod Software 3.1); and color display (iPod Software 1.2). My guess is that bookmark holding may still be problematic for these.

The Chapter marking also offers a potential solution for gapless playback (concerts, operas, etc).

[david1951]

Moriond, your last post crossed with my edit of my previous post.

Thanks for your comments about the variability of MP3 files, which I can confirm - there's a lot of deviation from the "standard".

Though all of this affects my program MarkAble, I'm actually quite excited by it, as it opens up a lot of new possibilities for additional features and third-party add-ons.

[moriond]

Originally posted by david1951
Moriond, your last post crossed with my edit of my previous post.

Thanks for your comments about the variability of MP3 files, which I can confirm - there's a lot of deviation from the "standard".

Though all of this affects my program MarkAble, I'm actually quite excited by it, as it opens up a lot of new possibilities for additional features and third-party add-ons.

Hi david, one query about Comrade Cruz's posting: I saw more than one sort of difference covering the same region of the mp3 file. (I just grabbed a number off different mp3 Podcast files off the feed and compared them to directly downloaded mp3 files). So there may be other different strings between "before" and "after". Haven't had a chance to play much with this (and don't have a 4G!). Main difference in this approach was using a range of Podcast mp3 files posted by other folks to iTunes.

And yes, this is kind of neat.

Addendum: It would be really great if we could "enhance" our own Audible books the way that one might enhance homemade books -- e.g., using some of the features described for PDF annotations for audiobooks, but having images, maps, etc, update as the books play. (On photo iPods these would display on the screen as the book plays). One can also imagine that web links (to lyrics, related text, and background notes) could be accessed from iTunes or, at some later date, a networked iPod. Unless Audible encodes story collections with Chapter Stops that also have section titles I imagine the DRM will keep us from adding our customizations.