External HD Issue

[bben411]

Hello everyone, 1st post

Although it's a problem i'm having, hopefully I could be of more use in the future to other members.

Well, I'm having an issue with my iTunes recognizing the path to my music that's stored on my external HD. It's quite strange because iTunes plays some files(very few) but the rest are showing the dreaded exclamation point indicating an invalid file path. Another strange occurrence is the fact that I haven't changed much since the last time this whole process has worked correctly. I've used the external in the past to play music in my iTunes but until today I haven't used iTunes for about 6 months. I decided to go to the iTunes website and download the newest version. I don't recall it being this complicated to get the external to be recognized and it's giving me a real headache. I haven't added or deleted music since it's worked last either.

The file paths I have selected are:

My music is stored in my external G:/Music so in iTunes media folder location I have that path set correctly. I also have selected the check boxes for: Keep iTunes Media folder organized as well as Copy Files to iTunes media folder...

I know there is something about 'consolidate files' but I was hesitant to proceed with that as I don't recall doing it the first time.

I have put the iTunes Library.itl, iTunes Music Library.xml and the Album artwork in the external as well. I put them in just G:/ I transferred these files from the C: Drive in my Music folder(The standard destination)

I have tried to select that iTunes Library.itl from the external upon startup of iTunes but the majority of the files are unrecognized.

Any help is greatly appreciated, I need my music!

Thank you very much.

[S2_Mac]

The "iTunes Media folder location" pref does not tell iTunes where to "look for" your music; that setting simply tells iTunes where to put files that iTunes creates -- CD rips, format conversions, Store purchases, app downloads, etc. For files that have previously been added to iTunes, the program depends on those files being at the same location they were in when they were added. Most likely, the paths to the files on your external have changed, and thus iTunes has no idea where they are.

First off, make sure the external has the same drive letter as it did 6 months ago (i.e. if it was F: half a year ago, it needs to be F: today). If the drive letter is the same, did you move/rename any of the folders containing your music files? If so, you've changed their filepaths to be different than the paths iTunes recorded for those files, and iTunes won't have a clue as to where they are; you'll have to undo those changes.

If you don't remember what the drive letter was before, or if you don't recall changing any folder names on the external, you can check where iTunes expects files to be by opening the "iTunes Library.xml" file (sometimes named "iTunes Music Library.xml") in a text editor and searching for any of the tracks that show as "missing"; look at the <location> item for the filepath that iTunes expects the file to have.

To get iTunes to use a library file on the external: Quit iTunes; copy the "iTunes Library.itl" file to the external; hold down the Shift key while launching iTunes; click the "Existing" button in the dialog that pops up, then naviagate to the "iTunes Library.itl" file on the external. There's better and worse places to put the library file on the external, but you can sort that out later. To minimize clutter and messiness, make sure the "iTunes Library.itl" file is inside a folder; there's a bunch of files that support the library file, and they all get created at the same directory level as the library file. If all these files and folders get created at the top level of the external (G:\\) things can quickly get messy and hard to manage.

The iTunes "Consolidate files" command is used to move content files to new locations and update their filepaths in the library. Consolidate won't help you right now, since so many of your files are "missing" (i.e. not in the place where iTunes expects them to be).

[bben411]

Well first off I'd like to thank you for the reply, it's very helpful.

It seems that the driver letter G: for the external hasn't changed so that can most likely be ruled out. On another hand you were absolutely right, after looking through the .xml file I found most of the file paths were incorrect for example:
<key>Location</key<string>file://localhost/C:/Users/Benjamin/Music/iTunes/iTunes%20Music/Johnny%20Cash/

I know that the location should read somthing more like /G:/Music/JohnnyCash

Some files are still in the correct location and are look like this:
<key>Location</key><string>file://localhost/G:/Music/Metallica/Metallica/04%20The%20Unforgiven.mp3

I'll go ahead and make a folder in the external drive G: called "iTunesMusic" where I will put the "iTunes Library.itl" file.
Is there a way to re-write multiple values in that .xml file so that I can change the incorrect ones back to the external drive G: ? It's quite a lot of music, around 16000 files.

Once again, thanks for the help. I really appreciate it.

[S2_Mac]

It seems that the driver letter G: for the external hasn't changed so that can most likely be ruled out.

Nuts; that's the easiest one to fix

...after looking through the .xml file I found most of the file paths were incorrect for example:
<key>Location</key<string>file://localhost/C:/Users/Benjamin/Music/iTunes/iTunes%20Music/Johnny%20Cash/

Double-nuts; that's the hardest one to fix.

Here's what probably happened: at some point long ago, you set the G:\\Music folder as your "iTunes Music folder location"; you ripped some CDs and the files went into that folder (such as with the Metallica album), and all was good. Then, one day, you fired up iTunes without having the external connected or turned on first. When iTunes launched, there was no G: drive; and since iTunes must have a place to put files it creates, it bailed on the G: drive and reset your pref to its built-in default location (the "iTunes Music" folder inside the "iTunes" folder in your user account's Music folder). Even when the G: drive was present on subsequent iTunes launches, the program didn't "remember" your old pref...iTunes just kept on using the default location on C:. (This has been a problem on the Windows side for years. The Mac version of iTunes has pretty much always remembered to revert back to the user's original prefs whenever possible, but the Windows version of iTunes 7, 8, and most of 9 all had this glaring fail. Not sure about iTunes 10, since I don't use it.)

Now, the big question this raises is where are the files that iTunes expects to be on the C: drive? They might still be on the C: drive. Or maybe you moved them by-hand onto the G: drive? (Perhaps when you copied the library and .xml files and Album Artwork folder to the external you also copied the "iTunes Music" folder?) Well, no matter where they are, make sure you've got them somewhere. (IOW, before working to fix the problem, make sure you've actually got all your music files...otherwise there's nothing to fix; ;-) I'm thinking you moved them to the external by-hand...since if they were still on the C: drive iTunes wouldn't be complaining that they're "missing".

Know where all your music files are? Great; on to fixing the problem. There are three approaches, and the one to use depends on how important "Date Added" is to you, and how important your playlists are to you. If you make use of Date Added -- for smart playlists, or perhaps just to have a sense of history about your music collection -- then you'll have to make the files' location match the expectations of the "iTunes Library.itl" file. This involves copying files back into the "iTunes Music" folder on the C: drive, at which point iTunes will be happy. From there you can use the "Consolidate files" to properly copy them to the external drive, and all will be well.

OTOH, if you're "meh" about Date Added, you can doctor the .xml file by editing filepaths and then importing the .xml file; this will p'bly result in lottsa duplicate listings in iTunes (which will require cleaning up), and you'll lose the original Date Added for all those files, but you'll be able to preserve other metadata like play counts and ratings. (iTunes doesn't "read" the .xml file; it only "writes" it for 3rd party software to have a look into the library. The only file that matters to iTunes is the "iTunes Library.itl" file...you can't fix the .xml file and have it automagically update the actual library file.)

Editing-and-importing the .xml file can be really easy -- in your case, you'd simply tell a text editor program to replace all instances of "file://localhost/C:/Users/Benjamin/Music/iTunes/iTunes%20Music/" with "file://localhost/G:/Music/", save the .xml, then import the .xml -- but the big drawback is that you'll lose tons of tracks from your static (i.e. regular, non-smart) playlists. (When you build a playlist in iTunes, the program is keeping track of actual files, not just song titles; if a track's file isn't where the playlist expects it to be, iTunes will remove it from the playlist.) There's a workaround for this, too...but it can be a real bear to implement, especially on Windows.

The third option is to simply start from scratch, and populate a new, empty iTunes library with all your files. You'll lose all metadata like play counts, ratings, playlists, and non-embedded art, but you'll have a 100% solid library going forward. For most folks, this is the least-desirable option.

That's a lot to think about (sorry! ;-)...but don't do anything until you think it through. Well, there is one thing you should do, right now: make copies of your "iTunes Library.itl" and "iTunes Library.xml" files. These files are updated every time you do something in iTunes, and preserving these files right now gives you a certain amount of "undo-ability" if you goof up or want to change direction. Whenever you're dealing with changing/experimenting with your ilbrary files, always work from copies!

(I've only teased you with outlines of how each fix works...there's not enough info here to actually accomplish a fix. So don't jump the gun! Pick an approach and post again for the how-to details.)

If it helps, here's the questions to ask yourself:
1) Is Date Added from the "missing" files A Big Deal?
2) Are your current non-smart playlists A Big Deal?
3) If there aren't thousands of music files on the C: drive right now, is there space on the C: drive for those files?
4) Is there enough space on the external for another complete copy of all your music files?

Decide what results you'd like to achieve, and answer those four questions; from there we'll figure out what's possible (and get it done ;-) Sorry to have gone on so long...unwinding these situations can be tedious. However, once fixed, you'll have a solid iTunes library that won't cause you more problems -- whatever route you go, you'll wind up with a library-and-files on the external drive that won't be subject to iTunes re-writing your prefs. As a bonus, you'll wind up a library-and-files that can be transferred to another external drive or another computer (even a Mac) by simply copying one folder, without losing a single drop of [meta]data.

[bben411]

Once again, thank you for such a thorough reply. Well I thought about what i'd do for a bit and i'm going to have to choose option 1. My C: drive has 115 GB free and the expansion drive G: has 1.15 TB free It seems i'll be able to move the contents of the 102GB Music folder without any problems (Yes they are sitting in the expansion drive G: where I manually moved them). I don't use smart playlists or deal with date added. To be honest, I don't even know what a smart playlist consists of... As for the non-smart playlists, I can always just screenshot them and re-add them manually. I made a backup copy of the whole iTunes folder containing the .itl on my C: drive and placed it in a folder on my desktop(hopefully that location won't interfere with anything?) I'm ready to begin the process whenever you are my friend. Thanks for the help I really appreciate it!

[S2_Mac]

Sounds like you're on Easy St. This should do the trick:
1) Quit iTunes; on the external, locate that "iTunes Library.itl" file inside the "iTunes" folder you saved from C:; make a copy of that .itl file. (Let's call that copied-from-C "iTunes" folder, "the original iTunes folder".)

2) Shift-launch iTunes to the .itl file inside the original iTunes folder. The music that was "missing" due to filepath problems should now be OK. Any files that iTunes had recorded as being on G: six months ago should also be OK. (This all assumes that you haven't moved files or folders around inside the "iTunes Music" folder you copied from C:)

The reason this should work is due to special treatment that iTunes gives to certain filepaths. If a music file's path is directly downstream of the library file, iTunes is willing to ignore everything upstream of the library file. This is what allows us to move an iTunes library from PC to Mac or Mac to PC, copied onto and run from an external drive, and other uses.

(I keep saying "this should work..." because I just don't know the condition of your files. If they're unchanged from back when the library was running off C:, this should work just fine... ;-)

3) If (most all) your files are showing up now, you're most of the way home. Now would be a good time to manually re-link any files iTunes still considers "missing". To easily generate a list of all missing files, do this:
* Create a regular playlist, name it "Non-missing Files".
* Display your Music list as a list; select everything in that list (Edit menu->Select All). Drag that entire selection onto the "Non-missing Files" playlist icon.
* Now create a smart playlist; put a checkmark in the box for "Match the following rule"; manipulate the popup menus to make the current rule Playlist is 'Music'; click the + button to add another rule and use the popups to set it to Playlist is not 'Non-missing Files'; put a checkmark in the Live Updating box and click the OK button to save the smart playlist. A good name for this playlist would be "Missing Files".
* The smart playlist now displays a list of all files that iTunes considers "missing". If it's just a handful or two, manually re-link them to iTunes; the smart list will de-populate as you fix things....

4) Now, Consolidate. Inside the original iTunes folder, create a folder named "iTunes Media". In iTunes Preferences->Advanced, set the "iTunes Media folder location" to the "iTunes Media" folder you just created on the external; also put checkmarks in the boxes for "Keep iTunes Media folder organized" and "Copy files to iTunes Media folder when adding to library"; save prefs.

Lastly, iTunes->File menu->Library->Organize library. If the box labelled "Upgrade to iTunes Media organization" will accept a checkmark, check it; put a checkmark in the "Consolidate files" box; click the OK button.

iTunes will now copy every file that it knows about, and can locate, into a folder named "Music" inside the "iTunes Media" folder, with everything neatly organized. Live with this for a few days; when you're satisified everything is working OK, move the old "iTunes Music" folder out of the original iTunes folder and verify that everything is still OK... If it is, you can delete the old "iTunes Music" folder and you're done.

What you'll wind up with is an "iTunes" folder on the external that runs the library and contains all your music. All music that you add in the future will go into this folder, all of it filed according to the "new" organizational scheme. You can take this "iTunes" folder to any other external drive and -- after Shift-launching to the library file -- your iTunes will work just like normal. You can also use this folder on an internal drive (say, on a new computer) or transfer it to a Mac. And, having everything in one spot will make backing up easier.

Post again with any questions/problems...

[bben411]

Ok, that got hectic really quick... I had about 6000 files missing that I had to delete and re-add the artists. As i'm typing it's consolidating the files into the iTunes folder that I had created in the external drive. I still have a few questions though. The folder I created in the external I named "itunesmusic"(the one I put the "iTunes Media" folder you told me to create in)... Should that folder have contained anything like the .itl or .xml from the C: drive? If so is it too late to put them in after or is iTunes going to create them again in that folder?

When you say this, 1) Quit iTunes; on the external, locate that "iTunes Library.itl" file inside the "iTunes" folder you saved from C:; make a copy of that .itl file. (Let's call that copied-from-C "iTunes" folder, "the original iTunes folder".) I think this may answer my 1st question but i'm quite confused with that "itunesmusic" folder I created in the external. What am I supposed to do with that .itl file that you told me to copy? Where should it be located and why did I need to copy it?


When you say this, "Live with this for a few days; when you're satisified everything is working OK, move the old "iTunes Music" folder out of the original iTunes folder and verify that everything is still OK... If it is, you can delete the old "iTunes Music" folder" Do you mean if everything works properly I then would be able to delete the folder or files containing all my music that I took from the external and put back on the C: Drive?

I'm trying to follow these instructions properly and we'll see how everything goes once it's done consolidating.
Well, once again thank you so much for your help. Hopefully I didn't confuse you haha

[S2_Mac]

Ok, that got hectic really quick... I had about 6000 files missing that I had to delete and re-add the artists.

Here's how it should have worked: by Shift-launching into the "iTunes Library.itl" file in the copied-from-C:-onto-the-external "iTunes" folder, the "missing files" problem should have been gone (owing to the special treatment iTunes gives to content files that have a folder in common with the library file). From there, creating an "iTunes Media" folder inside that copied-from-C:-onto-the-external "iTunes" folder would have provided a clean target for Consolidate to copy all your content files into. Once Consolidation was done, everything would have been contained in the copied-from-C:-onto-the-external "iTunes" folder -- library file, library support files, and all content files. The old, incomplete set of content files, in the "iTunes Music" folder of the copied-from-C:-onto-the-external "iTunes" folder, could be deleted whenever you felt like it.

The copy of the library file that I had you make at the beginning was just a backup -- if things had gone wrong, you'd still have that original library available for a do-over. As a rule of thumb, any time you're getting ready to take a "big step" with your library -- such as a massive Consolidation -- begin by making a copy of the library file. That way, if things go south you aren't stuck with the results; you can use the copied library file to un-do the recent changes.

From what you describe (The folder I created in the external I named "itunesmusic" (the one I put the "iTunes Media" folder you told me to create in)), you didn't Shift-launch into the copied-from-C:-onto-the-external "iTunes" folder...which most likely explains why the 6000 files had to be re-added. Big apologies if I didn't make that step clear enough; the whole point of that instruction was to save you the hassle of re-adding tons of music files and having to shuffle a bunch of library files around. Wups.

But, now that all your content files have been consolidated into G:\\itunesmusic\iTunes Media, all you need do is copy the library file and its "supporting" files into the "itunesmusic" folder, and you're done. Just make sure you pick the right library file to use! Here's a good way to do it:
1) Quit iTunes
2) Look inside whatever folder you think holds the "iTunes Library.itl" you were just using, and check Properties of that library file. If the Modification Date is less than a couple of minutes old, you've got the right file. Copy everything from that folder -- except for the "iTunes Music" folder -- into the "itunesmusic" folder.
3) Shift-launch iTunes into the "iTunes Library.itl" file in the "itunesmusic" folder; none of your tracks should have "missing" problems an everything should be well. Make double-sure everything is good by doing Get Info on a music file and verifying that it's living inside the "itunesmusic" folder (as in: G:\\itunesmusic\iTunes Media\The Beatles\Let It Be\The Long and Winding Road ;-)

(iTunes re-writes the library every time you quit the program, so if the Modification Date of "iTunes Library.itl" is older than a few minutes, you're looking at the wrong library file. Use Windows to search for "iTunes Library.itl" and sort the results by Modified Date; the most recent file is the one to copy into your "itunesmusic" folder. Make sense?)

(Another note....the only file that must be copied into the "itunesmusic" folder is the "iTunes Library.itl" file; all the other files and folders will be recreated as needed by iTunes. The other files/folders contain bonus stuff like listings of all the CDs you've ripped, any Genius info you've generated, cover art that was downloaded from the Store, and some other info. Not crucial stuff, but perhaps nice to have.)

[bben411]

Alright, I got everything in working order. Thanks to you my library is once again running correctly. Not only that but you've given me the bonus feature of having the library run entirely by it'self on the external. I can't express how thankful I am for your help.

I put the latest iTunesLibrary.itl and all the contents of the folder excluding the iTunesMusic folder (Which was from the original folder on the C: drive) into the the iTunesMusic folder in the external. Everything seems to be in working order.

Would it be a safe assumption to say that I can now delete all the music from my C: and the original music contents from the G: (External) drive?

Also, anytime I drag new music into iTunes it will automatically consolidate the new music into the new iTunesMedia folder on the external, correct?

Your instructions were very clear. The failure to understand was on my part, not yours. It just got a bit complicated at times but I understand the process now.

I am so grateful for your help, thanks once again. It truly saved my music library and my sanity!