Concerned my iTunes folder structure is different to that reported on iLounge

[dundas_sj]

I Have iTunes on a windows 10 PC consisting of an iTunes folder on c: and the data on my E drive. i have read lots of official and on-official posts about moving data and consolidation but my sittuation seems different causing me to lose data and have a non-standards file structure.
I want to move the data to a larger drive (G but wonder if I should somehow so something about the odd files structure first. I am aware I shouldn't manually move folders else I could lose links. I had in fact recently tried to move the media using iTunes options but the data worringly shrunk from about 250GB to 160GB so I restored the C: and data drive files from a backup made just prior to the attempted move! I'm wondering if there is an issue to correct prior to moving the media. I believe I made a mistake in moving the data then consolidating. I should have consolidated before the move I expect. I use iTunes 12.12.2.2. The structure currently is:

the c: iTunes folder has file structure as per 1st attachment plus the library xml and iTL files.
The music files are in a file on my E: called iTunes_Main. - see 2nd attachment

iTunes_Main looks like the 'Media' file and I was expecting it to have subfolders for music podcasts etc but it lists most of the artists amongst which are folders 'music' and 'podcast'. Most if not all of what is within 'music' appears in the list elsewhere but sometimes renamed or with a very slightly different file size.

So artists are listed within the c: iTunes folder (just a few) and on E: in the iTunes_Main folder and in the Music subdirectory of iTunes_Main.
How can I move all my data files to G: without losing any or having lots of duplicates? Also isn't the regular file structure to have single Music, Podcast, Film folders within a folder called Media? My gut feeling is that i should do a consolidate then change the media location to G: in advanced preferences (i forget does iTunes then move the data or do I need to copy it before changing preferences to G: ?)

iTunes is the centre of my universe which I rely on as a musician so I would be eternally grateful if you could help me.
Many thanks, Simon

 

[cjmnews]

First off, the attachments didn't make it. I am imagining the structure based on the text.

Second, there are a few ways of moving iTunes Music.

Since you have a C and E drive iTunes folder you can't use the Windows method to move everything. Which is just mapping the C drive folder to a new folder on a different drive (my iTunes is in a folder on my RAID E drive using this method, on my 6th computer too).

Consolidation should get you all of the files in a single location, at least all of the files that iTunes knows about. This process does a COPY not a move, which is why a lot of people can't use the Consolidate as they don't have enough space for 2 copies of their library.

The consolidation process will reorganize all the folders into the standard structure. iTunes\Media\Music\<Artist>\<Album>\<track>

You would need to be more clear on the "Most if not all of what is within 'music' appears in the list elsewhere but sometimes renamed or with a very slightly different file size.". This isn't making sense to me. You have the same tracks in different locations? You may need to see if the track is duplicated in iTunes or not. If not, check the location of the track in the Song Info menu (right click on the track) and see which one it is using. I think maybe a Consolidate was used before and you are seeing old (C Drive) an new (E Drive) locations. The Main vs Music is old iTunes (iTunes\iTunes Music\<Artist>) vs new iTunes (iTunes\Media\Music\<Artist>) and if the last Consolidate happened with new iTunes this would explain this too.

Glad to see you have a backup, that way if anything goes wrong you can start again.

You set the iTunes Library location in Edit->Preferences, Advanced tab, click the Change button. Put the path to the new drive (G drive) location, then use the File->Library->Organize->Consolidate option to copy the files to the new location.

 

[dundas_sj]

HI, thanks for taking the time to look into my issues. I'm not sure why the files didn't make it but here's a hopefully better description.

Issue 1 - My main concern is the fact that when I open E: drive folder iTunes_Main I see a list of artists but under 'm' there's an 'artist' called 'Music' which when I open has a similar list of artists and takes up about 75GB of the full 250GB or so total. I did as you say by searching the xml library file and found none of the 75 GB were mentioned ie. no directory path links. I'm happy to drop the non linked files as I've checked and to all intent and purposes they are duplicated. This must be an artifact - mix of new and old iTunes structure as you say. Am I right in thinking if I consolidate only linked files are kept so the non linked files will be let out of the new structure?

Issue 2 - Loss of about 100GB in size in my first attempt to move the data to G:. I had actually used the process you say - set new library location then consolidate. 75GB is accounted for by the unlinked files mentioned above and I the other 25GB seems to be and artist called 'Voice Memo'. In a past incarnation of iTunes I think this was a folder created and used by a little voice recorder device that plugged into the end of the iPod (non Apple!). Ued to record live music. Scanning the library xml there were only about 6 files referenced in it with directory path links. I need to investigate of the voice memos and if they are not in the iTunes data in any other place (eg. if renamed) I can reimport them. The iPod classic stopped being able to load wav files from iTunes quite a while ago but I could convert them to mp3. Once I've imported the voice memos I want to keep I should be ok to move to do the move to G: plus consolidation. all sound ok?

Issue 3 - Stray music files on C: (there aren't too many) can just be deleted after consolidation can't they as they wont be linked?
Issue 4 - My iTunes folder on C: looks odd - it has subfolders including 'iTunes Media', 'iTunes Music', 'Automatically Add to iTunes' along with the library files. 'iTunes Media' contains an empty folder called 'Automatically add to iTunes'. Are Media and Music just old and new iTunes bits? There are a just a few music files here which I presume will be copied to G: when consolidated. After consolidation I can delete music files not on G: Sound OK?

I really appreciate your advice on these issues.

thanks, Simon

 

[cjmnews]

When you create your folder for iTunes music and media on the G drive, keep the path fairly short. G:\iTunes would work G:\Music\iTunes would also work, just as well as G:\Music. Long paths can be troublesome for Windows and iTunes.

Issue 1: Yes only tracks that iTunes knows about will be copied. Anything that iTunes is not aware of will be ignored. So the 75GB of files will not be copied.

Issue 2: Voice Memo is still used by iPhones today. They record in m4a (aac) format using the Voice Recorder app. The WAV files cannot be imported into iTunes, but iTunes can convert them to mp3 or m4a for you. If you want these files to be part of the move, then converting and importing needs to be done before the Consolidate is completed. You can rename them in iTunes for easier to find and consume. The size should shrink considerably by converting them to a lossy format since WAV files are huge.

Issue 3: Yes, stray music on C and E can be removed after the Consolidate.

Issue 4:
It is hard to say for sure with these Library files. Are the dates on the Library files recent or old?

• If new (the date and time you last closed iTunes) on the E Drive, these may be the files that iTunes is using to hold your library information, these can not be deleted. You can delete the iTunes Library files or Previous iTunes Libraries folder or the Album Art folders from the C Drive (Not E Drive)
• If old, this may have been an attempt to copy from C to E in the past, and they can be deleted. You cannot delete the iTunes Library files or Previous iTunes Libraries folder or the Album Art folders from the C Drive as they will remain on the C Drive.

iTunes Media is from the new structure
iTunes Music is from the old structure
Automatically Add to iTunes is a standard folder. In the old structure it was under iTunes Music, in the new it is under iTunes Media. I don't recall it being at the iTunes folder level before.
Yes you can delete the iTunes Music and iTunes Media folders from the C Drive once the Consolidate is complete.

If you want the iTunes Library files to also be on the G Drive, you would copy the files from the active library location (usually C:\Music\iTunes, but could the E) to the top level directory where the music is stored (G:\Music or G:\iTunes or G:\Music\iTunes using my examples above) then hold SHIFT and start iTunes. Then navigate to the Library File on the G drive. Now you have moved the library off of C or E drive to G and your backup of G:\Music would get all of your iTunes data in one package instead two packages on different drives.

I think that covers it.

 

[dundas_sj]

All sorted and I am a very happy man. Thank you so so much for your expertise.

 

[dundas_sj]

sorry, something not quite right. After resetting the preferences for media folder to G: the move from E: to G: seemed to go fine , took ages copying. Then moved the library files from C: to G: and shidt/started itunes to pick the library file. iTunes looked fine when I opened it but it still said the media file was E: in preferences. This exact same thing had happened when I first attempted a move from E to G. There are 2 xml files in the G:itunes folder - itunes Library and iTunes Music Library both contain file references with E rather than G so iTunes is still using the old location (I hadn't deleted that folder yet).

Any ideas what's going on?

many thanks. Simon

 

[dundas_sj]

... very oddly i right clicked on a few files in the iTunes app and they reference G: Further investigation of the iTunes Library.xml file shows some files are being sourced on E: and some on G: The music data size on G: was what I was expecting. I had done a consolidation. Peculiar

 

[cjmnews]

I would expect the copy to take a long time, that is normal. Especially if G Drive is a USB drive.

The XML files don't mean anything they are actually written out by iTunes when it exits if the XML is enabled in iTunes. Mine is disabled for better performance.

The switch back to E drive in preferences is weird. It almost sounds like the Consolidate created a new library file on the G Drive and there was no need to move it from E. That would be new behavior. It has always updated the existing library file.

Is there a Library File on the C Drive? Is the date and time on it the same as the date and time of the move? If so, this was the library file that was updated by the Consolidate. Hold Shift and choose the C drive one and see where it thinks the music is located.

If the C Drive library is correct, then it can be moved to the G Drive and the Shift open can switch to this copy.

We can also force iTunes to find all the files in the G Drive by moving the C and E Drive folders (rename the top level folder of the music by adding _moved to the name) then play a track that was on the E drive and when it asks to locate it, find the track on the G drive, and when it asks if it should update all tracks with the new location answer Yes. Repeat for a C Drive track.

 

[dundas_sj]

Hi, thanks again for bearing with me. I think I will go back to square 1 and and restore the C: and E: files. I had deleted the iTunes library etc files on C: after moving them to G: in case the system and myself got confused by there being more than one. copy. iTunes asked me a couple of questions during the processes which I will write down next time. There were no library files on E: Anyway I'm about to restore the files. Is it worth doing a consolidate before changing the iTunes media preference to G: ? I was thinking I should do the following. Does it look like a sensible plan?

1. Restore C: and E: itunes files to their original state
2. consolidate to a) get all files into the c: library files and b) drop files not referenced - if there are any.
3. Set the media file location preference to G:\iTunes\ and tick the 'keep files organised box'
4. consolidate to get iTunes to move the files to G:
5. check all of files I expect to be copied have been (looking at the folder size and some spot checks on file paths using 'file\info'
6. if there is a new xml file formed on c: I will scan to check all directory references are E:
7. assuming all is well I will rename the E: tunes folder so it's definately not used
8. I will copy the c: itunes files to g:\itunes\ and rename the c: iTunes folder so it can't be used.
9. I will 'shift' open itunes and select the library file on G: I'd copied from C:
10. I will spot check some of the files in iTunes to check the paths refer to G:
11. I will close iTunes and if there is a v recent xml file I will scan it to check all directory paths are G:. If any are E: I will open iTunes and see if they play and what it says the directory path is using file/info
12. I'll check the media file preference is still set to G:

May have to implement your last paragraph action if things still not ok. and force iTunes to update the file directory names.

Sound OK?

 

[cjmnews]

My only concern is with step 8. You may want to close iTunes and see where the Library File was updated last. If C Drive copy is the one, then your steps 8 & 9 are correct. If there is a G drive copy that was updated when you close iTunes, then Steps 8 and 9 are not needed.

 

[cjmnews]

A late thought about Step 6. You have to exit iTunes to get the current XML written out (if enabled in the preferences)

 

[dundas_sj]

Thanks you've been very helpful. I hope this isn't driving you mad! The initial consolidation step does a lot of copying which creates a duplicates on the harddrive - the original folder structure and a new one all within iTunes_Main. Often these duplicated files are self recorded wav files which I treat as I would an mp3 populating the file info. But some ripped cd mp3 files do get duplicated as well. Anyway I realise that consolidate is probably not required for me as I always have the option 'import to iTunes media folder' checked. Likewise 'keep files organised' , which renames files in some cases to incorporate the disk and track number in to the song name (amaongst othe rthings) , is probably not required. Could i manually move the whole library to G: open iTunes and when prompted by 'missing file would you like to locate it' direct to G: and believe it might then ask to do this for future missing files. This will update file paths in the Library file. I now wonder if my use of blanks or the text 'Unknown' or 'Artist Unknown' 'Album Unknown' in file info is causing problems for iTunes and I could use some other text. For now I'm doing a bit more investigation.

 

[cjmnews]

This is pretty typical.

I am glad you are tech savvy enough to perform backups and restores. When I have to explain how to look at Song Info to see the path where the file resides it gets tedious.

Duplicates should not be happening unless you have duplicates in your library. Your investigation should show that by sorting by track name for example. You can also see if File->Library->Show Duplicates can identify some of the duplicates. This feature is not good with classical music.

Ripped MP3 files you didn't mention before. These need to have their song information updated in the files before you lose the information by playing them. Select all your MP3 tracks, Use File->Convert->Convert ID3 tags. Check the ID3 Tag version box, choose version 2.3 (if you have older devices) 2.4 (modern devices only, no older ones) and click OK to write the iTunes information into the MP3 files.

Yes having the Copy files to iTunes Media Folder checked is a great way to have all the tracks put into a centralized location in the iTunes folder.
BUT: If there were files outside of the iTunes Media Folder in your library before that was checked, they would remain outside of the iTunes Media Folder. That option is really for new music added to iTunes not for the existing tracks.

The Keep Files Organized just ensures the iTunes Artist/Album/Track organization scheme is used if this was unchecked, you could have a different scheme of organizing the iTunes Media Folder like Rock/Favorites/Artist/Album/Track or a big folder of all tracks.

The Consolidation is a cleaner way to move the files to the G Drive as it will ensure the latest version of the iTunes organization scheme is used for all your media, not 80% current, 20% old. (Mine is 100%old).

You can manually move the library and use the update trick mentioned above would get it to point to the G drive version, but if there is anything outside of the iTunes Media Folder it will still be referenced by iTunes.

The unknown artists/albums will cause some issues, but not a lot.

 

[dundas_sj]

I was a computer programmer which helps. Also a very inquisitive mind!
Copy to iTunes media folder has been checked since the year dot so I expect all files to be within the iTunes system according to Apple rules. As you've noted before there may be remnants of a couple of old incarnations of iTunes caused by past use of consolidation or 'keep files orgainised' but those old bits should no longer be used by iTunes (I will check again by looking for file paths other than E:iTunes_Main).

Ripped MP3 files are just where I've imported CDs in the regular way. I gert iTunes to get the tags fomr it
's database. If it doesn't find them I manually populate them. So tags should be fine for those.
The other major element in the library are wav and mp3 files I've manually recorded somehow and dragged in. Mostly I do populate the main tags like artist, song, album but sometimes if this isn't known I'll put 'Unknown'

There are known duplicates caused by variosu types of mistake and occasionally gettgin iTunes to create an mp3 version of an imported wav file. To follow what happens to an individual song I've been searching the iTunes main folder before and after processing eg. consolidation. As I say I will check the original iTunes incarnation for filepaths other than E: but I don't expect to find any.

I'm about to reset everything once again and have another attempt. First I'm going to populate any missing artist and album with 'missing' within the iTunes app.. As you say this shouldn't be a problem for iTunes but will do it just in case. 'Keeping files organised' bothers me that it renames the song name sometimes adding the track number before it. Consolidation while keeping the drive as E: still bothers me when i see it come up with a box saying copying and masses of file names flow by one by one. What is it doing? Copying implies duplication. I can understand if I've reset the media file location and want consolidation to copy from the old to the new. I don;t think i had the 'keep files organised' checked. I will get there but it's taking ages. I don't want to lose any files built up over 20 years or so.

 

[cjmnews]

Ripped MP3 do not have their data embedded in the song files by default if you set the data with iTunes. This is a huge shortcoming of iTunes. The Convert ID3 tags steps need to be performed to embed the tags inside of the files. If the MP3 file is played, it will forget the tags as the Music App will query the ID3 tags when it plays, and then the song will roll back to old, invalid or missing data. Even manually populated tags are stored ONLY in the iTunes database, not in the file. You'll be happier after the Convert ID3 tags process to have the song information stay consistent with iTunes always.

I have a few of those duplicates too. Mostly self made ringtones. For WAV files, I delete them once I get the MP3 to prevent issues. If you want the source to remain then put the WAV files outside of the iTunes folder structure but where a backup will get them. This will seem less confusing in time.

I like to have all tracks with Artist, Album and Track information so I highly recommend the clean up of the unknowns. The track number at the front of the name is just an iTunes thing. Since I use iTunes, I don't look at the track file names, so I don't care if it does that. My preference is to have it not add the track number to the track name. I suppose it helps when an artist reprises the same song on the same album making each track#_trackname not trackname and trackname_1 like it used to do. You can avoid it by not having track numbers in the Song Info, but the album order will be alphabetical not actual order if you do this.

Consolidate is copying each file from one location to another location. Every file that iTunes knows about will be copied to the new location, leaving the original location intact. Yes it is duplicating everything. Once it is complete and you know where the library file that represents the new location resides, the old locations can be removed to save space. The Consolidate process does take time, especially when the destination drive is USB connected.

The only way to lose a track is if iTunes does not know about it and it was located in one of the source locations that you delete at the end of the process.

 

[dundas_sj]

I just want to make sure i understand what you're saying:

1. ripped cd files and presumably other mp3 files I import without tags but add tags via iTunes \info do not have the tags of the source files changed? After when I play these files over and over I've never personally noticed say song/artist/label changing. Maybe your saying if i play the files outside of iTunes it wont have the tags - i have found that ie. the changes are only made to the iTunes library file.
2. do you mean there is a preference to add or not add track number to start of a song? If so where is it or is it the 'Keeps files organised option'

1. Consolidate - yes i see that consolidate will copy files into a single media folder so if i haven't changed the media folder location in adv prefs and it is still E: iTunes_Main then is the copying that I see in the iTunes app when i press consolidate implying not all source files are in E: ? Opening the library xml in firefox browser I searched for c: and only a handfull of files were found. A few files were in the iTunes_Main\music folder (which has masses of files also at the iTunes_Main level) but only a few were referenced. The vast majority were in the primary location e: iTunes_Main/ followed by artist and album. I'm still a bit in the dark as to why so many files where indicated by the app as being copied when consolidating.
2. when i got my original iTunes back from archive i did a file/library/export to get a tab delim file and reading it in excel almost 7,000 entries has a missing file pathname field. I tried to play one or 2 in iTunes and it couldn't find them so that matches though alarming. But when I did a file info there was a location - not sure why this info wasn't in the excel. Nevermind, I looked for the real file using windows explorer and it was in the location indicated by iTunes file/info BUT the filename was slightly different hence it wasn't being played. This was from a real CD almost certainly in the Apple info database to get tags. The real file was called Saccarine.mp3 but iTunes was looking for Saccarine 1.mp3. I spot checked a few more of these 7000 and iTunes was looking for a file with a '1' on the end. 7,000 presumably missing files of 17,000 total is a huge amount . Any idea how the library file got corrupted to be looking for files with a 1 on the end (actually a space then 1) and more importantly how I can correct it.

Could I be doing something wrong when I set up the archive files. I delete the c: iTunes folder and the e:iTunes_Main then restore both , to the same locations, using my Acronis archive app. I then open iTunes. I presume my iTunes settings will be restored as per when I made the archive as I restore the complete c: iTunes folder.
Sorry about this headache. Of course it may be all those files were corrupted when I did the archive and I hadn't noticed but it is more than half the collection so surely I'd have noticed some files not playing. It's rare that I ever come across missing files.

 

[cjmnews]

1. Some devices like iPods and iPhones will attempt to read the tags in the file and finding none or invalid tags will update the display to show what was in the ID3 tags. Yes iTunes has this data in the Library File, but it is only in iTunes. If you look at the MP3 file outside of iTunes, the tags will revert to unknown (if missing or empty) or the tags that were in the file originally. I recommend that all MP3s are updated with the Convert ID3 tags method to keep the data in the file.
-- The main reason is IF the iTunes Library file is lost, the usual recovery mechanism is to reimport all the music into a new iTunes Library. When this is done, the ID3 tags in the file are used to identify the track, and if you have several MP3 files with empty tags, the new iTunes Library will show all these MP3 files as Unknown Artist and Unknown Album and Unknown Track Name. Convert ID3 avoids this problem if done before the iTunes Library File loss.

2. Unfortunately there is no preference to stop the Track Number from being added to the front of the track name. BUT if you select 1 or more tracks and edit their Song Info, and set the Track Numbers to nothing, there is no track number data for the song. Then when the Consolidate happens the track number will not be applied to the track name. The downside of this is Album Order of songs will be alphabetical, not the correct track order. If you want to hear an album in the released order, you can't do that anymore. Choose: Track Number prepended to Track Name and have correct Album Order playback, or No Track Number on Track Names and Album Order is alphabetical.

1. Consolidate - Yes you can Consolidate to the same location. This will move the C and Drive files to the current location. It will also try to put everything into the new iTunes organization format. Meaning you no longer need to keep the old iTunes Music folder after it is done. It would copy all the files from iTunes_Main as well, so that folder would no longer be needed. The files may be organized in the legacy iTunes organization format which will cause lots of tracks to be copied.

2. Missing file pathname - Usually the " 1" added to the name is because there used to be a song without the " 1" in that location at one time. This could happen if a copy of folder iTunes already had imported was dragged and dropped into iTunes. iTunes following the Copy Music to iTunes Media Library folder copies the file to the appropriate folder, sees the duplicate and adds a " 1" to the name to write it there allowing you to have both copies. It can also happen if someone performs a Create MP3 version of MP3 files. If the original was removed, and the library was rolled back to before the mistake, this would cause the 7000 files to appear missing.

Based on what you said about the Acronis use, I don't see how it could be causing this.

You can use Playlists to find songs that iTunes does not know the location anymore.
File->New->Playlist to create a standard playlist, name it Good.
In Track view of the entire library, select all tracks, drag and drop onto Good.
File->New->Smart Playlist
Rules:
Playlist is Music (click the + button)
Playlist is not Good
Click OK
Name it Bad

All the tracks in Bad cannot be found by iTunes.
If you fix a bunch of them, then you can delete all the tracks from Good, then drag and drop all tracks onto Good again to see Bad get updated automatically.

I prefer this type of help, more fun for me to reproduce stuff at home.

 

[dundas_sj]

Sorry I wrote this before you'd posted your last reply but it failed to actually post as you were posting! without your having access to my PC you wouldn't have seen what I've just seen -

Partly solved**: my big error. iTunes was using the library files etc I left on G: when days ago I tried moving everything onto G:. I'd noticed iTunes contained data added after the archive date. I removed the G: files: and pointed iTunes to C: (archive) files when prompted. So now i'm properly back to square 1. All files tested so far play. Phew. I was starting to have kittens. Exporting the library to text then into excel I think i see there are a lot of files with missing artist or album. I beleive you said if I had edited the file info within iTunes it would remain tagged in iTunes but the source file would not be tagged. I must have inadvertently dragged all these files into iTunes forgetting to edit the info - or could there be another explanation? I will populate the tags before I progress. The excel is very handy to analyse the data. eg. all songs have a source file in E:iTunes_Main. Been on the pc far too long today and gettgn late so will read your words of wisdom tomorrow and carry on. tagging before I re attempt the move of everything from E: and C: to G.

 

[cjmnews]

Only MP3 files don’t get the updates in the file, when changes are made in iTunes.
AAC files get the updates applied directly to the file automatically.

In both cases, iTunes keeps the tags.

I agree Excel helps a lot.

 

[dundas_sj]

thanks that's v interesting. I'll switch to using AAC.