My script returns gibberish text: How to get it right. Help please!

[Scriptonite]

Hi there,

I have been building an applescript for iTunes to extract the author name from PDF tags and place it in the "Author" tag of my books (pdf) sitting in iTunes (I put back Books in iTunes and got rid of iBooks).
Actually, I have done little since my knowledge in applescripts is very incipient.
What I've been doing is modifying an existing script and modifying here and there and adding code that other friends around internet have gave me.

The script is almost complete, but has a problem: either reading or writing or both, the information, if the name in the tags of the PDF has especial characters (i.e. á, é, ñ, ü.....) it will write the iTunes tag with alphanumeric sequences where this special characters are: "Odtü Kutuphanési" the printed name in iTunes is: "Odtu\U0308 Kutuphane\U0301si

Could you please provide with the additional code to fix this problem?
Thank you very much in advance!

This is my code so far:

tell application "iTunes"
	
	set sel to the selection
	
	if sel is {} then display dialog "Please select something" buttons {"Cancel"}
	
	repeat with i from 1 to count of sel
		
		set the_track to item i of sel
		
		set the_posix_path to POSIX path of (location of the_track as alias)
		
		set authorfield to paragraph 2 of (do shell script "mdls -name kMDItemAuthors " & quoted form of the_posix_path & "|  awk -F\\\" '{print $2'}")
		
		set artist of the_track to (authorfield & "-") as string
	end repeat
	
	
end tell

 

[cjmnews]

This is my first time seeing AppleScript, I usually use javascript.
But AWK is one of my favorite languages, so this was interesting to me.

My guess is that the UTF-8 characters are not being handled by AWK correctly.

Try using

awk LC_ALL=en_US.utf8 -F\\\" '{print $2'}")

to see if that captures them correctly.

 

[Scriptonite]

Thank you very much, cjmnews!

I tried but I got
Result:
error "iTunes got an error: awk: syntax error at source line 1
context is
>>> LC_ALL=en_US. <<< utf8
awk: bailing out at source line 1" number 2

Just to let you know in case I did wrong, this is the way I insert your code:

set authorfield to paragraph 2 of (do shell script "mdls -name kMDItemAuthors " & quoted form of the_posix_path & "|  awk LC_ALL=en_US.utf8 -F\\\" '{print $2'}")

I hope it was my mistake
Otherwise I hope you are patient enough to fine a solution for me.

Thank you!

 

[cjmnews]

It may require quotes around the "en_US.utf8"

It may also require you set up the Terminal to have the locales correct.
I found this article: Setting locales correctly on Mac OSX Terminal application Remi Bergsma's blog
That talks about it with the use of FTP, it may work for Mac OS too.

Sometimes I wish I had a Mac to play with this stuff.

 

[Scriptonite]

Thank you again, cjmnews.
That didn't work either.
I read the link content and set the terminal prefs as they indicate, but no luck.
I wish everyone has a Mac

Cheers!

 

[cjmnews]

How about this I found?

Try adding or editing the ~/.profile file for it to correctly export your locale settings upon initiating a new session.

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

These two lines added to the file should suffice to set the locale [replace en_US for your desired locale, and check beforehand that it is indeed installed on your system (locale -a)].

After that, you can start a new session and check using locale:

$ locale

The following should be the output:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL="en_US.UTF-8"

 

[Scriptonite]

Thank you, cjmnews!
And thank you for the heads-up!
I know almost nothing about this, so I appreciate that you gave me that warn.

Now:
I back up every day, both music files and the records iTunes keeps (iTunes libraries (itdb, itl, and xml))
Also I make a backup in a compressed file next to the working one every time I start working with it.
The actual files are in a separate drive from the library files, and both are backed up every day.

I'm not gonna ask why this could happen. 1-It might be difficult for me to understand 2-Because I want to keep this on the subject.

Now, could you please indicate where exactly should I insert your new snippets?
I'm afraid to make a mess now

And...

I ran the command "local" in my Terminal and this is what I got:

LANG="en_US.ISO8859-1"
LC_COLLATE="en_US.ISO8859-1"
LC_CTYPE="en_US.ISO8859-1"
LC_MESSAGES="en_US.ISO8859-1"
LC_MONETARY="en_US.ISO8859-1"
LC_NUMERIC="en_US.ISO8859-1"
LC_TIME="en_US.ISO8859-1"
LC_ALL=

Thank you very much!

 

[cjmnews]

See your local is set to not be UTF-8 so it is changing those special characters to hex codes.

You probably have a .profile in your home directory, you can edit it with a text editor (e.x.: nano), and see if there are lines to set LC_ALL and LANG
If there are lines for that, change them from
"en_US.ISO8859-1" to "en_US.UTF-8"

If those lines don't exist, then add 2 lines:
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

Save the file. Open a new Terminal window and try the $ locale command again to see if it changed.

If it did, then I would log out and log in to make sure your whole profile gets the change, not just the one terminal window. Then try your code again.

If it did not change, try logging out and in again to see if the change gets applied.

 

[Scriptonite]

UPDATE:
I changed the text encoding prefs in Terminal to Unicode (UTF-8).

Now the result is:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

But the LC_ALL= value is still empty.

The result in the script is still the same. (it won't take the """ you suggested.)

Thank you.

 

[cjmnews]

Now that the environment is correct, your original code should work.

Remove the LC_ALL=en_US.utf8 or LC_ALL="en_US.utf8" that I had added in before.

 

[Scriptonite]

Thank you very much for your prompt answer, cjmnews!

It didn't work either. Still same gibberish
I tried running it as is in my very first post and without your snippet, but not luck.

 

[cjmnews]

OK, I understand it now.

You are seeing Combined Diacritic Marks : Combining Diacritical Marks ? Test for Unicode support in Web browsers

Basically the character is not one character, it is 2.
so for the accented e, you are getting the e and the accent as a separate character.

I wonder if this is an mdls issue.

Can you try altering it?
I am not sure how, but I found this on another site:
set mdlsResponse to (do shell script "mdls " & quoted form of POSIX path of myFile) as Unicode text

I think it may make the mdls convert to Unicode and that may fix this issue.
Passing Unicode to AWK with the Local set with Unicode may work.

 

[Scriptonite]

Thank you very much again, cjmnews!

I want to friendly remind you my introduction to this thread and myself.
My knowledge of code is very incipient.

Could you please tell me specifically what do you want me to do with that code?

Should I insert it in the main script? Where?
Should I replace part of my script with it? What?

Should I run it separately ? How?

I appreciate very much your help!
Thank you.

 

[cjmnews]

My best guess:

tell application "iTunes"
	
	set sel to the selection
	
	if sel is {} then display dialog "Please select something" buttons {"Cancel"}
	
	repeat with i from 1 to count of sel
		
		set the_track to item i of sel
		
		set the_posix_path to POSIX path of (location of the_track as alias)
		
		set authorfield to paragraph 2 of ((do shell script "mdls -name kMDItemAuthors " & quoted form of the_posix_path & ") as Unicode text |  awk -F\\\" '{print $2'}")
		
		set artist of the_track to (authorfield & "-") as string
	end repeat
	
	
end tell

 

[Scriptonite]

Thank you, cjmnews.

There is a problem with the code.
It won't even compile

 

[cjmnews]

I guess it didn't like the parenthesis.

I don't know this scripting language, and have no way to test it.
I separated the commands into 2 lines.

tell application "iTunes"
	
	set sel to the selection
	
	if sel is {} then display dialog "Please select something" buttons {"Cancel"}
	
	repeat with i from 1 to count of sel
		
		set the_track to item i of sel
		
		set the_posix_path to POSIX path of (location of the_track as alias)
		
		set authors to (do shell script "mdls -name kMDItemAuthors " & quoted form of the_posix_path " as Unicode text)
		set authorfield to paragraph 2 of (do shell script echo " & quoted form of authors & " |  awk -F\\\" '{print $2'}")
		
		set artist of the_track to (authorfield & "-") as string
	end repeat
	
	
end tell

 

[cjmnews]

I think I had a syntax error, an extra quote in the mdls line. Plus an the closing parenthesis was in the wrong place.

tell application "iTunes"
	
	set sel to the selection
	
	if sel is {} then display dialog "Please select something" buttons {"Cancel"}
	
	repeat with i from 1 to count of sel
		
		set the_track to item i of sel
		
		set the_posix_path to POSIX path of (location of the_track as alias)
		
		set authors to (do shell script "mdls -name kMDItemAuthors " & quoted form of the_posix_path) as Unicode text
		set authorfield to paragraph 2 of (do shell script echo " & quoted form of authors & " |  awk -F\\\" '{print $2'}")
		
		set artist of the_track to (authorfield & "-") as string
	end repeat
	
	
end tell

 

[Scriptonite]

Hi cjmnews,

Thank you!

The behavior of this last version is the same as the previous when I try to compile:

Syntax Error:
Expected “,” but found “"”.

I still have no luck in getting an answers from other sites I have requested help.

Thank you very much!

UPDATE:

I went ahead and checked for the common string is used when combined with "do script shell". This is: ...(do shell script "echo" &.....
I replaced it and the good thing is that it will compile, but it won't run.

Here is the result:

 

[cjmnews]

First off there should be a space after the word echo inside the quotes. That would allow the echo to run with the argument.

I think your code update was good.

This looks promising.

It assumes that there are 1 or more "/" characters coming from authors.

I would print out the value of authors with a linelike

display dialog authors

to see what I get

 

[Scriptonite]

Hi cjmnews,

I don't know what to do with: display dialog authors", but I set the extra space.
The result is: