Topic: https://brettterpstra.com/2011/04/02/mirror-your-pinboard-bookmarks-with-openmeta-tags/
hide preview

What's next? verify your email address for reply notifications!

Jerry Krinock 13y, 3h ago

Brett, Apple's launchd documentation is ambiguous in many places, and I've misinterpreted it more than once.

What that Apple engineer was telling me is that there is no way to fix automatically fix or avoid the problem of launchd trying to launch your program every 10 seconds if the executable has been removed. The only fix is for the user (or some watchdog that you've installed) to notice the problem, and then unload and uninstall the launchd task. But the watchdog is not a solution because then you need a super watchdog to watch the watchdog, etc., ad inifinitum.

What's weird in the case of your script is that it or some part of it must still be there, because it's hitting Pinboard. But launchd keeps kicking it every 10 seconds anyhow. I've often wondered, but could never establish consistent results, on what happens if a task launched by launchd returns other than 0. I know that launchd keeps track of return values. Maybe your script is returning other than 0. Or maybe it's timing out. There are a couple different timeouts in launchd, with particularly ambiguous documentations.

Anyhow, I think the solution is to somehow convince launchd that everything executed OK, so it will stop.

hide preview

What's next? verify your email address for reply notifications!

Jerry Krinock 13y, 7h ago

A user of my app BookMacster reported that he was unable to import bookmarks from Pinboard. After a week or so, with help from Maciej Ceglowski (the Pinboard guy), we found that the problem was this script, which the user had installed a few months ago. It was hitting Pinboard every 10 seconds and asking for all bookmarks (posts/all). This caused Maciej's server to throttle his account, so that any other app or script trying to get them would get an HTTP 429 error. Quickly skimming this blog post, I see that this script is installed so that it is triggered by launchd. One of the features of launchd in OS X is that if a job doesn't work, it keeps retrying every 10 seconds. I thought that this only happens if the script is missing, but apparently something was there. I haven't studied the package here in detail, but something like that that must have been happening.

Brett, maybe you could build in some kind of fail-safe mechanism for the next version so this does not happen to other users.

Thanks,

Jerry Krinock

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 7h ago

I'll still work on a failsafe measure, but I found a launchd key that should fix the issue:

<key>ThrottleInterval</key>
<integer>600</integer>

Add it to the list in the LaunchAgents plist and it should prevent the script from running more than once every 10 minutes. (not fully tested yet)

remark link parent
hide preview

What's next? verify your email address for reply notifications!

Jerry Krinock 13y, 7h ago

I don't that's the way it works. ThrottleInterval is how often it will normally launch your script. That 10-second thing is a feature. I once had an email discussion with an Apple engineer, suggesting that maybe it would be a good idea to limit that, because it often happens that people uninstall scripts or programs and that stupid thing will keep trying to launch it every 10 seconds for the next 10 years, and if people don't look in their Console they don't even know. But the Apple engineer said something like Nope, when a user says to launch a program/script, we keep trying, no matter what, and that it's an important part of Mac OS X.

remark link parent
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 6h ago

Like I said, not fully tested, but according to the documentation it should limit any spawning of the process. Misinterpretation of the documentation on my part is probable, though. If it works, I'll verify here.

hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 7h ago

Jerry, thanks so much for the pointer. I've actually been having the same trouble lately, and the launchd retry process would explain everything. I'll take a look at fixing this asap.

hide preview

What's next? verify your email address for reply notifications!

PJ Fagan 13y, 53d ago

Is your delicious script in the hands of anyone who still cares for it?

hide preview

What's next? verify your email address for reply notifications!

Mitchell Hislop 13y, 307d ago

I am getting an error on the second running of the app - I have the config file created, and when I go to run it again, I get:

/Users/mitchellhislop/.rvm/rubies/ruby-head/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in require': cannot load such file -- ftools (LoadError)
from /Users/mitchellhislop/.rvm/rubies/ruby-head/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in
require'
from ./getpinboard.rb:98:in block in <main>'
from ./getpinboard.rb:97:in
each'
from ./getpinboard.rb:97:in `'

Any ideas?

hide preview

What's next? verify your email address for reply notifications!

Irfan Hafiz 13y, 335d ago

Loaded up the script but seem to be getting the following repeating messages in the terminal

1579:1580: syntax error: Expected end of line, etc. but found “"”. (-2741)
2011-04-19 23:02:29.909 osascript[3505:903] Error loading /Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types: dlopen(/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types, 262): no suitable image found. Did find:
/Library/ScriptingAdditions/Adobe Unit Types.osax/Contents/MacOS/Adobe Unit Types: no matching architecture in universal wrapper
osascript: OpenScripting.framework - scripting addition "/Library/ScriptingAdditions/Adobe Unit Types.osax" declares no loadable handlers.
1396:1397: syntax error: Expected end of line, etc. but found “"”. (-2741)

The ~/bookmarks/bookmarks.stash file does not seem to get appended to either. At first I thought this may have something to do with the PDF part of the code but setting pdf_tag:false did not help.
Any ideas

Thanks

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 335d ago

The first chunk (up to "syntax error") is spewed when you have leftover Adobe Scripting Additions lying around. Delete the files at /Library/ScriptingAdditions/Adobe Unit Types.osax and it will go away. They're not used anymore, anyway.

The syntax error is likely because you don't have Paparazzi! installed. If that's the case, you can either install it, or comment out the whole chunk of the AppleScript section where it creates the PDF.

hide preview

What's next? verify your email address for reply notifications!

Ed Glazer 13y, 348d ago

Dude, you are awesomesauce! thank you for sharing!

hide preview

What's next? verify your email address for reply notifications!

Dan 13y, 350d ago

I figured out why I was getting an Applescript error, I figured I would post here to let anyone else know how to fix the same issue (the "Expected end of line" thingy).

Anyways, the Applescript calls Paparazzi. If you don't have this application installed, an error is thrown on the line with "capture "#{url}" min size minsize".

I removed this block with Paparazzi and everything now works perfectly.

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 349d ago

Thanks! That' s the same issue we were getting when Tags.app wasn't installed, but I forgot to compensate for Paparrazi! in the same way. I'll update the script and post a new download as soon as I can.

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 350d ago

Thanks a lot. For some reason I was thinking I had to add that in the getpinboard.yaml file >_<

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 350d ago

In the past I've been using setWeblocThumb with the -ni option so it doesn't add the site's favicon image to thumbnail preview. So with this script I disabled setWeblocThumb and then let Hazel run it with the -ni setting afterwards.

Is there any way to do this in the script so that I don't have to run this separately through Hazel afterwards?

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 350d ago

Sure, just add the -ni parameter right after setWeblocThumb in line 754.

hide preview

What's next? verify your email address for reply notifications!

Mike M 13y, 350d ago

Thanks! Believe it or not, this scratched an itch I was just thinking about last week. Well done.

hide preview

What's next? verify your email address for reply notifications!

HelgeG 13y, 351d ago

I have Tags.app (v2.02) installed, but no Bookmarks.plist file is stored in the folder ~//Library/Application Support/Tags/. I have gotten the script to work by using openmeta as tagmethod, but I am curious as to why I do not have the Tags.app database in the prescribed location. Tags.app works flawlessly by the way.

Oh, and thanks very much for your scripts and geekery. It is much apreceiated on this end.

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 351d ago

Have you tagged web pages before? I don't think it generates the plist file until you've tagged one… if that's not the problem, maybe it exists somewhere else on your system? Try a Spotlight search for "filename:Bookmarks.plist"

What's your location? There may be another issue with localization, too…

remark link parent
hide preview

What's next? verify your email address for reply notifications!

HelgeG 13y, 351d ago

You nailed it! I hadn't tagged a webpage before. As soon as I did, the Bookmarks.plist file appeared. As for location I am in Norway, but the UK date layout works for me.

Thanks for the help!

hide preview

What's next? verify your email address for reply notifications!

Giovanni Lanzani 13y, 351d ago

Hi Brett, is there a way to have the script save the bookmarks in a .html bookmark file (like Safari "Export Bookmarks" feature)?

In this way I could point Launchbar to that .html file, and be able to search my pinboard bookmarks with it.

Thanks

Giovanni

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 351d ago

See the demo script I linked in the last paragraph under Notes. It can take the database that the script creates and parse it to create a basic HTML file. You could pretty quickly extract just the parts of the script that create the database and then incorporate something like that, if you wanted…

remark link parent
hide preview

What's next? verify your email address for reply notifications!

Giovanni Lanzani 13y, 349d ago

You're right, I think this will do it

https://gist.github.com/905312

I put tag='' to have them all into the bookmarks file. Thanks Brett!

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 351d ago

All running fine now (after your emails and the new version with UK date formatting).

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 352d ago

Thanks a lot for your help Brett.
I did recently trash the bookmarks database for Tags, but did tag a couple of web pages to recreate it. I've restarted Tags which will hopefully help.

Just trashed all files and settings again and set it up with this latest version. It is downloading lots of webloc files but I'm also getting lots of error messages in Terminal such as:

258:589: execution error: Invalid date and time date 03-13-2011 08:31PM. (-30720)

and the odd one such as:

435:436: syntax error: Expected “"” but found unknown token. (-2741)

I've been getting these on and off since using the first version of the script.

Not sure if this is relevant but I'm in the UK and we write the date differently to you. For example today (3rd April 2011) you write it 4/3/11 and we write it 3/4/11. Could that be causing a problem?

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 352d ago

So, for the record, AppleScript DOES handle dates differently in the UK than in the US. New setting added to the config file... download updated.

hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 352d ago

Those are all AppleScript errors, and the occasional one is, unfortunately, to be expected as I didn't cover every possible edge case with the script. The one you're getting the most often is odd, though, as the date format coming from the Pinboard XML should be standard, and AppleScript should accept the date as is, no matter where you are. That's really odd.

I assume that the ones throwing errors do not have webloc files being created… I'll need to see the XML you're getting for one of these to figure out what's going on… I'll email you instructions for grabbing that directly.

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 352d ago

I've been having problems getting it running properly, so trashed everything and started again with this version.

But still having problems. After setting it all up and running it I get the following message in Terminal:

/Users/Jono/Documents/getpinboard.rb:778: undefined method []=' for nil:NilClass (NoMethodError)
from /Users/Jono/Documents/getpinboard.rb:722:in
each'
from /Users/Jono/Documents/getpinboard.rb:722

Only loads a couple of bookmarks now (was loading a few hundred intermittently previously).

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 352d ago

Ok, two things. First, that error is happening because the plist for updating the Tags.app database is coming up empty… which probably means you don't have a Tags.app database where it expects it. Do you have Tags.app installed and is there a database at ~/Library/Application Support/Tags/Bookmarks.plist? If there was and now there isn't, check for the Bookmarks.bak file in the same folder. Let me know, I'd like to figure out what happened there.

Second, I should have had some error checking on that line that it died on, so I added that and version 1.0.3 is on the download link now.

hide preview

What's next? verify your email address for reply notifications!

Barry 13y, 352d ago

Great idea! Unfortunately, after setting up the configuration file and running the script a second time, I get a long series of errors like the following:

607:611: syntax error: Expected end of line but found identifier. (-2741)

When I set the debug flag it spits out the syntax error following every "Grabbing ..." line.

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 352d ago

Hey all, thanks to Barry's sleuthing, the latest version (1.0.2) should now work better when you don't have Tags.app installed. I think. If you were having trouble, try the new download and let me know how it goes.

hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 352d ago

Barry, I sent an email reply, but haven't heard back. The error displayed is an AppleScript error, which is hard to track down because it could be happening anywhere in the osascript block that the script runs. If you could show me the configuration you're using (minus username and password, which I'll assume you've tested and are correct), I'll test it out and see if I can pinpoint the problem.

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 353d ago

Yes, that's worked. It's now downloading the the .weblocs.
So cool, thanks!

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 353d ago

…the last should read:
from /Users/Jono/getpinboard.rb:707

remark link
hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 353d ago

Try the download again (should be 1.0.1 now). Stupid mistake, please let me know if the new version helps.

hide preview

What's next? verify your email address for reply notifications!

Brett 13y, 353d ago

I see what happened there. I'll get a fix out in a little bit.

hide preview

What's next? verify your email address for reply notifications!

Jono 13y, 353d ago

Thanks a lot. Really been looking forward to giving this a try :)

Aftern I run the script for the first time the yaml file file pops up all I have to do is enter my pinboard details, save it and then run the script again, right?

Once I've done that I run the script from Terminal and it displays the following message:

/Users/Jono/getpinboard.rb:641:in needs_update?': undefined method[]' for nil:NilClass (NoMethodError)
from /Users/Jono/getpinboard.rb:7

hide preview

What's next? verify your email address for reply notifications!