Bye-bye Yahoo! Pipes

Someone, apparently trying to perform a really anemic denial-of-service or just trying to waste bandwidth, has written a Yahoo! Pipes application to repeatedly grab my front page. It shows up as a request for robots.txt with the user-agent “Yahoo Pipes 2.0″, and is followed by a lot of requests for “/” from hosts matching htproxy[num].ops.re[num].yahoo.net (htproxy3.ops.re4.yahoo.net, htproxy2.ops.re4.yahoo.net, etc.).

It doesn’t appear to be beefy enough to affect availability, so it’s more of an annoyance than anything.  It’s either someone too frightened to take his or her problems up with me directly, opting instead to hide behind a slow proxy, or it’s simply someone with a tragic bug in their Yahoo Pipes app.  I was willing to give it the benefit of doubt for a couple of days, but the latter possibility seems to be less and less likely as Pipes’ steady march continues.

What’s more, Yahoo Pipes’ abuse email (pipes-abuse@yahoo-inc.com) listed here bounces. I’ve sent my inquiry along to security@ and pipes-bd@, so hopefully someone there can fill me in more on what’s going on.

In the meantime, I’ve taken steps to block Yahoo Pipes from this site.  We’ll see how well that works.  If you are running a legitimate Pipes app that uses this site (doesn’t seem to be a lot of other Pipes activity in my logs), then this may break it, and you’ll just have to hang tight for a while.

Edit: Looks like a couple of people at yahoo have taken a look at the logs I emailed them.  I guess we’ll see if they write me back

Edit: Got a couple of responses.  The requests are coming from the Yahoo Query Language (a sister project of Pipes), and they’re looking into the problem.

Oh Bother Edit Again: Yahoo Query Language (YQL) is all kinds of messed up.  It spoofs a Firefox user agent for most of its requests, making it look like some skiddie tool when it freaks out and runs up over 9000 requests in 3 days.  I’d recommend robots.txt’ing it off, but there is/was/for-how-long-was-it-anyways a bug in it where it incorrectly parses it and goes to town on your site anyway.  That’ll be sorted out by the time you get to this, hopefully.