Site Outage due to Google AppEngine change

Sorry for the recent site outage. Google AppEngine made an incompatible change to their system last week, and I was out of town and unable to correct the problem until today. Hopefully everything is working OK now.

URL Shortener Ranking - Sept. 2009

Many new sites have been added for the first time in September; our URL-Shortener index now includes a list of over 300 services. In this month's list you'll also find data on the amount of increase of activity that each site is seeing in September as compared to my last index published in July 2009.

A new feature of the index this month is the ability to click on the "Tweets per Day" number next to any URL Shortener, and view what links people are tweeting right now.

As an aside, Google just shipped a web commenting tool called SideWiki. They are currently getting 3,400 Tweets per day from SideWiki users (about 1/3 of the digg.com links). It will be interesting to see if SideWiki fades away, or garners more and more adoption.

A Distributed Facebook?

I think we're seeing Google's take on competing with Facebook. If you combine SideWiki with user Profiles as they have done, you're really starting to get into the same territory as sharing links through your social graph - except Google's approach is decentralized; you don't have to visit Facebook to see social content - you get it "in-situ" while your browsing.

in reference to:

"feed"
- Google Sidewiki Allows Anyone To Comment About Any Site (view on Google Sidewiki)

Casual Gestures for Content Sharing Sites

If you're a user of Google Reader, you may have noticed all the ways you can interact with the content you're reading. I think most users are pretty confused about these options - and how they should be using them.

• Add Star - I believe most people use this as a form of favoriting - they want some mechanism to remember this particular post, and find it easily later.
• Share - Google using this to indicate that you want to share this item with friend that follow you on Google Reader, or in your publicly shared collection of stories.
• Like - This is a form of light-weight commenting on a story. Whenever someone has visibility to this item, your name will appear below the item as someone who has "liked" it.
• Share with note - This is actually quite different from "Share". When you "Share" an item, you redistribute the original item to your followers and public shared page. If you "Share with note", you are creating a new item which links to the original, and has your personal comment attached at the top.
• Comment - Once an item is shared (by you or someone else), you and your followers can add comments to it. This can be a complete comment thread at the bottom of the story.

This is a lot of options, and seems pretty confusing for users; especially when you consider all the possible combinations of options that a user has. FriendFeed and Facebook have fewer options: basically just "Like" and "Comment"; the Share option is basically implied by these actions.

On Faves.com, we have the option to Share ("Fave It"), and Favorite (a side effect of an Up Vote), as well as adding a comment (if the author is a friend of yours). Faves actually has an even finer grained permissions model once you do share something; you can save a Fave for "me", your "friends", or the "world" (public).

I would rather see a more streamlined version of this interface - one that puts the various options on a continuum and reduce the number of options that the user is presented with. I would order them this way:

1. Favorite (or Star or Subscribe) - This should be a private gesture to help the user save the item for future reference. I view this action as an explicit "subscription" to this item. You are indicating that you want to remember this, and would also like to be able to see if there is future activity related to this item. It would also be useful to allow users to add their own tags to favorited items, to further help in searching or browsing for saved items.
2. Like - It's very useful to have a casual gesture that is like a one-click comment. By Liking and item, you should, as a side effect, become subscribed to it just as if you had starred it. This gesture is viewable by other users who have visibility to the item.
3. Comment - Just like "Like", with additional comment text added to your annotation. There is arguably a need to enable private comments, or those that are restricted to an inner circle of friends or co-workers.
4. Share - This option is really only needed for "off-site" publishing of the item. For example, if you want to send it via email, or post it to twitter (if you don't automatically tweet all your Liked items.
5. Report - In any social community, there will be some users who abuse it for self promotion or SPAM. Users should have a simple gesture to mark innappropriate items that should not be included in the current stream.

Google Automated Spam Filter Blocked My Spreadsheet

Google finally replied to me about "blacklisting" my spreadsheet. Apparently there are one or more link shorteners that are marked as spam sites. By putting links in my spreadsheet to them, an automated filter singled out my spreadsheet to be blocked. Here's the email I finally received:

Based on the spreadsheet URL you provided in your email, it looks like
some of the links in your spreadsheet were detected as spam by our systems
and caused your spreadsheet to be blacklisted. When putting links in a
spreadsheet it would be best to make sure that they do not redirect to
spam sites.

We have restored your spreadsheet and will use this case to help inform
our systems moving forward.

Please accept our apologies for any inconvenience.

Sincerely,

Christine
The Google Team

I have a few observations to make about this incident:

1. Google can be responsive to problem reports - and a real person will investigate reported problems.
2. I am happy that Google is working to reduce the spread of spam and mal-ware on the Internet.
3. I would have appreciated some form of prior notification before my spreadsheet was blocked. Notice that Google has never explicitly identified which link they object to. Are they worried that this would allow spammers to game the system? Are they afraid of libel suits or allegations of restraint of trade ?
4. I should take a bit more care in adding sites to this list. I have reviewed most of the domains to make sure they are bona fide URL shorteners; it can be difficult or impossible for me to determine if they are used as vectors for spam or malware.

I appreciate that Google apologized for unpublishing my data - and I accept their apology. But note that Google's Terms of Service allows them broad latitude in removing users content when they use Google products to publish it:

8.3 Google reserves the right (but shall have no obligation) to pre-screen, review, flag, filter, modify, refuse or remove any or all Content from any Service

This teaches that if you want to express yourself freely on the Internet - you should refrain from using Google publishing sites.

Tr.im Planning a Shut-Down, then Donates All to "The Community"

Eric Woodward has reversed his decision to shut down link shortening site, tr.im by the end of the year, and instead donate all the code and data assets to "the community" to continue to operate in the public domain.

It's a magnanimous offer, and one which, I'm sure, will gather some supporters. But these announcements are coming out pretty fast, and it does seem that some of their decisions are being made too rapidly to carefully plan the transition.

But Eric has silenced all doubters by personally guaranteeing the financial viability of the service from his own personal funds. While he does not specificially address the structure of the new tr.im, it sounds like he will be setting up an independent not-for-profit foundation to own and operate the assets of tr.im (though the source code will be open-sourced).

It's an interesting wrinkle in the story. Will the "open" mojo be enough to materially impact the position of tr.im amongst url shorteners? This is reminiscent of social bookmarking site, Ma.gnolia (which subsequently lost all of their user data), and micro-blogging site, Jaiku - (now an open-source Google Service being operated as a 20% project).

Open-sourcing is not a panacea, but it may inject enough life into a foundering project to remain a valuable piece of the internet infrastructure and community.

Google Censoring Me?

I was surprised to see that the spreadsheet I shared in my most recent post has been blocked from the public. When I visit the page I see this warning:

This spreadsheet is currently blacklisted, and not visible to the public. You are seeing this spreadsheet because you are on an internal IP.

What does THAT mean. Unfortunately, Google won't tell me why my sheet was blacklisted. When I go to their problem report page, there is one link about the DMCA there. But my spreadsheet only includes domain names of other sites, and data I have generated myself. Did someone file a complaint with Google about it?

This points out a very serious problem with cloud computing and Google's policies in particular. If Google is going to try to censor my speech on the internet, should they not at least provide me with visibility to the reasons? I received no prior warning (NO notification at ALL, in fact) that they were going to blacklist this page, nor do I have an opportunity to confront my accuser - whomever that may be.

I am a huge Google fan in general - so this behavior comes as quite a shock to me.

Bit.ly Dominates TinyURL and 213 other Link Shorteners

Bit.ly continues to dominate the link shortener usage on Twitter through July 2009. Bit.ly has doubled the number of links being shared per day on Twitter from 800,000 in June to 1.7M by the end of July. In the same time, TinyURL lost half of their traffic, dropping from 500,000 to under 250,000 links shared per day.

No other link shortener has more than 1/4 of the traffic of the two leaders, and in fact, the next two leaders, is.gd and tr.im lost about 30% of their traffic in the last month.

New Link Sharing Site - Quip-Art.com

I've created a companion link-sharing service to Go2.me - this one more with fun in mind more than simply utility. Quip-Art lets you share any web page that has an image of your choice overlayed on top of it. You can choose from artist-generated images that are already on the site, or upload one of your own. Here's a sample (you can continue reading this post underneath the following Quip-Art link:

http://www.quip-art.com/37

I'm hoping some talented artists and designers will have some fun with Quip-Art and be motivated to create some fun overlay "stamps". As an incentive, every link that is created with your image will have a credit link back to your own site. You can even get a customized bookmarklet that will let you quickly stamp your image over any web page.

URL Shortener Battle Heats Up

There have been some really interesting battles going on in the URL-Shortener space since I've started compiling data on all the competitors.

The two giants are TinyURL.com and Bit.ly. Interestingly, Twitter changed their default shortener from the classic tinyurl, to the up-and-coming bit.ly (following their announcement of venture-back financing). As a result, bit.ly has surpassed tinyurl in daily usage for June.

The rest of the pack is orders of magnitude behind these two leaders in usage. Still, there are some interesting battles in the back of the field. Digg.com is ramping up following their innovative (but much maligned) toolbar. Ow.ly also seems to be getting some good traction on the strength of their Twitter application suite, HootSuite.

Amazon has an Integrated URL Shortener - AMZN.COM

Kyle Mulka just tweeted an AMZN.COM url to a product page. I was amazed that this didn't get converted to a longer URL, but that Amazon allows for this alternate page representation. Here's an example:

http://amzn.com/076790818X

I started playing with a number of other shortcuts besides product codes and found some other URL's that do redirect to other Amazon.com URL's:

• amzn.com - You can go to amazon's home page with this shorter version.
• amzn.com/books - Each of the major sections of the Amazon store seem to have their own shortcut (other examples: amzn.com/bestsellers, amzn.com/kindle, amzn.com/electronics, etc.
• amzn.com/bill - Information about your Amazon bill.
• amzn.com/help - Amazon's Help page.

Interestingly, I note that almost no one seems to be twittering these short URL's directly (less than one per day on Twitter right now - and only 15 people have sent updates including them in the last two weeks).

I also note that Google does not have any pages in it's search index using the amzn.com domain.

As Twitter becomes more popular, I think more sites will be incorporating URL shorteners into their applications, enabling their users to share their pages without resorting to 3rd party services. Amazon's tool, like Digg's "Digg Bar", is also a handy address-bar shortcut; I can envision training myself to type amzn.com/kindle instead of visiting amazon.com and then navigating to the kindle section.

Update: Analysis of AMZN.COM shortcuts

My curiosity piqued, I decided to see what mysteries lay behind the AMZN.COM domain. I wrote a program to search through an English dictionary looking for valid shortcuts. I found that they break down into a few distinct categories.

Feature Pages

The biggest category (496 shortcuts) I found are "feature" pages. These seem to be highly customized pages about a collection of product types. I'm assuming these are hand crafted by Amazon's editors.

accessories, adobe, ads, advertise, advertising, airborne, aldo, allied, angel, apparel, apple, archery, artisan, artists, arts, ati, atlanta, atlantis, att, austin, auto, automotive, autos, av, avon, baby, badminton, baking, ballet, bargain, bargains, baseball, basket, basketball, beauty, bedding, benefit, bernardo, beverages, bibles, blackberry, blankets, bleachers, blizzard, block, boating, bob, bodybuilding, bolt, bombay, books, borders, bosch, boss, bostitch, bostonian, brass, braun, broadband, broadway, brother, business, calendars, camera, cameras, camping, campus, canning, cannon, canon, car, careers, cars, catcher, cellular, celtic, champs, charities, charity, chat, cheap, chicago, chocolate, choice, christian, christianity, christy, citizen, classical, clearance, clifford, climbing, clothes, clothing, coffee, college, columbia, comics, command, computer, computers, concert, concord, condiments, confessions, construction, contractor, cooking, corporate, costume, costumes, country, crafts, crane, creative, crew, cricket, criterion, dallas, dance, decca, decks, decor, delta, denver, detroit, diamonds, discount, discounts, diving, dolls, dove, drapers, drugstore, drums, dummies, easter, eastman, educational, ei, electronics, emerging, engagement, engineering, entertaining, entertainment, essentials, et, exercise, eyewitness, fans, fantasy, faulkner, feed, feeding, fencing, festivals, financial, financing, fishing, fitness, flash, flashlight, flip, flowers, focus, food, football, footprint, footwear, fossil, fox, fragrance, fragrances, franklin, fresco, freud, fuji, furniture, galaxy, games, gaming, garden, gardening, gateway, gear, gilbert, ginkgo, giving, global, golf, gourmet, gps, graduation, green, griffin, grizzly, groceries, grocery, guess, guitars, gymnastics, hallmark, halloween, halo, hannah, hannibal, hardware, harry, health, hearts, heaters, hefty, hi, hiking, hips, history, hitchcock, hobbies, hobby, hockey, holmes, home, hoover, horizon, horror, household, hue, hulk, hunting, ice, indies, industrial, ink, insomnia, instruments, ironing, irwin, isi, jazz, jello, jet, jewelry, jobs, journey, kaiser, kids, kitchen, kraft, landscaping, latin, leadership, leapfrog, learning, leatherman, lee, levels, levis, librarian, libraries, licensing, lighting, literature, lodge, lucky, lush, macintosh, magazines, majestic, marvel, maverick, may, meats, medical, memory, metal, method, mi, microwaves, milwaukee, mini, minneapolis, mixers, monster, monsters, motorcycle, movies, mozart, murray, music, nabisco, nara, nautilus, naxos, nec, nfl, nick, nikko, nonfiction, nordstrom, notebooks, nuance, nursery, nursing, observer, office, olympics, olympus, oneida, oracle, organic, orvis, oshkosh, outdoors, outlet, paint, palm, paramount, parsons, parts, patio, patrician, patton, pc, pearls, penny, performance, pets, philadelphia, phillips, philosophy, phoenix, photo, pillows, planters, plumbing, po, polar, polaroid, polo, pools, practices, prepare, preschool, presto, princess, processing, professional, progressive, projects, puma, puzzles, pyrex, read, reese, reference, relic, religion, remington, replacements, resolutions, revere, ricci, rival, rockport, rocks, rodeo, romance, roots, rousseau, royal, rpg, ruby, running, russell, safety, sage, sale, salter, sandpaper, sands, sante, sauces, save, scholastic, science, scientific, seafood, seattle, security, sees, services, sesame, sewing, sex, shakespeare, sharp, shoes, shorts, shun, silhouettes, silver, simmons, simplicity, singer, skating, sledding, snacks, soccer, software, solutions, spanish, spices, sports, spreads, spring, squash, stanley, stones, storage, strings, summer, sunbeam, sunshine, survivor, swimming, switch, tang, tape, target, taxes, tech, teens, television, tennis, textbook, textbooks, thanks, theater, thomas, timex, toner, tools, topics, torino, toys, training, transformers, travel, trinity, triumph, truck, trucks, trump, tuna, tuttle, tv, twilight, umbra, universal, university, unlocked, upgrade, urban, used, verve, video, vikings, vinyl, vivo, viz, volleyball, vs, wallace, warner, watches, waterskiing, weber, werner, west, westinghouse, wheeled, wheels, wiley, winds, wire, wireless, wood, woodworking, wrangler, wrestling, write, xerox, yoga, zenith, microsoft

Product Pages

For some reason, Amazon has chosen to create shortcuts to individual products in their catalog. I found 100 shortcuts in this category.

america, apartment, arthur, badlands, balls, barrels, barry, beans, beats, body, bottom, brazil, breakaway, breakthrough, buckets, century, chaplin, chef, commitment, confidence, conversations, cookie, corner, cracker, dash, dating, deadline, deceit, details, dow, drawing, dungeon, eades, earl, espionage, evolve, excel, fish, fit, garcia, gunslinger, hillman, hive, icon, inclusion, indispensable, inspire, intrusion, israel, joss, kimball, kindle, leo, loopholes, millionaire, mira, mummy, noel, nostradamus, nourish, oasis, organizing, oyster, palmer, paralysis, passion, passionate, peacock, persuasion, poe, portfolio, presence, profitable, prosper, revved, santa, sarah, screenplay, seal, self, selling, service, ship, sloan, spies, startup, stocks, stranger, swimsuit, talent, toxic, traveler, waste, we, whistle, wind, winners, wired, wizard, wolves

Sports Team Pages

I found 178 custom pages dedicated to products related to sports or sports teams.

aggies, akron, alabama, angels, arizona, arkansas, army, arsenal, athletics, auburn, avalanche, baylor, bears, bills, blues, bradley, braves, brewers, broncos, brown, bucks, buffalo, buffaloes, bulls, butler, cal, california, campbell, capitals, cardinals, centenary, charleston, chiefs, cincinnati, citadel, colorado, colts, connecticut, cornell, cowboys, coyotes, creighton, cubs, dartmouth, davidson, delaware, devils, dodgers, dolphins, drake, drexel, ducks, duke, eagles, evansville, flames, florida, flyers, georgetown, georgia, giants, hampton, hartford, harvard, hawaii, hawks, heat, hoops, houston, howard, idaho, illinois, indiana, indians, iowa, irish, islanders, jacksonville, jets, kansas, kentucky, kings, lafayette, lamar, liberty, lightning, lions, longhorns, louisville, magic, maine, manhattan, marquette, marshall, maryland, massachusetts, mavericks, memphis, mercer, mets, miami, michigan, minnesota, mississippi, missouri, monmouth, montana, nationals, navy, nebraska, nets, nevada, niagara, northeastern, northwestern, ohio, oilers, oklahoma, orange, oregon, orioles, pacers, pacific, packers, panthers, patriots, pennsylvania, phillies, pirates, pistons, pittsburgh, portland, princeton, providence, raiders, rangers, rays, reds, rice, richmond, rider, rockets, rockies, saints, senators, sharks, southern, spurs, stanford, stars, steelers, stetson, suns, syracuse, temple, tennessee, texans, texas, tigers, titans, tulane, tulsa, twins, ucla, utah, vermont, virginia, volunteers, wagner, warriors, washington, wild, winthrop, wisconsin, wyoming, xavier, yale, yankees

Search Pages

14 pages are shortcuts for product searches.

bowman, browning, buckley, choose, converse, curling, lauder, olympic, pickup, plot, poker, razor, rink, slugger, steiner, thermostat, thermostats, wilson

Website

20 additional shortcuts send users to pages not on the primary www.amazon.com domain.

affiliates, associates, auction, cell, de, developer, fresh, investor, meters, news, phrase, phrased, phrasemaking, phraseology, phrases, pr, press, queue, soap, wine

Special

114 shortcuts defied categorization. They are generally special features of the Amazon web site.

access, advantage, agreement, anywhere, atonement, audible, bach, bank, barney, bill, birthday, blacks, cancel, capote, card, cards, cart, chronicle, claim, claims, classics, communications, connect, contact, coop, current, dad, daily, deals, delivers, direct, directory, discover, education, environment, episodes, feedback, garage, gifts, goldberg, greenwood, habitat, hd, heart, help, holiday, honor, horton, howell, images, inaugural, inauguration, international, jaws, journeys, kaplan, labels, library, lippincott, locker, marketing, marketplace, martian, mcgraw, mobile, mom, montreux, morningstar, nolo, nonprofit, occasions, orders, packaging, passes, password, phone, pin, pioneer, prime, privacy, productivity, publishers, push, quiver, refunds, reporting, return, returns, rewards, search, sell, shades, sos, sox, spa, studios, summons, tags, ted, today, tracking, trading, usability, valentine, vault, visa, visual, vote, wastewater, waters, wedding, wishes, zen, zing

Other

Perhaps most interestingly, I found 68 shortcuts that no longer seem to function specially (they default to the home page), or they reference a secured page not intended for Amazon customers

alerts, alien, ash, awards, beta, blocks, board, brooks, buick, community, credit, dell, documents, egghead, errors, es, evening, exec, favorites, fiesta, friends, game, ginger, go, ibm, input, java, join, lifetime, local, manage, mutter, new, orchestras, phantom, preferences, profile, recommendations, reminders, review, reviews, schwab, sellers, star, status, store, tag, text, traffic, wolf, yellow,
anniversary, communities, continue, damages, dexter, discussion, discussions, dylan, forum, forums, japan, japanese, links, madonna, mae, patents, supernatural

Time-Score - A New Algorithm for Calculating Time-Weighted Scores

One of the reasons I created Go2.me was as a test platform for a new social scoring algorithm I've developed called "Time-Score". You'll note that Go2.me calculates the most popular links on the site and updates it real-time.

Each time a user interacts with content on Go2.me, the site accumulates points as follows:

• Sharing a Link - 2 points.
• Favoriting an Link - 2 points.
• Commenting on a Link - 7 points.
• Viewing a Link - 1 point.

By time-weighting the scores, actions taken today are twice as valuable as actions taken yesterday (we say the "half-life" of a link-score is 1 day). So, if 100 people visit a link today, it will rise to the top of the popular list (with over 100 points). By the same time tomorrow, the score will have decayed to just over 50 points if no further people visit the link. Each day, it will be cut in half, so that after a week, it will effectively have less than 1 point, and more recently active links will rise to the top of the popularity list.

This ensures that the most popular links have to continually get the attention of the Go2.me audience to stay at the top. This scoring algorithm also applies to every tag applied to Go2.me links. For example, the most popular videos are shown using the same scoring algorithm.

I'm applying for a patent for this algorithm now, as I think it has several advantages over other scoring algorithms that I've seen or implemented in the past:

• Real-time updates - Scores are instantaneously updated and can be compared against all other scoring values in the database.
• No Batch Processing - The scoring decay function (half-life) does not require any batch processing of older scores to normalize them for subsequent days.
• Combines scoring events at distinct times - There is a well defined relation to combine scores at different times, based on the characteristic half-life of the score.

Since most sites don't publish how they do popularity ranking, I can't be sure if my algorithm is unique. Blog posts about Reddit's algorithm indicate some similarity - but they lack a mechanism to combine scoring events at different times (they calculate all events as if they occurred at the time the post was first submitted).

The code for the Time-Score algorithm is quite simple. I have granted a royalty-free license for all non-commercial users. If you're interested in licensing this technology for your commercial application, please contact me at info@go2.me.

Exhaustive List of URL Shorteners

Go2.me started as a small hobby project, and has grown in scope as I've found many possible features to add.

There are a wide variety of Link Shorteners in the market today. The first products appeared in 2001, and people have used them to shorten 100's of millions of URL's to date. With the advent of micro-blogging sites, like Twitter, the demand for URL shorteners has grown even greater.

TinyURL is the first URL Shortener to gain wide popularity. Started in 2002 by Kevin "Gilby" Gilbertson, TinyURL remains the single most popular URL Shortening service, with a claimed 145 million links shortened to date, and 1.5 billion clicks per month (nearly 600 per second) on their shortened links (as of January, 2009).

There are a number of reasons why people are using URL Shorteners:

• Fixing Links - Original links may be too long to fit on a single line in an email message, causing some email clients to wrap them, making the link unusable.
• Beautifying Links -Original links are long and ugly - a shorter one looks better in email or IM messages.
• Space requirements - Twitter messages only allow 140 characters - long URL's won't fit, or don't allow enough explanatory text to accompany a shared link.
• Spam and Phishing - Spammers sometimes use URL Shorteners to disguise the true destination of a link. They can claim it will take a reader to one destination, when if fact it goes to another. This can make it more difficult for spam filters to recognize spam messages, and can also fool users into falling for a phishing scheme.
• Link Tracking - Individuals and marketers who are sharing links, often like to know how many people are clicking their link, or know what site people are clicking from.

TinyURL defined the baseline functionality by which we can judge follow on competitors. As an early entrant, the feature set is basic, with a couple advanced features.

• Redirection - Redirects from a short URL back to the original.
• Web Form - Type a link into a web form to shorten it.
• Custom Alias - Select your own text (if available) for a tiny link (e.g. http://tinyurl.com/blog-go2me.
• Bookmarklet - Install a bookmarklet to create a short URL from the current page.
• Relative Redirection - TinyURL links can be appended by additional URL path text to redirect to a "sub page" of the original shortned URL.
• Link Preview - Turn on preview mode (remembered by cookie) to display the target link BEFORE redirecting to it (used to confirm link is not spam, adult, or phishing).
• API - TinyURL supports a simple web request that will return the tiny URL from a long link - very simple but it works for many applications, for example Twhirl.

In a future post, I'll review the most popular (stand-alone) URL Shorteners (and throw in a comparison to the Go2.me service as well); sites that do URL shortening as an added feature (e.g., FriendFeed's "ff.im") will not be considered.

"Complete" List of URL Shorteners

The following list is EVERY URL shortener I have found to date (and I'll update it as I find more - if you know of additional shorteners, please send mail to shortener@go2.me). As a measure of their relative popularity, they are listed in order of their published Alexa Ranking (most as or Jan 2009, but some updated in Apr 2009). The sites that are highlighted will be the subject of my next post - a competitive review of the features of the most popular URL Shorteners.

Update 1: I've estimated the number of Tweets per Day that each service is being used to send. The front runners on the list are sorted by that statistic, rather than Alexa rank as it seems a purer measure of popularity.

Update 2: Twitter Rate numbers from May 2009 - also showing relative change in rank of each shortener from last statistics in April.

Update 3: Updated June 11, 2009. Note Stumble Upon's new shortener su.pr is already getting lots of mentions on twitter, even though it's still in limited beta.

The LAST Update: I'm going to post updated stats in a fresh blog posts from now on.

Thanks to Chris Koss for doing the competitive research for this post.

Feature Update

As many Go2.me users will have noticed, I launched a pretty major new release of Go2.me on Feb 28, 2009. I had been working on a large set of changes for about 6 weeks, so I felt it was finally time to release what I had.

I must admit that some of the changes have been a little rough around the edges, as I rushed to complete features by my self-imposed deadline. But I feel that overall, the new version is a big improvement, though much more "heavy weight" than the original vision for Go2.me.

Here's a highlight of the what you'll find new in this release:

1. New Design - Yes, it's RED. But a bit more designed than the previous site. I tried to make it easier to understand that you can install the bookmarklet to get started.
2. Link Page Sidebar - The Go2.me comment area has moved to a side-bar (rather than the top of the page) where most people have extra real-estate. Anyone using a wide monitor should see more of the target web page without having to scroll. You can also click the expand/collapse triangle in the header of each sidebar panel to hide sections to make even more room for the Chat window.
3. Real-time Chat - Go2.me is now a fully AJAX application which allows you to chat in real-time with readers visiting your links. You can also see how many people are currently looking at your page (hidden feature - if you hover your mouse over the thumbnail of a user image in the chat window, it will tell you where in the world that person is located).
4. Sponsored Link Pages - While not an end-user "feature", per se, Go2.me is now accepting sponsorship to help pay for the costs of providing our service. If you get annoyed by the ads, you can shrink the Sponsored By panel so you don't have to look at it.

I had to punt a few features that were under development but not yet ready. These should be coming soon:

• Formatting Fixes - There are some page formatting problems I know about - especially in IE. Most of my users are on Firefox or Safari, but I will be addressing the cross-browser problems shortly
• Private Chat - You'll be able to create a link that only you and your friends know about to have a private chat about a web page.
• RSS feeds - Subscribe to personal or tag-based pages of the mosts popular links.

What's a "Real User"? Twitter Bots are Crawling Go2.me links.

One of the features of Go2.me is that it can tell you how many users are visiting your link. One problem I've been noticing lately, is that I am counting users that aren't real people.

I experimented with twittering a link, and then looking at the viewers of that link shortly after. I'm seeing a number of Bots crawling twittered links right away. From my server logs I see the following request that got counted as real users:

66.249.65.99 - - [10/Mar/2009:11:57:13 -0700] "GET /xL HTTP/1.1" 200 8416 - "Mediapartners-Google,gzip(gfe),gzip(gfe)"

67.202.28.213 - - [10/Mar/2009:12:00:25 -0700] "HEAD /xL HTTP/1.1" 200 9468 - "AideRSS 2.0 (postrank.com),gzip(gfe),gzip(gfe)"

xx.xx.xx.xx - - [10/Mar/2009:12:00:56 -0700] "GET /xL HTTP/1.1" 200 9468 - "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11 FirePHP/0.0.5.26,gzip(gfe),gzip(gfe)"

66.29.162.122 - - [10/Mar/2009:12:01:22 -0700] "GET /xL HTTP/1.1" 200 9469 - "WWW::Shorten::TinyURL/1.90,gzip(gfe),gzip(gfe)"

89.151.116.52 - - [10/Mar/2009:12:01:35 -0700] "GET /xL HTTP/1.1" 200 9469 - "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot,gzip(gfe),gzip(gfe)"

216.121.78.20 - - [10/Mar/2009:12:08:03 -0700] "HEAD /xL HTTP/1.1" 200 9468 - "Chat Catcher - (http://chatcatcher.com/bot.htm),gzip(gfe),gzip(gfe)"

91.121.89.155 - - [10/Mar/2009:12:08:33 -0700] "HEAD /xL HTTP/1.1" 200 9470 - "PycURL/7.19.3,gzip(gfe),gzip(gfe)"

Note that the link was posted at 11:57 - so these requests came in from 6 seconds to 10 minutes after Tweeting the original link. Only one of these listed was a "real human". So, Go2.me is counting 6 unique users that are actually Bots.

To fix this, I'm going to be changing my counting algorithm to require that you've successfully received a Go2.me cookie (as a normal web browser would do), AND that your browser has successfully made an AJAX request to the server to receive a Chat update. This should eliminate all Bots and Crawlers as falsely being reported as "Vistors" to Go2.me links.

Go2.me Goes International

This morning, Go2.me was "discovered" by Web 2.0 directory Go2Web20.net. As a result, there was a burst of activity, especially in Europe, of people visiting Go2.me and writing blog posts about it (in French and Italian).

It's been a pleasant surprise to see a bunch of new faces trying the service out, and twittering about Go2.me.

Three Must-Have Bookmarking Tools

I'm amazed at people who get by just using their browser to surf the web without any 3rd party tools for saving and sharing bookmarks. For anybody that visits a lot of web sites, and wants to share them with friends and co-workers, I find three web sites to be invaluable (full disclosure - I worked on 2 of the 3 products - but I genuinely love what they do for me).

1. 

   Faves.com - Faves is a Social Bookmarking web site. Which means, it allows me to save Bookmarks I find interesting in my own on-line database. I primarily use Faves for two reasons: a) I can save web sites for myself for easy recall in the future, and b) sharing interesting things I find with my friends and family (and by sharing - I mean this is a two-way street; I find cool stuff for them, and they expose me to cool stuff they find). Faves has a really good built-in search engine so I can quickly find any Fave I've saved in the past (over 6,000 of them so far!) - or even search across the bookmarks all my friends have saved.

   Faves is one of the first places I go to look for a recommendations for a resource on the web, or even product recommendations from my friends. Faves also sports a Facebook application, so my friends on Facebook get notified when I share something, as well as friends who are also using Faves directly.

2. 

   Go2.me - Go2.me is a link shortening service with a twist; it not only can convert any long URL into a very short one (just 16 characters long), it creates a landing page where you and your friends can exchange comments. If you share links with friends over Twitter or email, you don't really know how many people are actually visiting your link, or have a good way to have a conversation about that link. Go2.me adds a wrapper around your shared page that shows you the number of visitors, and allows them to leave comments for you.

3. 

   Foxmarks - Unlike the socially oriented tools, above, Foxmarks is a great utility for roaming your very frequently used bookmarks. Since I frequently use 4 different computers, Foxmarks can synchronize my Bookmarks toolbar across all of my machines. I have about 30 sites that I go back to over and over again in the course of my day - this way I can organize my Bookmark toolbar in one way, and Foxmarks will propogate my changes to each of my machines. You can also access your bookmarks directly from the web, if you are traveling and using a computer at a hotel, library, or Internet cafe.

   I'm a pretty new convert to Foxmarks; the setup was really simple, and it even did a decent job of merging the disparate collection of bookmarks I had on three separate machines.

All of these tools are free - hopefully they'll make your Internet usage a richer and more productive experience. Feel free to add recommendations for other Bookmark-related tools in the comments.