Promenade

CLUAS in 'Google-likes-us-again' shocker

Fri, 16 May 2008 15:35:31 GMT

Two weeks ago I posted a blog entry about how Google had, all of a sudden, dramatically reduced the number of CLUAS.com pages it crawls in a day (it dropped from an average of thousand pages a day to about 25 a day, see the graph below).

I put this down to be something to do with the fact that CLUAS stopped running Google ads for 3 weeks in April. I predicted that once the Google ads were back up and running (as they were two and a bit weeks ago) all would return to normal. A quick check earlier today in CLUAS.com's Google "Webmaster Tools" account and I saw that (for once!) a prediction of mine was on the money. Google is once again crawling a daily average of 1000 pages on CLUAS. Check out the graph below for the evidence that Google is still all chummy-wummy with us.

Reviving the CLUAS newsletter

Fri, 02 May 2008 09:24:39 GMT

Up to about 4 years ago I used to send out a CLUAS email newsletter every two or so weeks. It was very successful in regularly drawing to the site many visitors who would otherwise stop by very irregularly, if at all. The overall effect was that the newsletter helped drive traffic levels upwards. By the time I stopped sending the newsletter out it had about 3000 subscribers. However I had to stop sending it as emailing so many people simply got more and more difficult (as ISPs started, with the rise of email spam in general, to severely limit the number of emails one could send).

But, after a bit of research in the last few weeks I today relaunched the CLUAS newsletter with the help of YMLP.com, an excellent and reasonably priced third party mailing list provider. In choosing a company to go with I was conscious of the possibility of ending up with a company that would turn out to be some dodgy non-EU, non-USA backstreet outfit that were in this game to harvest email addresses to sell on to spammers. I am confident however that's not the case with the company we are using (who are a legitimate business operating out of Belgium).

It turns out that the number of email subscribers on CLUAS has in the last 4 years greatly increased - we now have just over 6000 emails (basically the original 3000 + over 2500 new registered users of the site + a few hundred who have chosen to register just for the newsletter in the last four years (either via our newsletter subscription form or via our end of year readers' polls voting form when we gave voters the option to sign up for the newsletter). My guess is a very big proportion of these 6000+ emails will be dead/dormant accounts (even up to 50%, some of those subscribers do after all go back to 1999!). Thankfully the mailing list service we are using automatically filters out dormant emails (based on delivery error messages received from "dead accounts") so the list will be cleaned up quick enough over time. Even so I think the revived CLUAS newsletter will regularly pull in a healthy number of visitors who otherwise might miss out completely on the CLUAS site.

As an act of curiosity I looked into the database of 6000 subscribers to see how they break down, in terms of where the subscribers come from and what email services they prefer to use. Below is a sample of some of what emerged from my rooting in the list of subscribers (without of course compromising the identity of any individual subscriber)...

Most popular email accounts / ISPs among CLUAS newsletter subscribers

Hotmail (1761 subscribers)
Yahoo (810 subscribers)
Gmail (403 subscribers)
Eircom.net (356 subscribers)
CLUAS.com (239 subscribers)
AOL (100 subscribers)
Ireland.com (50 subscribers)

Most popular university email addresses

TCD (48 subscribers)
DCU.ie (46 subscribers)
UL.ie (16 subscribers)
NUI Galway (12 subscribers)
UCD.ie (12 subscribers)
UCC.ie (6 subscribers)
QUB.ac.uk (5 subscribers)
DIT.ie (3 subscribers)
WIT.ie (2 subscribers)
LIT.ie (2 subscribers)
CIT.ie (1 subscribers)

Subscribers based on country of origin of email address

.ie email addresses (895 subscribers)
.uk (374 subscribers, although 255 are Yahoo.co.uk addresses)
.de (32 subscribers)
.fr (23 subscribers)
.it (15 subscribers)
.au (13 subscribers)
.nl (12 subscribers)
.ca (10 subscribers)
.es (7 subscribers)
.pl (6 subscribers)
.be (5 subscribers)
.nz (4 subscribers)
.pt (3 subscribers)

Top companies whose employees used their work email address when subscribing

Microsoft (32 subscribers)
RTE.ie (13 subscribers)
Gov.ie (11 subscribers)
Eircom.ie (7 subscribers)
Dell.com (7 subscribers)
IBM (6 subscribers)
Intel (6 subscribers)
Ericsson (3 subscribers)
HP.com (3 subscribers)
Pfizer (2 subscribers)

How Google sees CLUAS (part 1)

Thu, 01 May 2008 21:06:10 GMT

(First up - apologies for my absence from this blog in recent weeks, I won't burden you with a long list of protracted excuses, suffice to say I'll be about this place a bit more often. So, moving swiftly along...)

With thousands of its pages indexed by Google, CLUAS today receives a healthy chunk of its traffic from the world's leading search engine. The number of visitors they send our way can vary greatly from day to day, from week to week, but it is safe to say that we get a minimum of several hundred vistors a day coming from Google. Behind this fact lies plenty of interesting info and observations about how Google sees CLUAS, stuff I have been keeping my eye on for years but which now (cue collective groan) I am going to explore in a series of blog entries...

Casual users of Google wouldn't be aware (nor do they need to be) of the fact that Google shares out, for free, considerable amounts of information to webmasters about how Google sees their website(s). They do this via their Webmaster Tools service and all you need to do to get this info for your website is to prove to Google that you are indeed the owner of the domain name. They then dish out all sorts of info that any conscientious websmaster would be mad for, like:

Search queries that most often returned pages from your site, and which of them were clicked,
Which pages on your site have links pointing to them from other sites,
The number of pages on your site that Google indexes per day,
The average time it takes Google to download a page,
Pages that it has trouble accessing.

Exciting stuff, eh?

Anyway I've been checking in with the Google Webmaster Tools service for well over a year now to keep tabs on how Google is interacting with CLUAS. Last week when I logged in I noticed something unexpected. Google all of a sudden had dramatically reduced the number of CLUAS.com pages it crawls on an average day. It dropped from an average of thousand pages a day to about 25 a day (see the graph). My first reaction was "WTF?" before calming all the way down.

There are many reasons why Google would suddenly reduce dramatically the number of pages it crawls on any site: the site might not be updated often enough to merit 'deep crawling', the site might not be receiving enough new links from other sites, the site might have started using all sorts of frowned upon practices to deceive search engines. There could be any number of reasons. However I was reassured when I saw that CLUAS articles are still appearing in Google news within an hour of them being published. Somehow I don't think we are in the Google doghouse.

My own feeling is that this is something to do with the fact that, starting for a 3 week period on April 5, CLUAS stopped running Google ads on the site (so that we could run a banner ad campaign for Independent Records). I'm not saying that I think Google went "ahhhh, those CLUAS lads, they stopped running our ads, off with their heads, etc." Here's why. See, when you visit a page with a Google ad a few things happen in the blink of an eye. Simplifying it greatly, you visit the page, the page tells a Google ad server "there's a visitor on this page", Google grabs the page, checks its content and then serves up a ad relevant to content on the page. My guess is that once we stopped running Google ads for the 3 weeks, Google during this period - obviously - stopped "grabbing pages" to check content in order to decide what ads it should run on the page. And this is what has made our "pages crawled" stats plummet (background info: Google last year bundled together the task of checking a page for ad content with checking a page for possible inclusion in its search result pages).

Maybe I am wrong and Google just thinks CLUAS is not worth crawling as often as it did before for some other reason. Now that the Google ads are again running across the site I'll soon be able to see if my theory is right. I'll report back in a few weeks with an update on what happens.

Betcha just can't wait.

Top Irish Albums Ever: the poll of polls

Fri, 07 Mar 2008 15:15:12 GMT

Back in 2004, as part of its 5th birthday celebrations, CLUAS ran a readers' poll to identify the top 50 Irish albums of all time. Over 1300 votes were cast by CLUAS readers, giving the poll some sort of statistical legitimacy and indeed the majority of the albums that made the final top 50 were clearly deserving candidates.

Nonetheless the final results were not immune to raising a few eyebrows especially now, 4 years later, when you look through the list. For example, a Frames fixation among a certain part of the CLUAS readership back then resulted in a voting bias that helped push ~~one~~ ~~two~~ ~~three~~ four Frames albums to make the top 20. Indeedy. It's also curious to see that 10 of the top 50 40 albums came from the Paddy Casey-Frames-Bell-X1-Mundy-Damien Rice axis, but no surprise there considering the strong overlap of these acts' fanbases in Ireland in 2004.

Then a week ago the Irish Times "Ticket" supplement published its own list of the top 40 Irish albums of all time, as chosen by four of their journos. Again this was a solid list but one that also had its imperfections and its own bias. For example there is a rather strong affinity with albums released in one rather narrow 7 year window (1984 to 1991) by Dublin-based bands who ploughed the Baggott Inn-International Bar-McGonagles circuit (a total of six of their top 40 albums were released in this period by The Stars of Heaven, The Blades (2 albums each), Something Happens and A House).

Reading through the two lists and noting their various strengths/weaknesses it occurred to me that a merging of the polls might just eliminate many (if not all) of their respective biases. The principle I set out with was a simple one - create a new 'Best Irish Album' listing made up only of albums that appeared in both the CLUAS and The Ticket polls. It turns out there are twenty albums common to the two lists so I went about ranking them in order of their total score (which in turn was based on their respective placings in the CLUAS & Irish Times polls, see note on allocation of scores below).

The resulting top 20 is below, I'll avoid any dumb temptation to declare it as some sort of definitive list. But I do think it is a credible listing of 20 thoroughly excellent Irish releases that marries the 'wisdom of the crowd' with the considered views of 4 seasoned music hacks. For everyone there will be at least one glaring omission in the list. For me it is Sinead O'Connor. She didn't make the cut as the one album of hers that was chosen by the CLUAS readers (the quite thrilling 'Lion & the Cobra' ) differed to that picked by the Irish Times journos ('I do not want what I haven't got').

Nonetheless the top 20 as its stands is properly distributed over the decades with 3 albums from the noughties, 6 from the 90s, 5 from both the 80 and 70s and 1 from the 60s. Nor does it reflect any bias I can see in terms of scene or genre. All in all, a balanced and credible list. And if you disagree, feel free to let it rip via the comments below.

The Top 20 Irish Albums Ever (poll of polls)

My Bloody Valentine 'Loveless'
Van Morrison 'Astral Weeks'
U2 'Achtung Baby'
Whipping Boy 'Heartworm'
A House 'I am the Greatest'
U2 'The Joshua 'Tree' (actually joint 5th with A House)
Thin Lizzy 'Live And Dangerous'
The Undertones 'The Undertones'
Bell X1 'Music In Mouth'
The Pogues 'Rum, Sodomy and the Lash' (actually joint 9th with Bell X1)
Van Morrison 'Moondance'
Damien Rice 'O'
The Frames 'For The Birds' (actually joint 12th with Damien Rice)
Microdisney 'The Clock Comes Down the Stairs'
The Pogues 'If I Should Fall From Grace With God'
Ash '1977'
Planxty 'Planxty'
My Bloody Valentine 'Isn't Anything'
Therapy? 'Troublegum'
Rory Gallagher 'Live in Europe'

Note on scoring: Scores used in the ranking of the 20 albums that were common to the two original polls were calculated as follows: if a common album was a number one on either poll it got 50 points for its entry on that poll, if it was a number two it got 49 points, etc right down to it getting 1 point if it was a number 50. For each of the albums the scores it got for its position on each poll were then added up to get its total score.

The one flaw in this approach is that there were 40 albums in the original Irish Times poll, but 50 in the CLUAS poll. I had actually initially looked for albums that were common only to the top 40 of both polls – this threw up 18 albums. By extending the 'comparison check' to the full top 50 of the original CLUAS poll the number of common albums increased to the nice neat number of 20.

The Skinny & the Bloated - revisited

Tue, 04 Mar 2008 16:36:19 GMT

A month ago I surveyed over 30 leading Irish music blogs and websites in terms of how heavy they were when it came to the number of kilobytes visitors would have to download once they hit the site's home page. As you do, like.

The results showed a huge diversity across the sites: the slimmest (that'll be CLUAS) stepped in at a super-svelte 97 KB, while the heaviest (egocentric) was over 20 times heavier with its scale-busting 2.1 MB of data that each visitor had to download just to access its home page.

Alas, the truth is there are far more people out there using dialup to access the internet then we'd ever imagine (Mulley only yesterday wrote about there still being 200,000 dialup users in Ireland alone) and for such users your site will take an eternity to load if it is weighed down with 100s of kilobytes of data. It might not be terribly sexy but, until broadband is ubiquitous, keeping an eye on the 'payload' of your site or blog is really the smart thing to do.

So how are Ireland's leading music sites doing sites doing one month on after my original, er, exposé? Did any of them implement the recommendations I provided and trim themselves down? The answer, in short, is a definite 'yes' for a large number of them.

The table below provides an update of the sites' weights. Of the 32 sites originally surveyed, 21 have reduced their 'weight' in the intervening month (several by massive amounts). A month ago only 8 of 32 sites made into the 'green zone' (i.e. weighing in at 250 KB or less). Today 11 of the sites now merit for a privileged spot in the said zone.

At the other end of the scale, the number of sites in the 'red zone' of the table below (i.e. sites for which visitors would have to download a whopping 1 MB or more of data to access their homepage) has dropped to 4, compared to 7 a month ago (although Indie Hour - which appears to have eaten a few proverbial pies in the last few weeks - was just 1 single kilobyte away from a spot in the red zone).

Overall, the average weight of the 32 sites is today 20% lighter than it was 4 weeks ago (the average weight of a site was 631 KB at the beginning of February, today it is 515 KB). Some sites have made amazing progress in the last month and merit a special mention:

www.Donal.ie has lost a mind-boggling 1374 KB in the last month, allowing it to slim down from a hyper obese 1.9 MB to a cuddly 526 KB. Amazing work. No, really.
Egocentric has lost a similarly impressive 1255 KB in the last month, slimming down from a modem-melting 2.1 MB to a mere pudgy 845 KB. Keep drinking that virtual slimfast though as 845 KB is no walk in the park for those 200,000 Irish dialup users.
Cheebah has gone from 1.2MB to 278 KB, a huge drop of almost 1 full megabyte which has taken it out of the red zone and parachuted itself right to the edge of the hallowed 'green zone'.
Matt Vinyl and Asleep on the Compost Heap each having shaved off about 400 KB from their payloads.

Check out the full table below and see how your favourite music sites are doing (and I for one am pretty pleased to see that CLUAS.com's 3 entries are all in the top 5). The 4 colour coded categories correspond as follows:

Less than 250 KB: ("Optimal balance of content and page size")
251 KB to 500 KB: ("Total nightmare for dialup users")
501 KB to 999 KB: ("High risk of testing the patience of broadband users")
Greater than 1 MB: ("Clinical cases of inexcusable hyper cyber-obesity")

Ireland's Top Music Site's & Blogs
(ranked in terms of page size - updated 4 March 2008)

Current ranking	Previous ranking	Site / Blog	Components of webpage (in KB)					Total page size	Weight loss / gain?
Current ranking	Previous ranking	Site / Blog	HTML	Images	CSS	Flash		Total page size	Weight loss / gain?
1	(1)	CLUAS (*)	16	39	12	3	0	71 KB	-26 KB
2	(4)	Thrill Pier	22	1.5	74	4	1	102 KB	-75 KB
3	(2)	Test Industries	18	80	1	17	0	117 KB	no change
4	(7)	Key Notes	21	90	59	13	0	182 KB	-52 KB
5	(6)	French Letter	19	107	56	13	0	198 KB	-27 KB
6	(3)	Music Road	16	160	16	2	8	204 KB	+43 KB
7	(13)	House is a Feeling (*)	66	118	24	2	0	211 KB	-135 KB
8	(8)	Hot Press (*)	62	144	6	7	0	219 KB	-29 KB
9	(10)	Thumped	7	130	51	32	0	220 KB	-50 KB
10	(18)	The Red scrapbook	20	146	75	4	0	245 KB	-194 KB
11	(12)	Phantom FM (*)	50	144	34	13	9	250 KB	-53 KB
12	(5)	On the Record	19	175	38	26	0	259 KB	+70 KB
13	(28)	Cheebah	19	232	12	16	0	278 KB	-922 KB
14	(19)	Muse (*)	41	166	94	19	1	321 KB	-153 KB
15	(14)	Kilkenny Music (*)	46	178	24	26	52	328 KB	-26 KB
16	(16)	State	7	264	86	31	0	389 KB	-33 KB
17	(17)	Nialler9	17	280	96	22	6	420 KB	-6 KB
18	(9)	Sinead Gleeson	16	411	0	7	2	437 KB	+184 KB
19	(23)	I Prefer the Obscure Mix	18	583	33	2	6	444 KB	-194 KB
20	(20)	Indie Limerick	17	314	81	4	28	445 KB	-87 KB
21	(25)	Asleep on the Compost Heap	16	394	69	4	0	484 KB	-416 KB
22	(15)	MP3 Hugger	132	183	151	4	40	510 KB	+118 KB
23	(26)	Matt Vinyl	18	340	70	4	82	515 KB	-485 KB
24	(31)	Donal O’Caoimh (*)	32	405	81	8	0	526 KB	-1374 KB
25	(11)	Una Rocks	33	400	75	4	15	527 KB	+234 KB
26	(21)	The Torture Garden	17	551	13	4	3	588 KB	+4 KB
27	(32)	Egoeccentric	28	530	198	4	84	845 KB	-1255 KB
28	(24)	The Indie Hour	15	946	35	5	2	999 KB	+162 KB
29	(27)	Magoo (*)	43	1010	16	11	2	1.08 MB	+80 KB
30	(29)	Off the Rocker (*)	54	1387	65	11	2	1.5 MB	-100 KB
31	(22)	Cubik Music	30	702	16	12	832	1.6 MB	+400 KB
32	(30)	Stuart Bailie (*)	220	1756	1	0	0	1.98 MB	+180 KB

Note on the above:

Figures above are the sizes of each site's main page as surveyed between 29 February 2008 and 1 March 2008 (according to the Web Page Analyser service). It just represents a snapshot in time. The sizes above are dynamic and will fluctuate whenever new content is added to, or older content removed from, these sites' home page.

The Skinny & the Bloated (part II): Technology blogs

Fri, 08 Feb 2008 09:08:03 GMT

Yesterday I posted a blog entry ranking Irish music web sites, from the skinniest, to the most bloated. For those of you who missed it (or don't have the time to check it out) I basically started off riffing about how, despite the growth in broadband penetration in Ireland, website owners and bloggers should remain acutely sensitive to the size of their blogs as: a) there is still about 7% out there who are on a dialup modem and b) broadband users are an impatient bunch you don't want to annoy with bloated pages that test the outer limits of their download capabilities. I then ranked the top 30 or so Irish music sites and blogs in terms of their page size.

Today I'm turning my (weight-obsessed) attention to Irish technology blogs, namely the the 21 blogs longlisted for the category of "Best Technology Blog" in this year's "Irish Blog Awards" (this blog is also one of the longlisted technology blogs).

The table below ranks all these blogs in terms of their total size of their main page. As I did for the music sites, I have also in the table broken out the "payloads" for each different 'component category' that typically makes up a web page (i.e. the HTML part, images, Javascript files, Flash files and CSS files). The results are then clustered into 4 colour-coded categories, based on overall page size:

Less than 250 KB: ("Optimal balance of content and page size")
251 KB to 500 KB: ("Total nightmare for dialup users")
501 KB to 999 KB: ("High risk of testing the patience of broadband users")
Greater than 1 MB: ("Clinical cases of inexcusable hyper cyber-obesity")

Topping the list is here is Bill de hÓra super slim blog, barely detectable on the scales with its impressive 39 KB. And hats off to to James Corbett whose blog is the only other one to come in under 100 KB. At the other end of the spectrum are Ina O’Murchu and Matt Vinyl's blogs, both of which strain the scales with their 1 flabby megabyte of content (a payload that would require a poor dialup user to hang in there for about 4 minutes to complete the download).

All in all, these technology blogs are a slimmer bunch than the music blogs: over 3/4s of them coming in under 500 KB, a feat only 3/5s of the music sites were able to do. The average weight of the technology blogs is 390 KB, compared to a 630 KB average for the music blogs.

The 5 tech blogs falling into the orange and red categories below would do well to reduce their payload by pursuing tactics I yesterday recommend to the music bloggers i.e., reduce the size of images (or in the case of Tom Raftery and Ken McGuire remove the Multimedia flash content that massively bulks out their blogs) and then reduce the number of blog entries on their blog home page. All of them (except Ken McGuire's blog) have 'server side' compression already activated, the other piece of advice I offered yesterday.

Ireland's Top Technology Blogs
(ranked in terms of page size)

Rank	Blog	Components of webpage (in KB)					Total page size
Rank	Blog	HTML	Images	Javascript	CSS	Flash	Total page size
1	Bill de hÓra	18	15	0	5	0	39 KB
2	James Corbett	11	27	45	4	0	87 KB
3	Dave Northey	14	20	63	14	0	112 KB
4	Karlin Lillington (*)	30	85	0	7	0	123 KB
5	Chris Horn	34	2	89	4	0	129 KB
6	John Collins	14	65	71	11	0	161 KB
7	Haydn Shaughnessy (*)	49	117	3	6	0	175 KB
8	Michelle Gallen	68	114	26	2	5	215 KB
9	Niall Larkin (*)	67	173	0	7	5	252 KB
10	Michele Neylon (*)	57	165	17	15	0	255 KB
11	Inside View	13	236	26	11	0	286 KB
12	Donncha O’Caoimh	17	309	26	18	0	371 KB
13	Promenade	25	257	85	13	1	381 KB
14	Robin Blandford (*)	77	307	37	16	0	438 KB
15	What I think (*)	94	287	62	9	0	451 KB
16	Pat Phelan	15	343	58	18	39	473 KB
17	Ken McGuire (*)	66	95	51	8	447	667 KB
18	Tom Raftery	18	274	38	20	358	707 KB
19	Alexia Golez	127	579	6	10	62	785 KB
20	Ina O’Murchu	15	960	41	2	17	1.03 MB
21	Matt Vinyl	26	746	71	4	195	1.04 MB
(*) These sites/blogs do not have compression activated on their web server, for more detail on server-side compression see note 2 below.

Notes on the above:

Figures above are the sizes of each site's main page on the morning of 7 February 2008 (according to the Web Page Analyser service). It just represents a snapshot in time. The sizes above are dynamic and will fluctuate whenever new content is added to, or older content removed from, these blogs' main page.
Of the 21 sites in the above list, 15 are configured so that text files (such as HTML) are compressed by the webserver before being sent to the requesting visitor (whose browser then automatically decompresses them when the file is received). The advantage of this being that the amount of text data to be downloaded is reduced (the reduction in size of a HTML file that can be expected with such compression is usually in the order of 70-80%). In such cases it is the size of the HTML file after it has been compressed that is cited in the table above. The 10 sites who do not have this function activated (indicated above with an asterisk) are strongly advised to do so.
The above list does not include the 'Digital Sole' blog (one of the 26 blogs longlisted for the Best Music Blog at the Irish Blog Awards 2007) as it was inaccessible at the time I was checking the sizes of the blogs / websites.