News Numbers

An ounce of Prevention

I exercise regularly and don't smoke, but I avoid reading Prevention.com. It's just too annoying to be reminded of all of the other "smart ways to live well."  When I see "by the editors of Prevention.com," I have this picture in my mind of a bunch of really healthy people popping out cheerful stories like "5 Vitamins Your Bones Love" and "10 Reasons You're Always Exhausted." 

Now I have another (annoying) vision, thanks to MinOnline's story about how well Prevention is using web analytics.  Of course a staff that is so pragmatic and probably always mentally alert would resist "going for the cheap link grab and traffic spike" - the junk food of web analytics.

I haven't been sleeping well because I think too much (the top reason people don't get enough sleep and are therefore exhausted), so I'll just plop in these two paragraphs verbatim from Steve Smith's MinOnline story, "At the Building Prevention.Com, Only The Abs Are Flat."

Prevention stays "on its own brand message and [courts] the kinds of audiences that it and its advertisers really want. 'We got back to engaging with our customer in the ways we knew they wanted us to engage with them,' says [vp/digital Bill] Stump. Fishing for any and all eyeballs and courting simple traffic spikes in the search-driven universe doesn’t pay off in the end. 'You get waves of traffic, but the tide goes back out and what are you left with?' Instead, by keeping to the needs of the 'core customer' in everything that goes out to syndication or into the e-mail newsletters, prevention.com is courting the people who tend to stay.

"Now, each big wave raises the sea level for all of prevention.com’s metrics, says Stump. In the last two years, overall page views climbed 60%. In the last year, the number of visits per user went up 12%. But it is the engagement metrics of which Stump is proudest. 'The number that warms my heart,' he says, 'is page views per visitor that are up 49%.' That means the new visitors are sticking with the site and drilling much deeper than they ever have before. 'In general, advertisers want an engaged audience. They want the metrics that show that people value your brand and come to you for something that is unique. We own natural health and fitness and beauty. We are the authentic voice.'"

 

Wasting time

Time spent on a site or a visit ranks right up there with total page views and monthly unique visitors as widely quoted metrics masking as indicators of success for news organizations.

No, it's not a crime to misuse a metric, but isn't a shame to waste your time on something that's not absolutely essential to your site's success? 

Plus, the way that time spent is calculated is flawed.  All web metrics are flawed somewhat, but time spent is really misleading.

More on the ugly methodology later - let's tackle time spent's uselessness first.  In other words, if the methodology were acceptable would time spent still be a key performance indicator?

Advertisers have always made decisions based on the level of engagement a news org's audiences have with its brand and content.  But both content and the ways people use and interact with content are different - and thus the way engagement is measured is different, too. 

In the past, time spent was an important measure of engagement for news orgs and advertisers.  People spent whole chunks of time with one medium or another.  Readership surveys measured time spent per day or per week.

Because these were surveys, time spent was based on self-reported information.  It was what people said they did vs. what they actually did.

But it didn't matter whether what people said matched with what they did.  What mattered was how engaged people felt.  People who reported they spent an hour a day with Monday's newspaper but actually only spent twenty minutes believed they spent a large chunk of time and attention with a news org.

In stark contrast, web advertising decisions depend on knowing actual behavior as reported via rows upon rows of numbers ruthlessly pouring out every second.   Among many other things, advertisers track the number of times their ads come up and are clicked upon. Sites and audiences are more niche and are highly segmented.   The algorithms for and definitions of "engagement" vary for every site and every company.  

Time spent just isn't a good indicator of engagement.  Someone who spends five minutes a day on a site, goes to five different stories each visit and adds comments twice a week is clearly more engaged than someone who comes onto a site for 30 minutes a week and clicks idly on a few pages while talking on the phone.    

How many times have you spent 30 minutes or so on a site, flipping and flapping through what seems like a million page views in a fruitless attempt to find something?  Maybe you spent 30 minutes in such a visit once - and never went back.

A news org's success in the long-term will be based not on how much time people spend on a site but what they do once they're there.

--------------

How Time Spent on a Site is Calculated

AOL analytics

In the late 1990s, "America Online" was the shiny new company everyone watched, feared and tried to copy.  Just "AOL" now, it's hardly as fresh or inspiring. With its new CEO, logos and use of web analytics to select the stories it covers and evaluating its reporters, has AOL once again become a news organization to watch?

AOL's announcement that it will employ "judicious use of Web-analytics software" sparked the expected flutter of coverage.  It's admitted to using data to inform (dictate?) news decisions, so you could be led to believe that AOL is adopting a true audience-based approach.  However, after reading the Feb. 22 story in BusinessWeek and the reactions gathered by Media Post News, it seems like AOL is still using a traditional advertising-based mass media strategy.  It's still trying to be all things to all people.  It's just using web analytics to decide what those things are.

"Audience growth and audience engagement have to be the things that we judge the most off of our journalist investments," AOL CEO Tim Armstrong is quoted as saying.  So far, so good. 

Armstrong also said that "brand ads should be a lot bigger on the Internet today," talking about how online advertising revenue should pick up.  But there was no mention about AOL's own brand strategy, something that would answer the question of "What is AOL?" for audiences and advertisers once and for all.  On which niches will it focus?  How much of its content will be unique and compelling enough to those niche audiences so that they'll come back regularly?

Patricia Handschiegel, who blogs as Daily Patricia, sums it nicely:

"The right approach to the content business is to KNOW YOUR AUDIENCE, or the people that come to your site, and create a product for THEM. AOL’s approach is clearly not centered on this....it’ll drive up page views and therefore, revenue but that’s not likely to last as the industry becomes more analytics savvy. Today, a million uniques with zero session times, high bounce rate and no repeat visitors isn’t seen as a sign of a lack of audience but in the not too distant future it will."

I haven't researched AOL myself, so I don't know if all of the details in the BusinessWeek and Media Post News stories really reflect what AOL is doing.  So I'll just note some some things news orgs should think about when using web analytics to inform news decisions and evaluate journalists.     

• Evaluating success (either a site's or a journalist's) by total page views doesn't work. A large number of page views may just indicate visitors got there by mistake or clicked around trying to find something.   Plus, dynamic content (Flash, etc.) will not be counted as page views. Page views can be a useful metric, but only when combined with other metrics - such as ratios - that give context. 

• Engagement can't be determined by web traffic or behavioral data alone.  Attitudinal research is essential to find out why or why don't people come to a site regularly, what they want and what they're not finding. 

• If journalists are going to be held accountable for web traffic and audience engagement will they also have control over the factors that drive traffic, such as design, navigation and marketing? Or will they just submit their stories and hope for the best?

• "AOL is even considering sharing a portion of quarterly profits with staffers whose work fetches the most page views."  BusinessWeek

How will traffic goals be set?  If journalists will be rewarded for generating "traffic" (however it's defined), will they be fired if they don't? Will the benchmarks or starting points - and the time journalists have to reach the required traffic levels - be based on whether a topic is already established or whether it's one a news org wants to nurture and grow because the topic is essential to achieving its strategy?

• "Tacked to the newsroom walls in AOL's downtown Manhattan headquarters are pages and pages of Web traffic data." BusinessWeek
  
  Uh, this would cause even me to shut down.  It's definitely not "judicious use of Web-analytics software."  Does AOL have a few key performance indicators that everyone understands and on which they can focus as a team?
  
  Software and reports don't make decisions; people do. Successful use of web analytics depends on the decision-makers understanding and using the information correctly.  If news orgs believe the success of their websites depends on being truly audience-focused then they must also ensure the analytical resources and processes are there as well.  

AOL may stumble again but at least it's trying something different.  I look forward to learning from AOL whether it succeeds or fails. 

Video metrics for everyone!

YouTube's become a verb and a household name, but I'll always see it as an organization that's brought metrics into the lives of the common people (those who have broadband Internet, anyway).  The "Most Popular" and "Featured Videos" are seen worldwide, sometimes garnering millions of views.  "Hey, did you see...." is usually accompanied by something like "...and it has x million views on YouTube!"

Number of views is great for little else other than bragging rights.  It's one of the  "famous" metrics (web analytics guru Avinash Kaushik's term) that "are staring you in the face when you crack open any analytics tool" but "barely contain any insight."

Yep, for anyone in the content business, number of views is right up there with hall of famers number of page views and monthly unique visitors.  

YouTube has pushed all of its account holders  - no matter how amateur - to use meaningful metrics. In March 2008 it launched Insight, its "video analytics tool for all users," along with some almost-preachy instructions on how to use metrics to get more people to watch your videos and, of course, come more often to YouTube.

The Insight tool allows you to track "community engagements" (there's that word again) in terms of ratings, comments, and favorites.   YouTube doesn't want you to settle for people just watching your video.  People have to show, in a measurable way, that they not only watched it but also reacted to it. 

At the very least people should give a star rating (one is bad, five is good).  Rating is easy, quick and anonymous.  Tagging a video as a favorite is the next rung.  And if they're really engaged, they'll leave comments. 

But, as anyone who's ever spent any time at all on YouTube knows, many comments are spam, obscene and irrelevant - just noise.  But the value of social media metrics is in looking beyond what James Kobelius in Information Management points out is an "often low and laughable" signal-to-noise ratio. 

Kobelius notes that "if you crawl, correlate, categorize, mine, and explore it with the right tools....[this unstructured information] can yield unexpected insights....The intelligence value of any individual tweet [or comment] in isolation is negligible....Intelligence emerges from the aggregate."

If you can stomach a few obscenities, look at this thought in Encyclopaedia Dramatica about YouTube view fraud and how the ratio of VPC, or views per comment, "is the most accurate way to determine if anyone" cares.  "A high VPC usually means view fraud has been committed." 

The example in ED shows that a video with 136,097 views and 3,529 comments has a VPC of 38.7, a low number that indicates this is a video "that people actually find funny."  The video with 296,413 views, 541 comments and thus a VPC of 547.9 is probably something nobody really cares about.

I calculated some VPCs from this week's "Most Popular" videos and came up with some numbers that I don't know what to do with yet.  To see if VPC can be used as a key performance indicator, I'll need to calculate VPCs and crawl through the cacophony of a variety of news videos.  VPC may never be  "famous," but it might be insightful. 

Always a bridesmaid, never a bride?

It's almost Valentine's Day, so let's muse again about what it means to be "engaged."  In this day of figuring out whether people will pay for web news, defining success by measuring engagement is more important than ever. 

It doesn't matter whether you love or hate a news organization if you're engaged with it - as demonstrated by behaviors such as going to the site frequently, contributing content, e-mailing a story, rating a video or paying a monthly subscription fee.

Many worthy people have come up with all kinds of complicated mathematical formulas for measuring and tracking engagement.  Nothing's stuck.  In other words, just because a number was produced ("Disaster!  Our engagement rating was 14 last month but our goal was 19!") doesn't mean site traffic and other key performance indicators move in conjunction with it.  A metric is just a number if it doesn't move up or down as a result of some action or mistake on your part. 

Although measuring engagement still eludes us, I hope news orgs will still adopt an engagement philosophy and an audience-focused culture that will guide the decisions that do lead to measurable results.

A philosophy still needs some definition.  I like this quote from Dave Smith, CEO of Mediasmith, a digital advertising agency.  The interview is in "Digital Engagement," a book by Leland Harden and Bob Heyman.

"Engagement is an unconscious tick of the mind that causes you to think differently about and notice a brand differently in the future."

In the same interview Smith also quotes Erwin Ephron, perhaps the "founder of modern media planning," as saying that "Media engagement and advertising engagement are very different things....Historically, media are measured by audience delivery.  Advertising is measured by response.  Engagement-based ratings would measure media by response."

In other words, it's not enough now just to put content out there and hope your audiences will like it.  Traditional audience research that produces various numbers for loyalty and satisfaction isn't enough either.  Audiences can't just tell you how they feel.  They have to show you.      

Twitter strategy revisited

It's been fun to, ahem, follow all of the rather serious Twitter research (isn't The Science of ReTweets just the best title?). The studies that seem to get the most attention are those that examine the ego-boosting or deflating number of followers - what's too many, too few or just right. 

"In Praise of Obscurity," Clive Thompson's column in the Feb. 2010 issue of Wired, states the obvious to anyone who's ever been in a class of 15 people and then in one that has 500: 

"Once a group reaches a certain size, each participant starts to feel anonymous again, and the person they’re following — who once seemed proximal, like a friend — now seems larger than life and remote....Social media stops being social. It’s no longer a bantering process of thinking and living out loud. It becomes old-fashioned broadcasting."

At his Twitter Boot Camp last June, Tim O'Reilly chided the New York Times, saying that "just using Twitter as an RSS feed for your site is a missed opportunity."  Twitter's supposed to be all about building communities by having two-way conversations between the followers and the following.

At the time I instantly agreed with him.  But Twitter is no longer an amusing recreational hobby.  It's now a business juggernaut - and that implies a company should use it tactically any way that it wants.

The NYT has 2.3-million-plus followers, is supposedly following a measly 193, and doesn't ever seem to retweet or respond to a follower.  But every tweet has a link to the NYT site.

The NYT doesn't claim it wants to be your friend and talk to you.  Its Twitter bio states right up front that the NYT is "Where the Conversation Begins."  It's not where conversations continue, are facilitated, passed on or anything else connected to being a personal relationship builder.

I do think news orgs must build closer relationships with their audiences.  But Twitter isn't - and shouldn't be - the only way to do it.   What is important, however, is transparency in how a news org is using its main Twitter handle.  If a news org is just not that into you, it should say so.

A Tablet Day thought

All the talk this week about Apple's new tablet has me picking up my Amazon Kindle and hugging it.  After all, it's real, it's here, and it didn't cost $1,000. 

I really love my Kindle, but the metrics side of me cringes every time I read an article from the Los Angeles Times, New York Times or New Yorker subscriptions that appear silently - magically - every day or every week and are always at my fingertips.  

News orgs complain about the pennies they get from each Kindle sub, but that's the least of their problems in the long run.  The bigger problem is that they can't track how many subscribers click on an issue, when they do it and what stories they read, if at all.   

This is just like the good old days!  A newspaper pumps out a seven-day home-delivered product and collects a few subscriber dollars so it can say it has a paying audience.  It then sells some ads based on a mass media theory (otherwise known as "spray and pray").   In other words, it has no idea of what works for subscribers.

Web sites give news orgs detailed information about what engages audiences.  The info is sometimes painful ("What?  No one's reading my story?"), but it's essential for survival today.  

For a reader, Kindles and tablets are great. What's great for readers should be great for news orgs, too, but only if they know why. 

Black + white = grey

Which of these things are not like the others?

a.    Black vs. white
b.    Mass vs. niche
c.    News vs. advertising
d.    Site-centric vs. person-centric
                   OR    
       Census server vs. audience panel data
e.    None of the above (all are alike)  

The answer is (d).  Really?

When I plunged into the huge world that is web analytics, I decided to focus on site/server-centric or census analytics, or the data you get from software like Google Analytics or Omniture.

After all, site-centric data gives you what people actually did.  The comScore or Nielsen data comes from panels of people reporting what they said they did. 

It makes sense that there are two different types of data, because there are two different needs:  Advertising needs overall site data that gives some demographic information and allows competitive comparisons, but the newsroom needs data that tracks traffic for every story in every section (otherwise known as "excruciating detail").

Here's the smug little chart I've been using in presentations.  (It got a nice comment from someone on Slideshare:  "The best way I've seen yet to describe the fundamentals of web data structures!")

Things might not be so left-side vs. right-side now.  comScore and other panel vendors claim they're now combining the best of both.  Josh Chasin of comScore asserts in Online Metrics Insider that its hybrid methodology is "helping to resolve what once seemed to be irreconcilable differences between two ways of counting....But this convergence doesn't mean there isn't still room for two distinct metrics disciplines....The function and purpose of these two data streams remains separate, and both are essential....panel data can make census server data better, and...server data can make panel data better." 

I think I'll still focus on census server data.  It's too overwhelming to go deep into the panel side when there's still so much to learn from the census side. 

However, I'll start following the evolution of hybrid measurement systems.  More data often just leads to more "so what" reports.  But we might be on our way to a better understanding of our audiences, which both the newsroom and the business sides desperately need.

Is your "in" crowd big enough?

Defining success in Twitter, Facebook and other social media services starts with identifying distinct niche communities based on shared interests and beliefs.  Each community member has different levels of participation and prominence.  And in each community there's an "in" crowd whose actions determine whether others stick around.   

How do you know who these influencers are and whether you have enough of them to keep a network alive and thriving? 

Success in social media depends on both listening and doing.  It's difficult to measure the listening part, but you can definitely measure how many people are doing something, and what they're doing. 

Success also depends on who is contributing.  However, a prominent person's mere presence doesn't mean he or she is an influencer.  A true influencer posts content, starts conversations, organizes meet-ups and otherwise engages community members.

For a metric to be useful, you need to have a starting point and a goal.  Setting a goal can start with a pure guess that you adjust once you have some data.  Or, your initial goal can be based on benchmarks from a source that makes sense for your strategy and objectives.

This eMarketer story, "Harnessing Active Brand Advocates," summarizes findings from a social media survey done by Synovate, a research company.  The findings can help you determine whether you have a healthy number of influencers.

The survey found that 26 percent of U.S. Internet users posted online ratings or reviews.  Twenty percent contributed to online forums.  Eighteen percent attended a meet-up that originated online; almost 6 percent took an active role in organizing one.   The numbers didn't vary much between men and women, but they did vary by age group, as you would expect.

It won't be easy to gather this info from all of the multiple accounts you have on Twitter, Facebook and other services.  Measuring social media is art and science grounded in a thorough knowledge and understanding of what and who makes a particular social network work.

Before even counting influencers, I'd start with coldly assessing whether your news org is in--or out.  What would it take to be in--and stay there?  Will it ever be possible? 

I'm not sure it's worth the time and resources for a news org to be in a network if it's not an influencer.  Does just being a follower help or hurt?  Or do community members feel insulted if they perceive a news org is just putting in a half-hearted, token effort? 

I suppose it would vary by community and news org.  Oh, great - that's one more thing to measure and track.

 

Twitter: new medium, new metrics

It's a new year, so I'm hitting the reset button on my attempt to come up with a Twitter metrics methodology relevant to news organizations.

There's no better place to start than with Avinash Kaushik's November 2009 blog post.  Among other things, Kaushik is Google's "analytics evangelist."  He's not only an innovative web analytics thinker but also someone who really cares whether the common everyday professional understands this stuff - and uses it.

Why should news orgs measure their Twitter efforts?  Why don't they just tweet away and just count how much traffic Twitter sends to their sites?

Twitter is essential for news orgs.  News orgs won't be able to attract, build and engage audiences if they don't use social media successfully.   If news orgs don't believe this then...(insert your fave cliche about ostriches or whatever here).

Twitter takes a lot of time and effort.  At the very least, news orgs should use Twitter metrics so they can use their journalists' time and efforts effectively.  Who tweets?  Who doesn't?  Who should - and shouldn't?  Who should tweet more?  Less?  On what?  With whom?  Are news orgs reaching and engaging the audiences they need and want?

So, here are the key thoughts I have from Kaushik's blog. 

First, just as you shouldn't apply traditional mass media metrics to the web, you shouldn't use traditional web metrics to measure social media. 

Example: Total circulation/readership is a key performance indicator for print, but monthly unique visitors isn't.  You can count anything, but you shouldn't waste time on counts that don't directly lead to specific actions.

Kaushik:  "One of the biggest mistake companies and brands make about Twitter is that they think it is one more 'shout channel' like TV and radio and magazine ads or press releases. Twitter is not that. Twitter is a 'conversation channel,' a place where you can find the audience relevant to you (and your company and products and services and jihad) and engage in a conversation with them. It is not pitching, it is enriching the value of the ecosystem by participating."

For Twitter, Kaushik likes Klout's methodology for assessing reach, demand, engagement and velocity.  Klout gives metrics on each of these areas from which you can "pick and choose according to the objective/action/decision needed."

Klout also gives a total score or compound metric, which Kaushik warns against.  Compound metrics "can be subjective, inapplicable to many and efficiently hide the insights you need to understand what actions to take."

Instead of the simplistic follower/following ratio that many use to define Twitter success, Kaushik likes total retweets, number of retweets per thousand, messages per outbound message and churn.  These metrics measure conversation, not "just yelling."

About all of those followers:  Kaushik's intrigued (as am I) by GraphEdge's assertion that those followers who are following more than 2,000 people aren't "legitimate" because they aren't really monitoring your feed.  

I don't think those following more than 2,000 should be completely discounted.  After all, someone could be following you and not following the other 1,999 people.  And that someone could be really important to your targeted audience.

GraphEdge and other tools show much promise in figuring out what news orgs need to do with Twitter.  But it's going to take some time and a lot of effort.  Kaushik:  "Be willing to work hard. Be willing to put in the sweat equity. Be willing to try 45 things (tools/metrics/strategies) to find the three that work for you."

Ack!  Forty-five things to find three?  Unfortunately I think (actually, I know) he's right.  I'll be digging into Klout and GraphEdge, but I don't know if I'll be looking into 43 more things.  For news orgs, it might take more than that to find the magic three.