question

No Best Answer Selected, Tip Refunded
1 answerers thought this was unfair.

answers (19)

causticwin...
-1
Votes
causticwindow  |  February 02, 2009 08:28 PM
Woopra(woopra.com) looks very good check out the video of it at. http://www.geekbrief.tv/introducing-woopra-youre-gonna-want-it
Comment
sushisean
0
Votes
sushisean  |  February 02, 2009 08:29 PM
I use siteuptime and am very happy with it.
source(s):
Personal Experience
Comment
robbrown
0
Votes
robbrown  |  February 02, 2009 08:33 PM
SiteUptime.com is very good - I use them for small projects. They will provide the transparent data that you are looking for. If all you're monitoring is Mahalo, than it's free.

However, if you would like to promote yourselves as a technological leader, I might look into utilizing raw SNMP data that does not rely on a 3rd party service.

It is very easy to code a simple SNMP query with PHP. Using a simple query, you could work around the multi-server environment you're likely using and poll all of the servers as one for a unilatteral view on public website uptime. The strongest advantage to this (beyond provisioning) is whipping up your own code would allow you to schedule downtime and mark events.

The ability to schedule downtime in an uptime system can lead to a number of revelations. Namely, it can show you irregular patterns in traffic when compared against your web analytics.

The ability to mark events will give you baselines to measure performance and landmark goals, successes and trends.

My suggestion is to code it yourself. While certainly not a high priority on the development tasklist, having the ability to poll your own SNMP devices is by far one of the most powerful and useful things that a site in beta can do.  The beauty of this solution is that once you code your "SNMP Monitoring Program", you can use it for various other projects to provide the same insight and importantly, accurate statistics.

For more information:
http://ca3.php.net/snmp
http://www.activexperts.com/activsocket/howto/snmp/php/

Hope that this helps, Jason!
source(s):
I'm an I.T. pro with over 15 years of experience. I have been involved in many facets of website development and have specific experience in web hosting and networking environments.
Comment
philipy
philipy  |  February 02, 2009 08:37 PM
Whatever metrics you use, make sure that they take account of when only certain parts of the site are not available, while others are up.

For example I've found times when Spy or Profile is not available, but the main Answers page is ok/
robbrown
robbrown  |  February 02, 2009 08:45 PM
This really would be one of the pro's of taking the time to author an SNMP program. You can watch certain things at certain times and even tell how they're preforming.

If you get really fancy, you can plug in your web analytics and show how some things really affect traffic: network utilization, load, etc... they all play a part when you're dealing with a steady volume of traffic.
mikehayes
0
Votes
mikehayes  |  February 02, 2009 08:36 PM
The only software available today that monitors and optimizes your site, based on choice modelling (nobel prize winning mathematics http://en.wikipedia.org/wiki/Choice_modelling) is Accenture Digital
Optimization which can help optimize websites, paid search, display media,
email and direct mail campaigns as well as other online and offline
direct response channels.

http://www.accenture.com/Global/Consulting/Marketing_and_Sales_Effectiveness/Digital/Transformation_Suite/AMSOptimization.htm
Comment
seanpower
seanpower  |  February 03, 2009 11:17 PM
Based on the web page you just linked, Accenture's Digital Optimization product seems like a multi-variate tool that allows you to figure out which marketing campaigns are most effective. It has nothing to do with web performance or web monitoring.
seanpower
3
Votes
seanpower  |  February 02, 2009 08:40 PM
Jason,

Great question, and I'm sure you won't be suprised that the answer isn't as simple as you may have hoped for.

When it comes to performance monitoring, there are really two ways that you can approach it.

A) Is my site loading relatively fast from different places in the world (known as 'synthetic testing').

B) Is my site loading relatively fast for all of the users interacting with my site (known as 'real user monitoring').

Let's start with synthetic monitoring.  Basically, you set up a series of tests.  These tests "synthetically" load your website from different places around the world and tell you if it's fast or slow from those places.  This is great, because not only do you get an idea of how well (or abysmal) your site is performing, but it also tells you if it's performing slowly for some and well for others.  Synthetic testing can give you great daily, weekly and monthly reports that will give you a very good idea of where you pain points are (especially when you make site changes that may have had an adverse effect on the users of your site).  

But then the question really comes down to "where do I monitor from".  By checking your web analytics tool (you use Google Analytics), you can get a list of "Users by Country" and even "Users by City" which will give you an idea of where the largest percent of your site population lives.  You should probably test from those TOP 5 cities if possible.

Want to see how synthetic tests look like?  Easy.  Go to http://performance.webpagetest.org:8080/

Want to see how a 'professional' synthetic package looks like?  Check out Keynote's free "KITE" utility here, though you'll need to sign up and risk getting badgered by sales guys/girls for the rest of your life.
http://my.keynote.com/newmykeynote/multiinstant.aspx

If you want to do synthetic monitoring for free, download PageTest by AOL (http://pagetest.wiki.sourceforge.net/) and get a few servers up at different places around the world to test your pages.

If you've got a budget (at your scale, I'd strongly start thinking about it), talk to the following vendors in the space: AlertSite, Gomez, Keynote and WebMetrics.  They all offer great service, but you'll need to pay a bit of $.

Now, on to real user monitoring.  The problem with synthetic monitoring is that you're really sending "fake" tests.  They "emulate" the browser and they're really just one browser from one location testing your site at regular intervals.  It doesn't necessarily give you an idea of the real impact of changes you make on your site.  That's where real user monitoring tools come in.  RUM tools 'sniff' your traffic and give you performance details for *every single user* interfacing with your site.  If you think about it, that means that you can build reports like "Give me the slowest section of my site overall" or "Give me the slowest section for users from the East Coast" or "Give me a list of the slowest cities only for subscribed users".   This technology is very, very cool, but if you're hosted in the cloud, it might be harder to find a provider.

I don't know of any decent RUM tool that's open source.  You'll need to talk to vendors like Coradiant (which I have previously worked for), Gomez, Keynote, Tealeaf and Symphoniq.

I know, you were probably looking for an easy answer, but there's no real 'widget' that you can install that will do it all for free.  Avoid the temptation to install feel good "you page loaded in 5.3 seconds" counters and go with a real end user experience management tool that can make a difference.  You'll really find that you'll get actionable data back.

Jason, I'm currently writing a book for O'Reilly about this very subject (http://oreilly.com/catalog/9780596155131/index.html). ; If you want, i'd be more than happy to talk about this further.  I'm on twitter (www.twitter.com/seanpower).   I'd be more than happy to give you an advanced copy of the book as well if you want.

EDIT: I want to make it clear, Web Analytics tools will *not* give you performance related information.  If they do, it will be clumsy, at best (and sometimes outright wrong).  Also, avoid installing basic "ping tests" .. they're going to yield very little insight compared to full page testing.

The following screenshot are from Coradiant and Keynote. 
http://www.watchingwebsites.com/wp-content/uploads/2009/02/0802.png
It was unfair to choose no best answer
I had a pretty well thought out answer ;)
Comment
robbrown
robbrown  |  February 02, 2009 08:57 PM
Sean, this is a topic that I am very interested in.

I thought that you had a good answer. However, I've always questioned the value of the commercial services that offer "synthetic" testing.

When measuring the performance of a site across the globe, you're really measuring the effectivness of your bandwidth provider. The best hosts / bandwidth provers use proper peering and a wide, distributed network. Understanding that network and measuring it as it changes and grows through your partnership will lead to a deeper understanding of synthetic results.

In the past, I have simply observed the network, asked for a topological map and used SNMP to regularly query points to determine the relative speed that users will receive.

Regarding RUM measurement. I've come to understand current RUM tools as somewhat of a hindrance to a site rather than a real tool. I'm much more interested in interface path mapping, click points, and other tangible factors. I have not however visited these tools in 2 years. Has there been a marked improvement?
seanpower
seanpower  |  February 02, 2009 09:10 PM
Great questions. Let me address each, Rob.

A) synthetic testing measures the effectiveness of the bandwidth provider.

I disagree. Every single user travels on a carrier of some sort. It's not really possible to remove this from the equation. Furthermore, there are many other things to worry about such as DNS time, and page load time, how long it takes for a POST to process and so on, all of which are clumsy and hard to monitor with SNMP queries. Don't get me wrong, I believe that SNMP is *essential* to understand what I call "in-house availability", whether your hardware is working the way it should be, but there are many things going on in the browser (like the DOM) that SNMP can't see.

2) RUM Measurements.

I'm not exactly sure what you mean by path mapping (path analysis? extremely easy to measure with RUM tools) and click points (where users are clicking? Tealeaf is an expert at doing this). RUM tools give extremely refined visibility that synthetic-only providers only try to replicate (literally, that's the whole reason to run many synthetic tests in the first place). If you give me a bit of a better idea of what you're asking regarding RUM, i'll be more than happy to answer.

Thanks!
robbrown
robbrown  |  February 02, 2009 09:37 PM
Reading your answer and your direct reply (thank-you!) leads me to a common trend in statistics analysis. Currently, most companies are focused on providing very granular data surrounding web utilization, monitoring and traffic. In my experience (which yours could certainly challenge), I've always been a fan of customized, exact data that is tailored to my exact needs.

1) SYNTHETIC
"I disagree. Every single user travels on a carrier of some sort. It's not really possible to remove this from the equation. Furthermore, there are many other things to worry about such as DNS time, and page load time, how long it takes for a POST to process and so on, all of which are clumsy and hard to monitor with SNMP queries."

I closely monitor traffic that comes from the Netherlands. As a result, I have a dedicated server polling SNMP data from my U.S. based web server that is located on a host with direct peering to Planet Internet. The results of the SNMP history is rich and informative to me.

Would I get anything more by using a 3rd party synthetic service?

2) RUM Measurements.

What are the server load and process requirements of a RUM installation?

Again, this is really awesome to be able to question you Sean. Thanks a lot. I really appreciate any back-and-forth that will break me of my old ways and provide a better solution.
mikegero
mikegero  |  February 02, 2009 09:38 PM
Jason:

I'd like to add a bit to Sean's comments. Note: I'm a former colleague of Sean's, having worked at Coradiant as well.

Sound's to me like you may be stuck somewhere between "trying to do it quickly and cheaply" and "getting valuable, actionable data." You also need to be careful about confusion between web analytics and performance analytics, as Sean points out.

Web analytics will tell you how well your site/app is working from a "behavior" perspective. You can track things defined as conversions (e.g. registering on the site), create reports that show the paths most users followed through the site and understand at what parts of the site users most often leave (as just a few examples)

EUM provides performance analytics including measures of user interactions with the site, like e2e response time (and a breakdown of it's components like host time, network time,...), errors, etc.... In a nutshell, it'll let you know if users had either performance or availability issues, and provide you with the forensic data needed to pinpoint and resolve these issues - many times before users call you to tell you about them.

Web analytics are generally the domain of Marketing teams, whereas EUM or performance analytics are usually the domain of Web Operations.

As Sean indicated, there are a number of vendor options. That said, if you're looking for a solution that deploys quickly and easily, won't introduce any performance overhead or security issues, and will give your Web Ops folks everything they need to keep your site humming, you should take a serious look at Coradiant. It won't be your cheapest alternative, but will provide you with valuable insights into your user's site experience, that'll grow with you as needed.

For any more info, I'm on Twitter as @mikegero
seanpower
seanpower  |  February 02, 2009 10:30 PM
Rob,

Cascading threaded comments would be helpful, wouldn't they :)

1) "I closely monitor traffic that comes from the Netherlands. As a result, I have a dedicated server polling SNMP data from my U.S. based web server that is located on a host with direct peering to Planet Internet. The results of the SNMP history is rich and informative to me.

Would I get anything more by using a 3rd party synthetic service? "

Again, SNMP is great at collecting performance related networking stats, and simply can't move up the stack where HTTP lives. You won't be able to get DOM information, and see if javascript is being parsed or not with SNMP tools. SNMP is blind to this information. So yes, you'll gain alot more visibility in the layer that matters to the end user : "did the page get to me fast, and can I use it". Synthetic and RUM tools provide this type of information (though admittedly, any tool that can do javascript instrumentation (or captures the browsers information by artifically replicating it, like a synthetic test) can see more client-side information). That's why you'll see network teams using SNMP (because they care about the pipes), and web operations / web development teams using synthetic monitoring (because they care that HTTP is performing well, and that the HTML/CSS/JS rendered properly).

Think of it this way. A synthetic monitoring tool will get you A) Network level information AND B) DNS information AND C) HTTP related information AND D) Page related information (HTML, etc) AND E) Presentation-layer related information (CSS/JS) all in one shot (that you can trend over time).

2) "What are the server load and process requirements of a RUM installation?"

Well, that's the fun part. RUM tools are generally passive. They usually attach to a network tap and read a copy of the traffic. So from a deployment standpoint, it hardly affects the network.

Again, they're mostly valid if you host your own infrastructure, because you need physical access to the links so that you can TAP 'em.
jasonschum...
3
Votes
jasonschuman  |  February 02, 2009 08:55 PM

Having personal experience for the last 12 months with these, I'll share my findings and let you be the judge.  (Although there are many other services out there, these are the 5 that stand-out and that I'd recommend to anyone)

BasicState.com - http://basicstate.com/ - Free monitoring for the novice. Their alerting is as fast as some of the higher paid services. Very basic "uptime" alerting / nothing fancy - perfect for the part-time blogger.

Pingdom http://www.pingdom.com/ -- Good tool at an incredible price you are only interested in a handful of results.  Good for the beginner website (without substantial traffic).

SiteUpTime - http://siteuptime.com/ -- A nice step-up from Pingdom, you begin to have a little more control over the types of things you wish to monitor and how often to monitor.  They do have a free (1 url test) option.  Cost is fair and worth trying at least for a few months.

AlertSite - http://www.alertsite.com/ -- Best overall tool, best support, and the most cost-effective for your mid-level website (under 5 million uniques a month).  They do have a free-trial I would encourage anyone serious in monitoring to check out.  These guys have a Firefox Plugin that allows you to record a script, upload it to AlertSite, and playback across the world for transaction-based testing.  Their interface is easy to use and best of all, I don't need a sales guy if I need more monitors! They have an On-Demand system where I can add testing in any amount at any time.

Gomez -  http://www.gomez.com/ - Very Good, but very expensive.  They can provide a screenshot of a failure (should one occur) and other object data.  The downsides: XML Data feed for historical purposes will cost you extra. Their service is one of the most expensive. Over the last 12 months their technical support has gone downhill (my biggest complaint). 

Keynote - http://keynote.com/.  From the last 12 months of my personal experience, they are the most expensive but most compelling service you can get.  You get fine-grain detail of how your website is performing in ways no other service provides. The level of alerting (including escalation alerting) surpasses all competition.  For any mid- to large website (5m uniques /month or more) this would be the best possible solution, in my opinion.

 

Who do I use today?  I actually use a combination of AlertSite, SiteUpTime, Gomez, and an internal IpMonitor tool from Solar Winds.  Each external-facing server I manage has different uptime requirents.  Combining AlertSite with SiteUpTime ensures I have overlap in monitoring from around the world.  My strategy may not be the same as yours, so take that into consideration when you're making a decision.

 

(I just noticed others who were suggesting Woopra - While they have fancy graphs, that doesn't translate into great uptime or page rendering.  I worked with their service for 6 months to try and improve the page-load speed - they just couldn't do it.  If you use Woopra, expect your page load times to increase an additional 12-19 seconds.  To their defence, Woopra is still in Beta - I really hope they get the load times fixed.

Comment
geeknum88
0
Votes
geeknum88  |  February 02, 2009 09:44 PM
I currently use site24x7 for monitoring which is owned by the people that brought you the Zoho office
Comment
grantmepea...
0
Votes
grantmepeace  |  February 02, 2009 09:56 PM
Statcounter by far the best. Check it out, link below.
Comment
pdizzlle
0
Votes
pdizzlle  |  February 02, 2009 10:11 PM
Webmetrics (Neustar) is pretty good too. It is better than Keynote and cheaper than Gomez. 
all the other ones (Pingdom, Siteuptime, Alertsite, etc) are pretty basic with limited agents, technical support and lack of advanced functionalities/reporting
source(s):
www.webmetrics.com
Comment
chrisfaron
0
Votes
chrisfaron  |  February 02, 2009 10:23 PM
I've used the free version of siteuptime for 2 years & I'm very happy with the (basic) ping test from 2 locations (1 USA, 1 London), just see uptime %
Great answer Sean ;-)
Comment
seanpower
seanpower  |  February 03, 2009 11:26 PM
Thanks Chris! Glad you liked it :)
rfelix
0
Votes
rfelix  |  February 02, 2009 11:03 PM
What about the newcomer on the block? Check us out a  http://aremysitesup.com
Comment
jadecarlo
0
Votes
jadecarlo  |  February 02, 2009 11:51 PM
Another source to check is http://www.globalnetwatch.com ; especially if you care whether there is a problem accessing your web site from certain regions.
Comment
tmlf
0
Votes
tmlf  |  February 03, 2009 01:27 AM
Jason,
Montastic (http://www.montastic.com) is great for a minimalist solutio.  I used it as a server admin in the financial industry.
Comment
svrider
0
Votes
svrider  |  February 03, 2009 01:30 AM
I prefer Nagios as a ping monitor.  It is free open source software and does everything www.pingdom.com can do.  It can be setup to ping, poll SNMP data, verify services are running (i.e. http, ssh, ftp, etc), and so on.  Also, it can alert you via e-mail, SMS, pager, etc.  I've been using it for three years now and it has never failed me.

I also use Cacti which is also open source software that allows you to graph bandwidth, CPU, memory and various other data through SNMP.  I only use it to monitor bandwidth of various routers and switches but it is a great tool.  The user interface is a little tricky to learn, though.

If you set up a Linux box you can have both of these services up and running very quickly and they are not resource intensive so you do not need a super fast server to run these.

source(s):
www.nagios.org
www.cacti.net
Comment
stevenfoe
stevenfoe  |  February 04, 2009 04:10 AM
The company I work for has used and still has Nagios installed. The only thing is it can be a pain to configure fully and include all the options available. I know from experience there, because my first job was to configure this system and I knew nothing about servers at the time (and very little about *NIX).

We are currently working on a transition to OpenNMS, which is a lot easier to configure monitoring in. Nagios relied on text files you had to write by hand. OpenNMS will let you give the web interface an IP address and it will look for all services running on that IP from its list of over 30 standard services. Beyond the standard services, you can configure custom services that it will also look for.

One thing we added to our Nagios install was NRPE and NSClient++ depending on whether the server we were monitoring was *NIX or Windows. This allowed the Nagios system to see number of users, RAM/Swap usage, processor usage, hard drive space, etc. I haven't configured the custom service yet; but from the documentation I have read, OpenNMS can use the same interfaces to poll the same thing as Nagios in that respect.
lennysan
0
Votes
lennysan  |  February 03, 2009 04:24 AM
One more vote for Webmetrics.com for anything beyond a mom and pop online presence.
source(s):
www.webmetrics.com
Comment
charlie74
0
Votes
charlie74  |  February 03, 2009 06:47 AM
I use WatchMouse (http://www.watchmouse.com) for 4 years now. Very reliable and quick with their new features.
Especially their reporting. I can schedule PDF reports to get send to my customers so they can see how their website and my code is performing.

I know they have also more tools, like DNS, ping, traceroute, etc.
Comment
hippyhop
0
Votes
hippyhop  |  February 03, 2009 08:13 AM
Thanks to Sean for the detailed reply! I would go for the book when it comes out!

Regarding the topic and my experience. There are three types of monitoring tools I recognize:

1. Basic ping tools (pingdom, host-tracker, siteuptime and more). They just provide uptime monitoring with some functionality. A lot of site owners are happy with them. Personally, I do not use them, because they just test HTTP or other daemon for connection, some do not handle HTTP errors though. So you risk to have "uptime 100%" with permament "500 internal error"

2. Advanced monitoring tools. For example:
http://www.uptimeinspector.com and http://www.websitepulse.com
We use both. They provide a lot of advanced options, enough for the most of our tasks. They are hosted, they provide monitoring and performance analysis from many locations around the globe. Finally, they are easy to use. Nothing to download, nothing to install.

3. Downloadable solutions such as Nagios. Very advanced. System administrators are happy with such tools. You can monitor almost everything, but you should also monitor nagios itself :)
Comment
ales
0
Votes
ales  |  February 03, 2009 11:14 PM
I am surprise that nobody mentioned Wormly. I love these guys and they are cheaper than most services that are overkill for what I need. http://www.wormly.com
Comment
ericwastak...
0
Votes
ericwastaken  |  February 05, 2009 04:14 AM
For 24x7 monitoring of servers (ports and protocols configurable) and websites (url with "string" or even transaction success failure) I use http://websitepulse.com/.

I have been using their system for almost 4 years and it is superb, very configurable (including pricing) and very cost effective. They can send alerts based on your criteria to email, SMS, even an automated voice call. You can configure what a "failure" means to you (response time, missing strings, etc) and you can choose multiple monitoring locations worldwide! You can even establish a notification tree of your choosing... for example email on the first failure, SMS on the second failure and voice call on the third, etc. You can also save performance metrics for one year or more and you can even automatically publish those metrics if you choose.

A complete review of server monitoring tools would not be complete without checking out http://websitepulse.com/ in my opinion.
Comment
140

ask any question

Top of Page
Buy Mahalo Dollars
WITH CREDIT CARD OR PAYPAL

Please log in to use this function.