Synthetic Monitoring Vs Real User Monitoring

In order for an information system to be useful, it must be available to authorized users. In fact, one of the primary objectives of information security is to protect the availability of information systems. Availability simply means that systems and applications are up and running, and authorized users have uninterrupted access to resources when they are needed. Apart from denial-of-service attacks, in which the performance of a system or web-based service is maliciously rendered unavailable, some of the most fundamental threats to availability are non-malicious in nature. These include hardware failures, application errors, unscheduled downtime, resource utilization, and bandwidth issues.

In today’s digital economy, an application isn’t just there to serve a business, in most cases, it is the business itself. A lot of businesses rely heavily on web and mobile applications for delivering value to customers and generating revenues. Ensuring the availability of those applications is a high priority for those businesses.

Making the contents of websites or web applications load as fast as possible is crucial for increasing customer retention and engagement. Any disruption to service availability for even a short time can lead to loss of revenue, customer dissatisfaction, and reputation damage. Users that rely on those services will get frustrated if the service is not reliably available. This is why Web Performance Monitoring is so important. Monitoring the performance of a website or a web application is key to knowing if the service is performing as expected.

What is Web Performance Monitoring (WPM)?

Web Performance Monitoring (WPM) is a collection of tools and processes designed to monitor key application performance metrics of websites or web applications in order to ensure that they meet performance standards. It should also provide a valuable user experience (UX) such that contents are displayed as quickly as possible, pages are loaded at consistent speeds, and don’t suffer unexpected down-times.

WPM is essential in ensuring that a web service is available to end-users, and in improving their digital experience. Other key benefits include:

• Website monitoring helps benchmark the performance of the website against others in the industry to help determine how well it is performing in relation to competitors.
• Website monitoring can be used to hold web hosting providers accountable to their service-level agreement (SLAs).
• Website response time and availability–a key metric in WPM, is also used as a metric for search engine rankings.

WPM metrics cut across areas that are critical to the functioning of the web application such as network and database connectivity, bandwidth and computer resource utilization (RAM, CPU, and disk space), and many other areas.

The primary purpose of WPM is to determine how end-users are actually experiencing your web application, and how such data can be translated into actionable insights to achieve business goals. WPM data can also be used to measure the actual quality of service delivered to customers or end-users, and to determine how well the site is responding to improvement measures.

There are two main approaches to Web Performance Monitoring: Real User Monitoring and Synthetic Monitoring.  We will now discuss them in detail.

Synthetic Monitoring

Synthetic Monitoring is an approach to Web Performance Monitoring that employs active monitoring technology whereby scripts are deployed to simulate an action or path an end-user might take through a web application, and then providing reports about the performance for further analysis.

Those paths are then continuously monitored at specified intervals for performance such as functionality, availability, and response time measures. This type of monitoring does not require actual web traffic or user interaction; rather it uses synthetically generated traffic to collect data on page performance, hence the name synthetic monitoring.

Synthetic Monitoring is valuable because it enables a webmaster or system administrator to identify and resolve problems on the site even before it affects actual end-users or customers. it enables companies to test new applications or features prior to deployment in a production environment. Because synthetic monitoring is a simulation of typical user behavior, it is often best used to monitor commonly trafficked paths and critical business processes. Synthetic monitoring will report a wide array of performance metrics depending on what is being monitored or measured.

Synthetic Monitoring uses a computer application that conducts the performance checks on a routine basis and generates reports and alerts. Other key activities that can be performed by Synthetic Monitoring include:

• Creating a baseline for performance trends across countries and regions
• Monitoring critical database queries for availability
• Detecting problems caused by third-party scripts
• Alerting you when your database or app goes down
• Performance monitoring over low-traffic periods

Synthetic monitoring works by deploying scripts that conduct automated periodic performance checks on a routine basis. Every now and then, the script sends a request to the website or web service, measures the response times, and generates reports and alerts based on the site’s performance.

Synthetic monitors operate like bots that connect to websites, web services, APIs, and servers to verify availability and performance using a network of checkpoints external to the website’s own servers from various geographical locations. These monitors generate alerts when they identify downtime or any form of degradation in performance. With these capabilities, system administrators can proactively identify availability problems or major execution issues that can affect users’ digital experience. Synthetic monitoring answers the following key questions:

• Is my website available?
• How fast is my site at this moment?
• How is my website performance at the moment?
• Is there downtime or error?
• If yes, where is it?

 Advantages of Synthetic Monitoring

Deploying Synthetic Monitoring in your organization has a lot of advantages. Highlighted below are some of them:

• Find and fix issues proactively: The proactive approach of Synthetic Monitoring helps to provide feedback about possible performance issues and application errors even before anyone is aware of them. Synthetic helps to detect and troubleshoot shorter-term performance issues even in the absence of real user traffic. Thus, you can easily find and fix issues before they impact end-users.

• Setup baselines and benchmarks: Synthetic monitoring affords you the ability to set up baseline measures and comparison benchmarks which enables you to detect changes in the web performance once established baselines are exceeded, or set up comparison benchmarks with competitor’s websites. The data obtained can then be analyzed to gain insight into areas for improvement.

• Emulate customer transactions: Synthetic Monitoring allows you to emulate business processes and customer transactions such as order cancellation, password reset, and logging in and out, all from different geographical locations. It can replay user sessions and track transaction paths to uncover potential problems. This gives you the opportunity to formulate and implement performance improvement plans.
• Monitor the performance of third-party apps: Unlike RUM, synthetic testing also gives you the opportunity to monitor the performance of the third-party plugins, services, and APIs that you use on your web app. The ability to monitor the performance of these third-party services is especially important for eCommerce applications and websites that often rely on third-party plugins, shopping carts, and payment modules.

Disadvantages of Synthetic Monitoring

Although Synthetic Monitoring has a lot of advantages, it also leaves gaps in key end-user experience monitoring use cases. Highlighted below are some of its disadvantages:

• Does not always align with reality: Synthetic Monitoring has one major weakness: It does not always align with reality. Due to the fact that the nature of the traffic used for Synthetic testing is not organic but synthetic, and isn’t truly representative of real user experience at any given time; their outcome and performance are rather predictable. Because Synthetic Monitoring uses simulated actions, it is unable to predict the actions of real-life customers.

• Time-consuming for incident resolution: When helpdesk personnel receive complaints from end-users, Synthetic Monitoring products would tell them nothing about what the end-user was actually doing or experiencing. You would have to painstakingly analyze performance for every business-critical app, and create scripts for synthetic testing in order to uncover possible issues, and this can sometimes be time-consuming.
• Narrow view of performance: Synthetic Monitoring can’t match the rich diversity of performance variables that exist in the real world. Because you are not monitoring actual users, you aren’t really getting any data on how your product is used in the real world. All you have is simply a narrow view of application performance.

Real User Monitoring (RUM)

Real User Monitoring is an approach to Web Performance Monitoring that employs passive monitoring (the RUM application collects website performance metrics without affecting the operation of the site) technology that continuously observes, captures, and analyzes all user interaction with websites or web applications; tracking availability, functionality, and responsiveness.

With RUM, organizations can easily capture data about how their application is being used, as well as the actual performance of the application and users irrespective of device, browser, or network. RUM breaks down all the performance metrics (connectivity, load time, uptime, etc.) for each user, aggregates the data, and allows you to view your site’s performance based on user location, browser type, operating system, and device. RUM can also be used to test or anticipate changes in a live website or web application using A/B testing or other techniques.

With the increasing popularity of cloud services along with web and mobile applications, it has become more and more important to monitor the usage, behaviors, and performance of these applications.  By leveraging RUM, organizations can achieve the following:

• Understand its users and identify key areas on its site that require the most attention.
• Capture real end-user experiences from different devices and browsers, and understand the geographic or channel distribution trends, and other key usage trends of end-users.
• From a monitoring perspective, knowing your user trends allows you to identify areas to target for performance improvements and optimization.
• Correlate user engagement and business KPIs with application performance, and obtain reports on load time, network duration, backend/frontend duration, download time, and much more.
• Utilize historical usage data to forecast web performance trends and business outcomes.

RUM technology works by capturing a website or web app performance metrics directly from the end-user browser or client application. A tiny JavaScript code embedded in each page collects this data from users as they explore the app or web pages, and sends them back for analysis. Data visualization techniques such as graphs and charts are then employed to make it easier to glean actionable insights from those data. Real User Monitoring answers the following key questions:

• What is the current performance of my website/web application?
• How fast should my website or web app be in terms of response time, load time, etc?
• What are the performance issues currently experienced by end-users?

Figure 1.0 RUM injects JavaScripts in browsers to collect performance metric

Advantages of Real User Monitoring

Adopting RUM as your WPM strategy offers a lot of benefits. Below are some of the advantages of deploying RUM in your organization:

• Gain user insight: Because RUM allows you to monitor actual users and UX in real-time, you tend to gain actionable insight about how your product is used, and issues affecting users in the real world. RUM helps with understanding long-term trends based on actual usage patterns and lets you see the issues appearing from the end-user perspective.

• Automate reporting process: Most users who encounter bugs, unexpected crashes, or other performance tend not to report them even when automatically prompted.  However, with RUM, you can easily implement tools that automate the entire reporting process without depending on users.

• Prioritize Issues: With RUM, you can replay user sessions and track transaction paths. This helps you see real issues and problems that users encounter on your app.  You can easily prioritize and focus on resolving issues that improve the overall user experience, or issues that have the potential to cause the biggest losses for your business.
• Measure SLA and KPI targets: RUM offers real-world measurement of key targets such as agreed service levels and KPIs, by tracking actual visits, user behavior, and performance levels.

Disadvantages of Real User Monitoring

For all its benefits, RUM does have several limitations. Below are some of the disadvantages of using RUM:

• Not suitable for benchmarking: With RUM, it is difficult to set up baselines and comparison benchmarks with competitor’s websites or web apps. Because RUM relies on real traffic, there may be inconsistencies in terms of the timing of the data that comes in. Almost every user has their own unique set up variation. This makes it difficult to get accurate benchmarking data.
• Dependence on user traffic: RUM won’t work for you if you don’t have any traffic. This dependence on real user traffic in order to produce actionable reports and insights is not always advantageous, especially in situations where you want to detect and fix bugs and other issues before the users encounter them. It is therefore not suitable for testing the impact of changes that have not yet been rolled out.
• Large amount of accumulated data: Having loads of data can be a great advantage, but it can quickly become a disadvantage when you’re looking for specifics. Because RUM monitors virtually everything, there is the likelihood of accumulating unwanted (noisy) data. Filtering and sieving through this large amount of data can be daunting.

Table 1.0 Synthetic Monitoring Vs. RUM: How they compare

Synthetic Monitoring Vs. RUM: How they compare

RUM and Synthetic Monitoring provide different types of insight about your application performance and have different areas where they are best suited or individually stand out. Deploying one or the other will help analyze performance in different ways. Where Synthetic Monitoring falls short, RUM succeeds; and where RUM falls short, Synthetic Monitoring succeeds. When used simultaneously, RUM and Synthetic data can provide deeper insights that can be used to improve your customers’ digital experience.

Stated below are some of the areas or use cases where they are best suited or individually stand out:

• Synthetic is suitable for spot-checking performance or testing for specific errors or behaviors. This provides a narrow but needed view of performance to guide decision making, but it doesn’t actually reflect what users are experiencing in real life. RUM, on the other hand, is well suited in situations where you want to see things from the end user’s perspective and gain a deeper understanding of actual usage patterns.
• Synthetic Monitoring is the most effective solution to identify issues before customers encounter them. On the other hand, RUM would be the most helpful tool to measure application responsiveness, especially during peak hours. It also provides essential insight into real business metrics and how it impacts the bottom line.
• Synthetic Monitoring is best suited in situations where you want to measure the immediate impact of changes or new features to the application before roll out. RUM on the other hand can be used to measure the long-term impact of changes based on actual usage patterns.

Table 2.0 Use cases where Synthetic and RUM are best suited

In conclusion, therefore, both RUM and Synthetic Monitoring are complementary tools that play important roles in monitoring and improving the performance of web applications. In order to have a comprehensive view of the performance status of your website or application, both technologies should be considered in your overall monitoring strategy. Combining both results in a more robust monitoring strategy that gives you a better picture of the performance issues associated with your web application, which can then be used to improve the quality of UX.

Related posts:

• Real User Monitoring Guide
• Synthetic Monitoring Guide
• Best Real User Monitoring Tools