After the alarm: where monitoring is taking us
by Simen Frostad, Chairman of Bridge Technologies
An alarm wakes you up. Asleep, awake – it’s a binary response. There’s nothing nuanced about it: a noise interrupts your sleep, you become conscious and able to begin another day.
But even the basic wake-you-up alarm can be something more; with some basic data gathering your smartphone can tell where in your sleep cycle you are, and avoid waking you when you are in a deep sleep.
In monitoring applications for digital media, the alarm has been a dominant paradigm for many years. Monitoring systems have been thought of as tools for telling us if things are working, or if they aren’t. When errors occur, the system sounds the alarm, and engineers get to work tracing the fault and remedying it.
But like the bell that wakes us up, this use of alarms is reductionist – a more or less binary approach to monitoring. And it’s what has become the default model for monitoring systems, because it’s impossible for human engineers to eyeball the massive volumes of data gathered from a digital media operation and make sense of it. In order to stay on top of the operations’ status, you have to set parameters for acceptable performance, and leave the monitoring system to draw your attention when these parameters aren’t met.
In the real world, parameters of acceptable performance are difficult to define. Many are based on standards that are themselves compromised, and may be out of date. The real world is a slippery place, and without continuous clear-eyed adjustments, it’s difficult to get where we want to go. It would be hard to imagine how we could, for example, drive a car with only alarms as our primary source of feedback. How would we steer, accelerate and brake, without the multi-dimensional and subtly interacting sources of information that come from the instrumentation and our own visual, auditory and tactile senses?
If we think of a digital media operation as a complex organism for making profit through the provision of entertainment services, we can see that there will almost certainly be many monitoring systems keeping it pointing in the right direction. Cash-flow and financial models will be used for forward planning, market research tools for defining audiences and matching the right content to them, and so on. You can’t really envisage a business existing successfully without monitoring of this nature. But equally, you can see that if the finance department had to operate only on the basis of alarms that occurred when something went wrong, or content planning only took place on the basis of a binary ‘working/not working’ status test, the business would be doomed.
So why then, is the industry satisfied with a monitoring model for the delivery chain that is based around alarms, with little or no capacity for more subtle analysis, open-ended data correlations, and projections for forward planning?
The fact is that with the advanced IP and RF monitoring tools that exist today, we have the ability to gather, collate, model, and analyse project information on an unprecedented scale. But if we simply use this mass of gathered data to trigger alarms that set engineers scurrying to fix the fault, we are missing an enormous opportunity.
In the connected media consumption landscape we have now, there’s an extraordinary amount of data to be accessed and understood by the digital media operator. With hundreds of thousands of OTT clients, and millions of STBs to analyse, an operator has a priceless resource. But these volumes of data can only begin to be valuable if the tools exist to make sense of them. And the key to extracting this value is to use advanced visualisation in a creative way, with the ability to correlate data from heterogeneous sources as a way of discovering new patterns and connections.
Both data gathering and data presentation are core functionalities for monitoring and analysis systems, and if they gather data comprehensively and present it intelligently, they make the most useful tools. If data are gathered with poorly-defined criteria and then presented without being filtered and moulded into intelligible form, then information becomes misinformation, and the usefulness of the system is reduced. The combination of the right criteria to select and filter data and easy-to-read graphic metaphors for presenting it adds up to an effective decision-making tool.
The ability to correlate data from multiple different sources is important because it opens up the possibility of much more informative monitoring: beyond the mere confirmation of system status as OK or not OK, it allows us to observe system status data in a real-world context. As a very simple example, take the correlation between RF signal degradation at a terrestrial transmitter site against weather data for the location: by itself, the signal degradation would trigger an alarm in a ‘dumb’ monitoring system, and engineers would have to investigate, but cross-correlated with weather data showing very heavy rainfall, the RF degradation can be understood as transient, self-correcting, and not justifying action by engineers. A smart system could intelligently make this inference, and avoid raising a spurious alarm under these conditions.
One of Bridge Technologies’ most successful correlations to date has been between the transport stream continuity counter and the inter-arrival latency (IAT) of packets. This created for the first time the capability to see packet loss of UDP packets over an IP network and at the same time a patented visualisation technology called the MediaWindow.
Correlation is a very interesting field, and when used in our microAnalytics system, which gathers complex data sets from viewer devices, this type of heterogeneous correlation opens up the possibility of exploring such topics as: what type of phone the customer is using compared to the type of content viewed; or, which provider is the customer on, and does the content consumption pattern vary by provider? This type of correlation opens up very clear new ways of analysing behaviour and this will be very valuable to operators.
These types of analysis take users way beyond what alarms can tell them, into a new realm of data exploration. The monitoring systems already have the ability to deliver the data, and new presentation layers are in the pipeline that will give customers meaningful insight into the new patterns and correlations.
The presentation layer is where the most exciting developments will be. It’s of little use to an operator to be sitting on a mine of information, without the ability to extract it and exploit its value. Real-time visualisation tools that allow operators to ‘see through’ the data from any angle will take the digital monitoring system way beyond the status of an engineering tool, and make it a key business development resource.
Some of this functionality will be used by operators in an elective analysis; they will use the visualisations to analyse consumption patterns against market segmentation, geographical breakdown, mobility data and many other criteria. The data and the visual analysis for this will be available in real time, but it will also be possible to scroll through the data on a historical timeline of any length, with visual analysis to understand correlations. This kind of analysis exercise will allow operators to take a considered and extremely well informed approach to forward planning.
But the visualisation tools in the presentation layer will also find their way into real-time status displays in the mainstream of minute-by-minute monitoring activity. To deliver the most value here, an intelligent ability to present context-sensitive variations of the display is vital. In a normal status overview operators will be able to see which kind of viewers are looking at what, with which providers, on what kind of bandwidth, and what kind of technology mixture is in use. Error rates are displayed in a graphical way, and usage differentiations plotted as a group of easy to interpret curves. The data that feeds this overview is also available for statistical analysis at any time.
But the configuration of the monitoring display would change intelligently if some of these correlations point to dramatically increasing error rates in one particular set of the correlation; more detail will be shown on this area, to shed a more detailed light on what’s happening there. In this close-focused view, a more nuanced display of patterns and trends will be available to the engineer, and the system will have drawn attention to an area of concern while providing much more information in a readily-appraised way. This kind of content sensitivity (monitoring by exception) is very open-ended and customisable to reflect the user’s infrastructure and operating conditions, and it’s extremely valuable.
To explore the data, engineers can log in with a standard web interface, go into more detail in the database to pinpoint a correlation pattern of interest, and even get data from individual devices if really needed. But by with the visualisation tools in the presentation layer they will be able to track and understand what's happening much faster than by any other method.
This capacity is exceptionally useful in every kind of digital media situation. With the ability for example to look at RF data with all the different data types inter-correlated in real time, engineers will very easily be able to see trends, and engineering departments that want to keep a close eye on behaviour patterns at particular sites will have a much more comprehensive grasp of the situation.
Bridge Technologies introduced some of this technology at IBC 2014 in a technology framework that allows the addition of more information into widget based displays, with the ability to showcase third-party data into the same display technology. Customers will be able to add weather data, mapping information, and other sources of data into their correlations to create monitoring and analysis environments that really deliver far more than alarms ever could, and allow a new proactive approach to using monitoring.
