Tracking your assets

In today’s multi-channel world keeping track of where your media assets are used is of paramount importance. Making sure that your intellectual property is only used in authorised places, times and formats is now a major challenge. James Nuttall talks to Jeremy Bancroft, CEO of Advanced Broadcast Technology, a division of OmniBus Systems, to find out about a powerful new monitoring tool: fingerprinting.

  • E-Mail
By  Daniel Anderson-Ford Published  December 9, 2001

Tracking your assets|~||~||~|In today’s multi-channel world keeping track of where your media assets are used is of paramount importance. Making sure that your intellectual property is only used in authorised places, times and formats is now a major challenge. James Nuttall talks to Jeremy Bancroft, CEO of Advanced Broadcast Technology, a division of OmniBus Systems, to find out about a powerful new monitoring tool: fingerprinting.

It was 100 years ago that criminal scientists in England first discovered that each person’s fingerprints are unique. A match of loops, whorls and ridges means a positive and undeniable identification of a suspect. Media fingerprinting takes that simple idea and applies it to media assets, using unique sequences of elements such as luminance and colour, which can later give a completely reliable identification of the material.

Jeremy Bancroft was one of the founder directors of OmniBus Systems, a leader in transmission and news automation. “Asset management became a massive talking point a few years ago,” he told me. “As automation became more widespread, and particularly with the advent of multi-channel television, we had to keep track of large amounts of video.

“In a transmission automation system this is a relatively simple problem to address,” he continued. “The material may be on video tape, in one of a number of servers, or on data tape, but it will be in a single, fixed form. All you have to do is make sure it is available for transmission comfortably in advance. In practice this is not as simple as it sounds, but it is a clearly defined problem.

“But what happens when the possibility of post-production is introduced?” Bancroft asked. “This is an everyday occurrence in news, with incoming and library material constantly being re-purposed for different bulletins, continuing stories and reviews. Material goes out of the managed environment of the automation network, is edited, and comes back as a new asset. How do you link this with its source material?”

This is one of the biggest challenges in asset management in news. Some material in the archive will have been shot by the broadcaster’s own crews; the rest will have been contributed by freelancers, news agencies or libraries. This external material will probably be subject to licensing restrictions, including how many times the clip can be broadcast, and what fee is payable.
The initial OmniBus project was set up to look at ways of tracking media, which has been through an external process such as editing or graphics, to enable the new asset to be reconciled with the source material.

At that time digital watermarking was being developed by a number of manufacturers. This adds a code to the digital video stream, usually corresponding to an asset number and maybe the date of creation. Proponents of digital watermarking systems claim that their system is invisible to the viewer.

“We were concerned about an invasive procedure like watermarking,” Bancroft told me. “Our customers – broadcasters – were unhappy about introducing an additional signal process to add the watermark, and were unconvinced of the absolute transparency of the watermark.”

These are two important considerations. As well as imposing more digital signal processing electronics, the nature of watermarking demands that it is added at the time of ingest, making it difficult to implement where there is an extensive library of existing material.
The developers of digital watermarking systems are also emphatic that the code remains invisible to the viewer, but there are inevitably concerns that it might not remain transparent through all of the processes to which a video signal might pass, including concatenation of digital formats through storage, editing, delivery and transmission systems.

Recent trials by the European Broadcasting Union suggest that no more than three watermarks can be added to a signal before it becomes visible, and a single “high-energy” watermark – used to ensure readability where the programme will be highly processed or compressed – is visible after a single impression. This could be a significant problem where source material is subsequently edited to create new assets.

Of course there are important applications for digital watermarking, not least when you want to identify a particular copy of the material to trace piracy. The asset management application is significantly different, though, and that is why Jeremy Bancroft and his team elected to develop a completely non-invasive system, one which relies on the content of the material itself to identify it. Because it looks for and records the unique shape of the picture, it quickly became known as Media FingerPrinting.
||**||The Fingerprint|~||~||~|
Like many clever ideas, Media FingerPrinting is essentially very simple. The video frame is divided into 192 regions. In each the total values of red, green and blue are added. If the result is above a template value then the bit corresponding to that region is set at 1, if it is below then it is set to 0. The 192 bits are formed into a 24 byte sequence, and a further four bytes for universal time code are added.

In moving images the amount of change between adjacent frames is then measured. Anything above a preset threshold – that is, when there is a significant change in picture content – triggers a new fingerprint. Typically this occurs as often as five times a second, but it does depend on the nature of the content.

It is extremely unlikely that a single fingerprint could be relied upon to identify a media asset. It is the sequence and timing of a series of fingerprints which provide the unique match.

According to Jeremy Bancroft “we have found that the minimum duration of typical material needed to provide a unique match is four to six seconds, although in some cases if could be just a few frames.”

Because there are instances when the soundtrack is as important as the picture content – dubbed versions, commercials which use different voiceovers with identical pictures, or material for which the pictures are licensed but the commentary is protected – there is a companion audio fingerprint technology. This uses an equally simple and robust technique, and creates the same sort of data rates.

The data rate is one of the most impressive things about this technology. An hour of material generates only one to two megabytes of fingerprint data, which is stored alongside the rest of the archive’s metadata. Because it is dependent upon the content of the material, the fingerprint size is constant whether you are analysing HDTV or Real Video. And, as we will see when we look at wider applications, it is a simple matter to transmit the fingerprint, even over the Internet.

For the application, which was the original driver behind the development of Media FingerPrinting – tracking media assets which are repackaged into new assets – a simple database search is performed. As the new asset is ingested into the system it is fingerprinted and the file compared with the database. Because fingerprint files are very small, this is a simple and quick operation.

What is significant about Media FingerPrinting is that it is quite resilient to changes in the picture content. An overall change in the colour balance, perhaps because the asset is being used as a background under a tint, does not affect the way in which the relative colour levels change, and so would probably be identified by its fingerprint. Similarly, adding captions or borders does not affect the fingerprint, so its use would still be detected.

Most important, because the fingerprint is based on a grid evenly distributed across the frame and independent of the picture’s resolution, moving between video formats does not affect detection at all. The source material might have been digitised and fingerprinted at Digital Betacam or DV resolution; the edited version might use a completely different compression scheme; storage might be on servers with yet another format; it might even have been through a standards converter. It is the picture content which generates the fingerprint, not the absolute digital signals.

All of these attributes have led to a number of additional applications for the technology, which are of real importance to broadcasters today.

An extension of the internal asset tracking system is to use fingerprinting to police against piracy. A broadcaster or producer who suspected an unauthorised transmission of its programming could confirm it by comparing the off-air signal with the database. Again, because the fingerprint is not tied to a particular format or digital stream, but the content of the video and audio, it is equally accurate with off-air signals.

A broadcaster who has invested heavily in, say, the rights to a major sporting event will be unhappy to find it distributed via web streaming by a pirate. Even a 33kB Windows Media stream will generate a fingerprint which matches the original. Using web crawling techniques broadcasters can quickly identify unauthorised retransmissions and web streaming, and equally quickly move to stop it and protect the revenue stream.
||**||Remote Monitoring|~||~||~|
There is another group of applications for broadcasters concerning perfectly legal transmissions. One of the challenges of multi-channel transmission centres is in monitoring off-air signals to confirm that the right material is going to the right delivery system.

By fingerprinting the transmission output, then fingerprinting and comparing the off-air receiver, you get a completely automatic, completely reliable means of checking that the signal is reaching the viewer. As soon as the fingerprints do not match an alarm can warn the operator to look for downstream faults.

This is also the solution to a previously impossible challenge: monitoring the off-air signal of remote broadcasts. Without investing in expensive return video paths, international broadcasters have no way of knowing what is actually being transmitted outside of their immediate area. A Gulf regional broadcaster based in, say, Dubai, would have no confirmation of what was reaching the audience in, say, Qatar.

This is where the very compact nature of fingerprint files becomes a tremendous benefit. A fingerprint stream can be comfortably delivered via the Internet, needing only around 5kB/s. In the remote area, the broadcaster simply sets up a receiver and another fingerprinting station. The hardware, by the way, is a 1U PC, so this is not a major space requirement.

The remote unit generates a fingerprint from the off-air reception, works out the required time delay, and compares it with the reference from the broadcaster. If the two do not match, it sends an alarm back to the transmission centre. More than just an alarm, it also sends back thumbnails of the wrong output to help the broadcaster’s engineers to identify where the fault lies. Even an incorrectly set aspect ratio converter will be spotted this way.

The remote fingerprint device can also be switched by the transmission automation to the reverse mode during local opt-outs. If the distant service should be showing something different, perhaps during a commercial break, it will warn the transmission engineers if it is still showing what is leaving the master control.

Advertisers, too, can benefit from fingerprinting. Agencies can have their own fingerprint systems to generate automatically an as-seen log, which they can check against the broadcaster’s as-run log.
They can also use it as an analytical tool when looking at the competition. A manufacturer of, say, soft drinks might want to estimate the advertising spend of its rival. Because the fingerprint is derived solely from the content of the material it is perfectly possible to capture the rival’s commercial, then track every time it is transmitted. Quickly, and without expensive personal monitoring, the campaign details of a rival can be built up, and its effectiveness checked.

“To be really useful, a media asset management system has to provide a reliable means of tracking the material, whatever happens to it,” concluded ABT’s Jeremy Bancroft. “Media fingerprinting is resilient to quite significant changes in the format and content which makes it reliable; it is completely non-invasive so there are no quality of service issues; it can be retrospectively applied to archive material at any time; and it is simple to implement and operate, and relatively inexpensive. We believe any asset management system will benefit from the addition of fingerprinting.”

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code