Scene-Level Television Metadata — Tagging TV — is The New Oil in the Industry

written by: Richard Kastelein

Share this Article

Historically, TV Metadata has been used to supply Electronic Programme Guides (EPGs) and therefore has been adequate for description at a show level. Typically when the industry talks about TV metadata, they talk about snippets of information and images that is provided by companies such as Rovi and Gracenote that can be used for the descriptive editorial information, images and multimedia on one show as a whole.

But what about at the scene level? And why is scene level metadata — or Tagging TV — the new oil? It's now all about applying metadata not just to a whole piece of content, but individual chunks within it, such as a movie scene or song. Of course, this can be relevant both for production and search/discovery... but the real value lies in providing contextual data on the second screen — whether that be curated or automated, factual or commercial.

Over $200 billion is being spent annually in global TV ad spend — but, viewers are increasingly watching TV along with their portable devices, resulting in more and more viewer attention directed away from TV spots and towards their laptops, tablets and smartphones. This means those 30 second linear TV spots that agencies convince brands are worth millions is very likely to become less valuable in the future. As the second screen companion mobile device draws attention from commercials they will become hugely important in the future disruption of the broadcast industry... not only because viewers are drawn there to discover and share their content, but also the simple fact is — it's bi-directional, like the web, and will give IP metrics that the old school TV value chain could only dream about in the past — giving a deeper understanding of consumers and their behaviour.

Let me give you some links that make my case.

Is there life after the 30-Second Spot? Yes, and scene-level tagged TV metadata is the key.


Think about it. When Jennifer Aniston shows up on the red carpet at Cannes wearing Lanvin of Paris - and that dress has been tagged as metadata in the timeline of the show — that tag can then become a trigger for an action on the second screen. Such as, 'Save for Later and Buy' or 'Learn More'. When a Porsche shows up in a movie scene — perhaps it can trigger a second screen call to action by offering a free test drive? Perhaps even a different model can be shown depending on whether it's known more about the demographic of the user — throwing in more targeted advertising to boot.

Now this all sounds great and even perhaps easy. But it's not. And that's why it's the new oil.

The lack of standardisation in the area of television programme information (or television "metadata" as it is called) poses increasing problems right through the television value chain. Everybody loses, from content producers, broadcasters, advertisers and network operators to television viewers. Production companies are chock full of creatives — they don't find this extra work appealing in any sense and are not doing it. There's another chance to create contextual temporal tags at the broadcaster level — as they buy the scripts. But the infrastructure and common standards are just not there yet in the playout systems. So in many cases, it's third parties that are trying to solve this problem outside of the old school.

There are, essentially two ways to tag video entertainment. Curated and automated. And both have their pros and cons. Manually tagging millions of programmes and shows is going to take a decade of Mechanical Turks but this really offers up the best metadata. There are companies that are using technologies to automate the process such as Speech to Text, Video Recognition technologies, Audio Fingerprinting, Natural Language Processing techniques, and when available, Closed Caption data to create temporal tagging of content to provide a clear view of what is happening when within a piece of video.

Probably the best way is to do a combination of both. Automated then moderated/curated by humans.

There have been attempts to create common XML standards for the industry around EPG metadata — three broadcast industry initiatives have been started to tackle the problem. The earliest was DVB-SI, which is an integral part of the digitalisation of television in Europe and other regions of the world. Two other especially promising television metadata standards that build on the precedent of DVB-SI will soon be finalised. TV-Anytime, the first of these, addresses the needs that arise from high volume low cost storage (e.g. Personal Video Recorders (PVRs) and Video-on-Demand (VOD) services). The second, MPEG-7, is much broader in its scope, seeking to provide tools for describe all forms of multimedia content delivered by the broadest possible range of networks and terminals.

The Brussel's funded FP7 EU NoTube project aims to show how Semantic Web technologies can be used to connect TV content and the Web through Linked Open Data, as part of the trend of TV and Web convergence. They are focussing on BMF 2.0 (Broadcast Metadata Exchange Format), the rather outdated TV-Anytime, as an internationally agreed and accepted metadata schema in the TV consumer domain and another barely used but interesting egtaMETA from the commercial side for advertisements. NoTube is a European research project exploring the future of television in the ubiquitous internet than includes the BBC and IRT as well as a slew of university researchers from across Europe.

Essentially noTube will allow disparate metadata interoperability within the NoTube platform creating metadata transformations that are required to translate metadata of external sources to TV-Anytime. In the course of the metadata enrichment process in NoTube, additional metadata is then added to the TV-Anytime metadata sets, therefore pushing to that standard.

At Agora Media Group, the founders of we work with a number of commercial vendors in this space and are helping a number of broadcasters, cellcos, and production companies understand this new realm. Thanks  to others on the team - particularly Paul Johnson - who helped shape the ideas in this article! Feel free to contact us at This email address is being protected from spambots. You need JavaScript enabled to view it.



About the Author

Richard Kastelein
Founder of The Hackfest, publisher of TV App Market and global expert on Media & TV innovation, Kastelein is an award winning publisher and futurist. He has guest lectured at MIT Media Lab, University of Cologne, sat on media convergence panel at 2nd EU Digital Assembly in Brussels, and worked with broadcasters such as the BBC, NPO, RTL (DE and NL), Eurosport, NBCU, C4, ITV, Seven Network and others on media convergence strategy - Social TV, OTT, DLNA and 2nd Screen etc.

He is a Fellow of the UK Royal Society of Arts (RSA) and UK Royal Television Society (RTS) member.

Kastelein has spoken (& speaking) on the future of media & TV in Amsterdam, Belfast, Berlin, Brussels, Brighton, Copenhagen, Cannes, Cologne, Curacao, Frankfurt, Hollywood, Hilversum, Geneva, Groningen (TEDx), Kuala Lumpur, London, Las Vegas, Leipzig, Madrid, Melbourne, NYC, Rio, Sheffield, San Francisco, San Jose, Sydney, Tallinn, Vienna, Zurich...

He's been on advisory boards of TEDx Istanbul, SMWF UK, Apps World, and judged & AIB awards, Social TV Awards Hollywood, TV Connect & IPTV Awards.

A versatilist & autodidact, his leadership ability, divergent and synthetic thinking skills evolved from sailing the world 24000 miles+ offshore in his 20′s on sailboats under 12m.

He spent 10 years in the Caribbean media & boating industry as a professional sailor before returning to Europe, to Holland.

A Creative Technologist and Canadian (Dutch/Irish/English/Metis) his career began in the Canadian Native Press and is now a columnist for The Association for International Broadcasting and writes for Wired, The Guardian & Virgin. His writings have been translated into Polish, German and French. 

One of Kastelein's TV formats was optioned by Sony Pictures Television in 2012. 

Currently involved in a number of startups including publishing TV App Market online, The Hackfest and Tripsearch TV. As CSO for Worldticketshop he helped build a $100m company.

Other Sites

Social Media


About Us

TV Hackfest London 2013
TV Hackfest San Francisco 2014
M2M Hackfest 2013

Hackfest Twitter
TV App Market Facebook
TV App Market Linkedin
TV App Market Google+

Developer News
Enterprise Apps News

Digital Marketing News
Telecoms News
Cloud Computing News

This email address is being protected from spambots. You need JavaScript enabled to view it.
About us
Write for TV App Market