What a Wonderful Piece of Work is Opus

When we talk of free software, we typically think of things like GNU/Linux, Apache or Firefox. But one aspect that often gets overlooked is that of multimedia codecs. There's a good reason for this: most of them are patent-encumbered, which...


When we talk of free software, we typically think of things like GNU/Linux, Apache or Firefox. But one aspect that often gets overlooked is that of multimedia codecs. There's a good reason for this: most of them are patent-encumbered, which makes using them with free software hard – well, hard if you want to do it legally. In practice, most people have employed implementations of dubious legality, and the licensors have taken the sensible view that they are hardly losing millions from this kind of activity, and have turned a blind eye.

However, that's obviously a deeply unsatisfactory state of affairs, from multiple viewpoints – legal, ethical and technical. That has led to efforts to create unencumbered codecs for use in free software – things like Ogg Vorbis.

Apparently, there's been another major project aiming to do the same, although to my shame I hadn't really registered it until now. It's called Opus, and it's just been standardised as RFC 6716:

There was both skepticism and outright opposition to this work when it was first proposed in the IETF over 3 years ago. However, the results have shown that we can create a better codec through collaboration, rather than competition between patented technologies. Open standards benefit both open source organizations and proprietary companies, and we have been successful working together to create one. Opus is the result of a collaboration between many organizations, including the IETF, Mozilla, Microsoft (through Skype), Xiph.Org, Octasic, Broadcom, and Google.

That's impressive (not least having Microsoft on board, albeit as a result of its Skype acquisition). Equally so are the specs:

Unlike previous audio codecs, which have typically focused on a narrow set of applications (either voice or music, in a narrow range of bitrates, for either real-time or storage applications), Opus is highly flexible. It can adaptively switch among:

Bitrates from 6 kb/s to 512 kb/s
Voice and music
Mono and stereo
Narrowband (8 kHz) to Fullband (48 kHz)
Frame sizes from 2.5 ms to 60 ms

Most importantly, it can adapt seamlessly within these operating points. Doing all of this with proprietary codecs would require at least six different codecs. Opus replaces all of them, with better quality.

An accompanying graphic shows just what they mean: Opus is claimed to produce better quality output than Vorbis, AAC and MP3, and yet span applications from the bottom to top end. I've not heard it in action other than on the audio sample page, so I can't corroborate this, but I imagine it's true – it would be easy to disprove, and would make its creators look stupid were it not the case.

Talking of creators, these are employed by Mozilla – so kudos to all concerned:

Keeping the Internet an open platform is part of Mozilla's mission. When the technology the Web needs doesn't exist, we will invest the resources to create it, and release it royalty-free, just as we ask of others. Opus is one of these technologies.

Mozilla employs two of the key authors and developers, and has invested significant legal resources into avoiding known patent thickets. It uses processes and methods that have been long known in the field and which are considered patent-free. As a result, Opus is available on a royalty-free basis and can be deployed by anyone, including other open-source projects. Everyone knows this is an incredibly challenging legal environment to operate in, but we think we've succeeded.

"Challenging" is something of an understatement: once again, patents have contrived to ruin this field by making it hard to create innovative codecs without being sued for alleged infringement of patents. There's an interesting page listing which groups have made patents available to the Opus project, and under what terms. Here's the summary:

You can encode or decode Opus-compatible streams for any purpose at no cost.

You can integrate the reference Opus encoder and decoder in any application, program, or product, even commercially, at no cost.

You can create your own compatible implementations of the Opus specification and give them away or sell them.

You may not do these things if you engage in Opus-related patent litigation against any user of Opus.

I suspect that last clause will become more common over time, since it's a great way of punishing those who abuse the patent system and yet doesn't involve more litigation.

Since Opus is still quite new, support is limited: Firefox, GStreamer, FFMpeg, foobar2000, K-Lite Codec Pack, and lavfilters, with support due in VLC, rockbox and Mumble. However, it does form a key part of a major IETF initiative called WebRTC:

Opus is primarily designed for use in interactive applications on the Internet, including voice over IP (VoIP), teleconferencing, in-game chatting, and even live, distributed music performances. The IETF recently decided with "strong consensus" to adopt Opus as a mandatory-to-implement (MTI) codec for WebRTC, an upcoming standard for real-time communication on the web. Despite the focus on low latency, Opus also excels at streaming and storage applications, beating existing high-delay codecs like Vorbis and HE-AAC. It's great for internet radio, adaptive streaming, game sound effects, and much more.

All-in-all, Opus does sound rather splendid. And even better, there's more to come:

Opus is the first state of the art, free audio codec to be standardized. We think this will help us achieve wider adoption than prior royalty-free codecs like Speex and Vorbis. This spells the beginning of the end for proprietary formats, and we are now working on doing the same thing for video.

That's really great news, since the lack of a really good, widely-used unencumbered video codec becomes more of a problem for open source by the day. Let's hope the Opus video plan, er, works.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

"Recommended For You"

Why Google Isn't Evil (Today, at Least) Wikimedia mulls support for 'patent-encumbered' video file format