The secret of smart plugins: AI-powered tools for mixing, mastering and more
Table of Contents
Now well into the 2020s, we’re awash with ways to make release-quality music at home, via user-friendly DAWs, expansive sample libraries and fine-tuning mix plugins. But, despite access to tools that enhance audio and tackle the mixing and mastering process solo, these disciplines still call for some hefty prerequisite knowledge.
While learning the intricacies of music production is something we’re always keen to promote on MusicRadar, companies like Zynaptiq and iZotope noticed that for the many time-devouring processes, the application of AI could not only provide footbridges over the numerous pitfalls of production, but also solve issues that human beings found a challenge.
“The terms ‘AI’ and ‘Smart’ are really primarily used for marketing purposes, which is useful as the terms convey the product idea very clearly on a level of intent,” Denis Goekdag, Zynaptiq’s CEO explains to us. “You can read it as ‘we aim to make a better solution that takes some of the burden of your task off of you by means of state-of-the-art statistics software tech from the field of artificial intelligence’.
Software can do things analogue never could
This helps clarify, too, that the software doesn’t aim to reproduce analogue gear or old-school workflows etc, which was all the rage before companies like Zynaptiq started pushing the use of AI/smart stuff from 2012 onwards. Software can do things analogue never could, and allows imagining solutions that were simply unimaginable even just 20 years ago.”
How smart plugins think
Today, the release of production software which houses deep-seated machine learning or features a handy AI assistant, pulling the strings, is something of an everyday occurrence. Take Sonible’s widely beloved Smart plugin suite.
This popular range spans many mixing applications, each of which are finely tailored to intelligently hone in on particular audio issues. Among their wares, the extraordinary Smart: EQ 3 and the content aware Smart: Limit, each of which are directed by deep-coded virtual thought processes to fulfil their aims.
Alexander Wankhammer, Sonible’s CMO and co-founder, explained to us how the company first took their fleeting first steps into AI. “Our first software product ‘Frei:Raum’ was released in January 2015. It already had the option to automatically correct the spectral deficiencies of a signal by observing its spectral and temporal characteristics.
Any system that is capable of performing tasks normally requiring human intelligence can per definition be called an ‘AI-based system’
At this time, we were mainly using ‘classical’ statistics-based machine learning algorithms, though later updates of our Smart: Filtering technology started to incorporate deep learning (deep neural networks).” Alexander elaborates on deep learning, and tells us that this is typically what people are referring to when discussing AI, confirming Denis’s assessment: “Any system that is capable of performing tasks normally requiring human intelligence can per definition be called an ‘AI-based system’ – no matter if some other machine-learning-based approach is used to solve a certain task.”
Alexander goes on to tell us how the deep learning algorithms within Smart: EQ 3 are actually programmed. “[Smart: EQ 3] uses a system that is mainly trained by huge amounts of data. In the case of Smart: EQ 3, the system learned to transform ‘bad data’ (eg signals with spectral deficiencies) into ‘good data’ (eg signals with a nice spectral balance).
To do so, we used deep learning (in our case a specialised convolutional neural network architecture) and presented a spectro-temporal representation of the bad data to the input of the network. We then defined the ‘good data’ as the target for the network’s output. By doing this thousands and thousands of times, the network learned how to correct problems in bad data samples. Once a network has been trained, it’s basically a black box doing its thing.”
The cat question
This same principle of intensely hammering the algorithms into shape beforehand lay at the heart of Zynaptiq’s stable. Denis Goekdag explains: “The network might learn to output ‘CAT = TRUE’ when it is presented with a picture of a cat, and ‘CAT = FALSE’ if the picture presented to it contains no cat (but maybe a beer glass).
Labyrinthine neural networks can effectively target problems that human brains would struggle to isolate
In simplified terms, you would train the network to achieve this by showing it 1 million cat pictures (and stating that these pictures should result in TRUE being output), and 1 million pictures without a cat (which should give FALSE as output). At Zynaptiq, we use pattern recognition in many of our products; in source separation products like Unmix: Drums, for example. It is used to figure out which parts of the input spectrum are drums, and which aren’t.” From such technically straightforward beginnings, labyrinthine neural networks can be forged, which can effectively target problems that human brains would struggle to isolate.
Both Sonible and Zynaptiq’s intelligent tools have been undeniable trailblazers when it comes to the widespread understanding of how AI-assistive technology can dig deeper, more quickly, into mix issues. Recent, solid, releases from the likes of Oeksound, Baby Audio and Soundtheory have all taken their cues from their respective stables – leaning on painstakingly programmed models in their role as the ultimate time-saving, mixing problem solvers for this generation of producers.
1. Sonible Smart:EQ 3
Platforms: Mac/PC | Price: £107 | Buy (opens in new tab)
At the summit of Sonible’s respected Smart series, sits Smart: EQ 3. This perceptive software carefully considers a sound or full mix’s characteristics, and effectively EQs it to perfection. Its intelligent cross-channel processing allows it to understand how multiple channels are affecting each other.
Platforms: Mac/PC | Price: £169 | Buy (opens in new tab)
One of our favourite plugins of recent years, soothe2’s sharp mix detective work allows it to be considered within the bracket of ‘intelligent’ in our view. Scanning your mix, soothe2 isolates problematic resonances and gives them the boot, and saves time laboriously notching the EQ by hand.
Platforms: Mac/PC | Price: £49 | Buy (opens in new tab)
Sitting on the boundary between ‘meticulously programmed’ and ‘intelligent’, Mixroom takes a look at the picture of your sound on a per-channel basis or the master bus, and applies clever EQ filters to help bring out your track’s oomph. The ability to import reference tracks for it to try and emulate is a useful touch, too.
Platforms: Mac/PC | Price: £95 | Buy (opens in new tab)
The ultimate go-to for vocal manipulation, iZotope’s Nectar 3’s 12 vocal processing modules, unmasking controls and super intelligent Vocal Assistant will set EQ nodes and scrub out plosives and sibilants. The Unmask function lets the plugin scan the whole mix and push out intrusive frequencies from any other channels affecting the vocal’s clarity.
5. Zynaptiq Adaptiverb
Platforms: Mac/PC | Price: £180 | Buy (opens in new tab)
One of the clearest examples of the benefits of AI in music production, Adaptiverb adapts its reverb-creating algorithm to the source audio, and smartly preserves the definition and clarity of the mix, applying its reverb subtly and with more nuance than a traditional application would. Genius – particularly its Bionic Sustain Resynthesizer
Platforms: Mac/PC | Price: £150 | Buy (opens in new tab)
Another EQ game-changer. The nearly self-aware Gullfoss takes complete control over your mix and uses five controls to enhance and sculpt your mix. Via its computational auditory perception, Gullfoss listens and prepares the audio to isolate what aspects of the mix are competing for attention, and there’s little it can’t fix.
7. iZotope Neutron 4
Platforms: Mac/PC | Price: £171 | Buy (opens in new tab)
While pre-determined algorithms form the foundation of many of today’s most perceptive mixing plugins, often their capabilities can be expanded by the learning engendered by new data. An interesting example of an adaptive, self-refining algorithm can be found in iZotope’s Neutron 4. The company’s premiere mixing suite features a gamut of algorithms designed to continually refine what they suggest to the user.
The Mix Assistant also uses ‘Inter-plugin Communication’ to speak to other iZotope plugins being used on other tracks in a mix, to understand what they’re specifically doing to the respective frequencies and tones of those channels. This helps Neutron 4 take the reins and intelligently adjust each track’s levels, bringing focus to the sounds that should be foregrounded and hiding those that need to recede into the mix.
Platforms: Mac/PC | Price: £95 | Buy (opens in new tab)
One of the modern mastering engineer’s most trusted tools, Ozone 9 can automatically create EQs that match the frequency spectrum of your project, as well as intelligently handle loudness levels on the fly. Aside from its advanced brain, Ozone 9 also provides a huge array of fine-tuning options for the skilled engineer.
2. Zynaptiq Master Bundle
Platforms: Mac/PC | Price: £450 | Buy (opens in new tab)
A suite of three high-powered AI mastering plugins from the innovators at Zynaptiq. The three plugins – Unfilter (opens in new tab), Intensity (opens in new tab) and Unmix: Drums (opens in new tab) – rectify EQ, beef up the loudness and clarity as well as underline your drums respectively. Manual control is available too, but you’ll be impressed by the suite’s ‘auto-magic’ abilities.
3. Leapwing DynOne
Platforms: Mac/PC | Price: £179 | Buy (opens in new tab)
With its intelligent and very discerning attack and release settings, DynOne knows how to make the best from your mix from the get-go. Leapwing’s smart dynamics controller also puts five bands of quality parallel compression at your fingertips, as well as transparent and adjustable filters to tame your track.
4. eMastered
Platforms: Browser-based | Price: £14 per month | Buy (opens in new tab)
Should you want to turn over your mix entirely to an AI-mastering engineer, then you can do a lot worse than the skillful eMastered. Developed in tandem with a community of Grammy-winning engineers, the platform allows users to upload reference tracks to more precisely build a mastered final product with impact.
5. Cloudbounce
Platforms: Mac/PC | Price: £7 per track | Buy
Cloudbounce uses its internal algorithms to assess the best EQ fixes and final dynamic balance for your track. Cloudbounce also allows users to dial in aspects such as volume, brightness, warmth and bass by hand, as well as choosing the appropriate genre to tailor the master to. It’s now available as a standalone desktop app.
6. LANDR
Platforms: Browser-based | Price: £7.50 per month | Buy (opens in new tab)
One of the major names in online mastering. LANDR is driven by its sophisticated Synapse AI engine. Synapse is hugely adept at deciding how to EQ, compress, enhance and define anything you throw its way. Launched in 2014, LANDR continues to grow its membership – and its capabilities – with each passing year.
Platforms: Mac/PC | Price: £7.45 per month | Buy (opens in new tab)
An acclaimed collection of uniformly smart tools that are designed to handle such processes as noise removal, voice deepening, auto-EQing, reverb removal and de-clipping. Accusonus’ swiss army knife of sonic sculptors provide a perfect control centre for audio repair, with the Audio Clean-Up Assistant granting a top-down view of its wares.
Platforms: Mac/PC | Price: £95 | Buy (opens in new tab)
Perhaps the closest to being the ultimate example of AI-as-audio engineer. RX9 is well-equipped to both judge and fix even the muddiest, messiest and most low-res audio. With a whole armada of smart plugins in its stable, the Repair Assistant is continually on hand to give instant feedback on what’s been detected, and action needed. It’s not been stumped yet.
3. Adobe Audition CC
Platforms: Mac/PC | Price: £19.97 a month | Buy (opens in new tab)
Adobe’s Audition CC applies the same remarkably ambidextrous approach that the company previously applied to images (via Photoshop) to audio. Audition offers multitrack, spectral and waveform displays of your source audio, and the current incarnation brings DeReverb and DeNoise plugins. These can cleverly suss out how best to apply themselves to the audio.
4. Accentize Voice Gate
Platforms: Mac/PC | Price: £120 | Buy (opens in new tab)
Excelling at its sole objective of removing the audio from audio served up to it, Accentize’s Voice Gate uses its artificial neural network to determine any and all human speech in the audio and, well, eradicate it. Accentize are keen to update their algorithm, so if you come across any vocal-sounds that Voice Gate hasn’t detected, feel free to email them to refine their algorithm further!
Platforms: Mac/PC | Price: £107 | Buy
The debut AI product of Sonibile, Frei:Raum detects disturbing resonances and frequency notches, and cleverly gets to work correcting them, while its deep EQing abilities enable it to swiftly compensate for bad room acoustics or dialogue recording hiccups. Frei:Raum sets a precedent for intelligent, ever-learning plugins that Sonible continues to push.
6. Zynaptiq Repair Suite
Platforms: Mac/PC | Price: £550 | Buy (opens in new tab)
Also known as the ‘Un’ series, Zynaptiq’s Unveil, Unfilter and Unchirp present smart ways to get rid of resonance, roll-offs and poorly encoded audio amongst other things, enhancing and sprucing up whatever is thrown their way. Driven by a desire to improve at what they do, Zynaptiq’s audio restoration suite is a reliable future-proof audio-repair arsenal.