326 lines
17 KiB
HTML
326 lines
17 KiB
HTML
|
<!DOCTYPE html>
|
|||
|
|
<html lang="en-us">
|
|||
|
|
<head>
|
|||
|
|
<meta charset="utf-8">
|
|||
|
|
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
|||
|
|
<meta name="viewport" content="width=device-width,minimum-scale=1,initial-scale=1">
|
|||
|
|
<meta name="theme-color" content="#FF8E03">
|
|||
|
|
<meta property="og:title" content="HLE Audio Comes to Citra · Citra" />
|
|||
|
|
<meta property="og:site_name" content="Citra" />
|
|||
|
|
<meta property="og:url" content="https://citra-emu.org/entry/hle-audio-comes-to-citra/" />
|
|||
|
|
<meta property="og:description" content="Special thanks must be given to fincs and the rest of the 3DS community for their work reverse-engineering the DSP firmware. Without that work, Citra would not be this far with audio emulation.
|
|||
|
|
As of May 19th, 2016, Citra now has preliminary High Level Emulation (HLE) audio support! This means that users playing on Citra no longer have to listen to the deafening sound of silence in many titles. To get to this point was a huge reverse-engineering effort done by multiple people, with much of the reverse-engineering and the final implementation for Citra coming from MerryMage." />
|
|||
|
|
<meta name="description" content="Special thanks must be given to fincs and the rest of the 3DS community for their work reverse-engineering the DSP firmware. Without that work, Citra would not be this far with audio emulation.
|
|||
|
|
As of May 19th, 2016, Citra now has preliminary High Level Emulation (HLE) audio support! This means that users playing on Citra no longer have to listen to the deafening sound of silence in many titles. To get to this point was a huge reverse-engineering effort done by multiple people, with much of the reverse-engineering and the final implementation for Citra coming from MerryMage." />
|
|||
|
|
<meta property="og:type" content="article" />
|
|||
|
|
<meta property="og:image" content="https://citra-emu.org/images/banners/hle-audio-comes-to-citra.png" />
|
|||
|
|
<meta property="og:article:published_time" content="2016-05-20T18:04:00-05:00" />
|
|||
|
|
<meta property="og:article:tag" content="feature-update" />
|
|||
|
|
|
|||
|
|
|
|||
|
|
<link rel="icon" href="https://citra-emu.org/favicon.ico" />
|
|||
|
|
<link rel="shortcut icon" href="https://citra-emu.org/favicon.ico" type="image/x-icon" />
|
|||
|
|
<link rel="canonical" href="https://citra-emu.org/entry/hle-audio-comes-to-citra/">
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
<title>HLE Audio Comes to Citra - Citra</title>
|
|||
|
|
<link href="https://fonts.googleapis.com/css?family=Ubuntu|Dosis" rel="stylesheet">
|
|||
|
|
|
|||
|
|
<link rel="stylesheet" href="https://citra-emu.org/scss/style.min.css" />
|
|||
|
|
|
|||
|
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.1.1/jquery.min.js"
|
|||
|
|
integrity="sha256-hVVnYaiADRTO2PzUGmuLJr8BLUSjGIZsDYGmIJLv2b8=" crossorigin="anonymous"></script>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
|
|||
|
|
<script>
|
|||
|
|
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
|||
|
|
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
|||
|
|
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
|||
|
|
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
|||
|
|
|
|||
|
|
ga('create', 'UA-73966905-1', 'auto');
|
|||
|
|
ga('send', 'pageview');
|
|||
|
|
</script>
|
|||
|
|
|
|||
|
|
</head>
|
|||
|
|
|
|||
|
|
<body>
|
|||
|
|
<nav class="navbar navbar-default navbar-wrapper navbar-fixed-top">
|
|||
|
|
<div class="container">
|
|||
|
|
<div class="navbar-header">
|
|||
|
|
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar"
|
|||
|
|
aria-expanded="false" aria-controls="navbar">
|
|||
|
|
<span class="sr-only">Toggle navigation</span>
|
|||
|
|
<span class="icon-bar"></span>
|
|||
|
|
<span class="icon-bar"></span>
|
|||
|
|
<span class="icon-bar"></span>
|
|||
|
|
</button>
|
|||
|
|
<a class="navbar-brand" href="/"> </a>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div id="navbar" class="navbar-collapse collapse">
|
|||
|
|
<ul class="nav navbar-nav">
|
|||
|
|
<li class="index-1 first"><a href="/" title="Blog Posts">Blog</a></li>
|
|||
|
|
<li class="index-2"><a href="/download/" title="Download Citra">Download</a></li>
|
|||
|
|
<li class="index-3"><a href="/help/" title="Help">Help</a></li>
|
|||
|
|
<li class="index-4"><a href="/wiki/faq/" title="Frequently Asked Questions">FAQ</a></li>
|
|||
|
|
<li class="index-5"><a href="/game/" title="Game Compatibility">Compatibility</a></li>
|
|||
|
|
<li class="index-6"><a href="/wiki/home/" title="Wiki Homepage">Wiki</a></li>
|
|||
|
|
|
|||
|
|
<li class="index-7 dropdown">
|
|||
|
|
<a href="#" class="dropdown-toggle" data-toggle="dropdown"
|
|||
|
|
role="button" aria-haspopup="true" aria-expanded="false">
|
|||
|
|
Social <span class="caret"></span>
|
|||
|
|
</a>
|
|||
|
|
|
|||
|
|
<ul class="dropdown-menu">
|
|||
|
|
<li class="index-1 first"><a href="/rules/" title="Community Rules">Rules</a></li>
|
|||
|
|
<li class="index-2"><a href="https://community.citra-emu.org/" title="Forums">Community Forums</a></li>
|
|||
|
|
<li class="index-3 last"><a href="/discord/" title="Discord Server">Discord Chat</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
<li class="index-8 dropdown">
|
|||
|
|
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button"
|
|||
|
|
aria-haspopup="true" aria-expanded="false">
|
|||
|
|
Media <span class="caret"></span>
|
|||
|
|
</a>
|
|||
|
|
|
|||
|
|
<ul class="dropdown-menu">
|
|||
|
|
<li class="index-1 first"><a href="/screenshots/" title="Screenshots">Screenshots</a></li>
|
|||
|
|
<li class="index-2 last"><a href="https://www.youtube.com/channel/UC_dcdgzuapBtAY4ol3x-90Q"
|
|||
|
|
title="YouTube Channel">Videos</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</li>
|
|||
|
|
|
|||
|
|
<li class="index-9 last"><a href="https://www.patreon.com/citraemu" title="Patreon">Patreon</a></li>
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</nav>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div id="mainContainer" class="container" >
|
|||
|
|
<div class="row">
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="row row-fluid">
|
|||
|
|
<div id="content" class="col-xs-12 col-sm-12 col-md-9 col-lg-9 pull-right">
|
|||
|
|
|
|||
|
|
<a href="https://citra-emu.org/entry/hle-audio-comes-to-citra/">
|
|||
|
|
<div class="entry-embed-header">
|
|||
|
|
|
|||
|
|
<div class="entry-embed" style="background: url('https://citra-emu.org/images/banners/hle-audio-comes-to-citra.png')">
|
|||
|
|
<header>
|
|||
|
|
<h1>HLE Audio Comes to Citra</h1>
|
|||
|
|
</header>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="entry-meta" style="position: absolute; top: 5px; right: 20px;">
|
|||
|
|
May 20 2016
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="entry-meta" style="position: absolute; top: 45px; right: 20px;">
|
|||
|
|
feature-update
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
</a>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="entry-content">
|
|||
|
|
<p><em>Special thanks must be given to fincs and the rest of the 3DS community for their work reverse-engineering the DSP
|
|||
|
|
firmware. Without that work, Citra would not be this far with audio emulation.</em></p>
|
|||
|
|
<p>As of <a href="https://github.com/citra-emu/citra/commit/af258584d978f02d462743012491a273c61b067e">May 19th, 2016</a>, Citra now
|
|||
|
|
has preliminary High Level Emulation (HLE) audio support! This means that users playing on Citra no longer have to
|
|||
|
|
listen to the deafening sound of silence in many titles. To get to this point was a huge reverse-engineering effort
|
|||
|
|
done by multiple people, with much of the reverse-engineering and the final implementation for Citra coming from
|
|||
|
|
MerryMage. This undertaking has required many months of development but the end result brings sound to the masses.</p>
|
|||
|
|
|
|||
|
|
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
|
|||
|
|
<iframe src="https://www.youtube.com/embed/8LCUlyjvTJU" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<h2 id="technical-details-on-how-audio-works-for-the-3ds">Technical Details on how Audio Works for the 3DS</h2>
|
|||
|
|
<p>Audio processing and output is done by a specialised coprocessor. These kinds of coprocessors are called Digital Signal
|
|||
|
|
Processors (DSPs).</p>
|
|||
|
|
<p>Games that run on the 3DS need to communicate with the DSP in order to play audio. They do this by two ways: via the
|
|||
|
|
the <code>dsp::DSP</code> service and via a shared memory region. The <code>dsp::DSP</code> service provides service calls for initialization
|
|||
|
|
of the DSP hardware including firmware upload. The shared memory region is used for communication between the game on
|
|||
|
|
the CPU and the firmware on the DSP.</p>
|
|||
|
|
<p>In order to emulate audio, Citra must emulate the <code>dsp::DSP</code> service and also understand the layout of the DSP shared
|
|||
|
|
memory region. One must understand what writing to various addresses in the shared memory region does. One must also
|
|||
|
|
understand what happens between data being fed to the DSP firmware and audio coming out of the speakers.</p>
|
|||
|
|
<h2 id="early-reverse-engineering">Early Reverse Engineering</h2>
|
|||
|
|
<p>With this known MerryMage set out to trace reads and writes to shared memory that games did. She eventually ended up
|
|||
|
|
playing these back on hardware and figured out what the appropriate firmware responses were. This eventually lead to
|
|||
|
|
the early implementations of audio output that originally were shown in January by various users. Many other aspects,
|
|||
|
|
including ADPCM decoding, took a while to figure out.</p>
|
|||
|
|
|
|||
|
|
<div style="position: relative; padding-bottom: 56.25%; height: 0; overflow: hidden;">
|
|||
|
|
<iframe src="https://www.youtube.com/embed/1c_A7gpAZ8A" style="position: absolute; top: 0; left: 0; width: 100%; height: 100%; border:0;" allowfullscreen title="YouTube Video"></iframe>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<p>MerryMage had raw audio output working but she still didn’t understand how various parameters were applied to the audio
|
|||
|
|
by the firmware. It was at this point where she discovered that the firmware writes back what is output to the speakers
|
|||
|
|
into the shared memory region. This discovery made future work with audio effects much easier as bit perfect audio
|
|||
|
|
could be dumped as the 3DS produced it without any extra hardware. Having real hardware audio output on hand meant
|
|||
|
|
MerryMage could also apply signal processing techniques like system identification to figure out what internal
|
|||
|
|
processing the firmware does.</p>
|
|||
|
|
<p>With this newfound knowledge and this new set of tools MerryMage rapidly conquered what the firmware was doing.</p>
|
|||
|
|
<h2 id="time-stretching">Time Stretching</h2>
|
|||
|
|
<p>Emulation speed can vary a lot between games or even parts of games. To accomodate this, time streching was added as an
|
|||
|
|
audio enhancement. This post-processing effect adjusts audio speed to match emulation speed and helps prevent audio
|
|||
|
|
stutter. This is an effect completely separate from emulation and is only to alter and improve audio played back when
|
|||
|
|
the emulator is not going full speed.</p>
|
|||
|
|
<h2 id="future-plans-for-audio">Future Plans for Audio</h2>
|
|||
|
|
<p>Audio is still not complete! There are still a number of unimplemented features and accuracy improvements to have. Many
|
|||
|
|
of these features have been reverse engineered already but simply aren’t implemented. This includes reverb, delay, and
|
|||
|
|
other minor audio effects. Some features require more reverse engineering work, such as looped buffers and surround
|
|||
|
|
sound.</p>
|
|||
|
|
<p>While the black-box reverse engineering approach has served well so far, further improvements in accuracy can more
|
|||
|
|
easily be made by decompiling the firmware and perhaps implementing Low Level Emulation (LLE) audio. This comes with
|
|||
|
|
its own set of challenges especially as the DSP architecture is not well known and there is little documentation on it.</p>
|
|||
|
|
<p>Until then, we at Citra hope that everyone enjoys this initial HLE audio implementation!</p>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="entry-written-by">
|
|||
|
|
|
|||
|
|
<a href="https://community.citra-emu.org/u/merrymage">
|
|||
|
|
<img src="https://community.citra-emu.org/user_avatar/community.citra-emu.org/citrabot/120/28_1.png" class="avatar">
|
|||
|
|
</a>
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
<p>Written by <a href="https://community.citra-emu.org/u/merrymage">Merry Mage</a> on Friday May 20, 2016</p>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
<div class="entry-comments">
|
|||
|
|
<div id="discourse-comments"></div>
|
|||
|
|
<script type="text/javascript">
|
|||
|
|
DiscourseEmbed = { discourseUrl: 'https://community.citra-emu.org/', topicId: "38" };
|
|||
|
|
(function() {
|
|||
|
|
var d = document.createElement('script'); d.type = 'text/javascript'; d.async = true;
|
|||
|
|
d.src = DiscourseEmbed.discourseUrl + 'javascripts/embed.js';
|
|||
|
|
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(d);
|
|||
|
|
})();
|
|||
|
|
</script>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
<div id="sidebar" class="col-xs-12 col-sm-12 col-md-3 col-lg-3 pull-left">
|
|||
|
|
<div id="advertisement" class = "ad">
|
|||
|
|
<h3>Advertisement</h3>
|
|||
|
|
<ins class="ad adsbygoogle"
|
|||
|
|
style="display:block"
|
|||
|
|
data-ad-client="ca-pub-4126545610079023"
|
|||
|
|
data-ad-slot="4223809695"></ins>
|
|||
|
|
<script>(adsbygoogle = window.adsbygoogle || []).push({});</script>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="tagcloud hidden-sm hidden-xs">
|
|||
|
|
<h3>News Tag Cloud</h3>
|
|||
|
|
<ul>
|
|||
|
|
|
|||
|
|
<li><a class="taxonomy-citra-release" href="/tags/citra-release">citra-release</a></li>
|
|||
|
|
|
|||
|
|
<li><a class="taxonomy-feature-update" href="/tags/feature-update">feature-update</a></li>
|
|||
|
|
|
|||
|
|
<li><a class="taxonomy-progress-report" href="/tags/progress-report">progress-report</a></li>
|
|||
|
|
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div id="twitter" class="hidden-sm hidden-xs">
|
|||
|
|
<a class="twitter-timeline" data-tweet-limit="3" href="https://twitter.com/citraemu?ref_src=twsrc%5Etfw">Tweet Feed</a>
|
|||
|
|
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div id="footer" class="container">
|
|||
|
|
<div class="row">
|
|||
|
|
|
|||
|
|
<div class="col-md-2">
|
|||
|
|
<h1>Citra</h1>
|
|||
|
|
<a href="/entry">Blog</a>
|
|||
|
|
<a href="/download/">Downloads</a>
|
|||
|
|
<a href="/screenshots/">Screenshots</a>
|
|||
|
|
<a href="https://www.patreon.com/citraemu">Patreon</a>
|
|||
|
|
<a href="/donate/">Donate</a>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="col-md-2">
|
|||
|
|
<h1>Documentation</h1>
|
|||
|
|
<a href="/help/">Help Documents</a>
|
|||
|
|
<a href="/game/">Compatibility</a>
|
|||
|
|
<a href="/wiki/home/">Wiki</a>
|
|||
|
|
<a href="/wiki/faq/">FAQ</a>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="col-md-2">
|
|||
|
|
<h1>Blog</h1>
|
|||
|
|
<a href="/entry">News & Articles</a>
|
|||
|
|
<a href="https://citra-emu.org/index.xml">RSS 2.0</a>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="col-md-2">
|
|||
|
|
<h1>Social</h1>
|
|||
|
|
<a href="https://www.youtube.com/channel/UC_dcdgzuapBtAY4ol3x-90Q">YouTube</a>
|
|||
|
|
<a href="https://www.facebook.com/citra.emu">Facebook</a>
|
|||
|
|
<a href="https://twitter.com/citraemu">Twitter</a>
|
|||
|
|
<a href="https://community.citra-emu.org/">Forums</a>
|
|||
|
|
<a href="/discord">Discord</a>
|
|||
|
|
<a href="/chat">IRC</a>
|
|||
|
|
</ul>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<div class="col-md-2">
|
|||
|
|
<h1>Get Involved</h1>
|
|||
|
|
<a href="https://github.com/citra-emu/citra">GitHub / Source</a>
|
|||
|
|
<a href="https://github.com/citra-emu/citra/issues">Issues</a>
|
|||
|
|
<a href="https://github.com/citra-emu/citra/pulls">Pull Requests</a>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
</div>
|
|||
|
|
<div id="footer-bottom">
|
|||
|
|
<div id="footer-brand"></div>
|
|||
|
|
<div id="footer-legal">Copyright © 2024 Citra Emulator Project</div>
|
|||
|
|
</div>
|
|||
|
|
</div>
|
|||
|
|
|
|||
|
|
<script src="https://citra-emu.org/js/script.js"></script>
|
|||
|
|
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/baguettebox.js/1.10.0/baguetteBox.min.css" type="text/css" />
|
|||
|
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/baguettebox.js/1.10.0/baguetteBox.min.js" type="text/javascript"></script>
|
|||
|
|
|
|||
|
|
<script type="text/javascript">
|
|||
|
|
window.addEventListener("DOMContentLoaded", function() {
|
|||
|
|
baguetteBox.run('.is-img-preview');
|
|||
|
|
});
|
|||
|
|
</script>
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
</body>
|
|||
|
|
|
|||
|
|
</html>
|