{"id":69,"date":"2019-12-11T20:08:00","date_gmt":"2019-12-11T20:08:00","guid":{"rendered":"https:\/\/imitone.com\/blog\/?p=69"},"modified":"2024-05-03T18:05:27","modified_gmt":"2024-05-03T18:05:27","slug":"cause-for-calibration","status":"publish","type":"post","link":"https:\/\/imitone.com\/blog\/index.php\/2019\/12\/11\/cause-for-calibration\/","title":{"rendered":"cause for calibration"},"content":{"rendered":"\n<p><em>This post was transcribed from imitone&#8217;s kickstarter page. <a href=\"https:\/\/www.kickstarter.com\/projects\/evanbalster\/imitone-mind-to-melody\/posts\/2707712\">click here for the original post<\/a>, or <a rel=\"noreferrer noopener\" href=\"https:\/\/soundcloud.com\/evan-balster\/cause-for-calibration-imitone-devlog\" target=\"_blank\">click here for an audio version of this post.<\/a><\/em><\/p>\n\n\n\n<p>I&#8217;m calibrating! &nbsp;I&#8217;m calibrating!! &nbsp;I&#8217;m fine-tuning a training model, generating and analyzing billions of datapoints, calculating probabilities and building a time-critical pitch tracker capable of \u2014 ah \u2014 ah, wait, I shouldn&#8217;t get ahead of myself.<\/p>\n\n\n\n<p>I&#8217;ve been busy. &nbsp;Following the plan from our last update, I have devoted most of this year to research and development on the next generation of imitone&#8217;s technology \u2014 probably the most advanced pitch tracker in the world. &nbsp;A few weeks from now, it will be in your hands as part of imitone beta 0.10.<\/p>\n\n\n\n<p>Most exciting of all, it will be&nbsp;<em>evolving<\/em>.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/890\/a72335abbb13cbd94bae209c20324443_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043259&amp;gif-q=50&amp;lossless=true&amp;s=fc909857e929efb941bb78e10eefd5b7\" alt=\"\"\/><figcaption>The new imitone technology uses statistics and machine learning to follow your voice&#8217;s pitch more precisely, especially when the sound is noisy or unclear.<\/figcaption><\/figure>\n\n\n\n<p>As imitone&#8217;s new &#8220;brain&#8221; develops, it will respond quicker, cleaner and more accurately to a wider range of voices and singing styles. &nbsp;It will work in noisier spaces. &nbsp;It will consume less CPU power \u2014 a big consideration for our mobile apps. &nbsp;Finally, it will lay the foundation for important features like automatic scale detection, automatic pitch range and the intelligent note selection described in our last update.<\/p>\n\n\n\n<p>It has taken a long time and a&nbsp;<em>lot<\/em>&nbsp;of work, but this is a huge step toward the dream.<\/p>\n\n\n\n<p>The coming update will bring a few of those benefits, and will be followed by several smaller updates with gradual improvements. &nbsp;Next week, we&#8217;ll show you a preview of the new version and our plans for 2020.<\/p>\n\n\n\n<p>For now, read on to learn about what we&#8217;ve been doing in 2019, and how we&#8217;re building the new imitone.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:evans-2019\">Evan&#8217;s 2019<\/h1>\n\n\n\n<p>Over the last year I&#8217;ve traveled across the USA in a motor home while continuing my research.<\/p>\n\n\n\n<p><strong>January<\/strong>: &nbsp;Finished the first version of our custom machine learning tool, the &#8220;calibrator&#8221;. &nbsp;Showed imitone at NAMM 2019 with the MIDI Manufacturers Association.<\/p>\n\n\n\n<p><strong>February<\/strong>: &nbsp;Contributed to pitch-related features in MIDI 2.0. &nbsp;Made plans for 2019 research, development and marketing. &nbsp;Shot&nbsp;<a href=\"https:\/\/www.youtube.com\/watch?v=8_-wg63Lgn8\" target=\"_blank\" rel=\"noreferrer noopener\">tutorial videos<\/a>&nbsp;with a new team member.<\/p>\n\n\n\n<p><strong>March<\/strong>: &nbsp;Contributed to MIDI 2.0 Property Exchange. &nbsp;Designed a&nbsp;<em>secret<\/em>&nbsp;planned feature. &nbsp;Made another tutorial. &nbsp;Developed marketing plans. &nbsp;Attended GDC.<\/p>\n\n\n\n<p><strong>April<\/strong>: &nbsp;Lived in the Mojave desert. &nbsp;Resumed work on the calibrator, training synthesizer, theory of pitch tracking and the ability to run imitone&#8217;s machine learning on cloud computers. &nbsp;Made social media plans.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/907\/473ae20701b072cd2a4deea74746c0f0_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043485&amp;gif-q=50&amp;lossless=true&amp;s=f90b317c6a8efab19ab4b1280e43fad7\" alt=\"\"\/><figcaption>Mount San Jacinto, California, seen from Joshua Tree state park. In late April I took a spill in this landscape and sustained a concussion. I continued research, but shelved most side-projects for a while including my work with MIDI 2.0. I&#8217;m told that the best thing to do after a head injury is to avoid screens and intense focus for a few weeks&#8230; Whoops!<\/figcaption><\/figure>\n\n\n\n<p><strong>May<\/strong>: &nbsp;Lived on solar power in the mountains of California. &nbsp;Tested several experimental SAHIR designs. &nbsp;Developed mathematical techniques for high-fidelity filtering.<\/p>\n\n\n\n<p><strong>June<\/strong>: &nbsp;Visited with computational creativity experts in Santa Cruz. &nbsp;Integrated the new filtering methods into imitone&#8217;s technology. &nbsp;Used&nbsp;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Attractor\" target=\"_blank\" rel=\"noreferrer noopener\">attractor theory<\/a>&nbsp;to improve imitone&#8217;s learning system.<\/p>\n\n\n\n<p><strong>July<\/strong>: &nbsp;Investigated and fixed several design flaws in the new SAHIR. &nbsp;Started running imitone&#8217;s machine learning on cloud computers. &nbsp;Visited Crater Lake in Oregon.<\/p>\n\n\n\n<p><strong>August<\/strong>: &nbsp;Further improved the training tools and the ability to run them in the cloud. &nbsp;Made a significant improvement to the SAHIR&#8217;s pitch-following.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/912\/5b272c0c8b7acfd2558397adaa0b3ee3_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043554&amp;gif-q=50&amp;lossless=true&amp;s=827ddf593a7a4234b973f517cd75b46f\" alt=\"\"\/><figcaption>Crater Lake, Oregon, with Wizard Island in the background. Yeah, Wizard Island.<\/figcaption><\/figure>\n\n\n\n<p><strong>September<\/strong>: &nbsp;Developed a CPU budgeting algorithm for SoundSelf, which might be used in imitone too. &nbsp;Interviewed potential additions to the imitone team.<\/p>\n\n\n\n<p><strong>October<\/strong>: &nbsp;Had an early holiday with family and caught the flu. &nbsp;Planned out the 0.10 update. &nbsp;Investigated the new SAHIR&#8217;s CPU usage. &nbsp;Made a few &#8220;recipe&#8221; videos showing imitone techniques.<\/p>\n\n\n\n<p><strong>November<\/strong>: &nbsp;Made a pipeline for using the training data in the live SAHIR. &nbsp;Began testing and tuning the new tech for real-world use. &nbsp;Worked on 0.10&#8217;s interface design. &nbsp;Developed a new approach to vibrato tracking and pitch-guide.<\/p>\n\n\n\n<p><strong>December<\/strong>: &nbsp;Getting the new technology ready for action and putting together imitone 0.10!<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:taking-imitone-to-th\">Taking imitone to the Next Level<\/h1>\n\n\n\n<p>My focus with Phase 4 has been on unlocking the potential of imitone&#8217;s pitch tracking. &nbsp;To do this, I&#8217;m adding a layer of machine learning to imitone&#8217;s original technology, and building a collection of tools to help me oversee the process.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:the-fundamentals\">The Fundamentals<\/h1>\n\n\n\n<p>Let&#8217;s start with a quick summary of how imitone has always worked.<\/p>\n\n\n\n<p>If you sing beneath a concrete tunnel, into a plastic pipe or onto the strings of a guitar, these things will &#8220;resonate&#8221; with the tone of your voice. &nbsp;If you sing at just the right pitch, the resonance will be loud, bright and long-lasting. &nbsp;Otherwise, it will be quiet and dull.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Back in my day we did it like this\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/pNXgokSZdNM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption><em>Playing a mandolin with my voice \u2014 without imitone!<\/em><\/figcaption><\/figure>\n\n\n\n<p>Each string on a guitar has a specific pitch. &nbsp;If you sing a note that falls between those pitches, the strings won&#8217;t resonate. &nbsp;imitone&#8217;s early prototypes worked like&nbsp;<em>hundreds<\/em>&nbsp;of guitar strings \u2014 so that any pitch you could sing would resonate one of them. &nbsp;However, there was a problem: &nbsp;if the pitch of your voice is changing, it will only match up with each string for a moment \u2014 and none of them will resonate strongly. &nbsp;In practice, voice pitch wobbles and slides a&nbsp;<em>lot<\/em>.<\/p>\n\n\n\n<p>The piece at the heart of imitone is the &#8220;self-attuning resonator&#8221;, or&nbsp;<strong>SAHIR<\/strong>&nbsp;(&#8220;say-here&#8221;).<\/p>\n\n\n\n<p>The SAHIR resonates with your pitch, but also&nbsp;<em>follows<\/em>&nbsp;it \u2014 as if someone is tuning the guitar&nbsp;<em>really fast<\/em>&nbsp;to match your voice. &nbsp;When the SAHIR rings loud and bright, it has locked onto your pitch \u2014 and will track it faithfully. &nbsp;Because they&#8217;re flexible and precise, a handful of SAHIR work better than hundreds of simple resonators.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/931\/479a59b63920d7f1b79ad9727655c801_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043757&amp;gif-q=50&amp;lossless=true&amp;s=b13fef4bf3d60456b9adc467ae3e73ff\" alt=\"\"\/><figcaption>Comparing the detection and pitch tracking between imitone 0.9 and our new technology. The prototype uses a higher-fidelity signal and can detect the note about 12 milliseconds earlier. We&#8217;re still fine-tuning the pitch tracking in these first few moments.<\/figcaption><\/figure>\n\n\n\n<p>The original SAHIR uses special measurements to do its two jobs:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Attune: change pitch to lock onto and follow your voice.<\/li><li>Detect: determine whether the SAHIR is locked onto a tone.<\/li><\/ul>\n\n\n\n<p>Using these measurements and a little bit of fudge factor, we have managed to make one of the most responsive pitch trackers in the world. &nbsp;And&nbsp;<em>yet<\/em>\u2026 as we know, imitone 0.9 had room to improve. &nbsp;Each measurement is based on a simplified model of the human voice, and has a little bit of uncertainty to it. &nbsp;Add some background noise, or some growl, or a sound that doesn&#8217;t match up with our model, and the uncertainty grows, with messy results&#8230;<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:the-evolution\">The Evolution<\/h1>\n\n\n\n<p>\u2026And so, it became clear that the key was in statistics: the science of uncertainty. &nbsp;Applied statistics leads us to machine learning, where we can create knowledge from examples. &nbsp;I designed a completely new SAHIR \u2014 one that could eventually be trained to do the best job possible.<\/p>\n\n\n\n<p>I began studying how the SAHIR&#8217;s measurements behave with different sounds. &nbsp;Dozens of sounds, then thousands&#8230; then millions, billions. &nbsp;I built a new tool called the &#8220;calibrator&#8221; to run these musical exeriments, plotting the SAHIR&#8217;s measurements against the correct answers.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/939\/7fefc435b54e8a733d076d4dbc427fc9_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043842&amp;gif-q=50&amp;lossless=true&amp;s=f06f0aafba9cf1cec20423c688190f93\" alt=\"\"\/><figcaption>A calibrator chart depicting the test we use to distinguish musical notes from background noise. I&#8217;ve designed about 60 different charts to help me improve and fine-tune the SAHIR.<\/figcaption><\/figure>\n\n\n\n<p>Originally I expected this would be pretty simple: &nbsp;chart the measurements against some example sounds, transform those charts into statistical likelihoods and use those as calibrations in a fine-tuned pitch tracker. &nbsp;The reality has been a&nbsp;<em>little<\/em>&nbsp;more complicated&#8230; &nbsp;Who could have guessed?<\/p>\n\n\n\n<p>The first problem I ran into was\u2026 more like an opportunity. &nbsp;My new tool allows me to design plots comparing different measurements in different situations. &nbsp;I made dozens of these, and each gave me a new perspective on billions of datapoints. &nbsp;They gave me insight into behaviors of the SAHIR that were invisible with the tools I had before: &nbsp;Situations where it would get stuck, jitter, or fail to recognize a tone&#8230; &nbsp;I spent several weeks investigating newfound flaws and opportunities for improvement.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/945\/708d72dcdd85872d0e7d474cb4de217b_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043888&amp;gif-q=50&amp;lossless=true&amp;s=bf48e381c56771306e0b6cc2381c7d98\" alt=\"\"\/><figcaption>A much earlier version of the same test, with some serious problems. It accepted only a narrow range of sounds, and would interpret any sudden sound as a musical note.<\/figcaption><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:playing-fair\">Playing Fair<\/h1>\n\n\n\n<p>The second problem is an inevitability with machine learning. &nbsp;To teach imitone to recognize human voices, I need lots and&nbsp;<em>lots<\/em>&nbsp;of examples and counter-examples. &nbsp;I need to be very careful about how I choose these examples, because they create&nbsp;<em>bias<\/em>&nbsp;\u2014 and imitone needs to be ready for all the different voices it might encounter in the real world.<\/p>\n\n\n\n<p>I tested the original imitone constantly with my&nbsp;<em>own<\/em>&nbsp;voice, fine-tuning it to feel good when&nbsp;<em>I<\/em>&nbsp;sing at it. &nbsp;Because of that testing, it works well with my voice, and voices like mine. &nbsp;I&#8217;ve learned that there are other kinds of singing voices \u2014 even other kinds of vocal cords \u2014 which imitone doesn&#8217;t understand so well. &nbsp;Therefore, we can say that imitone has been biased toward my singing voice (a &#8220;modal baritone&#8221;) and against certain others (like those of heavy metal singers).&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/949\/c1b9d324e95ae28e7504dddce4656a75_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576043945&amp;gif-q=50&amp;lossless=true&amp;s=e595526122f55e08834b75a6911f576c\" alt=\"\"\/><figcaption>Calibrating the pitch tracker has been a long and challenging process, but I&#8217;m beginning to see the fruits of my labor. They look a bit like a peach and a mango.<\/figcaption><\/figure>\n\n\n\n<p>Many other sensitive technologies also have a habit of working a little better for people who look, sound or act like their creators. &nbsp;Cameras designed to recognize people may behave differently based on ethnicity. &nbsp;Airport security scanners regularly cause trouble for people whose body shape isn&#8217;t typical for their gender. &nbsp;Similarly, imitone has had its own biases toward certain singing voices and the western canon of music.<\/p>\n\n\n\n<p>To work against voice bias and build a truly &#8220;universal&#8221; music tool, I&#8217;ll be training imitone against an especially diverse collection of sounds. &nbsp;For now, I&#8217;m creating billions of artificial voice sounds which vary more widely than human voices do. &nbsp;If I use real voice to train imitone in the future, I will collect the most diverse collection of voices I can. &nbsp;Finally, future versions of imitone might be able to learn the unique nature of your voice while you sing \u2014 but we aren&#8217;t there quite yet.<\/p>\n\n\n\n<p>I&#8217;ll be working against imitone&#8217;s western bias by introducing more traditions of music, such as the Indonesian and Arabic scales mentioned&nbsp;<a href=\"https:\/\/kickstarter.com\/projects\/evanbalster\/imitone-mind-to-melody\/posts\/2391339\" target=\"_blank\" rel=\"noreferrer noopener\">in our last update<\/a>. &nbsp;(I&#8217;m interested in talking to experts in traditions around the world.) &nbsp;Our fifth and final phase of research will revolve around music theory and the way imitone chooses notes. &nbsp;That will result in big changes to imitone&#8217;s scales, keys, tuning and pitch correction.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"h:whats-next\">What&#8217;s Next?<\/h1>\n\n\n\n<p>I&#8217;ll be spending the next few weeks fine-tuning the new SAHIR and making some other much-needed improvements to imitone for a winter update. &nbsp;Once it&#8217;s in your hands, the team will be collecting as much feedback as we can, continuing to fine-tune and fixing any bugs that have crept in over the last two years.<\/p>\n\n\n\n<p>imitone will be exhibiting at NAMM 2020 with the MIDI Manufacturers&#8217; Association. &nbsp;If you plan to attend, don&#8217;t be a stranger! &nbsp;I also suggest keeping an eye out for announcements about MIDI 2.0 \u2014 I&#8217;ll talk about those in a future post.<\/p>\n\n\n\n<p>For the new year, I&#8217;m resolving to be less of a hermit. &nbsp;No, that doesn&#8217;t mean I&#8217;ll quit living in the the woods \u2014 but that I&#8217;ll be making efforts to be more transparent, more communicative and more active in updating the software. &nbsp;To get there, I&#8217;ll be adding some new faces to the imitone team. &nbsp;You might get to meet them in a future article (or at NAMM). &nbsp;If you feel&nbsp;<em>you<\/em>&nbsp;might have something to bring to the project,&nbsp;<a href=\"mailto:\/\/hello@imitone.com\" target=\"_blank\" rel=\"noreferrer noopener\">shoot us an E-mail<\/a>. &nbsp;We&#8217;re looking to organize, research, teach, design and build.<\/p>\n\n\n\n<p>Thanks for sticking with me on this long ride.<\/p>\n\n\n\n<p>Happy trails from the I.W.S. Coddiwomple,<\/p>\n\n\n\n<p>\u2014 Evan<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/ksr-ugc.imgix.net\/assets\/027\/475\/958\/d26ee144a5efd2997fd7f3899542acd4_original.png?ixlib=rb-4.0.2&amp;w=700&amp;fit=max&amp;v=1576044069&amp;gif-q=50&amp;lossless=true&amp;s=d822ad043c16fde80037ce32f2735279\" alt=\"\"\/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>This post was transcribed from imitone&#8217;s kickstarter page. click here for the original post, or click here for an audio version of this post. I&#8217;m calibrating! &nbsp;I&#8217;m calibrating!! &nbsp;I&#8217;m fine-tuning a training model, generating and analyzing billions of datapoints, calculating probabilities and building a time-critical pitch tracker capable of \u2014 ah \u2014 ah, wait, I [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/imitone.com\/blog\/index.php\/2019\/12\/11\/cause-for-calibration\/\">Read More&#8230;<\/a><\/p>\n","protected":false},"author":1,"featured_media":104,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-69","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devlog"],"_links":{"self":[{"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/69","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=69"}],"version-history":[{"count":2,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/69\/revisions"}],"predecessor-version":[{"id":105,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/posts\/69\/revisions\/105"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/media\/104"}],"wp:attachment":[{"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=69"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=69"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/imitone.com\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=69"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}