Posts  / #POST-225719
BLOG

I cloned my voice for $1

S
Apr 28, 2026 · 00:21

<p>I am a curious person. I want to figure out how things work. </p><p>I also want to start with an admission: <strong>I had no idea what I was doing.</strong></p><p>A few days ago, I was looking into the latest open-source AI models, specifically text-to-speech. One that caught my attention was OmniVoice. Not the most downloaded (although over a million downloads per month), but it has something most don&#8217;t: a really good voice cloning from just a short audio reference.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" href="https://substackcdn.com/image/fetch/$s_!AoQ4!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F35f94c30-bb3b-4b9d-b68a-d29b04d088f3_1200x648.png" target="_blank"><div class="image2-inset"><source type="image/webp" /><img alt="OmniVoice - a Hugging Face Space by k2-fsa" class="sizing-normal" height="648" src="https://substackcdn.com/image/fetch/$s_!AoQ4!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F35f94c30-bb3b-4b9d-b68a-d29b04d088f3_1200x648.png" title="OmniVoice - a Hugging Face Space by k2-fsa" width="1200" /><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image" tabindex="0" type="button"><svg fill="none" height="20" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5" viewBox="0 0 20 20" width="20" xmlns="http://www.w3.org/2000/svg"><g><title></title><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image" tabindex="0" type="button"><svg class="lucide lucide-maximize2 lucide-maximize-2" fill="none" height="20" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" viewBox="0 0 24 24" width="20" xmlns="http://www.w3.org/2000/svg"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>That got me thinking. <strong>Could I train it on my own voice? But, how do I train?</strong></p><p>I decided to figure it out. So, I asked Claude some dumb questions, and then some more. </p><p>I asked Claude to generate 50 simple sentences (5 seconds or less each) and recorded them all. It took me less than 15 minutes.</p><p>Then, it was time to fine-tune the model.</p><div><hr /></div><h4><strong>The heavy lifting</strong></h4><p>Training (fine-tuning) the model is the heavy lifting. In this phase, the model learns what my voice sounds like from the audio samples. It&#8217;s compute-intensive, it happens once, and it needs expensive hardware.</p><p>For example, my MacBook Pro would take 30+ hours for the 50 audio files totaling less than 250 seconds. That&#8217;s not a typo.</p><p>So, I looked into renting a GPU online, and within 10 minutes, I had an Nvidia A100 on JarvisLabs running at $0.99/hour.</p><p>Set-up and training took less than an hour (<strong>as a finance person</strong>)<strong> </strong>at a total cost rounded to $1.</p><p>The A100 retails for around $12,000. I rented it for $1. That&#8217;s a good deal for me, but also for the company renting it. Electricity aside, there are 8,800 hours in a year. My rough estimate is that renting at 30% utilization rate is already a good outcome for them. Once training was done, I could export the model and run it locally.</p><p><strong>Inference</strong> is using the model. Once trained, it runs locally. No cloud, no subscription, no heavy equipment. Some models today run directly on phones.</p><p class="button-wrapper"><a class="button primary" href="https://thefinancecorner.substack.com/subscribe?"><span>Subscribe now</span></a></p><div><hr /></div><h4><strong>Why does this matter for investors?</strong></h4><p>The quality of my cloned voice was really good, despite having low audio data. I will re-run this with 10x the audio and longer training. The cost will be around $15.</p><p>This simple experiment gives a glimpse into what conceptually OpenAI and Anthropic do (only at a scale that requires billions in infrastructure). </p><p>Training is the expensive part. Once it&#8217;s done, serving you a response (via ChatGPT or Claude) is relatively cheap. They also control the quotas, which means they control their margins. The heavy lifting is behind them.</p><p>The cost to build functional, customized AI (not frontier models, but useful ones) is low. If I can do it in under 2 hours for $1 without any prior knowledge, anyone can.</p><p><strong>Here are my two predictions:</strong></p><ol><li><p>At some point, the frontier models will hit a ceiling. More data and more compute won&#8217;t lead to quality improvement. Every major lab will get there eventually. Then the question will be, who can deliver the output cheaper? Who built the more efficient model?</p></li><li><p>In the long run, inference will be a significantly bigger market than training. The world doesn&#8217;t need to keep retraining from scratch, but it needs to run what&#8217;s already been built, billions of times a day.</p></li></ol><div><hr /></div><p>I started out as an idiot with no idea what I was doing. I am still the same idiot, just with way more questions. </p><p class="button-wrapper"><a class="button primary" href="https://thefinancecorner.substack.com/subscribe?"><span>Subscribe now</span></a></p><p></p>