All images by author.

Real Time Audio Wave Visualization in Python.

Or how to overcome Integration hell.

Keno Leon
Keno Leon
Jun 10 · 8 min read
Why read this? You want to visualize audio in realtime with Python and find the whole thing intimidating (like I did !) or you might be having problems integrating complex libraries and want to hear a tall tale of how I solve mine along with some tips.

What are we building ?

Spec sketch:

*Looks simple but when we add the complexities of streaming in real time from a microphone to both the GUI elements and getting the UI buttons to coordinate everything plus each libraries quirks and kinks, well let’s just say specs can be deceiving, making them can be easy but implementing them is a completely different challenge.
By necessity integration is a complex thing, not only do you have to know about the thing you are building but also have a good understanding of the libraries you are going to use to achieve your project, in this case I've written some introductory posts on the required individual components and leave them here as suggested/complementary reading.UIs in Python with PySimpleGUIIntegrating PyAudio & PySimpleGUICustom made plots in Python with PySimpleGUIIntegrating PyPlot and PySimpleGUIHearing for robots and AISo integration tip #1: Make and test individual components (the parts) that you think you will need to make your final project, you might not end up using all of them and that is fine.
It might not be obvious, but implicit in the representation of sound via waveforms there is a time dimension ( the horizontal axis ) , so each black segment is analogous to a frame in a movie and the whole waveform ( the bottom plot ) is equivalent to perceived movement or in this case a perceived sound.

It helps to know your data.

At the callback level:data = np.frombuffer(in_data, dtype=np.int16)
print(data) >>>
[-118 -115 -113 ... -104 -106 -104]
[-106 -106 -104 ... 102 105 109]
[111 114 116 ... 213 215 217]
[221 226 228 ... 273 266 261]
[261 262 260 ... -7 0 7]
So we get 1,024 ( the CHUNK or sample rate ) numbers in each array/list both positive and negative representing a slice in time which matches our expectations, all we need to do now is plot them in real time...
You basically clear and redraw the whole plot which you also draw from scratch, we’ll forego the labeled axes for now to focus on the integration bits and audio bits.

Integration Hell

Welcome to Hell ! let me show you around: I think integration can get hellish because you are trying to combine 2 or more different books from different authors into one coherent movie script with you as the director and producer, it can be a lot. Another level of hell comes from not having any help available, who's responsibility is it to help you when there are problems? Not the individual libraries that's for sure, it is YOU and this might not be a comfortable place to be (coming up with solutions to new problems with no outside help).
But worry not, I see this stage as one of the most valuable in software development and where truly new and useful things are made, so there are diamonds if you successfully navigate hell, whats more if you understand how difficult it is you can reframe it as a challenge and understand you might fail a lot before you start winning, it is simply the cost of making something new so don't despair.

Bad Code/ Challenge:

Integration tip #2 It's the small steps that count, don't despair, try something new, take a break, celebrate progress rather than perfection.
Note: I had to lobotomize my working example since I didn't keep the original crashing code, so this one crashes a little less and might even work for a few seconds, semi crashing code is the worst since you are close but no cigar and errors are sometimes hard to replicate. (see tips #2 & #3) 

Getting comfy where the paved road ends.

In my/this specific case I suspect the problem was related to threads and memory ( I could still research more, but time is still at a premium so I let it go for now), I miss-managed the audio stream callback which needs to be slim, I was drawing the plots there and having a memory blowout, so basically moving 4 lines of code and data outside of a callback and adding a variable, this might not make sense to you but the point is that after experimenting a bit and separating concerns (ie organizing your code) a solution presented itself. At some point you fully internalize your code (you can imagine all the moving parts) and your brain starts presenting solutions, thanks brain! ❤️ 🧠 Integration tip #3: Not crashing is good, but crashing and experimenting can be the norm, after all you are smashing things together until you know how to make them cooperate, see tip #2.  

Working Integration:

Just some coffee shop/street background noise, you might have notice my complete disregard for measurement units and that is mostly intentional, it makes things easier at this point.
You can recreate this by playing a 1 kHz sine wave tone in your phone while you run the script on your computer.
And it does ! It is obviously not perfect and could use a number of improvements like unit measurements on the axes, auto scaling, gain/volume control, etc, etc. But as hinted by the name integration prototype or simple integration, it is not meant to be the latest and greatest but minimal and working.Integration Tip #4: Don't be fuzzy, favor iterating, learning and progress before perfection.

Further integrations, PyPlot again.

Crashes less than my initial prototypes ( runs fine for short periods ), but still crashes 😕
Integration tip #5: A successful integration ( even if simple ) is usually the start of a new and unknown integration, avoid burnout and improve your chances of success by taking a moment to rest and document your progress so far, this will be your starting point for the next round.

Parting words

Don't despair, research & experiment, integration is tough. 

Geek Culture

Proud to geek out. Follow to join our +500K monthly readers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store