Use Python to turn a computer into a piano

Record a piece of audio, change its pitch 50 times and match each new audio to a keyboard key, and you can turn your computer into a piano!

A piece of audio can be encoded as an array (or list) of values, like this:

　　

We can change the speed of this audio to twice times by taking a second value in the array every second:

　　

So we not only halved the length of the audio, but also doubled its frequency so that it had a higher pitch (pitch) than the original.

Conversely, if we repeat each value in the array once, we will get a slower, longer period, or lower pitch audio:

　　

Here is a simple Python function that can change the audio speed by any factor:

Python

1

2

3

4

5

6

7import NumPy as NP

def speedx (Sound_array, Factor):

"" To multiply the audio speed by any factor ' factor ' ""

indices = Np.round (Np.arange (0, Len (snd_array), factor))

indices = Indices[indices < Len (snd_array)].astype (int)

return sound_array[indices.astype (int)]

The more difficult part of the problem is changing the audio length while maintaining its pitch (variable speed, audio stretching), or changing the pitch of the audio while maintaining its length (pitch shifting).

Variable speed

The variable speed can be achieved through the traditional phase sound encoder (phase vocoder, interested friends can read the Wikipedia page). First, the audio is decomposed into overlapping bits, and then the bits are rearranged so that they overlap more (the length of the sound will be shortened) or less (the length of the audio will be stretched), as shown in:

　　

The difficulty is that the rearranged bits can be very heavily interacting with each other, so there is a need for phase transformations to ensure that there is no effect between them. Here is a Python code, taken from this page (if you can't open it, you know.) --Translator's note):

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21st

22

23

24

25

26def Stretch (Sound_array, F, Window_size, h):

"" Stretch the audio by factor ' F ' ""

Phase = Np.zeros (window_size)

Hanning_window = np.hanning (window_size)

result = Np.zeros (len (Sound_array)/F + window_size)

For I in Np.arange (0, Len (sound_array)-(window_size+h), h*f):

# Two sub-sequences that may overlap each other

A1 = Sound_array[i:i + Window_size]

A2 = sound_array[i + h:i + window_size + h]

# re-sync the second sequence by the first sequence

S1 = Np.fft.fft (Hanning_window * A1)

S2 = Np.fft.fft (Hanning_window * A2)

Phase = (phase + np.angle (S2/S1))% 2*np.pi

a2_rephased = Np.fft.ifft (np.abs (S2) *np.exp (1j*phase))

# Add to Results

i2 = Int (i/f)

Result[i2:i2 + window_size] + = hanning_window*a2_rephased

result = ((2** (16-4)) * Result/result.max ()) # normalized (16bit)

Return Result.astype (' Int16 ')

Modulation

Once you have achieved the speed change, the pitch is not difficult. If you need a higher pitch, you can stretch the audio and keep the pitch constant, then speed it up so that the final audio will have the same length as the original audio, higher frequencies, or higher pitches.

Doubling the frequency of a piece of audio will increase the pitch by a factor of eight, which is 12 chromatic. Therefore, to increase the pitch by N-Chromatic, we need to multiply the frequency by the coefficient 2^ (N/12):

Python

1

2

3

4

5def Pitchshift (Snd_array, N, window_size=2**13, h=2**11):

"" To increase the pitch of a piece of audio by ' n ' and ' ""

Factor = 2** (1.0 * n/12.0)

stretched = Stretch (Snd_array, 1.0/factor, Window_size, h)

Return Speedx (stretched[window_size:], Factor)

Small Program: Computer piano

Let's play with our pitch shifter. We first hit the bowl to determine a "standard pitch high":

Next we create 50 pitch pitches based on the previous audio, from very low to very high:

Python

1

2

3

4

5from Scipy.io Import Wavfile

FPS, Bowl_sound = Wavfile.read ("Bowl.wav")

Tones = Range ( -25,25)

transposed = [Pitchshift (Bowl_sound, N) for n in tones]

Next, based on the order in this file, we match each audio to a key on the keyboard, as shown in:

　　

We just need to tell the computer in the code to play its corresponding sound when a key is pressed down, and then stop playing when the button is released:

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21st

22

23

24

25

26

27

28

29

30

31

32

33

34import Pygame

Pygame.mixer.init (FPS,-16, 1, 512) # too flexible;)

Screen = Pygame.display.set_mode ((640,480)) # Set focus

# Get a list of keyboard keys in the correct order

# "Keys" as arranged as [' Q ', ' W ', ' E ', ' R ' ...]

Keys = open (' typewriter.kb '). Read (). split (' \ n ')

Sounds = map (pygame.sndarray.make_sound, transposed)

Key_sound = dict (Zip (keys, sounds))

is_playing = {K:false for k in keys}

While True:

event = pygame.event.wait ()

If Event.type in (Pygame. KEYDOWN, Pygame. KEYUP):

Key = Pygame.key.name (Event.key)

if Event.type = = Pygame. KEYDOWN:

if (key in Key_sound.keys ()) and (not Is_playing[key]):

Key_sound[key].play (FADE_MS=50)

Is_playing[key] = True

elif Event.key = = Pygame. K_escape:

Pygame.quit ()

Raise Keyboardinterrupt

elif Event.type = = Pygame. KEYUP and Key in Key_sound.keys ():

Key_sound[key].fadeout (50) # Stop playback and 50ms fade out

Is_playing[key] = False

So we turned the computer into a piano! Now, let me show you a turkey march to express your appreciation for your patience in reading this article:

If you want to try it yourself, you can download all the files you need here. Because not all people use Python, I also use JAVASCRIPT/HTML5 (here) to achieve a computer piano, but not particularly ideal. It would be nice to have an experienced Html5/js/elm programmer to improve the improvement, or rewrite it from scratch.

What do you do next?

More generally, I find that computers are seldom used for performing performances. I understand that it is much easier to use a piano keyboard or to record directly from an instrument, but see what you can do with just one bowl and 60 lines of Python code!

Even a cheap computer has so much control to achieve a so-called music platform: You can sing to the microphone, gesture to the camera, use the mouse to modulate, and then use the keyboard to do the rest of the stuff. There are so many ways to express yourself, and there is a Python package in each of these ways ... Is there a great God with artistic talent to join?

Use Python to turn a computer into a piano