How I make my videos

This is high level overview of how I make my videos. I hope to produce further posts with more detail.

Not using Synthesia

One of the most popular programs used for doing music visualizations like the ones I produce is Synthesia. It’s an designed for learning the piano, and can render and play pretty much any MIDI file. For my purposes it has a major downside though, it only runs in real time. This means I have to let it run live and record it using some screen recording software. I then have to do some editing to cut out unwanted content at the beginning and end and clip out UI.

MIDI visualizer

Instead, I’m using the cunningly named MIDIVisualizer by kosua20 and ekuiter. It can be made to render offline and thus eliminates the problems with Synthesia. One minor downside is it doesn’t actual play audio, but that is easy to deal with.

MuseScore

MuseScore is software for writing and editing music scores, and has support for importing and exporting in various formats. I primarily use it for either creating the original music or cleaning up existing music. For the full score visualizations I also use it to generate the actual audio.

TiMidity

Timidity is a command line tool for synthesizing audio from a MIDI file. I use it to generate audio for the modes retuning videos.

FFmpeg

FFMpeg is the real workhorse of the process. It’s a command line too for manipulating audio and video. I use it for at least the following:

• Generate the video from the MIDIVisualizer output (which is just a series of PNGs of each frame).
• Add the text in the form of subtitles.
• Combine the video and audio.
• Pan-and-scan the individual parts for better spacing.
• Combine multiple parts together in a mosaic.

Wrapping it all up

All the above is combined together with a custom tool that runs all the individual bits in a pipeline.