Coding with a bumbling baffoon: bad apple in the CLI
https://github.com/cyano-dove/bad-apple-in-Odin
It’s the weekends, I’m trying to get my groove back on programming and picking up a new programming language while at it. I always wanted to try printing Bad Apple on something, so we will walk through that.
This is NOT a tutorial, but rather my thought process on how to do this somewhat blindly and document it hoping I succeed.
Game plan
This will be attempted without ffmpeg, but an audio package will be used.
First hour: attempting to do it this without ffmpeg
Understanding the .mp4 data
We will start by taking the .mp4 and print out in hex data
As you can clearly see, I don’t know what I am doing here. So let’s look at some resource.
https://dev.to/alfg/a-quick-dive-into-mp4-57fo
basically every .mp4 file has boxes (or atoms) stored in a container. There are boxes for video, audio, subtitle, etc. The first 8 bytes of a box are designated as the box header, 4 for the size of the box and 4 for the name of the box, from there you can figure that the data size is the box size - 8.
video.mp4
├───ftyp -------------------> FileType Box
├───mdat -------------------> Movie Data Box
├───moov -------------------> Movie Boxes
│ ├───trak ---------------> Track Box
│ │ ├─── tkhd ----------> Track Header
│ │ └─── mdia ----------> Media Box
│ │ └─── ...
│ └───trak
│ │ ├─── tkhd ----------> Track Header
│ │ └─── mdia ----------> Media Box
│ │ └─── ...
└───udta -------------------> Userdata BoxI am too broke to legally afford a copy of the ISO official documentation, but fortunately there is something available for free. The .mp4 file is based off Apple’s QuickTime format, and they have a free source of reference for the byte layouts. Sure, there are free resource on google but for now I want to specifically work with official stuff, plainly for no reasons other than my amusement.
https://developer.apple.com/documentation/quicktime-file-format
(also my university’s library probably have one or two books of the spec stored somewhere but I am too lazy to check for em, I usually go there just for the free AC and electricity)
Parsing the .mp4 visual data
To get started we will try to extract data from one frame, let’s say the frame at 0:05 of the video, and maybe we can reassemble it back into an ASCII frame in the terminal.
Let’s look at the boxes at the surface level with a simple loop
boxSize : u32 = 0
boxType : [4]rune
subIndex : u32 = 0
for value, index in data {
// note: does not account for the case of an atom that exceed 2^32 bytes
if subIndex < 4 {
boxSize = (boxSize << 8) + u32(value)
}
if subIndex == 3 {
fmt.printf("size: %d\n", boxSize)
}
if subIndex >= 4 && subIndex < 8 {
boxType[subIndex-4] = rune(value)
}
if subIndex == 7 {
fmt.printf("box type: %s\n", boxType)
}
if subIndex == boxSize - 1 && subIndex >= 7 {
boxSize = 0
subIndex = 0
continue
}
subIndex = subIndex + 1
}odin run . -- .\badapple.mp4
size: 32
box type: ftyp
size: 260457
box type: moov
size: 8
box type: free
size: 8042439
box type: mdatThe video says ~8,109KB on file explorer, so that looks correct
Problems from here
Foolish as I was, I did not account for how long it would take to decode the raw data without the usage of a decoder library, the apple QuickTime spec does not have enough information for me and even if I use the real thing from ISO written for .mp4 it would probably take ages for me to write a decoder. So that specific topic will be for another time.
Using ffmpeg
Fairly simple with ffmpeg and ffprobe, just have Odin run the commands. ffprobe could be skipped if I refine the .mp4 box parser that I wrote earlier but I will leave that for when if I would tackle on the decoder.
Frames
First thing to do is to create a folder consist of images and then parse the images one by one.
makeFramesFF :: proc(file: string) {
dot := strings.last_index_byte(file, '.')
os.make_directory(fmt.tprintf("%s_frames", file[:dot]))
libc.system(fmt.ctprintf("ffmpeg -hide_banner -loglevel error -stats -i \"%s\" \"%s_frames/%%04d.ppm\"", file, file[:dot]))
}When one of the frames gets put into a hex viewer, the first few bytes are the most notable, P6 is the “magic number” identifying the file type, 480 is the width of the image and 360 is the height of the image and 255 is the maximum color value, whatever that is.
Every .ppm file has a header like this with 3 newline, so it’s easy to gather width, height and offset the header out off the pixels data
frameHeader :: proc(data: []u8, print := false) -> (offset: int, width: int, height: int) {
newLineCount : int = 0
offset : int
frameW : int
frameH : int
for value, index in data {
if value == 10 {
newLineCount = newLineCount + 1
}
if newLineCount == 1 && frameW == 0 {
tempIndex := 1
for ; data[tempIndex+index] != u8(' '); tempIndex += 1 {
frameW = frameW * 10 + int(data[tempIndex+index]) - '0'
}
tempIndex += 1
for ; data[tempIndex+index] != u8('\n'); tempIndex += 1 {
frameH = frameH * 10 + int(data[tempIndex+index]) - '0'
}
}
if newLineCount == 3 {
offset = index
break
}
}
if print {
fmt.printf("offset: %d | width: %d | height: %d", offset, frameW, frameH)
}
return offset, frameW, frameH
}Now let’s try to print a frame
framePixelData :: proc(data: []u8) -> (pixels: []u8, w: int, h: int) {
offset, width, height := frameHeader(data)
r, g, b : u8
pixel := make([]u8, width * height) // avg the r, g, b value into pixel's brightness
for y in 0..<height {
for x in 0..<width {
p := y * width + x
index := offset + 1 + p*3
r = data[index ]
g = data[index + 1]
b = data[index + 2]
if (int(r) + int(g) + int(b)) / 3 > 128{
pixel[p] = 255
}
}
}
return pixel, width, height
}The below loop would print the frame perfectly, but ASCII would probably be too big for the terminal
for y in 0..<height {
for x in 0..<width {
if pixel[y * width + x] == 255 {
fmt.printf("#")
}
else {
fmt.printf(".")
}
}
fmt.println()
}So let’s make a version that scale down the pixel and add a grayscale ASCII, first I’ try and sketch out how that’d look like
Let’s turn that into code
printFrame :: proc(pixel: []u8, width, height : int, setScrWidth := -1, setScrHeight := -1) {
scaleWidth : f32 = 1
scaleHeight : f32 = 1
scrWidth := width
scrHeight := height
if setScrWidth != -1 {
scrWidth = setScrWidth
scaleWidth = f32(width) / f32(scrWidth)
if scaleWidth < 1 {
scaleWidth = 1
}
}
if setScrHeight != -1 {
scrHeight = setScrHeight
scaleHeight = f32(height) / f32(scrHeight)
if scaleHeight < 1 {
scaleHeight = 1
}
}
str := strings.builder_make()
defer strings.builder_destroy(&str)
if scaleHeight == 1 && scaleWidth == 1 {
// without scaling
for y in 0..<height {
for x in 0..<width {
if pixel[y * width + x] == 255 {
strings.write_byte(&str, '#')
}
else {
strings.write_byte(&str, '.')
}
}
strings.write_byte(&str, '\n')
}
}
else{
// with scaling
BRIGHTNESS := " .`',:;-~=+*#%$@" // shoutout to the C donut devblog https://www.a1k0n.net/2021/01/13/optimizing-donut.html
for scrY in 0..<scrHeight {
for scrX in 0..<scrWidth {
sum := 0
count := 0
for y in scrY*int(scaleHeight) ..< (scrY+1)*int(scaleHeight) {
for x in scrX*int(scaleWidth) ..< (scrX+1)*int(scaleWidth) {
sum += int(pixel[y*width + x])
count += 1
}
}
strings.write_byte(&str, BRIGHTNESS[int(sum/count) / 17])
}
strings.write_byte(&str, '\n')
}
}
os.write_string(os.stdout, strings.to_string(str))
defer delete(pixel)
}ALT_SCR_T :: "\x1b[?1049h"
ALT_SCR_F :: "\x1b[?1049l"
CUR_HIDE :: "\x1b[?25l"
CUR_SHOW :: "\x1b[?25h"
CLEAR :: "\x1b[2J"
HOME :: "\x1b[H"
...
main :: proc() {
...
os.write_string(os.stdout, ALT_SCR_T + CUR_HIDE + CLEAR)
defer os.write_string(os.stdout, CUR_SHOW + ALT_SCR_F)
printFrame(framePixelData(data), width, height)
libc.getchar()
}And we finally printed a frame
PS D:\coding projects\bad apple in Odin> odin run . -- .\badapple.mp4 80 30
Now let’s iterate through the frames in the folder, we already know the the video’s FPS is 30 and I don’t want to go to the hassle of getting it from the .mp4 at this point so that will be hard coded into the loop.
FPS :: 30
FRAME_TIME :: time.Second / FPS
iterateFrames :: proc(folder : string, width : int, height : int) {
for n := 1; ; n += 1 {
currTime := time.tick_now()
frame := fmt.tprintf("%s/%04d.ppm", folder, n)
data, err := os.read_entire_file(frame, context.allocator)
if err != nil {
break
}
printFrame(framePixelData(data), width, height)
delete(data)
free_all(context.temp_allocator)
os.write_string(os.stdout, HOME)
diffTime := time.tick_since(currTime)
if diffTime < FRAME_TIME {
time.sleep(FRAME_TIME - diffTime)
}
}
}This works, but when I play this video it will lag behind, might be a Windows thing. After a bit of looking around the problem seems to be the loop measuring being off. To resolve this, I put the time.tick_now() outside of the loop.
iterateFrames :: proc(folder : string, width : int, height : int) {
startTime := time.tick_now()
for n := 1; ; n += 1 {
frame := fmt.tprintf("%s/%04d.ppm", folder, n)
data, err := os.read_entire_file(frame, context.allocator)
if err != nil {
break
}
printFrame(framePixelData(data), width, height)
delete(data)
free_all(context.temp_allocator)
os.write_string(os.stdout, HOME)
nextFrame := time.Duration(n) * FRAME_TIME
elapsed := time.tick_since(startTime)
if elapsed < nextFrame {
time.sleep(nextFrame - elapsed)
}
}
}It works fine for me, so we will finish this with the audio and it is done
Playing the audio (using miniaudio package)
Updating the ffmpeg command to extract the .wav
makeFramesFF :: proc(file: string, folder : string) {
os.make_directory(folder)
libc.system(fmt.ctprintf("ffmpeg -y -hide_banner -loglevel error -stats -i \"%s\" \"%s/%%04d.ppm\" -vn \"%s/audio.wav\"", file, folder, folder))
}From here I will just play the audio in the background, if I ever revisit to parse the .mp4 without ffmpeg I will parse it, I had enough for this weekends. (I also moved the alternative screen toggle into the loop)
engine : aud.engine
result := aud.engine_init(nil, &engine)
if result != .SUCCESS {
fmt.println("could not start audio, press enter to continue: ", result)
libc.getchar()
return
}
defer aud.engine_uninit(&engine)
aud.engine_play_sound(&engine, fmt.ctprintf("%s/audio.wav", folder), nil)
os.write_string(os.stdout, ALT_SCR_T + CUR_HIDE + CLEAR)
defer os.write_string(os.stdout, CUR_SHOW + ALT_SCR_F)And that’s about it, a lot of compromise had to be made but in the end, bad apple is playable on the terminal, and that is pretty cool
Full source
https://github.com/cyano-dove/bad-apple-in-Odin
main.odin
package main
import "core:fmt"
import "core:os"
import "core:strings"
import "core:c/libc"
import "core:strconv"
import "core:time"
import aud "vendor:miniaudio"
ALT_SCR_T :: "\x1b[?1049h"
ALT_SCR_F :: "\x1b[?1049l"
CUR_HIDE :: "\x1b[?25l"
CUR_SHOW :: "\x1b[?25h"
CLEAR :: "\x1b[2J"
HOME :: "\x1b[H" // move cursor to row 1, col 1
FPS :: 30
FRAME_TIME :: time.Second / FPS
printFileHex :: proc(data: []u8, stop: int = -1) {
// unused
buffer : [16]rune
for value, index in data {
if stop != -1 && index == stop {
return
}
fmt.printf("%02x ", value)
buffer[index%16] = rune(value)
if (index+1) % 16 == 0 {
fmt.printf(" | ")
for value, index in buffer {
if value == 0 {
fmt.printf(" ")
}
else if value == 10 {
fmt.printf(" [nl] ")
}
else {
fmt.printf("%r", value)
}
}
fmt.printf("\n")
}
}
}
getBoxheaders :: proc(data: []u8) {
// unused (for now)
boxSize : u32 = 0
boxType : [4]rune
subIndex : u32 = 0
for value, index in data {
// note: does not account for the case of an atom that exceed 2^32 bytes
if subIndex < 4 {
boxSize = (boxSize << 8) + u32(value)
}
if subIndex == 3 {
fmt.printf("size: %d\n", boxSize)
}
if subIndex >= 4 && subIndex < 8 {
boxType[subIndex-4] = rune(value)
}
if subIndex == 7 {
fmt.printf("box type: %s\n", boxType)
}
if subIndex == boxSize - 1 && subIndex >= 7 {
boxSize = 0
subIndex = 0
continue
}
subIndex = subIndex + 1
}
}
makeFramesFF :: proc(file: string, folder : string) {
os.make_directory(folder)
libc.system(fmt.ctprintf("ffmpeg -y -hide_banner -loglevel error -stats -i \"%s\" \"%s/%%04d.ppm\" -vn \"%s/audio.wav\"", file, folder, folder))
}
frameHeader :: proc(data: []u8, print := false) -> (off: int, w: int, h: int) {
newLineCount : int = 0
offset : int
frameW : int
frameH : int
for value, index in data {
if value == 10 {
newLineCount = newLineCount + 1
}
if newLineCount == 1 && frameW == 0 {
tempIndex := 1
for ; data[tempIndex+index] != u8(' '); tempIndex += 1 {
frameW = frameW * 10 + int(data[tempIndex+index]) - '0'
}
tempIndex += 1
for ; data[tempIndex+index] != u8('\n'); tempIndex += 1 {
frameH = frameH * 10 + int(data[tempIndex+index]) - '0'
}
}
if newLineCount == 3 {
offset = index
break
}
}
if print {
fmt.printf("offset: %d | width: %d | height: %d", offset, frameW, frameH)
}
return offset, frameW, frameH
}
framePixelData :: proc(data: []u8) -> (pixels: []u8, w: int, h: int) {
offset, width, height := frameHeader(data)
r, g, b : u8
pixel := make([]u8, width * height) // avg the r, g, b value into pixel's brightness
for y in 0..<height {
for x in 0..<width {
p := y * width + x
index := offset + 1 + p*3
r = data[index ]
g = data[index + 1]
b = data[index + 2]
if (int(r) + int(g) + int(b)) / 3 > 128{
pixel[p] = 255
}
}
}
return pixel, width, height
}
printFrame :: proc(pixel: []u8, width, height : int, setScrWidth := -1, setScrHeight := -1) {
defer delete(pixel)
scaleWidth : f32 = 1
scaleHeight : f32 = 1
scrWidth := width
scrHeight := height
if setScrWidth != -1 {
scrWidth = min(setScrWidth, width)
scaleWidth = f32(width) / f32(scrWidth)
}
if setScrHeight != -1 {
scrHeight = min(setScrHeight, height)
scaleHeight = f32(height) / f32(scrHeight)
}
str := strings.builder_make()
defer strings.builder_destroy(&str)
if scaleHeight == 1 && scaleWidth == 1 {
// without scaling
for y in 0..<height {
for x in 0..<width {
if pixel[y * width + x] == 255 {
strings.write_byte(&str, '#')
}
else {
strings.write_byte(&str, '.')
}
}
strings.write_byte(&str, '\n')
}
}
else{
// with scaling
BRIGHTNESS := " .`',:;-~=+*#%$@" // shoutout to the C donut devblog https://www.a1k0n.net/2021/01/13/optimizing-donut.html
for scrY in 0..<scrHeight {
yStart := int(f32(scrY) * scaleHeight)
yEnd := int(f32(scrY + 1) * scaleHeight)
for scrX in 0..<scrWidth {
xStart := int(f32(scrX) * scaleWidth)
xEnd := int(f32(scrX + 1) * scaleWidth)
sum := 0
count := 0
for y in yStart..<yEnd {
for x in xStart..<xEnd {
sum += int(pixel[y*width + x])
count += 1
}
}
strings.write_byte(&str, BRIGHTNESS[int(sum/count) / 17])
}
strings.write_byte(&str, '\n')
}
}
os.write_string(os.stdout, strings.to_string(str))
}
iterateFrames :: proc(folder : string, width : int, height : int) {
engine : aud.engine
result := aud.engine_init(nil, &engine)
if result != .SUCCESS {
fmt.println("could not start audio, press enter to continue: ", result)
libc.getchar()
return
}
defer aud.engine_uninit(&engine)
aud.engine_play_sound(&engine, fmt.ctprintf("%s/audio.wav", folder), nil)
os.write_string(os.stdout, ALT_SCR_T + CUR_HIDE + CLEAR)
defer os.write_string(os.stdout, CUR_SHOW + ALT_SCR_F)
startTime := time.tick_now()
for n := 1; ; n += 1 {
frame := fmt.tprintf("%s/%04d.ppm", folder, n)
data, err := os.read_entire_file(frame, context.allocator)
if err != nil {
break
}
printFrame(framePixelData(data), width, height)
delete(data)
free_all(context.temp_allocator)
os.write_string(os.stdout, HOME)
nextFrame := time.Duration(n) * FRAME_TIME
elapsed := time.tick_since(startTime)
if elapsed < nextFrame {
time.sleep(nextFrame - elapsed)
}
}
}
main :: proc() {
if len(os.args) < 2 {
fmt.println("args: \"file\" width(optional) height(optional)")
return
}
file := os.args[1]
dot := strings.last_index_byte(file, '.')
if dot == -1 {
fmt.println("file needs an extension, e.g. video.mp4")
return
}
folder := fmt.aprintf("%s_frames", file[:dot])
defer delete(folder)
width : int = -1
height : int = -1
if len(os.args) > 2 {
width, _ = strconv.parse_int(os.args[2])
}
if len(os.args) > 3 {
height, _ = strconv.parse_int(os.args[3])
}
makeFramesFF(file, folder)
fmt.println("press enter to play")
libc.getchar()
iterateFrames(folder, width, height)
libc.getchar()
}