All 8 entries tagged Ffmpeg

Scripting discovery of random YouTube videos

(I just discovered this post as unpublished draft from about three years ago. No idea why I didn't publish it. The method described in it still works. All the code is bash script. There's a comment in the last bit of code "needs more refining" which I leave as an exercise for those so inclined.)

Do you now, or have you ever wanted to, script discovery of random YouTube videos? I did recently and couldn't find anything useful online. So I made up my own method.

If you're thinking YouTube videos are identified by 11 character strings so you can generate a random 11 character string and use that, you're not technically wrong, but it's not the way to go about it. As a test I generated 1000 and none of them were were valid. This isn't at all surprising given how many possible values those 11 characters provide. In my observation, each character in can be an lower or uppercase letter a number, or a -. That's 63 possible characters. A calculator tells me that 63^11 is 62050608388552830000. (If you want to say that out loud, say "sixty-two quintillion", then mumble a bit.)

function getVideoID  {
   local id="";
   while [ "${id}" = "" ];do
      id=$(curl -s https://www.youtube.com/results?search_query=$( < /dev/urandom tr -dc A-Za-z-0-9 | head -c4) | grep -o 'watch?v=[a-zA-Z0-9]\{11\}' | sort | uniq | sort -R | head -1);
   done
   echo "${id/watch?v=/}";
}

That gets you a valid id, such as dQw4w9WgXcQ. If you discover videos entirely at random some of what you find will be NSFW. Really. It will be. The method I use to filter out NSFW content uses youtube-dl

function getVideoUrl  {
   local url="";
   url=$(./youtube-dl --age-limit 0 --get-url "${1}");
   echo "${url}";
}

videoID=$(getVideoID);

videoUrl=$(getVideoUrl "${videoID}");

If ${videoUrl} is not zero length then, in my experience at least, the video is SFW and it's value is an url of the raw video which could be used as input value for ffmpeg or whatever. (To emphasis, it is *my experience* that this method filters out NSFW content.) If you just want to download the whole video, youtube-dl can do that for you. (youtube-dl will find the highest quality version of the video by default. You may want to change that depending on your available bandwidth or what you intend to do with the video.)

Some videos on YouTube have a video component that is just a static image. E.g. someone's ripped an album and then combined the audio with the album cover art to create something that can be uploaded to YouTube. Such videos are visually uninteresting and maybe you want to identify those videos and discard them rather than use them in whatever it is you're doing that involves random YouTube videos. I did, so I worked out a way of doing that too. The method I've used is to generate a bunch of images from the video, then compare them in a way which gets a value that represents how much the images differ by. If that value is less than a certain value, discard it. I've used GraphicsMagick for comparing the images. ImageMagick can be used to but is slower. (The less powerful your hardware, the bigger the speed difference is. ImageMagick output is slightly different to GraphicsMagick so you can't just remove the "gm", the awk and cut arguments would need changing.) To extract the images you obviously first have to download the video and in the below the downloaded video is theVideo.mp4

# generate an image at 2 second intervals
ffmpeg -loglevel fatal -i theVideo.mp4 -vf fps=1/2 -y foo__%02d.jpg

if [ $? -eq 0 ];then

  # get an integer value that represents how different the images all are to each other
  v=$(gm compare -metric MAE foo__*.jpg null:-  | grep Total | awk '{print $2}' | cut -d . -f 2);

  if [ ! -z "${v}" -a "${v:0:2}" != "00" -a "${v:0:2}" != "01" ];then **** needs more refining 018 019 OK 010 not OK maybe test 3rd char too
    # the video isn't a static image
    # do whatever it is you want to do with it
  fi

fi

I arrived at discarding videos where the first two characters of v are 00 after calculating v for a bunch of videos.

Mike Willis : 05 Dec 2019 13:50 |  Tags: Bash Ffmpeg Graphicsmagick Imagemagick Random Youtube |  Comments (0) |  Close comments |  Report a problem

Quick and easy way to get ffmpeg on Raspberry Pi runing Raspbian

The Raspbian repos have avonv instead of ffmpeg. If, like me, you want ffmpeg because it has some functionality not available in avconv then you can use static builds. Go to https://ffmpeg.org/download.html#build-linux click the 'Linux Static Builds' link and so on. If you have a Raspberry Pi 3 get the armhf build. If you have one of the first generation Pi you need the armel build. I don't have a Pi 2, but if you do why not try both builds and leave a comment about which one works.

There's a bunch of blogs posts about getting ffmpeg on Raspbian by compiling it from source. I don't know if that'd result in a binary more optimised for the Pi than the static builds referred to above. I've found the performance of the static builds adequate enough that I haven't bothered trying to build it. I suspect that doing little more than a git checkout and build, as all the guides I found described, would result in a binary with the functionality I want. It'd take a while to find out given how long a build would take on the Pi, especially a first generation. Though you could do cross compiling on an x86 machine if you were so inclined.

First generation Pi running Raspbian 8

pi@pione:~ $ ffmpeg-3.1.4-armel-32bit-static/ffmpeg
ffmpeg version 3.1.4-static http://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.1 (Debian 5.4.1-2) 20160904
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --enable-libmp3lame --enable-libx264 --enable-libwebp --enable-libspeex --enable-libvorbis --enable-libvpx --enable-libfreetype --enable-fontconfig --enable-libxvid --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvo-amrwbenc --enable-gray --enable-libopus --enable-libass --enable-gnutls --enable-libvidstab --enable-libsoxr --enable-frei0r --enable-libfribidi --disable-indev=sndio --disable-outdev=sndio --enable-librtmp --cc=gcc-5 --disable-ffplay
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.101 / 57. 48.101
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
pi@pione:~ $ 

Pi 3 running Raspbian 8

pi@pithree: $ ffmpeg-3.2.2-armhf-32bit-static/ffmpeg
ffmpeg version 3.2.2-static http://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.1 (Debian 5.4.1-3) 20161019
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-5 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gray --enable-libass --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzimg
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'
pi@pithree:~/RandomYouTubeTwitterBot $ 

Mike Willis : 11 Jan 2017 21:48 |  Tags: Ffmpeg Pi Raspberry Raspbian |  Comments (0) |  Close comments |  Report a problem

Webcam timelapse – 2013

Follow-up to Webcam timelapse – January 2013 and February 2013 from Mike's blag

Yep, this is the whole of 2013 through the University webcam. I say whole. There will be a few small gaps of a minute or so here and there because the machine on which the script that grabbed the images was running did not have 100% uptime. Also it seems that at 15:23:26 on 19th March the webcam crashed or something because all the images from that time until 11:35:01 on 21st March are the same.

As with previous timelapses, images were grabbed from webcam once per minute. The video is made with 48 images per second. Each day lasts about 29 seconds and the video is 2 hours 59 minutes and ~1.2GB. No, I haven't sat and watched it all the way through.

Download

I put the video together by making a video for each day then joining them up. It could be done all in one go but making separate videos means it's easier to spot issues. For example I noticed the video for 20th March had a considerably smaller filesize than the others and that the videos for 19th and 21st were also slightly smaller than average. It also reduces the risk of leaving something running, checking it two hours later and finding all the output is garbage.

I used ffmpeg. The command for each day's video looks like

$ ffmpeg -r 48  -pattern_type glob -i '*.jpg' -an -vcodec libx264 -f mp4 -threads 0  -b:v 1000k foo.mp4

It took about two hours to generate all the videos on a 2.8Ghz Intel Core 2 Quad. (A single video took about 16 seconds. On a 1.6Ghz Intel Core Duo a single video took about five and half minutes and on an ARM Marvell Kirkwood 1.2GHz it took about 42 minutes.)

To join them up you need to make a list of all the filenames

$ for i in mp4s/*;do echo "file '${i}'" ;done > list.txt

Then use ffmpeg's concat demuxer

$ ffmpeg -f concat -i list.txt -c copy  -movflags faststart webcam2013.mp4

The -movflafs faststart argument tells ffmpeg to 'Run a second pass moving the index (moov atom) to the beginning of the file.' This means that when the video is viewed in a web browser playback can start straight away rather than waiting for the entire video to be downloaded.

Mike Willis : 14 Jan 2014 12:34 |  Tags: Ffmpeg Webcam |  Comments (0) |  Close comments |  Report a problem

Webcam timelapse – January 2013 and February 2013

Follow-up to Webcam timelapse – Monday 10th December 2012 – Sunday 16th December 2012 from Mike's blag

This time, entire months. As before, images grabbed from webcam once per minute. Videos are made at 48 images per second.

January 2013. If you want to see snow skip to 6:00. There's several instances of snow. You can also see the snow hangs around for ages on the roof on the far right.

February 2013.

Mike Willis : 09 Mar 2013 18:24 |  Tags: Ffmpeg Webcam |  Comments (0) |  Close comments |  Report a problem

Converting FLAC to AAC/M4A

I found myself wanting to convert some audio files that were encoded with FLAC to AAC/M4A that iTunes can understand. Conversion using ffmpeg is easy enough but I couldn't find a way to make ffmpeg copy the artwork that's embedded in the FLAC files. So I ended up using atomicparsley as well.

$ for i in *flac;do of="${i/.flac/.m4a}";af=foo.jpg;ffmpeg -i "${i}" -y "${af}"; ffmpeg -i "${i}" -vn -acodec libfaac -aq 320k -f mp4 -y "${of}";atomicparsley "${of}" --artwork "${af}" --overWrite;rm -f "${af}";done

I had to tell ffmpeg to force the mp4 container otherwise atomicparsley wouldn't process the resulting file.

Mike Willis : 31 Dec 2012 16:11 |  Tags: Aac Atomicparsley Audio Convert Ffmpeg Flac Itunes M4A |  Comments (0) |  Close comments |  Report a problem

Webcam timelapse – Monday 10th December 2012 – Sunday 16th December 2012

Follow-up to Webcam timelapse – Friday 7th December 2012. from Mike's blag

Same deal as before but this time it's of an entire week and they've been optimised for streaming, so they'll start playing back straight away rather than you having to wait for your web browser to download the whole thing. Depending on your patience there's three versions, presented here in decreasing duration.

10 images per second, duration approx 16:38

24 images per second, duration approx 6:56

48 images per second (y'know, like Peter Jackson did for The Hobbit. Only without being anything like that at all), duration approx 3:28

I've just realised these videos don't play in Firefox on my Linux machine. I can play the videos on my Linux machine, (I made them on my Linux machine), just not in Firefox. Lack of H264 decoding capability I guess, which I would guess means they won't work on Firefox on Windows either. I'm not curious enough to boot Windows and find out. The day long video was made on and posted from my Mac, on which Firefox happily plays back the videos, but my Mac is old and ffmpeg only manages to encode the videos at about 2fps. So I built ffmpeg with libx264 support on my not-as-old Linux machine to encode the week long ones in a sensible time. I guess I could upload an flv fallback video. But that would mean making such a thing and I can't be bothered, at least not right now.

Edit: They do play on another Linux machine of mine with Firefox and Totem plugin. Totem uses gstreamer and on the machine in question there's a gstreamer plugin that supports H264.

Mike Willis : 20 Dec 2012 20:36 |  Tags: Ffmpeg Webcam |  Comments (2) |  Close comments |  Report a problem

Webcam timelapse – Friday 7th December 2012.

Friday 7th December 2012 as seen through the University webcam in approximately 2.3 minutes.

Images were grabbed at a rate of one per minute, which in theory gives a total of 1440 images. There's actually only 1420 as 20 images scattered throughout the day didn't get saved for whatever reason. Video is constructed using 10 images per second. Total run time should be 142 seconds. According to ffmpeg the actual length of the generated video is 00:02:21.90. Don't know why.

I just occured to me that the video isn't optimised for streaming so it all needs to download before you can start to watch it. There's something in ffmpeg to do such optimisation apparently so I should look in to that at some point.

Coming soon, versions that cover a week, a month and a year. Maybe. And for varying values of soon. Imagine how long it could take to get a year's worth of images! Obviously wouldn't be able to get a consistent one image per minute for an entire year, that's obvious just from the 20 missing images in a single day. It'd be an exercise in maintaining uptime really. Or having more than one machine grabbing the images. By my calculations a year's worth of images would be about 11GB. A video created using 10 images per second would be over two days long.

Mike Willis : 09 Dec 2012 14:04 |  Tags: Ffmpeg Webcam |  Comments (1) |  Close comments |  Report a problem

Why not mash clips of all your videos together?

A long time ago, in a galaxy street far not all that far away, there is a nightclub I visited on N occasions. You might know it as Kasbah because you're much younger than I am and don't remember when it was called The Colosseum. Anyway, it had a big screen at one end that showed random video clips of a few seconds each. Bits of cartoons, the Enterprise D going to warp, all sorts of things. For reasons that completely escape me I started thinking about that recently and wondering how easy it would be to make a video sequence like that. I have no practical use for such a thing, but it seemed like an interesting exercise.

So if you've been wondering how you can easily create a video that comprises randomly selected clips then this is the blog post for you. To create such a video you will need:

• One computer running a Unix-like Operating System. (I tested on Mac OS X 10.6 and openSUSE 12.1)
  
• Lots of video. The more video you have, the more random (for a given value of random) the clips will be.
  
• A copy of ffmpeg compiled with libx264 support. If you're using a Mac I highly recommend you have MacPorts installed or be prepared to hack the script a bit to make it work.
• Script that mashes up videos which you save in to a directory on it's own and then run. It will plunder your video collection and assemble a montage of random clips of total length defined by a variable in the script (default is 60 seconds). The number of clips and length of each clip varies on each run.
  

You'll want to read the comments at the start of the script before running it. The output is 1280x720 H.264 encoded in an mp4 container. Aspect ratios are not preserved. Any video clip from a source with an aspect ratio that isn't 16:9 will be stretched to 16:9. I did consider preserving aspect ratio but doing so would have meant that some clips had back bars either side of them and some wouldn't. That would be visually jarring. The output has no sound because it would be horrible if it had sound. I did try adding sound but it went hopelessly out of sync with the video and I couldn't be bothered figuring out making that work.

Ideally there would be a sample of the output here. In practise there isn't because of concerns about licenses, credits and all that sort of stuff.

Mike Willis : 02 Jul 2012 21:33 |  Tags: Ffmpeg Mashup Video |  Comments (0) |  Close comments |  Report a problem