Live Stream Your Pets with Linux and YouTube!

Live Stream Your Pets with Linux and YouTube!
Image
Shawn Powers Thu, 10/26/2017 - 06:53

Anyone who reads Linux Journal knows about my fascination with birdwatching. I've created my own weatherproof video cameras with a Raspberry Pi. I've posted instructions on how to create your own automatically updating camera image page with JavaScript. Heck, I even learned CSS so I could make a mobile-friendly version of BirdCam that filled the screen in landscape mode.

Recently, however, I've finally been able to create an automated system that streams my BirdCam live over YouTube. It starts when the sun comes up and stops when the sun goes down. And thanks to some powerful open-source software, I never have to touch the system!

Some of the tools I describe here have been covered in other articles, but this is the first time I've been able to create a stream that anyone can see utilizing bandwidth Google pays for!

Figure 1. Birds are always camera-shy. If you watch long enough, however, they come and steal peanuts!

My List of Ingredients

First off, I want to be clear about what sort of hardware and software is required in order to accomplish something similar to what I'm doing:

  • A Linux computer: if you plan to use USB cameras, this needs to be a physical computer. If your video source is network-based, this can be a virtual machine on your network. A Raspberry Pi isn't really powerful enough for the video work that has to be done, unless maybe it's low-resolution. I have an old i5 CPU running at 1.6GHz, and it's more than enough.

  • A video source: this can be pretty much any video source you have at hand. If you plan to use a USB webcam, you'll need to be sure you are using a physical Linux computer as noted above. I've used USB, MJPEG over http (see my old BirdCam articles), cheap wireless security cameras that have an RTSP stream, and most recently, I started using UniFi video cameras. In fact, if you are considering purchasing outdoor video cameras for a project like this, I can't recommend UniFi cameras enough. They are PoE, HD and the free software handles recording and provides RTSP streams that have both HD video and top-notch audio.

  • A YouTube account with Live Streaming enabled: you'll need to verify your account, and then enable live streaming here. It's not a difficult process, but without following those steps, you won't be able to use the free service.

  • Open Broadcaster Software: I've tried multiple ways to use a CLI solution to stream directly to YouTube with FFmpeg or mencoder, but I've never been able to make it work consistently. I was hesitant to use OBS, because it's a GUI solution and doesn't have a CLI interface, but I worked around that problem, and I'm actually happy to have the GUI now.

  • A web server to host your embedded channel: you could just share the URL to your YouTube channel, but embedding is much cooler, because you can integrate it into your own site.

  • Enough upstream bandwidth to support 1.5–2mbps while streaming: since YouTube is going to redistribute, the local bandwidth requirements don't change regardless of how many people are watching your stream. For some folks (like me, unfortunately), sacrificing that much bandwidth is difficult and sometimes causes issues. Just know that it takes a small, but not insignificant amount of constant upstream bandwidth to stream live video. That should be obvious, but it's something to consider.

  • A few other utilities like crontab and sunwait: the latter is only if you want to time your streams with sunrise and sunset. And, crontab is needed only if you want to automate the starting and stopping. Those touches really make a difference for me though, so I encourage you to consider it.

Gather Your Info

YouTube:

In order to live stream, you'll need a few bits of information. As I mentioned above, you'll need to verify your account to turn on streaming. Then you'll need to get your streaming key (Figure 2). It's important that you not share the streaming key, because it acts like your authentication. If others get your key, they can stream to your channel, even without your user name.

Figure 2. That's not my real streaming key, just FYI.

The other bit of information you'll need from YouTube is your channel ID. It's not easy to find the channel ID, but if you want to embed your video, you'll need it later. Head to this page, and find the line that looks like, "YouTube Channel ID: UCbUTB3bVg3cmeyJUtUC9DPA" (your channel ID will be different from mine). The long string of text is your channel ID, copy that somewhere easy to find.

Video Camera Feeds:

I can really give you only hints about what to look for here. You need to find the streaming video feed coming from your camera. Make sure you don't use the web page that has the stream embedded (most cameras have a rudimentary web server that embeds the stream). You need the raw feed itself. Google or the user's manual will be your best bet for figuring out the raw stream URL.

I have an Onvif-compatible video camera that has an MJPEG stream URL that looks like this: http://192.168.1.170:9090/stream/video.mjpeg.

One of my Foscam cameras requires a user name and password in the URL in order to get the stream. It looks like this: http://192.168.1.180:88/cgi-bin/CGIStream.cgi?cmd=GetMJStream&usr=admin&pwd=xxx.

And my new UniFi cameras actually use an RTSP URL that comes from the UniFi server instead of from the cameras directly. It looks like this: rtsp://192.168.1.16:7447/58cf11bef14c359f4b3c7b2e_1.

The point I'm trying to make is that finding your video camera's streaming URL often is challenging. If you do it before you start, it can save hours of frustration. An easy way to test if you've found the correct URL is to try opening it in VLC. I haven't found a video camera that VLC can't view, so if it complains about an invalid video source, you probably don't have the correct URL. Google, along with your camera's model number, is probably the best way to figure it out.

The Software

There are many scripts online claiming to stream from a camera source to YouTube using FFmpeg. I'm sure they work for someone, but I've never gotten them to work, no matter how many settings I tweak. In fact, I gave up for quite a while because I didn't want to rely on a GUI interface to stream. I wanted my server to do the dirty work and do it without my interaction. One day recently, however, I discovered that Open Broadcaster Software (OBS) supports command-line flags for starting streaming. That means I could have the server start streaming without the need to "click" anything.

One problem I had to overcome was the lack of the X Window System on my BirdCam server. There's no monitor connected to the server, but in order for OBS to work, it has to have a logged-in GUI desktop. I hooked up a monitor long enough to get a GUI installed and then set the system to log in automatically. I also disabled all power-saving features for the monitor, because I wouldn't have one logged in anyway. Once it was set up, I installed TeamViewer so I could control the system remotely if I needed to. There have been some issues with TeamViewer's security recently, so it might not be the software you choose for controlling the server, but it's what I have installed, and it works. Figure 3 shows my "server" controlled remotely via TeamViewer.

Figure 3. I don't normally have the security camera to my cat's litter cave on my live stream, but I wanted to show multiple cameras.

Installing OBS is simple. Head over to the OBS site and download the latest version, or simply install their PPA if you're using Ubuntu. The software has matured since I last mentioned it, and I didn't have any problems with dependencies, even when connecting over a remote session.

OBS also has the great feature of saving your last-used session. That means once you set up your cameras, you don't have to worry about readjusting them on the next launch. OBS just uses the same settings you had before. If you look back at Figure 3, you'll see there are multiple cameras added to the preview window. Without the need to save a layout, OBS just remembers from launch to launch how you had the cameras positioned.

In order to get the best results, you need to tweak a few OBS settings. Click the settings button, and then head over to the Video tab (Figure 4). This is a little confusing, but you have two different resolutions to set. The "canvas" is how big you want OBS to show on your preview window. The "output" resolution is what it scales your video to for streaming and recording. I just set them both to 720p, because I figure scaling takes CPU. You also set the frames per second (FPS) for the output video. I use 10FPS with the 720p size. You can adjust this if you want 1080p, or down if you don't have bandwidth.

Figure 4. The resolutions are flexible, but I like to keep it simple.

Next, click on the "Output" tab (Figure 5). My settings are visible, and I recommend keeping them close to mine, except for the bitrate of the video and audio. If you want higher quality video (and you can afford the bandwidth), this is where you set the average upload speed. You also can change the audio quality if you want higher quality. Keep in mind that the resolution you chose in the last step will work with the bandwidth you selected here to give you the video quality users will see. A video rate of 1500 (measured in kbps) works well with my 10FPS and 720p resolution. But if you try to stream 1080p, 30FPS video with 1500kbps, it's going to be really poor quality video. You'll have to experiment to find the sweet spot.

Figure 5. 1500 is the maximum my current internet connection can handle.

The "Stream" tab is where you configure the streaming service you want OBS to use (Figure 6). You should be able to select YouTube and then paste that stream key you got from YouTube earlier. (This is not the channel ID; it's that hidden key from back in Figure 2.) Once entered, you shouldn't need to make any changes in settings. OBS will keep all the settings, including streaming information.

Figure 6. It's truly amazing how well OBS does with YouTube streaming.

All that's left is to add the camera(s) to your preview screen. This is the nicest feature of OBS, well apart from actually being able to stream to YouTube. The setup is drag and drop, and you can resize cameras, overlap cameras and arrange them however you want. Since OBS supports so many types of inputs, you can get crazy with text overlays and so on. To add a network camera, click the + at the bottom middle of the main window, and select "media source" (Figure 7). Then uncheck "local file" and enter the camera URL in the "input" field (Figure 8). Once you click OK, your camera should appear on the preview window, and you can resize and move it. The interface also allows you to crop the section of the video you want to use. It's very powerful and incredibly user-friendly. Plus, as I mentioned earlier, OBS stores all your tweaks automatically, so the next time you start it, you'll get the same arrangement.

Figure 7. Media source isn't obvious as the choice for network cameras.

Figure 8. Be sure to uncheck the "local file", or you won't have an input field.

Once you have your camera(s) set up, you can decide whether you want to include audio if your camera supports it. The audio levels should appear in the column next to the list of cameras. Then just click "Start Streaming" to send your stream live to YouTube. It takes 30 seconds or so to show up in the YouTube dashboard, but now is the time to make sure streaming works.

Automation

I could just leave OBS running 24/7 and have it stream my bird feeders all night. Honestly, I'm not sure how YouTube would handle a 24/7 stream, but I don't want to do that anyway. I not only want to automate the starting and stopping of OBS, but I also want to make sure that if something crashes, it starts back up the next day without me needing to fix it. Cron was the obvious way to manage that, but since OBS is a GUI program, cron proved to be challenging. In the end, I was able to include environment variables in my crontab, and things worked smoothly. Here's what my OBS part of crontab looks like. Check it out, and I'll explain it afterward:


DISPLAY=:0
@reboot sleep 10; obs --startstreaming
0 5 * * * /usr/local/bin/sunwait civ up 45.3733N 84.9553W;
 ↪obs --startstreaming
0 16 * * * /usr/local/bin/sunwait civ down 45.3733N
 ↪84.9553W; pkill obs

First off, setting the DISPLAY environment variable to :0 means that crontab can launch a GUI application on the current desktop. I was embarrassed when I realized how easy it was to get cron to launch GUI apps. It is important to note that the user must be logged in, however.

The @reboot line starts OBS when the system boots. The simple --startstreaming flag tells OBS to launch and immediately start streaming. It's awesome. Really, if I had to figure out a way to automate actually clicking a button, we probably wouldn't be doing this project together.

The next two lines are a little confusing. First off, I have the program "sunwait" installed. It's an old program, but it's so incredible, I can't believe it's not in every distribution by default. I've mentioned it before in BirdCam articles, but basically, it's a C program that determines sunrise and sunset based on your longitude and latitude. The last version was released in 2004 (seriously), but it still compiles. You can get the source here.

Anyway, those two cron lines tell the server to start and stop OBS at sunrise and sunset. At 5AM, I tell sunwait to "wait" until the sun rises. It literally just waits until sunrise and then ends. Once it ends, OBS is started up. Then at 4PM, I tell sunwait to wait until sunset, and after the sunwait program ends, kill stops OBS. Why 5AM and 4PM? Well, in my part of the world, the sun never rises before 5AM and never sets before 4PM. There is the potential problem that if I reboot my server after 4PM, it will stream all night. But that potential problem doesn't concern me enough to make the logic more complicated.

Since my server doesn't have a monitor or keyboard connected, a random GUI application starting and stopping in the middle of the screen doesn't affect anything. Since I connect to my server's desktop only when I want to make a change to OBS, it's actually convenient that it's always running front and center on my desktop! I couldn't be happier with the current live stream setup.

Embedding the Stream

Not long ago, YouTube made a change so that every time a live stream starts, it gets its own embed code. That means if you simply use the "share" button on the live stream to get the embed code, it will work only for that current streaming session. For me, that means the next day it would show a recording of the previous day, but not the live stream. I'll be honest, that quiet change was very frustrating! Thankfully, there is a way to embed the actual live stream, so that any time you start live streaming, it becomes active—that's where the Channel ID you got earlier comes in.

Here is the embed code for my live stream at http://birds.brainofshawn.com">:




Obviously, you'll need to make the changes for your own channel, but it should be clear what the various things mean. I stuck with the 720p size even on my embedded page. Since this is embed code, you don't have to put it on its own page like I did; you could embed a tiny resolution version on your blog, for instance.

Setting up the live stream through YouTube is nice for several reasons. One, your bandwidth requirements don't change even if you have 10,000 viewers. Also, since it's YouTube, you can "cast" the video to a television or Chromecast device and show off your channel to your friends. I still hope to get more cameras and maybe set up camera rotation on multiple bird feeders, but for right now, I couldn't be happier. Enjoy!

Live Stream Your Pets with Linux and YouTube!

Anyone who reads Linux Journal knows about my fascination with birdwatching. I've created my own weatherproof video cameras with a Raspberry Pi. I've posted instructions on how to create your own automatically updating camera image page with JavaScript. Heck, I even learned CSS so I could make a mobile-friendly version of BirdCam that filled the screen in landscape mode. more>>

Working with YouTube and Extracting Audio

Working with YouTube and Extracting Audio
Image
Dave Taylor Tue, 10/10/2017 - 06:48

In my last few articles, I've been exploring the capabilities of ImageMagick, showing that just because you're working on a command line doesn't mean you're stuck processing only text. As I explained, ImageMagick makes it easy to work with images, adding watermarks and analyzing content far more accurately than with the standard Linux file command, and much, much more.

Continuing in a similar vein, I want to look at audio and video in this article. Well, maybe "listen" to audio and "look" at video, but again, I'm still focusing on the command line, so in both instances, player/viewer apps are required.

YouTube to MP3 Audio

As someone who watches a lot of lectures online, I'm also intrigued by the online services that can extract just the audio portion of a YouTube or Vimeo video and save it as an MP3. Listening to a lecture while driving is far safer than trying not to watch a video on the move, for example.

Since there are so many live concert performances online, many people also like to use a video-to-MP3 service to add those songs to their music libraries.

Note: be leery of copyright issues with any download and conversion of content. Just because it's on Vimeo, YouTube or other online service, doesn't mean you have permission to extract the audio or even download it and save it on your computer.

Let's start with the most basic functionality: downloading a video from YouTube so you can watch it on your Linux system. There are a lot of browser plugins and even websites devoted to this task, but who wants to risk malware or be plagued by porn site ads? Yech.

Fortunately, there's a terrific public domain program called youtube-dl on GitHub that covers all your needs. At its most basic, it lets you download video content from YouTube and a variety of other online video repositories, but as you'll learn, it can do quite a bit more.

You can grab a copy for your system here.

Let's start by downloading a copy of one of my own YouTube videos. It's a review of the splendid 1More quad-driver headphones, and its URL is https://www.youtube.com/watch?v=BFL1E77hTHQ.

As an aside: I have a YouTube channel where I review consumer electronics and gadgets. You should subscribe! Find all my videos at http://youtube.com/askdavetaylor.

YouTube has a bunch of ways it can assemble a URL, however, including using its URL-shortener youtu.be, but fortunately, youtube-dl can handle the variations.

Downloading a copy of the video to the current working directory is now as simple as:


youtube-dl 'https://www.youtube.com/watch?v=BFL1E77hTHQ'

The full output of the command is a bit, um, hairy, however:


$  youtube-dl 'https://www.youtube.com/watch?v=BFL1E77hTHQ'
[youtube] BFL1E77hTHQ: Downloading webpage
[youtube] BFL1E77hTHQ: Downloading video info webpage
[youtube] BFL1E77hTHQ: Extracting video information
[youtube] BFL1E77hTHQ: Downloading MPD manifest
WARNING: Requested formats are incompatible for merge and
will be merged into mkv.
[download] Destination: 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.f137.mp4
[download] 100% of 118.74MiB in 02:49
[download] Destination: 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.f251.webm
[download] 100% of 4.81MiB in 00:03
[ffmpeg] Merging formats into "1More Quad Driver In-Ear
Headphones Reviewed-BFL1E77hTHQ.mkv"
Deleting original file 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.f137.mp4 (pass -k to keep)
Deleting original file 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.f251.webm (pass -k to keep)
$

You can wade through the output messages, but it's the message from companion open-source program ffmpeg that's most important: merging formats into ... mkv.

In other words, the download format of the video is MKV by default. MKV is part of the increasingly popular Matroska Multimedia Container format, and it works with a lot of video players (including VideoLan, aka VLC, my favorite cross-platform video player).

A quick ls reveals the result and that the default filename is taken from the title of the video, something that might not be particularly desirable:


$ ls -lh *mkv
-rw-r--r--  1 taylor  staff   124M Jan 31 16:56 1More Quad
Driver In-Ear Headphones Reviewed-BFL1E77hTHQ.mkv

Do you prefer to specify the output name and have the output file in MP4 (MPEG4) format instead? That's doable:


$ youtube-dl -o 1more-review.mp4 -f mp4 \
    'https://www.youtube.com/watch?v=BFL1E77hTHQ'
[youtube] BFL1E77hTHQ: Downloading webpage
[youtube] BFL1E77hTHQ: Downloading video info webpage
[youtube] BFL1E77hTHQ: Extracting video information
[youtube] BFL1E77hTHQ: Downloading MPD manifest
[download] Destination: 1more-review.mp4
[download] 100% of 57.63MiB in 00:27

As a bonus, you get less ominous informational messages from the program too, so it's cleaner. And the output, sure enough, is in MP4 format:


$ ls -lh *mp4
-rw-r--r--@ 1 taylor  staff  58M Jan 31 16:57 1more-review.mp4

As a second bonus, it's also more efficient in its video encoding, so the MP4 version of the downloaded video is only 58M as opposed to the 124M of the MKV-merged version.

So how do you watch it? Most likely, do a double-click and it'll be up and running, as shown in Figure 1.

Figure 1. Downloaded YouTube Video Playing in Ubuntu Player

That's easy enough, but the original goal was to be able to extract just the audio component of a YouTube video, so let's look at that task.

Downloading Just the Audio Track

Since I've already started to delve into the command-line options for the youtube-dl program, it's not a leap to find out that there's yet another command-line option that lets you save just the audio portion of a video:


$ youtube-dl -x --audio-format mp3 \
    'https://www.youtube.com/watch?v=BFL1E77hTHQ'
[youtube] BFL1E77hTHQ: Downloading webpage
[youtube] BFL1E77hTHQ: Downloading video info webpage
[youtube] BFL1E77hTHQ: Extracting video information
[youtube] BFL1E77hTHQ: Downloading MPD manifest
[download] Destination: 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.webm
[download] 100% of 4.81MiB in 00:07
[ffmpeg] Destination: 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.mp3
Deleting original file 1More Quad Driver In-Ear Headphones
Reviewed-BFL1E77hTHQ.webm (pass -k to keep)
$ ls -lh *mp3
-rw-r--r--  1 taylor  staff   4.0M Jan 31 18:22 1More Quad
Driver In-Ear Headphones Reviewed-BFL1E77hTHQ.mp3

That's easy enough, and the output is delightfully small: 4MB total. The problem is, there's the same awkward naming issue, so the addition of -o output-filename definitely will be a win. But, really, youtube-dl makes these tasks trivially easy, as long as you're willing to figure out all of its command-line options.

Writing a Wrapper Script

Instead of worrying about the obscure command-line flag notation, let's just write a script that does the heavy lifting for you. I'm going to call it ytdl for "youtube download", and by default, it'll accept just a URL and output an MP4 format video file that has the same name as the YouTube shortcut (for example, the above video would become BFL1E77hTHQ.mp4).

Add a second parameter, and that becomes the output filename. Specify the -a flag, and it saves audio output only, in MP3 format instead.

Let's start with a usage block if the user forgets to specify anything or just needs a simple reminder:


if [ $# -eq 0 ] ; then
  echo "Usage: $(basename $0) {-a} YouTubeURL {outputfile}"
  echo "   where -a extracts the audio portion in MP3 format"
  exit 1
fi

That's easy enough. The script is also going to use some predefined combinations of flags to make it easier to write:


youtubedl="/usr/local/bin/youtube-dl"
audioflags="-x --audio-format mp3"
videoflags="-f mp4"
flags=$videoflags       # default set of command flags
audioonly=0             # default is audio + video

If the user specifies the -a flag, audioonly will be set to true (that is, 1), and the default flags will switch from video to audio:


if [ "$1" = "-a" ] ; then
  audioonly=1
  flags=$audioflags
  shift
fi

You'll recall that the shift command moves all the parameters "down" one to the left, so $2 becomes $1 and so on. It's an easy way to process and discard parameters in a script, of course.

The biggest block of code creates a default output filename from the YouTube URL:


if [ $# -eq 1 ] ; then
  # no output filename specified
  outfile=$(echo "$1" | cut -d= -f2)
  if [ $audioonly -eq 1 ] ; then
    outfile="$outfile.mp3"
  else
    outfile="$outfile.mp4"
  fi
else
  outfile="$2"
fi

This isn't the most robust code, because it assumes that the URL specified is in a format like the examples used herein, youtube-yadda-yadaa?value=shortcode. It extracts the shortcode and simply appends an appropriate filename suffix. There are better ways to do this, but that's okay, this'll work for now. Just realize that your output format might be a bit weird if you have a very different type of YouTube URL or a URL from another site.

And, finally, the actual invocation of the youtube-dl command:


$youtubedl $flags -o "$outfile" "$1"

That's it! Now you can download a video as simply as:


$ ytdl 'https://www.youtube.com/watch?v=5yXDzg_QDGw' wiper.mp4

And an audio portion with:


$ ytdl -a 'https://www.youtube.com/watch?v=5yXDzg_QDGw'

Nice, eh?

I've way overrun my space for this column, but this is such a fun and simple script atop a terrific, powerful program, that it's worth it, right? And now you know how to make YouTube work for you, rather than vice versa!