Capturing ATSC TS to DVD
From Catholicpenguin
This document describes how to capture over-the-air ATSC television and convert it to a DVD, in the same fashion as the hardware DVD recorders for sale in retail stores. This document assumes Windows, as many USB TV tuner cards only provide drivers for Windows. Despite that, only open source software is used, so assuming your capture device works on Linux, only minor modifications should be needed.
Contents |
Prerequisites
- An ATSC capture device with Windows DirectShow BDA drivers. (Most of them)
- VLC. Choose the most recent version, but any version >= 1.0 should work. Add the location of vlc.exe to your PATH environment variable.
- Java. You will need the JDK, not just the JRE, as ProjectX requires compiling. Once installed, find the location of the jdk folder in Program Files, then add a system variable called JAVA_HOME pointing to that directory.
- ProjectX. Extract the ZIP file someplace. From the command prompt, run the build.bat file, which compiles the Java source. Once complete, there should be a ProjectX.jar file in the same directory. Add this directory to your PATH environment variable. Create a file in the same folder called projectx.bat, with the following contents, modifying the location of the ProjectX folder as appropriate:
java -jar "C:\Program Files\ProjectX_Source_0.90.4\ProjectX.jar" %1 %2 %3 %4 %5 %6 %7 %8 %9
- mencoder. Depending on the exact distribution you download, you may have to extract the ZIP file someplace, or an installer may place the files in Program Files for you. Add the location of mencoder.exe to your PATH environment variable.
- dvdauthor. This site provides windows binaries. Extract the ZIP file someplace, then add the location of that folder to your PATH environment variable.
- cdrtools. This site provides windows binaries. Extract the ZIP file someplace, then add the location of that folder to your PATH environment variable.
Background, and Overall Flow
Background
Digital TV in the United States is, since June 2009, broadcast according to standards, primarily A/53 developed by the Advanced Television Systems Committee. These standards specify one or more programs of MPEG2 video, AC3 audio, and closed caption data, bundled into an MPEG2 Transport Stream. This transport stream is encoded for redundancy, then directly broadcast over the air. The video and audio streams are subject to constraints on image size and total bitrate. Many combinations are allowed by the standards. For video, common formats are 1920x1080 interlaced (1080i), 1280x720 progressive scan (720p), or 704x480 progressive or interlace (typically used for the lower quality 2nd or 3rd subchannel programs). For audio, AC3 is used, at varying bitrates and channel configurations. In any case, the total bitrate for video, audio, and metadata such as closed captioning must not exceed around 18 Mbps.
DVDs also contain MPEG2 video and AC3 audio, except the formats of these video and audio streams are limited by a strict set of constraints, and are stored in an MPEG2 Program Stream (as opposed to the transport stream used for ATSC). In particular, the allowed video sizes are smaller (720x480 interlaced scan is the largest size allowed, and the most common), and the total bitrate must not exceed around 9.8Mbps.
As shown above, although ATSC and DVDs use the same underlying MPEG2 and AC3 technologies for storing video and audio, due to different constraints on the formats allowed for the video streams, as well as different containers (transport versus program stream) used to store the video and audio, it is not possible to simply copy the digital data received off the air onto a DVD.
Ideal Conversion Flow
At the absolute minimum, the video and audio streams must be demultiplexed (extracted) from the transport stream, then multiplexed (combined) into a DVD compliant transport stream known as a VOB. If the video resolution and bitrate transmitted by the television station is compliant with the more limited DVD specifications, as may be the case for the lower quality 2nd or 3rd subchannel programs, theoretically this is all the processing that would be required to record ATSC TV onto a DVD. However, in practice, even if these constraints are met, this solution is impractical for reasons outlined below. More commonly, the user will wish to record a high definition video stream, which probably means 720p or 1080i. These video dimensions are not allowed on a DVD. In this case, the video will need to be resized and transcoded to fit within the constraints of a DVD, after which a DVD program stream could be created, and the recording process complete.
Unfortunately both cases only work if the data received from the television station is completely free of errors. Sadly, this is very rarely the case, as anyone familiar with watching over the air television is probably aware. Trees blowing in the wind, cars driving by, or simply walking around one's home can be enough to momentarily disrupt a weak television signal. When this happens, there will be errors in the MPEG2 transport stream received. Transport streams are designed to accommodate such errors, and when they occur, televisions simply skip past the bad data and move on. You may have noticed this effect as temporary discolored blocks appearing in the picture, or the audio momentarily dropping out.
MPEG2 program streams, used on DVDs, however, are not designed to accommodate errors, as it is assumed the underlying medium is generally reliable. Therefore, while in theory it is possible to place the damaged error-containing video and audio streams received over the air directly into a DVD program stream, the results would be unpredictable, especially when read by hardware DVD players not designed to handle erroneous video and audio streams. Therefore, copying video and audio streams directly from an ATSC program onto a DVD, even if the video is complaint, is generally a bad idea.
Ideally, the problem of errors in the incoming video and audio bitstreams could be mitigated during transcoding. Since transcoding involves first decoding the original files, during this decoding process, invalid video or audio could simply be replaced with, perhaps, a black picture or silent audio when creating the files for the DVD. While the resulting file would appear visually wrong in spots, it would not contain any errors in the actual video or audio bitstreams, which could then be recorded to a correct DVD. This sounds great in theory. This is in fact what hardware DVD recorders do.
One such tool which can, in theory, perform this entire chain of processing is the Video LAN Client, or VLC. In theory, it can receive a transport stream from over the air and transcode it to DVD compliant video, in realtime, all within one program. Annoying, almost all open source video and audio processing tools, including VLC, are broken in their ability to perform this type of transcoding properly.
Actual Conversion Flow
What actually happens is that almost nothing handles the transcoding of video and audio streams with errors properly. Without fail, every single transcoding tool I have tried, including VLC, ffmpeg, mencoder, and HandBrake fail to correctly handle audio/video synchronization in the face of errors. (I realize this is probably because all these tools utilize the libav* libraries to demux and decode the video/audio.) Once a portion of the video or audio is lost, the video becomes de-synchronized with the audio, and remains that way for the remainder of the stream. Transport streams include timestamps intended to allow video and audio to be resynchronized after transmission errors. Several tools, such as VLC and mplayer, utilize these timestamps when playing transport streams. It is a shame that none of these tools appear to utilize these timestamps during transcoding.
As if that wasn't bad enough, VLC, instead of inserting black video or silent audio when presented with invalid input, and subsequently generating correct transcoded bitstreams, actually sometimes carries these errors into the transcoded bitstreams, almost invariably producing invalid output unusable for recording to a DVD.
Upon discovering these problems, what followed was a long period of trial and error, attempting to discover which portions of the various tools functioned correctly enough to use, and which portions must be avoided. VLC is still good for receiving transport streams from the physical TV tuner device, so long as no attempt is made to decode or transcode the stream. ffmpeg, mencoder, and HandBrake, when presented with synchronized video and audio streams, even those with minor errors, transcode just fine.
The key breakthrough in this process is to utilize an external transport stream demuxer, such as ProjectX, although there are others such as xport. These demuxers extract the elementary video and audio streams from the transport stream, and in the process, remove the portions containing errors. If these errors would cause the video and audio to become desynchronized, the appropriate amount of data is removed from either the video or audio track to bring them back in sync.
In practice, the video and audio streams extracted from ProjectX still may contain minor errors, but they usually are not enough to crash the transcoder or desync the video/audio.
Capture ATSC channel to an MPEG2 Transport Stream
Remove the newlines in the VLC command, and convert this to a BAT file.
@rem %1 -> Filename
@rem %2 -> RF channel
@rem %3 -> Logical channel (the 3 in 3.1)
@rem %4 -> Subchannel (the 1 in 3.1)
@rem %5 -> Record time in seconds
"%VLC%" atsc://frequency=0000 --program %4 --run-time=%5
:dvb-bandwidth=-1 :dvb-caching=300
:dvb-frequency=0 :dvb-network-name= :dvb-create-name= :dvb-adapter=-1 :
dvb-inversion=2 :dvb-polarisation=H :dvb-network-id=0 :dvb-azimuth=0
:dvb-elevation=0 :dvb-longitude=0 :dvb-range= :dvb-lnb-lof1=0
:dvb-lnb-lof2=0 :dvb-lnb-slof=0 :dvb-srate=27500 :dvb-modulation=-1
:dvb-major-channel=%3 :dvb-minor-channel=%4 :dvb-physical-channel=%2
:dvb-code-rate-hp=-1 :dvb-code-rate-lp=-1 :dvb-bandwidth=-1
:dvb-guard=-1 :dvb-transmission=-1 :dvb-hierarchy=-1
:sout=#std{access=file,mux=ts,dst=%1.ts}
Transcode to a DVD
@rem Converts a MPEG TS to a DVD
@rem Call with the name of the TS file -> end up with name.iso
@rem Uses a temporary file named tmp.mpg, so "tmp" cannot be used as a file name.
cmd /C projectx -demux %1.ts
mencoder -oac lavc -ovc lavc -of mpeg
-mpegopts format=dvd:tsaf -vf scale=720:480,harddup
-srate 48000 -af lavcresample=48000
-lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=5000:
keyint=18:vstrict=0:acodec=ac3:abitrate=192:aspect=16/9:threads=2
-ofps 30000/1001 -o tmp.mpg %1.m2v %1.ac3
dvdauthor.exe -o dvd -x author.xml
mkisofs.exe -dvd-video -V %1 -o %1.iso DVD
del %1_log.txt
del %1.m2v
del %1.ac3
del tmp.mpg
rmdir /s/q dvd
For some reason, cropping 4:3 content (such as local news) which is contained in a 16:9 feed crashes mencoder upon receiving bad MPEG2 video, so there's no provision for using that. Actually, quite a bit crashes mencoder. It may be required to fall back to ffmpeg in these cases:
ffmpeg -i %1.m2v -i %1.m2v -target ntsc-dvd tmp.mpg
Note that the ffmpeg installed by port install ffmpeg on OS X, is as of this writing, roughly twice as slow as mencoder. Also, the ntsc-dvd target uses MP2 audio, which I've found doesn't handle glitches in the incoming data as well as mencoder using AC3. Windows binaries do not appear to have these problems.
Most builds of mencoder are not compiled with ffmpeg-mt, so the threads=2 doesn't help that much; it appears only a 10% speed boost, possibly because some of the encoding and decoding is done in different threads this way.
The mencoder command line flags were taken from their document on creating DVDs, under the 11.8.5.2. NTSC DVD section.
