API/interface to pull recorded show listings, metadata, and files off of the Tablo

@jskenney

I have applied minor changes to b33 so that the file names would work when MCEBuddy picks them up and then drops them to my Plex server.  If you would like to incorporate them, please let me know and I will send you the file.

I feel stupid asking this question but how do you set the path to FFMPEG on OSX? If I type FFMPEG in terminal, I am able to run it.

There are two options, one is to change the defaults by editing the script, you will see the lines

    'tvdir':    ['./tv',               'PATH',   'Save TV shows here'],
    'moviedir': ['./movies',           'PATH',   'Save movies to this directory'],
    'tempdir':  ['./',                 'PATH',   'Location of temp directory'],
    'faildir':  ['./fail',             'PATH',   'Location to save unknown files'],
    'existdir': ['./exists',           'PATH',   'Location to move duplicate files'],

just change the portion inside the quotes (ie ./tv or ./movies) to what you want them to be, this is how you can make anything permanent.

To change it via the command line, give it the options in this format

python tablo2go-2.0b33.py -tvdir:/mytvshows -moviedir:/mymovies

or use windows like paths

python tablo2go-2.0b33.py -moviedir:c:/movies -tvdir:"c:/tv shows"

use the option -help to see all of the available fields.  

Make sure you tell it where ffmpeg is.

@rmm805 You need to use the -ffmpeg:"/path/to/ffmpeg" option when calling the script or you can edit the script directly and set the path.

@mm805 I think you are wanting to know what the path is?  From that OSX Terminal command line, type:

type ffmpeg

And that will show  you the path.

Is there anyway to sort by the airdate/recorded date? … there are some instances where I want to say see what episodes of a daily tv show (eg. sesame street) where recorded the past week etc… just curious if there is a means to sort by (or display) recorded/air dates … (I know there is the -long switch but the output is ugly and not human friendly)


edit: i should add using the “-list” option is there a means to sort the output other than the default order?
Ok, I am getting towards the final stretch, with my time in front of the TV during the holiday quickly coming to an end.

@ericgus - sorting would make sense - date first than show, or show than date, I am leaning towards show then date.

@girimurthy - I think I finally figured out what I was doing wrong with closed captioning, it will require ccextractor ( as @cjcox noted) , and I think I can embed the subtitles into the videos (as a stream vice burn't in), I don't want to make this the default, as I have seen some really bad resulting .srt files.  Maybe an option to move the associated .srt file to where ever the file is moved???

@ericgus - I am finding that removing the last few minutes is harder than expected, the chunks are not of a guaranteed length...  Maybe for a later version.

@BoomerTheWeim - I think I am going to put off destructive options like deleting from the box after download, and commercial cutting a bit longer, maybe for version 3 later down the line.

Version 2b33 has been rather stable for me, I have transferred about 1.5GB of off of two tablos as a test without error.  I think I will finish up with some code cleanup, making the printouts a bit more reasonable by sorting the output queue (-list), and seeing if I can embed .srt streams or at least provide the files if ccextractor is available.

Items for a future holiday would be:
- Shaving padding off of a recording if it is excessive
- Implement some sort of name it as you want option
- Integrating Comskip for the adventurous
- Implementing some API level things (when it becomes more available - hint hint @tablotv or @TabloSupport) such as:
   - deleting a video from the tablo after it has been transferred and moved successfully
   - and other, more kmttg like, features
- Maybe a GUI for the less adventurous...

I'll shoot to have the (hopefully) final version 2.0 in the next few days.

@jskenney   I have tried the script but I’m not having any luck. The first thing I tried was the -list command which I thought would give me a listing of my recordings so i could be more specific about what I downloaded.

python tablo2go-2.0b33.py -list -long -ignore

I am on Windows 7 and it’s a WiFi connection. i wonder if this is supposed to take a long time or would i see some results very soon. This is as far as I get.

[Dec 26 22:19:21 2014] - Starting TabloToGo Version 2.0b33
[Dec 26 22:19:22 2014] - Loading download history
[Dec 26 22:19:22 2014] - 0 shows and movies have already been downloaded
[Dec 26 22:19:52 2014] [192.168.0.107] - Found Tablo named Tablo
[Dec 26 22:19:52 2014] [192.168.0.107] - Loading meta-data for 917 recordings

The next thing I want to do is download to my directories.

python tablo2go-2.0b33.py  -moviedir:k:/movies -tvdir:“k:/tv shows”  -ffmpeg:ffmpeg -debug -ignore -mp4

The result:
Dec 26 22:24:38 2014] - Starting TabloToGo Version 2.0b33
[Dec 26 22:24:39 2014] - Loading download history
[Dec 26 22:24:39 2014] - 0 shows and movies have already been downloaded
[Dec 26 22:24:57 2014] [192.168.0.107] - Found Tablo named Tablo
[Dec 26 22:24:57 2014] [192.168.0.107] - Loading meta-data for 917 recordings


I’m not sure if the above line is correct. I have been using the commands in a batch script to save retyping. Hopefully that doesn’t make a difference. Is this supposed to take hours to happen? I am not very experienced with python, so I may have made some fatal mistake that caused my command to fail. Is there anything obvious that I missed?

Thanks

Harry

                                                                                                  

"I am finding that removing the last few minutes is harder than expected, the chunks are not of a guaranteed length…  Maybe for a later version."


Thats unfortunate… thanks for giving it a shot.

Here’s my request : parse the .history file to remove lines that coincide with “Tablo Down” lines in the .log file.


Sometimes my Tablo goes “down”, and the python script considers the currently transferring show complete.  I have had success in removing those lines from the .history file manually, but it’s fraught with pain and error.  Maybe this should be a separate script.

But, oh my gosh, this script is so awesome !!!  Thanks !!! - Kurt

@kkoceski - can you post some of the Tablo Down errors, I have not seen this condition on my two tablos.

Or, it’s just me :slight_smile:  Thanks for looking into this !


Below, EP015344720007 was successful ( Penn + Teller … ).
But then EP005455792251 didn’t finish, and was marked as “processed” ( Jimmy Kimmel ).
Then a whole bunch started, but never really started.

So, when I come back, it thinks it has “processed” all of them, but never really did.
If I hit the up arrow to do again, only the new-new ones get processed.

Like I said, I could get all the failed ones to work, but I had to manually delete .history lines.

+++++++++++++++++++++++

[Dec 26 16:25:03 2014] [192.168.0.101] -   1019 MB,  96%, 0m 23s
[Dec 26 16:25:28 2014] [192.168.0.101] -   1059 MB, 100%, 0m 0s
[Dec 26 16:25:28 2014] [192.168.0.101] - Converting video to .mp4
[Dec 26 16:27:14 2014] [192.168.0.101] - Deleteing old .ts file
[Dec 26 16:27:14 2014] [192.168.0.101] - Moving to “./tv/Penn + Teller Fool Us/Season 1”
[Dec 26 16:27:14 2014] [192.168.0.101] - Marking EP015344720007 as processed
[Dec 26 16:27:14 2014] [192.168.0.101] - Downloading “Jimmy Kimmel Live - S12E111”
[Dec 26 16:27:15 2014] [192.168.0.101] - Video has 404 segments
[Dec 26 16:27:20 2014] [192.168.0.101] -      9 MB,   0%, 8m 24s
[Dec 26 16:27:35 2014] [192.168.0.101] -     34 MB,   3%, 8m 48s
[Dec 26 16:28:00 2014] [192.168.0.101] -     66 MB,   7%, 9m 22s
[Dec 26 16:28:22 2014] [192.168.0.101] -     93 MB,  11%, 8m 59s
[Dec 26 16:28:40 2014] [192.168.0.101] -    115 MB,  14%, 8m 8s
[Dec 26 16:29:00 2014] [192.168.0.101] -    137 MB,  18%, 7m 42s
[Dec 26 16:29:23 2014] [192.168.0.101] -    167 MB,  22%, 7m 28s
[Dec 26 16:29:51 2014] [192.168.0.101] -    207 MB,  25%, 7m 25s
[Dec 26 16:30:16 2014] [192.168.0.101] -    240 MB,  29%, 7m 8s
[Dec 26 16:30:39 2014] [192.168.0.101] -    267 MB,  33%, 6m 47s
[Dec 26 16:30:51 2014] - Unable to download specific segment
[Dec 26 16:30:51 2014] - Unable to download video segments
[Dec 26 16:30:51 2014] [192.168.0.101] - Marking EP005455792251 as processed
[Dec 26 16:30:51 2014] [192.168.0.101] - Downloading “Jimmy Kimmel Live - S12E116”
[Dec 26 16:30:51 2014] - Unable to retrieve segment information - Tablo Down
[Dec 26 16:30:51 2014] [192.168.0.101] - Marking EP005455792256 as processed
[Dec 26 16:30:51 2014] [192.168.0.101] - Downloading “WESH 2 News at 11 PM - 2014-09-11”
[Dec 26 16:30:51 2014] - Unable to retrieve segment information - Tablo Down
[Dec 26 16:30:51 2014] [192.168.0.101] - Marking SH007451400000 as processed
[Dec 26 16:30:51 2014] [192.168.0.101] - Downloading “Masters of Illusion - S01E08 - Under Water”
[Dec 26 16:30:51 2014] - Unable to retrieve segment information - Tablo Down

@kkoceski I will change it so that it doesn’t mark it as processed, I wonder why your tablo does this?  Maybe it needs a reboot…

Yeah, I’ve rebooted a few times.  I have it on WiFi, so it’s not that robust.  I just don’t have an antenna cable available anywhere near my router.  It could be that I check things too often, burdening the WiFi throughput.  As well as a couple Linux laptops, I also have a Nexus 7 and a new Kindle Fire HDX that I keep connecting to the Tablo.  Oh, and I have an WiFi Extender in the house to complicate matters.


Thanks for your change.  I hope it doesn’t mess anybody else up ( I can’t see how it could, though ).
@kkoceski, thanks for finding that bug.  I will get that fixed for the 2.0 version.  

On a separate note, take a look at http://www.newegg.com/Product/Product.aspx?Item=N82E16833181423 which are ZyXEL 1200Mbps Powerline Gigabit Ethernet Adapters, I use them for one of my guest rooms, and they easily sustain 80MB/s (640Mbps) which may be a good option for connecting your remote tablo to the rest of your house.  This is the first set of powerline adapters that has worked for me with rational speed.

@HarryR , when you ran

python tablo2go-2.0b33.py -list -long -ignore 
it never produced any output?  How long did you wait, what is your network configuration?  For 917 recording I would anticipate a few minutes…

@jskenny

Output found in “tablo” file

[Dec 27 23:02:10 2014] - Starting TabloToGo Version 2.0b33
[Dec 27 23:02:10 2014] - Loading download history
[Dec 27 23:02:10 2014] - 0 shows and movies have already been downloaded
[Dec 27 23:02:39 2014] [192.168.0.107] - Found Tablo named Tablo
[Dec 27 23:02:39 2014] [192.168.0.107] - Loading meta-data for 921 recordings

That’s as far as it ever gets.

Windows 7 WiFi 72 Mbps Router 300N
Run from C:\Python27 run from windows shell. I tried IDLE and batch files too.
I have waited a few minutes.  Is there supposed to be a file generated? Where would it be found?

Any ideas what I’m missing?


Python /Volumes/HDD/Tablo/tablo2go-2.0b33.py -tvdir:/Volumes/HDD/Tablo -ffmpeg:/usr/local/bin/ffmpeg -TempDir:/Volumes/HDD/Tablo/Temp

[Dec 27 23:50:17 2014] - Starting TabloToGo Version 2.0b33

[Dec 27 23:50:17 2014] - Loading download history

[Dec 27 23:50:17 2014] - 0 shows and movies have already been downloaded

[Dec 27 23:50:17 2014] [10.0.1.24] - Found Tablo named Tablo

[Dec 27 23:50:17 2014] [10.0.1.24] - Loading meta-data for 3 recordings

[Dec 27 23:50:17 2014] [10.0.1.24] - Failed to get metadata for 2 videos

[Dec 27 23:50:17 2014] [10.0.1.24] - 1 new TV shows and 0 new movies

[Dec 27 23:50:17 2014] [10.0.1.24] - 1 matching videos have been queued

[Dec 27 23:50:17 2014] [10.0.1.24] - Downloading "Anger Management - S02E41 - Charlie and Sean and the Battle of the Exes"

[Dec 27 23:50:17 2014] [10.0.1.24] - Video has 199 segments

[Dec 27 23:50:18 2014] [10.0.1.24] -      6 MB,   2%, 0m 30s

[Dec 27 23:50:21 2014] [10.0.1.24] -     41 MB,   7%, 0m 50s

[Dec 27 23:50:31 2014] [10.0.1.24] -    133 MB,  15%, 1m 16s

[Dec 27 23:50:41 2014] [10.0.1.24] -    238 MB,  22%, 1m 21s

[Dec 27 23:50:51 2014] [10.0.1.24] -    337 MB,  30%, 1m 17s

[Dec 27 23:51:01 2014] [10.0.1.24] -    437 MB,  37%, 1m 12s

[Dec 27 23:51:09 2014] [10.0.1.24] -    525 MB,  45%, 1m 3s

[Dec 27 23:51:20 2014] [10.0.1.24] -    635 MB,  52%, 0m 56s

[Dec 27 23:51:30 2014] [10.0.1.24] -    731 MB,  60%, 0m 47s

[Dec 27 23:51:38 2014] [10.0.1.24] -    806 MB,  67%, 0m 38s

[Dec 27 23:51:49 2014] [10.0.1.24] -    928 MB,  75%, 0m 30s

[Dec 27 23:52:00 2014] [10.0.1.24] -   1035 MB,  82%, 0m 21s

[Dec 27 23:52:10 2014] [10.0.1.24] -   1137 MB,  90%, 0m 11s

[Dec 27 23:52:19 2014] [10.0.1.24] -   1234 MB,  97%, 0m 2s

[Dec 27 23:52:22 2014] [10.0.1.24] -   1262 MB, 100%, 0m 0s

[Dec 27 23:52:22 2014] [10.0.1.24] - Converting video to .mp4

FFmpeg version CVS, Copyright (c) 2000-2004 Fabrice Bellard

Mac OSX universal build for ffmpegX

  configuration:  --enable-memalign-hack --enable-mp3lame --enable-gpl --disable-vhook --disable-ffplay --disable-ffserver --enable-a52 --enable-xvid --enable-faac --enable-faad --enable-amr_nb --enable-amr_wb --enable-pthreads --enable-x264 

  libavutil version: 49.0.0

  libavcodec version: 51.9.0

  libavformat version: 50.4.0

  built on Apr 15 2006 04:58:19, gcc: 4.0.1 (Apple Computer, Inc. build 5250)

/usr/local/bin/ffmpeg: unrecognized option '-loglevel'

[Dec 27 23:52:22 2014] [10.0.1.24] - Deleteing old .ts file

[Dec 27 23:52:23 2014] [10.0.1.24] - Moving to "/Volumes/HDD/Tablo/Anger Management/Season 2"

[Dec 27 23:52:23 2014] - Unable to move file, keeping it in tempdir

[Dec 27 23:52:23 2014] [10.0.1.24] - Marking EP015084700060 as processed

@rmm805 - your ffmpeg is very old, what type of Mac is it? Please go to http://www.ffmpeg.org/download.html#build-mac and download the newest version your hardware can support.

@jskenney it’s a 2011 iMac running Yosemite. The FFMPEG actually came from a FFMPEGX.dmg because it was precomplied. I can download the FFMPEG from the site you specified but I’m confused how to “install” it.

@HarryR - Please download b33debug I just posted and run that, it adds one line of extra debuging to look at your issue.