SurLaTablo.py - Python program to query and convert Tablo recordings

First release:
http://endlessnow.com/ten/SurLaTablo/

An alternative to the program released by @jskenney   This program does not attempt to be an ‘api’ library for python programmers.

Essentially a cleaned up version of some software I’ve been using for awhile at the house.  Written purely so that I could say I have written a Python program.  There will be updates and enhancements.  This release has not been tested under Windows.  Tested under Linux only using Python 2.7.3 under openSuse.  But it might work under Windows or other…

surlatablo 0.1 GPLv2


FIRST THINGS
------------
Edit this program and change the values for (near top of file):

    # Location of your ffmpeg program (optional if you won’t ever convert to mp4)
    FFMPEG = ‘/usr/bin/ffmpeg’

    # Location of your ccextractor program (optional)
    CCEXTRACTOR = ‘/usr/local/bin/ccextractor’

    # Location directory where meta db caches go
    SURLATABLO_ROOT = ‘/SurLaTablo’

    # List of your Tablo IPs, this program does not auto detect today
    TABLO_IPS = [‘192.168.1.73’]



UPDATE AND SEARCH
-----------------
Update cache db (need metadata db to do anything)
(typically you do this first, first run will take a bit)

    surlatablo.py

Update and search db

    surlatablo.py --query Green\ Acres

Do not update cache, just search db
(note matches Green Acres anywhere in the metadata)

    surlatablo.py --noupdate --query Green\ Acres

Search db by metadata

    # Find all for TV series called Green Acres
    surlatablo.py --noupdate --query series~=Green\ Acres

    # Find all Sports records
    surlatablo.py --noupdate --query meta_type~=Sports

Use a query format string to change output

    # Find all TV shows and user a query format for output
    # (note --queryformat will not do anything combined with --convert)

    surlatablo.py --noupdate --query meta_type~=TV --queryformat ‘${lair_date}  ${rec_id}       ${series}       ${title}‘

Display full TabloTV metadata for a specific recording id.

    # (note --rec_id can be used with --convert)
    surlatablo.py --noupdate --rec_id 28191

Note:  You can easily force a full refresh of local cached data by removing the files
at SULATABLO_ROOT/Tablo_IP/*



CONVERT OR PREVIEW CONVERT (ADD SUBTITLES/CC, MP4 METADATA, CROP)
-----------------------------------------------------------------

Locate a particular recording, convert with close captioning done as a subtitle. Sample interative output also shown.

    surlatablo.py --noupdate --query Fuji\ falls --convert --ccaption --clobber
   
    Working on:                             [./TV/McHale’s Navy/Season 3/McHale’s Navy - s03e15 - Fuji’s Big Romance.mp4]
     Retrieving Tablo Data (28191):         [####################] 100%
     Extracting CC as Subtitle:             [####################] 100%
     Transcoding:                           [####################] 100%

Locate all sports and use default season of game_year and use supplied episode start number by sport_type. Defaulting
starting with episode number 1 for unspecified types.

    surlatablo.py --noupdate --query meta_type~=Sports -E “NFL Football::3,NBA Basketball::5,1” --convert

Use the --debug (probably misnamed) to show what surlatablo.py would do, but don’t do it (option to be combined with --convert)

    surlatablo.py --noupdate --query Fuji\ falls --convert --ccaption --clobber --debug
    [ ./TV/McHale’s Navy/Season 3/McHale’s Navy - s03e15 - Fuji’s Big Romance.mp4 ]
    [ http://192.168.1.73:18080/pvr/28191/segs -> /tmp/TabloE1_zkh.ts]
    [’/usr/local/bin/ccextractor’, ‘/tmp/TabloE1_zkh.ts’, ‘–gui_mode_reports’, ‘-o’, ‘/tmp/TabloZdkhvf.srt’]
    [’/usr/bin/ffmpeg’, ‘-i’, ‘/tmp/TabloE1_zkh.ts’, ‘-f’, ‘srt’, ‘-i’, ‘/tmp/TabloZdkhvf.srt’, ‘-bsf:a’, ‘aac_adtstoasc’, ‘-c:v’, ‘copy’, ‘-c:a’, ‘copy’, ‘-c:s’, ‘mov_text’, ‘-metadata:s:s:0’, ‘language=eng’, ‘-metadata:s:a:0’, ‘language=eng’, ‘-metadata’, “title=Fuji’s Big Romance”, ‘-metadata’, ‘date=1964-12-25’, ‘-metadata’, ‘network=Antenna’, ‘-metadata’, “album=McHale’s Navy”, ‘-metadata’, “show=McHale’s Navy”, ‘-metadata’, ‘genre=Sitcom’, ‘-metadata’, ‘episode_id=s03e15’, ‘-metadata’, “synopsis=Fuji falls in love with a native chief’s daughter.”, ‘-metadata’, “description=Fuji falls in love with a native chief’s daughter.”, ‘-metadata’, “comment=Fuji falls in love with a native chief’s daughter.”, ‘-y’, “./TV/McHale’s Navy/Season 3/McHale’s Navy - s03e15 - Fuji’s Big Romance.mp4”]

Note: Better, full output of an actual --convert can be captured simply be redirecting output to a file instead
of to screen/terminal.  Useful for when things aren’t working (bugs in the software).

Note: Program uses Python tempfile functions.  To adjust your temporary directory place:
    tempfile.tempdir = '/path-to/my-tempdir’
For other notes on how Python finds your temporary file location, see Python documentation.

Full documentation is inside program, or surlatablo.py -h

Metakeys (see program, or surlatablo.py -h)

1 Like

A Moody Blues fan, perhaps? (your subject and app name)


Bummer, only have Linux at work, and Novell’s SUSE. 
No python anywhere unless you count Monty Python, then plenty of that. 

Since if a Tablo device blows you lose anything on your drive, something to periodically export recordings, back them up, or some ability to recreate a database of recordings for a replacement Tablo would be my priority.

But something like this, like your “app” here would be great to have also because I could have converted recordings on any device on my network and access them through my Blu ray player which will grab any shared content and play it through our TVs. 

I’ll get around to doing Windows testing… I promise.  But may have to undo some of the wiz-bang neato stuff… but it will be ok.

Oh… and I do like the Moody Blues, but actually, didn’t name it based on that… if anything brought back memories of an old “I Love Lucy” episode.  I really don’t speak French at all… except what I get here and there through media.


If you reserve IP in your DHCP server like I’ve done then you know your Tablo IP address so a search could be left out, IMO, saving overhead. Networks where these are used would likely be small, few devices and ideally one always knows what’s on their home network and where, they need to know what hardware, IP address and/or MAC for security in densely populated areas if they run Wi-Fi (or at least run the default security!

I also allow only devices with MAC listed in my MAC table. 
The tablo was a whole lot easier to troubleshoot when I reserved an IP address so if I am having issues, I can always try to ping the Tablo and see if it’s really still alive. 

Just commenting on your note that it doesn’t search for Tablo. I’d personally leave that for later if at all. 

I don’t know how much longer I’m going to keep Windows alive at home so maybe that would be a moot point for you - Windows testing. But for now it would be a cool tool if it did work, even if semi-crippled. 

First run 

Traceback (most recent call last):

  File "surlatablo.py", line 913, in

    os.mkdir(get_tablo_root)

OSError: [Errno 13] Permission denied: '/SurLaTablo'

-- EDIT


Sorry ignore that, I looked at the long help after ;)

Did a rudimentary convert under Windows with CC, no problem.  However, I did have to install pytz using pip.

The progress bar during Transcode isn’t shown and magically appears at 100% (when done).

Python 2.7

I was instantly reminded of 1988’s Sur la Mer (the last full album to officially include Moraz) when I saw the thread.


I may give Python a shot - glad I checked back to see which version you wrote and tested under. Not sure if the 3.xxx would play well with it or not. Since I don’t really use it otherwise it didn’t matter to me which, I was merely making sure YOUR script would work well.

Me, I’m an old-skool CMD and VBS person - with limited abilities there, however.

@cjcox I am interested in trying this. I would like to convert some shows to be played on the Plex server, but there’s a lot of stuff I’m not familiar with. I’m not sure what to paste in terminal. What steps to do first do I need to manually create a temporary database folder? Then search for a show? Then run something else to convert it? Just worried about screwing something up. Thanks

Install python. Install pip for python. Pip install pytz… unfortunately, those are the hard things… I may have to write a doc, but feel free to google…

Then you download the surlatablo file from the link in the first post.  It’s named as a .txt file to make sure it can be easily obtained.  But once you get it, you will want to rename it to surlatablo.py.

One of the first things you can do is run:  surlatablo.py --help

It’s a large help output, you need to pay particular attention the very first part which tells you the few things you need to change at the top of the surlatablo.py file.  In particular:

    # Location of your ffmpeg program (optional if you won’t ever convert to mp4)
    FFMPEG = ‘/usr/bin/ffmpeg’

    # Location of your ccextractor program (optional)
    CCEXTRACTOR = ‘/usr/local/bin/ccextractor’

    # Location directory where meta db caches go
    SURLATABLO_ROOT = ‘/SurLaTablo’

    # List of your Tablo IPs, this program does not auto detect today
    TABLO_IPS = [‘192.168.1.73’]


Then I sort of hope from the help doc you get the idea that you need to build your local cache first.   This is what makes searches fast because querying the Tablo device is hard, slow and not well suited currently for searching.  By default, whenever you run surlatablo.py (unless you supply -n or --noupdate), it will update the cache so that it is insync with the data currently on your device.

I’m going to stop here… and see if you able to make it this far.

Just fyi, the update 0.2 (probably tonight) will:

Allow
\t,etc. style escapes in --queryformat (aka -Q).  It will also print a friendly message instead of an error and stack trace when it cannot create the SURLATABLO_ROOT directory.  It also allows you to use --queryformat (-Q) to do output along with using the --convert (aka -c) option.  There will be a new piece of metadata (which means you may want remove and regen your caches) that will add metatype called ‘friendly_title’, which looks much like the default patterns used for file creation, it’s actually the names you see output when it’s doing a cache update.  Why?

Well, the friendly_name for TV shows looks like a Plex name… so you get something like:

Green Acres - s01e02 - Lisa’s First Day on the Farm

The advantage is that you can do an easier query for shows by season:

surlatablo.py -n -q friendly_title~=‘Green Acres - s01’

(which will return all of Season 1)

surlatablo.py -n -q friendly_title~=‘Green Acres - s01e02’

(which returns Season 1, Episode 2)

Okay I downloaded the FFMPG, CCEXTRACTOR, Python, downloaded the surlatablo.py changed the location for the FFMPG, CCEXTRACTOR, changed the IP address for tablo and set location to eastern. I think I have installed PIP.

This is what I have so far.

Mac-French:~ macfrench$ sudo easy_install pip
Password:
Searching for pip
Best match: pip 1.5.6
Adding pip 1.5.6 to easy-install.pth file
Installing pip script to /Library/Frameworks/Python.framework/Versions/2.7/bin
Installing pip3.4 script to /Library/Frameworks/Python.framework/Versions/2.7/bin
Installing pip3 script to /Library/Frameworks/Python.framework/Versions/2.7/bin

Using /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Processing dependencies for pip
Finished processing dependencies for pip
Mac-French:~ macfrench$ python /Users/macfrench/desktop/surlatablo.py
File “/Users/macfrench/desktop/surlatablo.py”, line 1
{\rtf1\ansi\ansicpg1252\cocoartf1343\cocoasubrtf160
^
SyntaxError: unexpected character after line continuation character
Mac-French:~ macfrench$ surlatablo.py
-bash: surlatablo.py: command not found
Mac-French:~ macfrench$ easy_install --upgrade pytz
Searching for pytz
Reading https://pypi.python.org/simple/pytz/
Best match: pytz 2014.10
Downloading https://pypi.python.org/packages/2.7/p/pytz/pytz-2014.10-py2.7.egg#md5=a6e26cf01e307d3505612d7386fee859
Processing pytz-2014.10-py2.7.egg
Moving pytz-2014.10-py2.7.egg to /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Adding pytz 2014.10 to easy-install.pth file

Installed /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pytz-2014.10-py2.7.egg
Processing dependencies for pytz
Finished processing dependencies for pytz
Mac-French:~ macfrench$ python /Users/macfrench/desktop/surlatablo.py
File “/Users/macfrench/desktop/surlatablo.py”, line 1
{\rtf1\ansi\ansicpg1252\cocoartf1343\cocoasubrtf160
^
SyntaxError: unexpected character after line continuation character
Mac-French:~ macfrench$

:slight_smile:

Switch your backslashes to forward slashes.

I have not tested on OSX.

I’ve got to get python installed on my Windows 7 machines and give this a shot. I know it’s going to be a whole lot different, though because the screenshots won’t even be close at all…

CMD window in Windows is a different animal and you DO use backslash for directories in Windows.

Just in case you’ve forgotten - >


@ShadowsPapa, just use the forward slashes… trust me.

However, the user in question was using OSX… but in any case…

Because it’s in the python environment or shell and not the command.com of Windows, right? More like managing FTP than Windoze.


OK, this is what I am looking for - and you may have the most simple solution at the moment:

I have no plex server (not saying never will, saying I don’t have now or near future.
I have multiple computers, a lot of decent storage, small or tiny compared to many folks here, but enough for me. 
I have a desktop, 64bit with 6 core processor, 10 gig ram and 1.5T drive for one system.
That’s likely where I’d handle conversions or imports or whatever. It’s connected through a Cisco switch to my router and the Tablo is connected via Ethernet cable to my router. So PC to switch, switch to router, router to tablo.
I run as of now Windows 7 64

I understand I need Python 2 series, I grabbed the 2.7.9 msi installer (Windows of course)
I need pip? I assume not one of Gladys Knight’s guys… but some other pip.
What else, if anything?

Ah, some say pip is part of 2.7.9 which is what I downloaded…