SurLaTablo.py - Python program to query and convert Tablo recordings

ok… You know I changed season 0 episode 0 scenarios to use local air date month for season and day for episode.  This actually works quite well in situations like “the news”… where original air date is nuked (which is why I chose lair_date)… you get something like:

surlatablo.py -n -q ‘Fox 4’ -Q '${friendly_title}'
Fox 4 News at 9 - s01e30 - Episode 2015-01-30
Fox 4 News at 9 - s01e29 - Episode 2015-01-29
Fox 4 News at 9 - s01e28 - Episode 2015-01-28


Which look great!!  Probably what you want for a daily show where there is no season or episode meta data.

But then, I found that all of the Gidget series I recorded also has no season or episode meta data… Sigh…  and you probably did not want (there’s only 1 season of Gidget):

Gidget - s10e26 - A Hard Night’s Night
Gidget - s10e26 - Ask Helpful Hannah
Gidget - s08e03 - Image Scrimmage
Gidget - s08e03 - Gidget Is a Proper Noun
Gidget - s12e14 - Chivalry Isn’t Dead


(yours my differ since the made up meta info for season and episode are based on lair_date)

Sure, we can always switch off the new behavior with NOZEROTV = False and then we’d get names that all have s00e00 in them.  Again, not ideal.

So… in 0.7 what I’ve done is augmented the -S and -E which used to be for making up meta info on Sports only to be able to handle TV overrides where instead of a sport_type, you’d specify a series_name override…  And of course instead of sorting by game_date, they’d be sorted by original_air_date.

Thus:

surlatablo.py -q series~=Gidget -S Gidget::1 -E Gidget::1 -C -c +Gif +Json

Would give us things like:
Working on:                             [./TV/Gidget/Season 1/Gidget - s01e01 - Dear Diary: Et Al]
 Retrieving Tablo Data (117065):        [####################] 100%
 Extracting CC as Subtitle:             [####################] 100%
 Transcoding (Mp4, .mp4):               [####################] 100%
 Transcoding (Gif, .gif):               [####################] 100%
 Executing (dumpJson, .json)

Working on:                             [./TV/Gidget/Season 1/Gidget - s01e02 - In God, and Nobody Else, We Trust]
 Retrieving Tablo Data (117067):        [####################] 100%
 Extracting CC as Subtitle:             [####################] 100%
 Transcoding (Mp4, .mp4):               [####################] 100%
 Transcoding (Gif, .gif):               [####################] 100%
 Executing (dumpJson, .json)

Working on:                             [./TV/Gidget/Season 1/Gidget - s01e03 - The Great Kahuna]
 Retrieving Tablo Data (122973):        [####################] 100%
 Extracting CC as Subtitle:             [####################] 100%
 Transcoding (Mp4, .mp4):               [####################] 100%
 Transcoding (Gif, .gif):               [####################] 100%
 Executing (dumpJson, .json)


…etc…

Which if we have all of Season 1 of Gidget, should ideally match up (except where shows were aired out of order for some reason).  Again, you’d only have to do this for scenarios where you had series, but for whatever reason no meta data was provided for the season number or the episode number.

Just some stuff I’m playing with… thoughts welcome.





It works both ways from command line.

I tried this and I'm getting % numbers and the name of my recording.
python c:\\SurLaTablo\\surlatablo.py -n -q 123781 -c

This is a good sign.


@HarryR, if you want to query by rec_id, use the -i option (your query will work though most of the time)

python c:\SurLaTablo\surlatablo.py -n -i 123781 -c

The -i is a special case in that it also dumps out the original meta data in pretty printed form (the original Tablo stored meta data).  But it still works as a valid query as well.  Updated:  Well… technically it’s not a query, it’s just a direct pull from the Tablo at that point.  One of the reasons why surlatablo exists is because most people don’t know the rec_ids.  If you give your Manual recording a title, you can use -q to search for it.




This works too and uses the query cache:

python c:\SurLaTablo\surlatablo.py -n -q rec_id~=123781 -c

However, by default the search is unbounded to the right… so 123781 will match 123781 and 1237315 (for example).  Most things surlatablo.py are strings, so you could terminate the search to the right with an escaped double quote.  For rec_id, it’s an integer, so just tack on a comma.

python c:\SurLaTablo\surlatablo.py -n -q rec_id~=123781, -c

0.7 will have sorting and file based templating

Almost there.  I fixed a bug in sortTitle which caused the metakey sort_title to not be populated (gets rid of leading article for sorting purposes, e.g. ‘a’, ‘an’, ‘the’).  So you’ll want to blow away your caches (sorry) and regen when 0.7 comes out.

Sorting is done and works like so:

-s series,sort_title

Which sorts by series name and then by sort_title.  You can reverse the order by prepending the option list with ‘-’ (just at the front of the list)… for example:  -lair_date,friendly_title  The default sort is -lair-date which means local air date decending (most recent recordings first).  Sort metakeys act as a filter, records not containing all of the sort keys will be ignored.

Preliminary CHANGELOG for 0.7 (almost done!):

 0.7 -
   1. -E, -S, now gerneralized and long formats renamed to --episode and --season.
      These can now be used to force season and episode data in a made up way
      for handling TV by series name values where no season/episode meta data
      was provided.
       e.g.   -S Gidget::1  -E Gidget::1
      When queried, converting, this arranges the series matches for Gidget
      by original air date and numbers season as “1” and episodes starting
      at “1” and auto-incrementing.
      The behavior for making up meta data for Sports shows has not changed.
   2. Removed some extraneous debug prints.
   3. Added the -B option to allow converting/processing of shows that are
      stuck on the Tablo in a non-“finished” state.  Not sure what causes this.
      Some say it happens when things go “bad”, but I think I’ve seen this
      just randomly.  Anyhow, if you have stuck shows that always say they
      are RECORDING/BUSY, this option will allow you to ignore that
      (use with caution… don’t ignore shows that you know are really
      recording, and if you do, avoid converting them until they are
      truly done recording).
   4. Fixed bug, sort_title was not working.  You will want to remove your
      SURLATABLO_ROOT/<tablo_ip> cache files and regenerate.
   5. Added -s (–sortkeys) option to allow you to specify what metakeys to
      to sort by (default is sort by lair_date descending).  Warning: this
      is also a filter.  Records that do not have all the metakeys specified
      will not be output.  For example if you have -q . (everything) and -s
      series, only TV shows will be shown since they are the only records
      containing series.

   6. Query format (-Q) may now specify urls.  In which case they are templated
      (that is, metakey substitutions are done).
      There are two formats.  url(‘location1’,‘location2’,…) and
      URL(‘location1’,‘location2’,…).  The second variant, URL, forces
      metakey substitution on each url prior to inclusion.  If the resource
      scheme begins with ‘1’ (e.g. 1http://…) then include that resource
      only once for all matches.  Resource schemes allowed are (http: and file:,
      some others might work…)


SurLaTablo   0.7 released

Q. I noticed SurLaTablo has H.265 encoding option (new versions of ffmpeg). I usually consider this a 4K option. Are there any OTA stations that transmit 4K? Can Tablo record 4K? Just curious.


Thanks,
Tom


It’s a transcoding option… output of h265 at about same quality of h264 can be considerably smaller.  But it is a trasnscode, and it’s pretty slow.  But could save tons of space.  Of course, assumes your clients can do h265.

You know, you could run comskip.exe on the file once it’s pulled out of Tablo and that would give you a .edl or .txt edit decision list that could be used with some media viewers to watch shows without the commercials. Or you could use something like MCEBuddy.

@Angus, something like that (but more portable) is in my queue :slight_smile:

I will likely run after .ts pull and preserve a file of the data that can be fed to the transcoders (for stop, start).


I am having great luck with this script. The only problem I am having is if there are characters in the title that windows doesn’t like. Like a : (colen) for example. ffmpeg will not convert these. Is there a way to override this behavior so I can convert these episodes.


Even deleting the title for these would be fine. Series name - season number & episode number - ext. would work.

Any help would be appreciated.

Thanks

Phil

P.S. I am still using version 2 of your script.

I fixed it. I just edited the script to fix the problem.


FILENAME_PATS={‘Default’:’${title}’,
    ‘TV’:’${series} - s${plex_season_number}e${plex_episode_number}’,
    ‘Sports’:’${sport_type} - s${plex_season_number}e${plex_episode_number} - ${title}’,
    ‘Movie’:’${title} (${release_year})'

Saved it with a different name so I can use it when I run into this problem.

Windows name filtering was added later. Recommend you upgrade to the latest version. 0.7

Thanks I will…

That worked great. I ran the new script on a few troubled TV programs & they converted just fine.


One more question please. Is there a easy way to change the temp directory that the .ts files get written to? I am running a ssd for a c: drive & would like to get the extra traffic moved to a different drive.

Thanks

Phil


From the Python documentation:

Python searches a standard list of directories and sets tempdir to the first one which the calling user can create files in. The list is:

  1. The directory named by the TMPDIR environment variable.
  2. The directory named by the TEMP environment variable.
  3. The directory named by the TMP environment variable.
  4. A platform-specific location:
    • On RiscOS, the directory named by the Wimp$ScrapDir environment variable.
    • On Windows, the directories C:\\TEMP, C:\\TMP, \\TEMP, and \\TMP, in that order.
    • On all other platforms, the directories /tmp, /var/tmp, and /usr/tmp, in that order.
  5. As a last resort, the current working directory.

Oh… and you can always set:

tempfile.tempdir = 'your-folder’

in the program or in your surlatablo.conf file.

Rats… can’t use surlatablo.conf for that.

You will need to add:

tempfile.tempdir = ‘your-folder’

after the import line containing tempdir

(sorry about that… I’ll fix in the next release)

“Commercial detection is a mystical black art (pun intended) that will surely leave you without a show or with commercials....”

Again thank you for your help. It is much appreciated…


Phil