SurLaTablo.py - Python program to query and convert Tablo recordings

@cjcox

I’m trying out your script via a VM in my OpenStack lab and am wanting to move the transcoded files to my NAS. I don’t have much room in the VM so I’m using the temp dir variable to store the files locally but then the exported video transcodes over the network to my Synology.

Any chance of getting a variable added to the conf file to have the transcoded file be put into the temp dir first and then moved into the output dir (which in my case is a symlink to the NFS mount in my VM).

Also, I don’t keep shows and delete them via plex after I’ve watched them. Whenever I finish a season this leaves empty season dirs in the TV show dirs. Would be nice to have a variable to enable/disable season subfolders.

Thanks.

Not sure about the transcoded to temp and then move… might have to wait until a 2.0 release or something like that (I will give it some thought though, note cross device moves are very expensive, same as a full copy).

With regards to how things are named, it’s completely configurable.

In your surlatablo.conf file you could have (for example, this strips out the Season subdir for meta_type TV):

BASE_DIRS[‘TV’] = ‘./${meta_type}/${Eseries}’

(Note: Eseries is used instead of series just to make sure you get something that is “ok” in Windows should the series name contain non-ASCII characters)

I apologize if this question has been asked before but I haven’t been able to find a solution for following query question:


How can I list all episodes of Gotham but only starting with episode #15 so that my output would include episodes 15, 16, 17 etc …

surlatablo.py -q 'Gotham - s01e1[5-9]'

That catches 15,16,17,18,19

Right now surlatablo only really has string match capability.  So to catch the 20’s…

surlatablo.py -q ‘Gotham - s01e2’

If you haven’t guessed,

surlatablo.py -q 'Gotham - s01e2’

matches Gotham - s01e2*, so it matches s01e200 if there was over two hundred episodes in a season :slight_smile:

While not needed in this case in particular, if you did want to limited it, a better query might be:

surlatablo.py -q 'Gotham - s01e2[0-9] -'

https://docs.python.org/2/library/re.html

@cjcox

Thx a lot. Works like a charm :wink:

@cjcox Just ran across this the other day, and it’s fantastic. Thank you for creating it! 


Commercial handling is less than stellar though (the relatively new -z option).

I’m thinking about makeing this two pass… that is, select option (let’s say -z) and that creates a buch of segmented .ts files.  Then you go in an remove the ones you don’t want, and the 2nd pass transcodes what is left into the final file.

Thus, it would not be “automatic” anymore, but in all fairness, it didn’t guess right more then 50% (probably less) of the time.

So in other words, it becomes a tool that makes it easy to strip out commercials… rather than a good guesser.  Still, I may leave the existing (or slightly modified for fixes) guesser code in… but it does have a lot of flaws.

There’s a reason people always reference “commercial skipping” (which is pretty easy) rather than commercial elimination.  It’s just too hard.

Yeah, MythTV did a commercial “flagging” run to video. Then in the front end you could edit the cut list. After that you transcoded or exported the video. The flagging was pretty good but not perfect.

@cjcox


Do you know if there is any detailed tutorial on:
1-how to install, configure and run python on my windows pc (for dummies preferably :slight_smile:
2-how to properly run your program

I’ve read this tread since the beginning and tried out with the information posted, but did not accomplish much. Any advice will be greatly appreciated - keeping in mind my “for dummies” level of knowledge on programing

@fzrxhq, it is on my “to do” list…  I have a busy weekend, but might be able to do something this weekend (not sure).

I started using Tablo and surlatablo is invaluable cjcox, many thanks for creating it. I am using it as a DVR replacement.


- At home I don’t get all the channels (not the 4 major networks anyway), but in my office I get a clear reception, so…
-  I have setup tablo in my office, and I am using a cron job + surlatablo to transfer the recordings over to my home NAS where they are indexed and viewed using Plex (+Roku).

It is working very smoothly so far so this is mainly a note of appreciation. 

The command I use right now is surlatablo-0.9-py  -n -q “.”  -c. 

As I understand it goes through all my converted recordings and all that exists on Tablo and only converts the recordings that don’t exist. That seems to be fine for now but as the libraries grow, that approach might not be scalable. Is there an option to only convert shows that were recorded since the last time surlatablo was run (or just simply newer than some configurable date/time?)

Many thanks again for providing this very nice utility.

-Vishal

@cjcox - Thank you for sharing surlatablo with the community.  I have been using it with great success for quite a while now - even the commercial zapping has been more often correct than not.


@VishaMisra - I use the following syntax in my daily cron job to get all the shows that locally aired either today, or yesterday.  It seems to work well, but if the metadata surrounding the locally aired date isn’t accurate, it will miss items (although I don’t think it has happened to me yet):

TODAY=date +"%Y-%m-%d"
YESTERDAY=date --date='yesterday' +"%Y-%m-%d"

/config/scripts/surlatablo.py --convert --query lair_date~="$TODAY|$YESTERDAY"

``

@VishalMisra actually… this may be easier than you think.  While it may seem somewhat of a waste, as long as you do not specify the -y option, surlatablo.py will not clobber files it has already retrieved (as long as you don’t rename things in a way that differs from what surlatablo named them).  So, you’ll get a whole bunch of really fast “won’t clobber” messages for things that exist.  It’s not that bad.  As you are satisfied and delete things off your Tablo, just make the surlatablo.py cache gets updated and that will shorten the list of existing “won’t clobber” items (-n is your friend, but not in this case).

So just:

surlatablo.py  -q ‘.’ -c

For example would convert everything, skipping things that exist because it won’t clobber existing files by default.

Now… with that said, if you are getting then moving files, sure, it won’t know quite what to do, because it looks like you have not converted the files yet.  But as long as you are alway converting to the same destination areas and are not moving or renaming things, it should work just fine.

With that said, I have thought about adding some more search features… we’ll see (date math?).

Since it’s really designed to convert straight into Plex compatible library areas, in general should not be a problem to configure surlatablo.conf to drop the files into those libraries.  Personally, I’ve setup a TabloTV and TabloMovie library just to keep my Tablo extractions separate from my other Plex libraries, but it’s really up to you.

Next version (maybe this weekend) will have better commercial zap handling (it will offer two algorithms and will correct the “out of sync” audio/video problem).

As I mentioned before though, commercial zapping probably should be a helper app that requires some human input into the process.  Thus it would break things apart, then “you” would tell it what to keep and what to toss.  I’ll leave the  automatic stuff in.  Likely new version will default to the new commercial zap algorithm, since the existing one uses some chaotic logic and thus can actually lock up in certain cases.  What can I say, I like probablistic algorithms.  With that said, the new algorithm is good and shouldn’t cause any lock up problems.






Thanks @cjcox - So far I am doing what you say above - all transfers between two set locations and it is working great. 


I will try your new version with the commercial zapping. Just curious about the automatic version - does it have “false positives” (i.e. zaps part of the actual show) or “false negatives” (misses zapping commercials occasionally). If the errors are false negatives then the automatic version is good enough - false positives would require a helper app definitely.

@ccridge - thanks for your crontab entry, helpful.

 


Routine has both issues.

Minor bug when title length is only 1

Traceback (most recent call last):

  File "C:\\DVDs\\TABLOTV\\surlatablo-0.9.py", line 2525, in
    rec_val['sort_title'] = sortTitle(rec_val['title'])
  File "C:\\DVDs\\TABLOTV\\surlatablo-0.9.py", line 1111, in sortTitle
    return title.split(' ',1)[1]
IndexError: list index out of range


I'm sure there is a better fix but I simply added a length check to work around this issue. 

 1103  # Return title minus any first word article, see articles
 1104  def sortTitle(title):
 1105      articles = ['a','an','the']
 1106      # Check first word against articles array
 1107      #  if matched, remove word and return the rest.
 1108      word = title.split(' ',1)[0].lower()
 1109      for article in articles:
 1110         if (article == word):
 1111            if (len(title) ==1):     #<--------------------
 1112               return title          #<--------------------
 1113            else:                    #<--------------------
 1114               return title.split(' ',1)[1]
 1115      return title

@alexbunk, thanks will do…

@cjcox - is there a way to delete the recording automatically from the tablo once successfully converted? 

@VishalMisra, while this is likely possible and likely more valuable to most than any other feature on this forum (ever)… I have not developed this, nor has anyone else at this time.