Working on 2.0b5 (sigh). This will add -U and -P, to skip unwatched and protected items respectively. The calls are expensive since it will make a live call to the Tablo for any matched query item to see if it should be included in the query results.
You will want to remove the cache files under your SURLATABLO_ROOT and do a full reindex of your Tablo.
2.0 - Fixes. Added -U and -P options. Metadata changes (refresh your caches)
Added -U which means skip query matches of things unwatched. Added -P which means skip query matches of things protected. Those are VERY expensive options as a check to the Tablo is done for each matched item to see if they should be skipped or not. These options might be valuable combined with the Delete or DeleteX “transcoders”. Because these are expensive, SurLaTablo defaults to deleting things even if protected. Therefore I recommend using at least the -P option when doing deletes. There are metadata changes, remove your files under SURLATABLO_ROOT and do full reindex.
1 Like
Installed and testing it now.
Request for comments…
So, dealing with null data. In particular TV without a season, without an episode. The problem is where there is a show that might air multiple times in a day and have hundreds of shows in a year.
Right now SurLaTablo tries to “fix” this with the hard code boolean NOZEROTV. It tries to put something into the season and episode for TV by default (NOZEROTV defaults to true).
But it’s anything but perfect and can lead to collisons (multiple shows with zero data on the same day).
I’m thinking about keeping the zeroes and moving NOZEROTV to some sort of option that is more dynamic (dynamic metakeys) and configurable, possible even on a show by show basis (that is, for a series). Thus the meta db would have the zeroes and only on extraction, based on defaults, and/or options set, would you be able to translate the zeroes into something meaninful. Including dumb incrmental, but could also have dynamic meta key (for example), of lair_date_yday (e.g. Dec 25, 2016 is day 360) and possible lair_date_yday_hhmm to make things “ok” for shows with multiple recordings on the same day. I’ve tested this “mess” of a format in Plex, and it seems to be ok with “Hairspray Live! - s2016e3421900” (means 342 day of 2016 at 1900) (btw, thetvdb doesn’t know about Hairspray Live! and themoviedb does… with multiple entries… but of course, the metadata seen by Tablo designates this as “TV” and not “Movie”")
I want to make sure we can still override for zero data using -S and -E options as well. So that if the zeroes are present, one way would be do say -S 2016 -E 1. This might allow matching (manually) what thetvdb might think of the show. For example, the special Once Upon a Time - Evil Reigns Once More, could be forced by command line to be s00e06, which would match what thetvdb thinks about it and make Plex happy.
I guess it’s the whole possibility of s2016e3421900 that is the “request”. Maybe that’s just too ugly. But as a default, it would at least “work” and make Plex happy, just wouldn’t get any matches of metadata from sources known to Plex.
Other examples of “bad” zero show is Carol Burnet and Friends on one of my local channels. While there were “seasons” as far as thetvdb goes, the metadata for Tablo for that show on my channel is not there. So it’s all got zero season and episode data. Again, it might allow for difficult manual naming and matching… sigh, I’d have to tie into thetvdb to attempt title matching to fill in the gaps.
Messy, messy, messy…
The tool could use some better usage examples. And thanks for pointing me at it. Much better than writing my own shell scripts. As to better examples…
These assume you have tablo and surlatablo.py working properly.
For a list of movies in a somewhat useful format:
surlatablo.py -q meta_type~=Movie --queryformat ‘${rec_id}\t${title} (${release_year})’
For a list of tv shows in a similar format:
surlatablo.py -q meta_type~=TV --queryformat ‘${rec_id}\t${friendly_title} (${original_air_date})’
With all the options and such that are in surlatablo, there are probably several hundred useful things.
A brief breakdown of the flags I used (these are all documented in surlatablo, I’m just putting them here for convenience:
-q - perform a query of the surlatablo database
meta_type~=Movie - do the query for entries with a “meta_type” field that looks like “Movie”
–queryformat - format the output of the query as defined in the following
’${rec_id}\t${title} (${release_year})’ – rec_id (which is needed to get videos off the tablo) followed by a tab followed by the name of the movie and then the year it was released in brackets. So, something like:
13534 Universal Soldier (1992)
12885 Blazing Saddles (1974)
If I documented everything, it would be a good size book. 
But, certainly something I want to do. But might be for SurLaTablo 3 (where everything will change).
It probably doesn’t make sense to document everything, beyond the man page documentation you’ve got in the script itself (impressive). But some examples of common things people are likely to do often in a quick reference format. Which is sort-of what I tried to start with my post.
Yep… even then, it would be a book
There are some example throughout the thread, mostly posts I’ve made.
Got a new $200 (ok $203.50) machine. E3-1246v3 cpu. So vs. my old (very old) dual E5440’s… on a commercial zap transcode:
2xE5440
Transcoding (Mp4zap1, -z.mp4): [###############] 100% Elapsed seconds 373.0
E3-1246v3
Transcoding (Mp4zap1, -z.mp4): [###############] 100% Elapsed seconds 200.0
Searching for commercials was more than a minute faster too!! Whoo hoo!
Was going to post some QSV number, but man, what a stikin pile of poo Intel has made. If anyone ever says Intel is pro-open source, tell them they are dead wrong. Evil stuff. Complete mess. Intel, just say no. I mean they couldn’t have made a bigger large stinkin pile of garbage. Amazed.
I’ve got it (SurLaTablo) running on an i3 (doubles as my Plex server).
I haven’t figured out how to do commercial detection (other than manually). What’s the trick?
I made a very simple wrapper script with my preferred command line options.
It wasn’t at all clear to me how to tell SurLaTablo to get a specific recording, so I use wget to grab all the .ts files.
Same problem with transcoding the .ts files into a video file. For that I just call ffmpeg directly from my script (with my preferred options) to re-package the .ts files into a single .mp4 file.
SurLaTablo indexes the data on your Tablo. You can use the recording id, but you can also search by any of the information about a show or shows. Make sure you modify surlatablo.py so that it knows where your ffmpeg executable resides.
The easiest way now with the 2.0 version to do a commercial rip is to specify a transcode option that has commercial zapping built in.
surlatablo.py -q Your-Show -c Mp4zap1
You can use the -q options to select by rec_id by doing -q rec_id~=123456 or by using the -i 123456 instead of a query.
Thanks for the pointer, @cjcox.
I tried it out but it ended up eating about 1/3 of the show I tested it on (Big Bang Theory). So, I’m still using my same process.
- copy the full recording and convert to a .mp4
- jump through the show in VLC to identify timestamps of show vs commercial.
- cut up the .mp4 to remove the commercials.
- splice back the bits I want to keep.
I occasionally have audio sync issues after splicing it back together but that’s my problem (I’m calling ffmpeg directly), not something surlatablo does.
Future version of SurLaTablo will insert chapter marks for where it thinks there might be commercials, etc. For players that know how to deal with chapters.
2 Likes
I was impressed at what is was able to pull off.
I suspect, there needs to be a database of known commercial fingerprints to help cut down on false positives. Possibly some audio analysis as well. Most commercials use a narrower set of audio frequencies (i.e. less base and treble) so that they can increase the apparent volume (how loud we perceive it) without increasing the actual volume (total energy of the sound).
Right now surlatablo just uses black frame detection and some guessing about that.
Did Tablo do an automatic update that disabled my ability to surlatablo my recordings? I have several systems (two) that can download from tablo, and they both seem to experience the same problem at the same time.
Uggghhhhh - allen
Well, I partially answered my question. In all cases, the failure seems to be an attempt to extract a particular episode of “Taken”.
@cjcox Well, this is definitely a growing problem. More and more episodes are failing to be extracted. I use other means to extract the recordings, so I know they are valid. But it keeps getting this kind of error:
Traceback (most recent call last):
File “surlatablo.py”, line 2970, in
search_data_json = urllib2.urlopen(search_data_url.format(search_rec_id))
File “/usr/local/lib/python2.7/urllib2.py”, line 154, in urlopen
return opener.open(url, data, timeout)
File “/usr/local/lib/python2.7/urllib2.py”, line 429, in open
response = self._open(req, data)
File “/usr/local/lib/python2.7/urllib2.py”, line 447, in _open
‘_open’, req)
File “/usr/local/lib/python2.7/urllib2.py”, line 407, in _call_chain
result = func(*args)
File “/usr/local/lib/python2.7/urllib2.py”, line 1228, in http_open
return self.do_open(httplib.HTTPConnection, req)
File “/usr/local/lib/python2.7/urllib2.py”, line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 9] servname not supported for ai_socktype>
Is this app going to continue to be supported, or should I be looking elsewhere? I hope I don’t have to, as I really like this app 
thx - allen
It is supported (well… by me anyhow). I take it you are running SurLaTablo 2.0?
surlatablo.py -v