Cool! I can’t wait. I actually have been working on my own web interface that will essentially execute the script based on some selections or options. Can you share some thoughts on what may be coming to the new version?
Plans include a web based gui including an integrated web service that is also standalone for taking out commercials.
1 Like
Could be awhile though… in the process of moving…
Oh… and I do have some more stuff for version 1. In particular srt generation will always be “on” (that is save srt) when closed captions are requested. This is for Plex since embedded subtitles only work when doing a transcode (albeit a lightweight one)… but this way you can have the srt file in the same dir and Plex can use that with Direct Play.
1 Like
Cool stuff! Can’t wait.
I would like to do this on my FreeNAS. Could you please share how you did this. I created a jail installed python and ffmpeg but that’s it. I have no idea what to do next so it will download only new shows everyday. I don’t even know where ffmpeg is located so I can edit the script location.
Thx
I don’t have a FreeNAS to experiment with. However if you can get to a shell prompt on that server you can do “type ffmpeg” (for example) that will reveal if ffmpeg is on your PATH (ditto python and ccextractor).
If that doesn’t show anything, you may have to hunt. Did you install any of them via some kind of packaging system (e.g. ports or otherwise)?
Trying to help… others that are more familiar with FreeNAS feel free chime in.
Sorry I was asking more of what the syntax should be to download only shows that haven’t been downloaded before. For example “python surlatablo.py” then what?
I would like to download every show to it’s own folder like this “/Arrow/season 1/Arrorw - S01E01 - The Grimm Identity.mkv” I just can’t figure out how to do it.
Also is it possible to remove commercials? If so what would I need to install and do you know if this can be done in FreeBSD. I’m trying to see if I can do all this in FreeNAS before I start over in windows.
Sorry for all the questions any help would be appreciated.
Thanks
e.g.
surlatablo.py -q '.'
Will show (query) everythin that is on your Tablo.
In your surlatablo.conf (default to one in your home directory) you can place options:
SURLATABLO_ROOT = '/opt/plex_root'
TABLO_IPS = ['192.168.1.73']
TRANSCODER_DEFAULT = [ 'Mp4', 'Gif', 'Json' ]
options['truncate_ts'] = 25
options['queryformat'] = '## ${friendly_title} ##\n\t${friendly_date}\n\t${channel_sign}(${channel_affiliate}) - ${channel_res_name}\n\n\tDescription:\n\t${Wlong_description}\n'
options['wrapwidth'] = 80
options['wrapindent'] = '\t'
options['wrapsubindent'] = '\t'
By default, surlatablo caches for speed, so without the option ‘-n’ SurLaTablo will check your Tablo for any changes (new or deleted things). This just means you can query the cache using ‘-n’ without contacting the Tablo at all (for example, to create a csv extract of your data… just change the queryformat above).
SurLaTablo won’t download/overwrite things that are already present. Thus, even if you don’t clean up your Tablo, you can run SurLaTablo to query all and convert all.
surlatablo.py -q . -C -c
(the -C cap-C is optional and will embed subtitles as a stream in the resulting transcode)
The final optional argument is a list of transcoders to use, but default to just Mp4, unless you override on the command line or using TRANSCODER_DEFAULT array, as in my config example, to specify the transcoders to run things through.
Removing commercials is very probabalistic, which means it may simple make you want to pull your hair out. You use the option -z, but you also need to use the transcoder Mp4z, or you will get audio/video sync isues. There are two tweakable algorithms to choose from. You can select the algorithm by setting a default in surlatablo.conf of:
options['zap_algorithm'] = 2 #Note: this default to algorithm 1
Or you can override options on the command line -o zap_algorithm=2
So, to convert and attempt to zap commercials:
surlatablo.py -q . -C -c -z Mp4z
Why you don’t want to do that? Because commercial extraction is primitive in the tool and to just try to always convert all media zapping commercial is probably unwise.
But let’s say you did the non-commerical get and convert everything, and now you want to see about zapping commercials. Note: this requires currently that the show still be found on your Tablo. Let’s say it the latest Supergirl…
surlatablo.py -q 'Supergirl.*s01e04' -C -c -z Mp4z
The resulting file for an Mp4z is named differently (suffixed by default with -z.mp4) so it won’t conflict with existing plain ole Mp4 conversions. Of course it will get placed in the same dir by default, so if it’s a dir tied to a Library dir of plex, you’ll get two playable options for the entry in plex (the original .mp4 and the new -z.mp4 file).
Oh… and again, to avoid cache sync, you can add -n to any of the commands above (that is, you’ve already done a plain ole surlatablo.py call and the cache is up to date). Just to make things faster.
Now, let’s say you want to automate… here we probably don’t care too much about caching delays… so during a idle time (try not to do these while your Tablo is doing anything… I may build in a check at some point)… so you want to create a cron entry on your box.
For example (doing crontab -e, add the following line to your crontab):
00 02 * * * surlatablo.py -q . -C -c >/var/log/surlatablo.log 2>&1
Thus every morning at 2am, try to convert everything that is currently on the Tablo. Remember it will skip anything it finds is already out there (SURLATABLO_ROOT).
Here’s an example of me converting the new Supergirl (I tend to not automate, and only pull things down when I want to). Note: I used a very general query for anything contain the word Livewire, it could of matched many things, but I ran a plain -q query first (without the -c) to see what came back:
surlatablo.py -n -q 'Livewire' -C -c -k @
## Supergirl - s01e04 - Livewire ##
Monday, November 16, 2015 at 07:00 PM
KTVT-DT(CBS) - 1080i
Description:
Kara's foster mother visits for Thanksgiving; Cat and Supergirl become targets
when an accident transforms a volatile CatCo employee into Livewire.
Preserved temporary (.ts) file location: [./TV/Supergirl/Season 1/Supergirl - s01e04 - Livewire.ts]
Working on: [./TV/Supergirl/Season 1/Supergirl - s01e04 - Livewire]
Retrieving Tablo Data (508018): [####################] 100% Elapsed seconds 217.0
Srt file location: [./TV/Supergirl/Season 1/Supergirl - s01e04 - Livewire.eng.srt]
Extracting CC as Subtitle: [####################] 100% Elapsed seconds 18.0
Transcoding (Mp4, .mp4): [####################] 100% Elapsed seconds 30.0
Transcoding (Gif, .gif): [####################] 100% Elapsed seconds 2.0
Executing (dumpJson, .json)
So… why -k @. You don’t have to do this, but this means I want to keep all transient data files including the .ts file download off the Tablo. This also preserved the .srt file which is useful with Plex (new version, not quite released makes the .srt generation mandatory when using -C).
Anyway, lots to absorb I’m sure. Keep prodding me if you get stuck.
Thanks
I said something wrong… SURLATABLO_ROOT is merely the location of the cache metadata. By default transcodes go into a plex friendly hierarchy at the present working directory. So change directory to your plex library root folder.
If you don’t like that, you can change the default base directories in your surlatablo.conf:
BASE_DIRS={'Default':'plex_root/${meta_type}',
'Sports':'plex_root/${meta_type}/${sport_type}/Season ${season_number}',
'TV':'plex_root/${meta_type}/${Eseries}/Season ${season_number}',
'Movie':'plex_root/${meta_type}/${Etitle} (${release_year})'
}Ok I got everything working except zap. I get “Warning: No zap compatible transcoder provided. Transcode may have audio/video sync issues.”
One other thing I’m not using Plex I’m using Emby if that make any difference.
Thx
Last parm (not a switch)… make it Mp4z or if you want to augment the default trancoders (do whatever the system has set or you’ve already specified in your conf) do +Mp4z (by default that would create both a .mp4 and a -z.mp4 file)
Thanks for the help everything is working now.
###1.3 -
- When using -C always save the .srt file into the transcode target
directory. Copy both into your plex area and then you can use the
SRT from Roku (for example). - PGM_VERSION was never changed in 1.2, fixed to 1.3 in this version.
There are some things not working quite right with the new Tablo changes (2.2.8). I am working on some adjustments. I will try to document things that aren’t going to work anymore.
SurLaTablo 1.4 Released (the sadness release)
###1.4 -
1.4 - (the sadness release)
- With removal of extended metadata by Nuvyyo, the following changes had to be made:
- gamedate (and gamedate_year, gamedate_month, etc.) is simply replace by the local
air date data. Which is definitely not the same, but the best we can now do. - long_description will now simply inherit the short or regular description.
- channel_* all channel metadata is gone.
- genre is gone.
- rating is gone.
- qualifiers_ list is gone.
- gamedate (and gamedate_year, gamedate_month, etc.) is simply replace by the local
- I have added season and episode metadata into ffmpeg transcoded files.
- Added a --ts option which allows makes the assumption that the pulled Tablo mpeg4_ts
file pull has already been done. Useful for trying things out without have to
do the slow re-pull of the mpeg4_ts off the Tablo.
Note: due to metadata changes, you may want to remove your cache (surlatablo will rebuild) if you have recorded any Sports type of program especially after Dec 1. Existing metadata for shows prior to Nuvyyo’s changes to the metadata are currently not affected and the elements mentioned above will still work for those shows.
I still believe there is some hope to get back some of the lost functionality.
Thanks for creating such a great program. I hope the company updates you with new API information.
I did have a question that does not appear to be version related to either Tablo firmware or SurlaTablo. I get these busy messages and it’s not currently recording any of these programs. Unless Tablo has a hidden 12 channel tuner inside I was wondering if anybody else have seen such messages?
RECORDING/BUSY Crazy Ex-Girlfriend - s01e04 - I’m Going on a Date With Josh’s Friend!
RECORDING/BUSY The Big Bang Theory - s08e01 - The Locomotion Interruption
RECORDING/BUSY Breakout (1975)
RECORDING/BUSY CBS News Sunday Morning - s12e06 - Episode 2015-12-06
RECORDING/BUSY The Office - s09e26 - Finale
RECORDING/BUSY NOVA - s11e25 - Einstein’s Big Idea
RECORDING/BUSY The Avengers - s06e01 - The Forget-Me-Knot
RECORDING/BUSY F Troop - s01e07 - The Girl From Philadelphia
RECORDING/BUSY My Favorite Martian - s01e30 - How You Gonna Keep 'Em Down on the Pharmacy?
RECORDING/BUSY The Andy Griffith Show - s04e12 - Opie and His Merry Men
RECORDING/BUSY The Office - s05e07 - Customer Survey
RECORDING/BUSY The Big Valley - s01e15 - Judgment in Heaven
I’ve noticed this same issue for some shows. It seems to be from a problem with Tablo “thinking” that this is the state of the recording. Sometimes this can happen if Tablo crashes (among possibly other reasons as well). Often, there is nothing wrong with the recording other than the state.
To force SurLaTablo.py to convert the show, I use the “–busyignor” option. Then, I delete the show from Tablo as I no longer need it on Tablo.
for example (based on my script that would convert all the shows I care about):
python surlatablo.py
python Surlatablo.py --noupdate --query “The Big Bang Theory” --convert --busyignore
python Surlatablo.py --noupdate --query “Breakout” --convert --busyignore
.
.
.
python Surlatablo.py --noupdate --query “CBS News Sunday Morning” --convert --busyignore
Hi, cjcox.
I was forced to discover surlatablo when a tablo update caused my other download app to fail. I LOVE surlatablo !!
I’m running plex media server on FreeBSD. There is one issue I can’t seem to resolve. Closed captioning is tantalizingly close to being resolved for me. I’ve spent 20+ hours trying to get it to work. I can’t seem to figure out how to install ccextractor (written for linux, but not officially ported to FreeBSD). I’ve tried using ffmpeg to “-scodec copy” to the mp4 file (it fails). The more elegant solution is to embed it into the movie file, but would be entirely pleased if it were a separate .srt boxcar file.
It seems that you have the same environment as me, but you’ve managed to figure out the CC solution. Any help you can provide would be IMMENSELY appreciated 
thx - allen