SurLaTablo maintains a cache of recording ids (so it can quickly know what is new/changed from the last run) and also keep a meta-data cache for each recording. That is the description, format, name, etc. for each recording.
The id cache looks like:
{
"ids": [
"/recordings/series/episodes/2263226",
"/recordings/series/episodes/2263225",
"/recordings/series/episodes/2260295",
"/recordings/series/episodes/2260294",
"/recordings/series/episodes/2258792",
"/recordings/series/episodes/2258790",
"/recordings/series/episodes/2258791",
"/recordings/series/episodes/2256694",
... etc....
The metadata cache is a combination of information from the Tablo and some augmented data, again, for each recording:
"/recordings/movies/airings/2130531": {
"air_date": "2019-07-05T16:00Z",
"air_date_day": "05",
"air_date_hour": "16",
"air_date_minute": "00",
"air_date_month": "07",
"air_date_string": "2019-07-05 16:00",
"air_date_year": "2019",
"air_date_year_day": "186",
"cast_": "William Powell;Myrna Loy;James Stewart;Elissa Landi;Joseph Calleia;Jessie Ralph;Alan Marshal;Penny Singleton;William Law;Sam Levene;George Zucco",
"channel_affiliate": "MVIES",
"channel_num": "27.2",
"channel_res_height": "480",
"channel_res_name": "480i",
"channel_res_width": "720",
"channel_resolution": "sd",
"channel_sign": "Movies!",
"description": "Recently returned home from vacation, private detective Nick Charles (William Powell) and his socialite wife, Nora (Myrna Loy), are back on the case when Nora's cousin, Selma (Elissa Landi), reports her husband Robert (Alan Marshal) missing. As the duo search for the disreputable Robert, Selma's friend David Graham (James Stewart), who secretly pines for her, aids the investigation. When the three begin to unravel Robert's secret life, the missing persons case turns into one of murder.",
"friendly_title": "After the Thin Man (1936)",
"genres_": "",
"lair_date": "2019-07-05T11:00-0500",
"lair_date_day": "05",
"lair_date_hour": "11",
"lair_date_minute": "00",
"lair_date_month": "07",
"lair_date_string": "2019-07-05 11:00",
"lair_date_tz": "Central",
"lair_date_year": "2019",
"lair_date_year_day": "186",
... etc...
The idea with SurLaTablo is you can query your recordings. And then, for a query, you can optionally have actions done. For example, extract each to an .mp4 file.
An empty (no parameters) run of SurLaTablo update the “caches” (databases) above. As it runs, it show the new shows added and also shows you shows deleted. Again, the “cache” represents the current state of your Tablo recodings (not the actually media recording, but basic info about what has been recorded).
No parameter run might look like:
prompt> surlatablo.py
Bless This Mess - s02e04 - Bang for Your Buck
blackish - s06e04 - When I Grow Up (to Be a Man)
This Is Us - s04e04 - Flip a Coin
*RECORDING/BUSY* blackish - s06e02 - Every Day I'm Struggling
Madam Secretary - s06e02 - The Strike Zone
This Week With George Stephanopoulos - s00e41 - Sen. Bernie Sanders (I-Vt.) 2019-10-13 09:30
Meet the Press - s00e39 - Former Defense Secretary James Mattis 2019-10-13 09:00
FOX News Sunday With Chris Wallace - s00e41 - Sen. Chris Van Hollen (D-Md.) 2019-10-13 08:00
The Masked Singer - s02e04 - Once Upon a Mask
Riverdale - s04e02 - Chapter Fifty-Nine: Fast Times at Riverdale High
Trying to remove /recordings/sports/events/1037482
REMOVED:{
REMOVED: "air_date": "2019-10-06T20:25Z",
REMOVED: "air_date_day": "06",
REMOVED: "air_date_hour": "20",
REMOVED: "air_date_minute": "25",
REMOVED: "air_date_month": "10",
REMOVED: "air_date_string": "2019-10-06 20:25",
REMOVED: "air_date_year": "2019",
REMOVED: "air_date_year_day": "279",
REMOVED: "cast_": "Green Bay Packers;Dallas Cowboys",
REMOVED: "channel_affiliate": "FOX",
REMOVED: "channel_num": "4.1",
REMOVED: "channel_res_height": "720",
REMOVED: "channel_res_name": "720p",
REMOVED: "channel_res_width": "1280",
REMOVED: "channel_resolution": "hd_720",
REMOVED: "channel_sign": "KDFW-DT",
REMOVED: "description": "NFL Football - Green Bay Packers at Dallas Cowboys",
REMOVED: "episode_number": 0,
REMOVED: "friendly_title": "Green Bay Packers at Dallas Cowboys (2019-2020)",
REMOVED: "game_date": "2019-10-06",
REMOVED: "game_date_day": "06",
REMOVED: "game_date_month": "10",
REMOVED: "game_date_year": "2019",
REMOVED: "genres_": "",
REMOVED: "home_team": "Dallas Cowboys",
REMOVED: "lair_date": "2019-10-06T15:25-0500",
REMOVED: "lair_date_day": "06",
REMOVED: "lair_date_hour": "15",
REMOVED: "lair_date_minute": "25",
REMOVED: "lair_date_month": "10",
REMOVED: "lair_date_string": "2019-10-06 15:25",
REMOVED: "lair_date_tz": "Central",
REMOVED: "lair_date_year": "2019",
REMOVED: "lair_date_year_day": "279",
REMOVED: "lang": "en",
REMOVED: "long_description": "NFL Football - Green Bay Packers at Dallas Cowboys",
REMOVED: "meta_type": "Sports",
REMOVED: "path": "/recordings/sports/events/1037482",
REMOVED: "qualifiers_": "cc",
REMOVED: "rec_id": 1037482,
REMOVED: "season": "2019-2020",
REMOVED: "season_number": 2019,
REMOVED: "season_type": "regular",
REMOVED: "sort_title": "Green Bay Packers at Dallas Cowboys",
REMOVED: "sport_path": "/recordings/sports/959330",
REMOVED: "sport_type": "NFL Football",
REMOVED: "tablo_ip": "192.168.1.231",
REMOVED: "teams_": "Green Bay Packers;Dallas Cowboys",
REMOVED: "title": "Green Bay Packers at Dallas Cowboys",
REMOVED: "tms_id": "EP000031282090",
REMOVED: "video_duration": 16637,
REMOVED: "video_durationh": "4:37:17",
REMOVED: "video_height": 720,
REMOVED: "video_offsetend": 5524,
REMOVED: "video_offsetstart": -15,
REMOVED: "video_size": 10732732416,
REMOVED: "video_sizeh": "10.0GiB",
REMOVED: "video_state": "finished",
REMOVED: "video_width": 1280
REMOVED:}
S.W.A.T. - s03e03 - Funny Money
Dancing With the Stars - s28e05 - Disney Night
The Flash - s06e02 - A Flash of the Lightning
FBI - s02e04 - An Imperfect Science
So, just a default run refreshes and synchronizes the cache info. Initial run on a new installation with no cache will sync the id and metadata for everything recorded on your Tablo. While syncing doesn’t take long, if you have a lot of recordings the initial sync might take a minute or two or three depending on just how many recordings you have. Future syncs, if done regularly will happen fairly quickly as it only has to handle changes (adds and deletes).
Querying the data is done with a query search parameters and optional formatting parameter. By default, without any formatting, you’ll get the pretty printed JSON data that is stored in the cache db file (the -n says to not do a sync, not needed if we just did one, you know?):
prompt> surlatablo.py -n -q Belushi
"/recordings/movies/airings/756977": {
"air_date": "2019-04-05T02:00Z",
"air_date_day": "05",
"air_date_hour": "02",
"air_date_minute": "00",
"air_date_month": "04",
"air_date_string": "2019-04-05 02:00",
"air_date_year": "2019",
"air_date_year_day": "095",
"cast_": "John Belushi;Dan Aykroyd;Lorraine Gary;Ned Beatty;Treat Williams;Nancy Allen;Robert Stack;Tim Matheson;Toshir\u00f4 Mifune;Christopher Lee;Warren Oates;Bobby Di Cicco;Dianne Kay;Murray Hamilton;Slim Pickens;Eddie Deezen;John Candy",
"channel_affiliate": "THIS",
"channel_num": "33.3",
"channel_res_height": "480",
"channel_res_name": "480i",
"channel_res_width": "720",
"channel_resolution": "sd",
"channel_sign": "ThisTV",
"description": "After Japan's attack on Pearl Harbor, residents of California descend into a wild panic, afraid that they might be the next target. Among them are Wild Bill Kelso (John Belushi), a crazed National Guard pilot; Sgt. Frank Tree (Dan Aykroyd), a patriotic, straight-laced tank crew commander; Ward Douglas (Ned Beatty), a civilian willing to help with the American war effort at any cost; and Maj. Gen. Joseph W. Stilwell (Robert Stack), who tries his hardest to maintain sanity amid the chaos.",
"friendly_title": "1941 (1979)",
"genres_": "",
"lair_date": "2019-04-04T21:00-0500",
"lair_date_day": "04",
"lair_date_hour": "21",
"lair_date_minute": "00",
"lair_date_month": "04",
"lair_date_string": "2019-04-04 21:00",
"lair_date_tz": "Central",
"lair_date_year": "2019",
"lair_date_year_day": "094",
"lang": "en",
"long_description": "After Japan's attack on Pearl Harbor, residents of California descend into a wild panic, afraid that they might be the next target. Among them are Wild Bill Kelso (John Belushi), a crazed National Guard pilot; Sgt. Frank Tree (Dan Aykroyd), a patriotic, straight-laced tank crew commander; Ward Douglas (Ned Beatty), a civilian willing to help with the American war effort at any cost; and Maj. Gen. Joseph W. Stilwell (Robert Stack), who tries his hardest to maintain sanity amid the chaos.",
"meta_type": "Movie",
"movie_path": "/recordings/movies/756978",
"path": "/recordings/movies/airings/756977",
"qualifiers_": "cc",
"rating": "pg",
"rec_id": 756977,
"release_year": 1979,
"sort_title": "1941",
"tablo_ip": "192.168.1.231",
"title": "1941",
"tms_id": "MV000000130000",
"video_duration": 9081,
"video_durationh": "2:31:21",
"video_height": 480,
"video_offsetend": 65,
"video_offsetstart": -15,
"video_size": 1766821888,
"video_sizeh": "1.6GiB",
"video_state": "finished",
"video_width": 640
}
So, maybe that is sort of ugly, thus you can specify a query format to make things simpler… again, so far, we just querying what is recorded on your Tablo, but because it uses synced data, the queries aren’t actually going to the Tablo (that is, we’re being device friendly) and rather is using the already obtained cached id and db data.
Now a query with format.
prompt> surlatablo.py -n -q Belushi -Q '${friendly_title}'
1941 (1979)
Smaller…but maybe we want more? At this point, maybe it’s good to discuss surlatablo2.conf which is a file where you can configure all sorts of things (btw, it must contain the IP addresses of your Tablos). We could add a much much more sophisticated query format at the default. Just to change things up a bit, rather than a query that matches a movie (above), I’m switching to something matches a TV episode (so you can see Plex naming convention fully at work).
Inside my surlatablo2.conf (which goes in your home directory):
SURLATABLO_ROOT = '/tmp/SLT'
TABLO_IPS = ['192.168.1.203','192.168.1.231']
options['queryformat'] = '## (${rec_id}) ${friendly_title} ##\n\t${friendly_date}\n\t${channel_sign}(${channel_affiliate}) - ${channel_res_name} [${video_durationh}]\n\tTablo: ${tablo_ip}\n\n\tDescription:\n\t${Wlong_description}\n\n\t${cast_}\n'
options['wrapwidth'] = 80
options['wrapindent'] = '\t'
options['wrapsubindent'] = '\t'
And query run for all Season 6 shows for Flash:
prompt> surlatablo.py -n -q 'Flash - s06'
## (1043659) The Flash - s06e02 - A Flash of the Lightning ##
Tuesday, October 15, 2019 at 07:00 PM
KDAF-DT(CW) - 1080i [1:01:17]
Tablo: 192.168.1.231
Description:
Faced with the news of his impending death, Barry's resiliency suffers as he
struggles to fight fate; Cecile's career as a D.A. conflicts with her identity
when she crosses paths with a villainous meta-human.
Grant Gustin;Candice Patton;Danielle Panabaker;Carlos Valdes;Tom Cavanagh;Jesse L. Martin;Tom Felton
## (1032654) The Flash - s06e01 - Into the Void ##
Tuesday, October 08, 2019 at 07:00 PM
KDAF-DT(CW) - 1080i [1:01:17]
Tablo: 192.168.1.231
Description:
While Barry and Iris deal with the loss of their daughter, the team faces their
greatest threat yet — one that threatens to destroy all of Central City; Killer
Frost has a brush with death that will change her relationship with Caitlin
forever.
Grant Gustin;Candice Patton;Danielle Panabaker;Carlos Valdes;Tom Cavanagh;Jesse L. Martin;Tom Felton
So far, all we’ve done is querying… no actions at all.
So, let’s do “something”. Originally the idea of the -c option was to “convert” or “create”, but really, this is an action trigger to do “something”. The default uses the predefined (and overridable via surlatablo2.conf) variable TRANSCODER_DEFAULT = [ 'Mp4' ]. Thus by default, we take the recording (extract) from the Tablo to an mp4 file. Unlike most extractors, SurLaTablo extracts and populates where possible the mp4 tag data with a myriad of information from the cache db.
Back to our Flash case, let’s extract those episodes to mp4 (the default action). Note that one of the most time consuming parts is just pulling the data off the Tablo itself. The default action, mp4, is really just a quick recontainerization of the h264 ts data off the Tablo.
prompt> surlatablo.py -n -q 'Flash - s06' -c
## (1043659) The Flash - s06e02 - A Flash of the Lightning ##
Tuesday, October 15, 2019 at 07:00 PM
KDAF-DT(CW) - 1080i [1:01:17]
Tablo: 192.168.1.231
Description:
Faced with the news of his impending death, Barry's resiliency suffers as he
struggles to fight fate; Cecile's career as a D.A. conflicts with her identity
when she crosses paths with a villainous meta-human.
Grant Gustin;Candice Patton;Danielle Panabaker;Carlos Valdes;Tom Cavanagh;Jesse L. Martin;Tom Felton
Working on: [./TV/The Flash/Season 6/The Flash - s06e02 - A Flash of the Lightning]
Retrieving Tablo Data (1043659): [####################] 98% Elapsed seconds 260.0
Transcoding (Mp4, .mp4): [####################] 100% Elapsed seconds 3.0
## (1032654) The Flash - s06e01 - Into the Void ##
Tuesday, October 08, 2019 at 07:00 PM
KDAF-DT(CW) - 1080i [1:01:17]
Tablo: 192.168.1.231
Description:
While Barry and Iris deal with the loss of their daughter, the team faces their
greatest threat yet — one that threatens to destroy all of Central City; Killer
Frost has a brush with death that will change her relationship with Caitlin
forever.
Grant Gustin;Candice Patton;Danielle Panabaker;Carlos Valdes;Tom Cavanagh;Jesse L. Martin;Tom Felton
Working on: [./TV/The Flash/Season 6/The Flash - s06e01 - Into the Void]
Retrieving Tablo Data (1032654): [####################] 98% Elapsed seconds 266.0
Transcoding (Mp4, .mp4): [####################] 100% Elapsed seconds 3.0
Rich meta examination with mediainfo:
prompt> mediainfo TV/The\ Flash/Season\ 6/The\ Flash\ -\ s06e01\ -\ Into\ the\ Void.mp4
General
Complete name : TV/The Flash/Season 6/The Flash - s06e01 - Into the Void.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 1.33 GiB
Duration : 1 h 1 min
Overall bit rate : 3 103 kb/s
Collection : The Flash
Season : 6
Album : The Flash
Part : 1
Track name : Into the Void
ContentType : TV Show
Law rating : TV-PG (L)
Writing application : Lavf57.83.100
Part_ID : s06e01
TVNetworkName : KDAF-DT
HDVideo : Yes
... etc...
So, that’s the basics. I’ve done my extract, can I delete without using the Tablo api? Yes, of course. Same query, but this time we’ll override on the command line the action to take.
prompt> surlatablo.py -n -q 'Flash - s06' -c Delete
## (1043659) The Flash - s06e02 - A Flash of the Lightning ##
Tuesday, October 15, 2019 at 07:00 PM
KDAF-DT(CW) - 1080i [1:01:17]
Tablo: 192.168.1.231
Description:
Faced with the news of his impending death, Barry's resiliency suffers as he
struggles to fight fate; Cecile's career as a D.A. conflicts with her identity
when she crosses paths with a villainous meta-human.
Grant Gustin;Candice Patton;Danielle Panabaker;Carlos Valdes;Tom Cavanagh;Jesse L. Martin;Tom Felton
Working on: [./TV/The Flash/Season 6/The Flash - s06e02 - A Flash of the Lightning]
Executing (deleteResource, .always)
About to delete [The Flash - s06e02 - A Flash of the Lightning], enter "Y" to confirm:
So, for each episode, we’ll get prompted to confirm the delete by entering Y. For large cases, let’s say there was 26 episodes of Flash Season 6 already, I might switch Delete to DeleteX which is a no confirmation delete. Useful for doing bulk high speed deletes off your Tablo.
What else can you do? Well, while you can see embedded text for the closed captions, sometimes a media player (even Plex) can’t deal with the embedded closed captions well. So… with an additional program, ccextractor, SurLaTablo can also produce a companion .srt. The options to do that is -C (capital c). So, to do both the Mp4 extract with srt subtitles, supply options -c -C. A popular choice if you have ccextractor installed.
SurLaTablo has rudimentary commercial removal. This features does require a full transcode, which is somewhat time consuming, but maybe interesting. You can use the transcoder Mp4z. Btw, you can do both, saying Mp4 Mp4z, or simply adding +Mp4z to the default (which by default is Mp4).
When doing multiple transcodes (actual manipulation of the recorded mpeg4 ts data), and wanting to do that with multiple calls to surlatablo, for efficiency sake you might want to preserve the Tablo mpeg4 ts data for subsequent processing. Now, this does take up extra space of course since you’ll have the mpeg4 ts, Mp4 and perhaps the Mp4z file (which are actually named file-z.mp4 vs. file.mp4).
You can see available transcoders (actions) using -t:
prompt> surlatablo.py -t
Mp4W Crop a 4:3 480i channel presentation of a 16:9 widescreen movie.
PlexRefresh Refresh Plex Library
Logging Log command to file
Gif Create an short animated Gif
Ts Testing, do not use. Use --keepdir @ instead.
h265 Product an HEVC h265 .mp4 file.
Flash Create a flash .flv file
Mp4zap1 Ultrafast Mp4 re-transcode that force -z 1.
Mp4z Ultrafast Mp4 re-transcode often used with -z.
cjcox testing user added transcode type.
Mp4zap2 Ultrafast Mp4 re-transcode that force -z 1.
Mp4Wzap2 Crop a 4:3 480i channel presentation of a 16:9 widescreen movie that force -z 2.
Mp4Wzap1 Crop a 4:3 480i channel presentation of a 16:9 widescreen movie that force -z 1.
DeleteX Delete from Tablo.
... etc...
(hmmm a typo bug, Mp4zap2 should say using -z 2 not -z 1)
If you haven’t figured it out yet, you can add your own transoders via surltablo2.conf.
How good is the commercial removal? Well, it’s reasonable, but does use black frame detection only (and some algorithmic decision making). You might get a feel for it by viewing the anti-comskip demo (that is, the opposite of commercial removal where I removed the content, btw, there isn’t really a built in option to do that).
(it’s The Bucket List, except without The Bucket List)
Anyway, barely touching the service. You can do all sorts of things with SurLaTablo.
(do I get a cookie for longest reply post?)