SurLaTablo.py - Python program to query and convert Tablo recordings

FlyingDiver · June 11, 2019, 9:04pm

Thanks. I had just figured that out from an older post in the thread. I was looking for some sort of compound query terms. Oh well.

cjcox · June 11, 2019, 9:08pm

It’s essentially a regex search over lines (each line is a recording) of JSON for recordings.

cjcox · June 11, 2019, 9:14pm

With that said, you can do complex “like” things doing multiple queries (multiple executions of SLT) and your shell (assuming you have a good shell).

I may have to improve querying so that you can do many things with just one execution of the program (again, likely for SLT 4.x)

FlyingDiver · November 5, 2019, 10:11pm

Is “Mp4zap1” the equivalent of “Shrink” plus --zapcommercials? Or should I just add the “–zapcommercials” option to the Shrink transform?

cjcox · November 5, 2019, 10:39pm

Correct. Use Mp4zap1 instead of --zapcommercials Because of how the code works, it’s a full transcode as it’s an assembly broken up mpeg4 ts data (which you can’t do normally, but transcoding it all makes it work). It tries to use a reasonable transcode set of options that isn’t necessarily designed to save space, but more focused on not taking absolutely forever to transcode.

And yes, other transcoders, like Shrink are safe to use with --zapcommercials. As long a true transcode is done, should be safe to use with --zapcommercials. You’ll see a zap section inside the code for transcoders that should work with --zapcommercials. You’ll also notice that Mp4zap1 hardcodes the zap option to algorithm #1.

bbaorbb · November 29, 2019, 11:50pm

I just installed SurLaTablo and am experimenting with commercial skip. Both examples below yielded a full recording and another recording that was about 1/3 the length of the original,(presumably the zapped version?). Am I doing something wrong?

cjcox · November 30, 2019, 12:23am

For game shows you’ll likely want to add -o postscript=True (and any show that might have a very short end piece at the end)

But other than that, looks like what you’re supposed to do.

Edit: Note, you can’t add options after the transoders (e.g. --clobber)

bbaorbb · November 30, 2019, 2:03pm

I’ve tried processing several shows and the commercial zapped version is always approximately 9-11 minutes. The unzapped version is always fine. Any thoughts what it might be?

cjcox · December 1, 2019, 1:59am

Make sure you’re running the latest version. surlatablo.py --version, should be running 2.3. Can you say what OS you’re using and what version of ffmpeg you’re using? ffmpeg -version

bbaorbb · December 1, 2019, 3:20am

Windows 10
Do I need to put ffmpeg in my path? I actually have two different ffmpeg versions on my PC but I point to the one below in the conf file. I have actually tested with both with the same results. I will remove the one I no longer use.

cjcox · December 1, 2019, 4:23am

Looks like your ffmpeg is a dev build out of git. Which I would think would be ok. Not sure why things aren’t going right. But again, while I suspect maybe something else, black frame detection can make mistakes, but requires sporadic black frames in the true content.

I think in order for me to debug further, I’m going to have to work with you to preserve the original .ts file. I can walk you through some of that and provide a way for you to upload it to a place where I can retrieve it. At that point, we’ll want to take this to private messaging.

deerewright · December 5, 2019, 4:01pm

Is there a way to use this to extract the recordings directly from an old hard drive? I made the mistake of resetting the Tablo, trying to get it to recognize a new hard drive. I have all of the data (/rec/nnnnnn/segs/*.ts) files still intact on the original hard drive. The drive will work with the tablo, but none of the recordings show up.

cjcox · December 5, 2019, 5:23pm

No. Concepts are “there”, I just haven’t looked at the raw drive contents, I’m banking on what others have said about “where things are stored”… but since I haven’t looked at it myself, can’t say what is or is not possible. That is… surlatablo doesn’t handle this, and I haven’t looked at the “raw” data on a harddrive to determine what can be made out of it apart from recorded content without any meta data (this should be possible). There are many posts about how to concat the ts files and turn that into a singular mp4.

Patrick_Wade · January 19, 2020, 7:56pm

First, Thx for the work here! Trying to adjust tmp directory. I see the note about tempfile.tempdir = ‘/path-to/my-tempdir’ but where does this get adjusted? Is there a specific options?

Thanks,
Patrick

cjcox · January 19, 2020, 10:08pm

Put it in your surlatablo2.conf.

Patrick_Wade · January 20, 2020, 3:11am

thx for the quick reply…

yep I did…
SURLATABLO_ROOT = ‘/mnt/usbd/media/SurlaTablo’
TRANSCODER_DEFAULT = [ ‘Mp4’, ‘Gif’, ‘Json’ ]
TABLO_IPS = [‘10.100.10.154’]
tempfile.tempdir = ‘/media/usb1/SurLaTablo/transcode’
options[‘truncate_ts’] = 25
…

but when I run with debug still shows /tmp

python surlatablo-2.3.py -n -d -c -z -q Last\ Man\ Standing\ -\ s08e05 Mp4zap2
Working on: [/media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways]
[ /media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways ]

RETRIEVE FROM TABLO
[ -> /tmp/TabloTxR1K1.ts]
[’/usr/local/bin/ffmpeg’, ‘-i’, ‘/tmp/TabloTxR1K1.ts’, ‘-vf’, ‘blackdetect=d=.03:pic_th=1:pix_th=0.10’, ‘-an’, ‘-f’, ‘null’, ‘-’]

Mp4zap2
[’/usr/local/bin/ffmpeg’, ‘-threads’, ‘0’, ‘-i’, ‘/tmp/TabloTxR1K1.ts’, ‘-preset’, ‘ultrafast’, ‘-metadata:s:a:0’, ‘language=eng’, ‘-metadata’, ‘title=The Office; Mysterious Ways’, ‘-metadata’, ‘media_type=10’, ‘-metadata’, ‘album=Last Man Standing’, ‘-metadata’, ‘show=Last Man Standing’, ‘-metadata’, ‘season_number=8’, ‘-metadata’, ‘episode_sort=5’, ‘-metadata’, ‘episode_id=s08e05’, ‘-metadata’, ‘hd_video=1’, ‘-metadata’, ‘network=WFLD-DT’, ‘-metadata’, ‘genre=’, ‘-metadata’, “synopsis=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, ‘-metadata’, “description=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, ‘-metadata’, “comment=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, u’/media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways-z.mp4’]

cjcox · January 21, 2020, 4:51pm

Sorry about the delay… I’ll take a look at this tonight.

Patrick_Wade · January 21, 2020, 5:02pm

I think the issue it my conf file is not being used. But not sure why…
Running on Raspbian Jessie
Normal conversions do work with Mp4 in /tmp directory
but want to move to diff temp directory for final deployment.

cjcox · January 21, 2020, 5:06pm

Default for where the conf is is set near the top of the program to:

SLT_GLOBAL['CONF'] = '~/surlatablo2.conf'

That means, surlatablo2.conf in your “home directory”.

But you could manually edit that if you need to.

Patrick_Wade · January 21, 2020, 5:12pm

Yep…I did change it

Use option --help for detailed usage.
“”"
SLT_GLOBAL = {}
SLT_GLOBAL[‘PGM_VERSION’] = ‘2.3’

Sur la Tablo (on the Tablo)

SLT_GLOBAL[‘PGM_NAME’] = ‘surlatablo’

License

SLT_GLOBAL[‘PGM_LICENSE’] = ‘GPLv2’

Configuration file

You can override option variables below by putting your into

this file.

SLT_GLOBAL[‘CONF’] = ‘/media/usb1/SurLaTablo/conf/surlatablo2.conf’

raspberrypi:/media/usb1/SurLaTablo $ cd conf/
raspberrypi:/media/usb1/SurLaTablo/conf $ ls
surlatablo2.conf
raspberrypi:/media/usb1/SurLaTablo/conf $ pwd
/media/usb1/SurLaTablo/conf

but same results…

python surlatablo-2.3.py -n -c -d -z -q Last\ Man\ Standing\ -\ s08e05 Mp4zap1
Working on: [/media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways]
[ /media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways ]

RETRIEVE FROM TABLO
[ -> /tmp/TabloKAznLT.ts]
[’/usr/local/bin/ffmpeg’, ‘-i’, ‘/tmp/TabloKAznLT.ts’, ‘-vf’, ‘blackdetect=d=.03:pic_th=1:pix_th=0.10’, ‘-an’, ‘-f’, ‘null’, ‘-’]

Mp4zap1
[’/usr/local/bin/ffmpeg’, ‘-threads’, ‘auto’, ‘-i’, ‘/tmp/TabloKAznLT.ts’, ‘-preset’, ‘ultrafast’, ‘-metadata:s:a:0’, ‘language=eng’, ‘-metadata’, ‘title=The Office; Mysterious Ways’, ‘-metadata’, ‘media_type=10’, ‘-metadata’, ‘album=Last Man Standing’, ‘-metadata’, ‘show=Last Man Standing’, ‘-metadata’, ‘season_number=8’, ‘-metadata’, ‘episode_sort=5’, ‘-metadata’, ‘episode_id=s08e05’, ‘-metadata’, ‘hd_video=1’, ‘-metadata’, ‘network=WFLD-DT’, ‘-metadata’, ‘genre=’, ‘-metadata’, “synopsis=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, ‘-metadata’, “description=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, ‘-metadata’, “comment=Mike debates what to get Vanessa for a business anniversary gift; Kristin and Mandy try to get out of Kyle’s Outdoor Man HR orientation; Vanessa is secretly obsessed with Jen’s new video game.”, u’/media/usb1/media/TV/Last Man Standing/Season 8/Last Man Standing - s08e05 - The Office; Mysterious Ways-z.mp4’]