Addition of a remove_format similar to query_format and a formatted_remove boolean to trigger it. This gives you the option of creating a better format for the db cache REMOVES than the normal full JSON dump.
This will likely become the official 2.0 release. So please test. Thank you.
@cjcox: Had some time to check out your new release 2.0b3 this evening. It’s again, a fabulous script but I’m getting some errors early on when trying to create the initial cache files. It’s not a blocker as I can work around these issues but still wanted to let you know in case you want to properly catch these errors. I have a huge library and there currently are two offending episodes that your script chokes on when trying to process them:
Frequency - s01e03 - The Near Far Problem
Traceback (most recent call last):
File "C:\TabloTV\surlatablo-2.0b3.py", line 2774, in <module>
rec_val['description'] = new_data_meta['episode']['description'].encode('utf-8').strip()
AttributeError: 'NoneType' object has no attribute 'encode'
Hogan's Heroes - s04e09 - Guess Who Came to Dinner?
Traceback (most recent call last):
File "C:\TabloTV\surlatablo-2.0b3.py", line 2811, in <module>
if (not rec_val['season_number']):
KeyError: 'season_number'
Hogan's Heroes - s04e09 - Guess Who Came to Dinner?
Traceback (most recent call last):
File "C:\TabloTV\surlatablo-2.0b3.py", line 2815, in <module>
if (rec_val['season_number'] >= 0 and rec_val['episode_number'] >= 0):
KeyError: 'season_number'
Hogan's Heroes - s04e09 - Guess Who Came to Dinner?
Traceback (most recent call last):
File "C:\TabloTV\surlatablo-2.0b3.py", line 2955, in <module>
print(' ' + rec_val['friendly_title'])
KeyError: 'friendly_title'`
I’ve also attached a screenshot of the offending shows’ meta data.
Thanks, I’ll look into these and make fixes.
BTW, I’m making an assumption that the traceback errors are from the show printed immediately before the error is shown. If that’s not correct, then I’ve attached the wrong meta data.
Would you mind sending me the result of:
surlatablo.py -i 287642
I’m trying to get surlatablo’s view through the new API of the meta data (you’re giving me the WebSQL DB view I believe).
"/recordings/series/episodes/287642": {
"airing_details": {
"channel": {
"channel": {
"call_sign": "METV-SD",
"call_sign_src": "METV-SD",
"major": 20,
"minor": 2,
"network": null,
"resolution": "sd"
},
"object_id": 27142,
"path": "/recordings/channels/27142"
},
"channel_path": "/recordings/channels/27142",
"datetime": "2015-05-05T04:00Z",
"duration": 1800
},
"episode": {
"description": "Hogan is worried that an underground contact is a double agent.",
"number": 9,
"title": "Guess Who Came to Dinner?",
"tms_id": "EP000021060101"
},
"object_id": 287642,
"path": "/recordings/series/episodes/287642",
"qualifiers": [],
"season_path": "/recordings/series/seasons/27141",
"series_path": "/recordings/series/27140",
"snapshot_image": {
"has_title": false,
"image_id": 298342
},
"user_info": {
"position": 0,
"protected": false,
"watched": false
},
"video_details": {
"duration": 2120,
"height": 480,
"schedule_offsets": {
"end": 305,
"start": -15
},
"size": 531173376,
"state": "finished",
"width": 704
}
},
"/recordings/series/seasons/27141": {
"object_id": 27141,
"path": "/recordings/series/seasons/27141",
"season": {
"name": "4",
"number": 4
},
"season_counts": {
"airing_count": 26,
"failed_count": 0,
"protected_count": 0,
"unwatched_count": 23,
"watched_and_protected_count": 0
},
"series_path": "/recordings/series/27140"
},
"/recordings/series/27140": {
"object_id": 27140,
"path": "/recordings/series/27140",
"series": {
"awards": null,
"background_image": {
"has_title": false,
"image_id": 732686
},
"cast": [
"Bob Crane",
"Werner Klemperer",
"John Banner",
"Robert Clary",
"Richard Dawson",
"Ivan Dixon",
"Larry Hovis",
"Kenneth Washington",
"Cynthia Lynn",
"Sigrid Valdis"
],
"cover_image": {
"has_title": true,
"image_id": 732685
},
"description": "Perhaps the funniest show ever made about fictional hilarity in a Nazi P.O.W. prison, this series focuses on Stalag 13, a camp set aside for U.S. resistance fighters and overseen by the strict but bumbling Col. Klink. Although the prisoners have an abnormal number of perks at the camp, they are constantly trying to mess with the Germans and help the Allied war effort from within.",
"episode_runtime": 1800,
"orig_air_date": "1965-09-17",
"thumbnail_image": {
"has_title": true,
"image_id": 732684
},
"title": "Hogan's Heroes",
"tms_id": "SH000021060000",
"tms_series_id": "184204"
},
"show_counts": {
"airing_count": 169,
"failed_count": 0,
"protected_count": 0,
"unwatched_count": 136,
"watched_and_protected_count": 0
},
"user_info": {
"up_next": "/recordings/series/episodes/141557"
}
},Well, the first one is certainly a “bug”. But I’m a bit confused on the season_number error. The meta data looks ok… and I think my code is ok… so not sure. Maybe that first bug is messing something up? (wouldn’t think so though).
I’ll put in two fixes… one that is more legit than the other. Because season_number should be there, in fact you see it in the friendly_title there. The last one is also puzzling.
Surlatablo should die at the first error and not try to continue. I’m thinking some of the errors are due to the program trying to continue. What version of python are you using?
You could be right. Those style of failures should prevent a friendly_title from being successfully contructed for printing out.
Let me push out a 2.0b4.
Just to be clear. Surlatablo exited out after each error and did not continue. I then hacked the script to skip the error. That way I got the next error and so on. Until I didn’t get any errors anymore.
Python 2.7.12
Download and test with: http://endlessnow.com/ten/Temp/slt-test-py.txt
You’ll want to delete your SurLaTablo cache files before running.
I have a couple more changes… then I’ll push out an official 2.0b4.
You will want to remove the cache files under your SURLATABLO_ROOT and do a full reindex of your Tablo.
Addition of options[‘titles_from_description’] which defaults to false. If set, if there is a resource with no title, it will see if it can construct something from the description up to the first comma or semi-colon. Otherwise and by default title will set to lair_date YYYY-mm-dd. Sports with no season will no longer have empty parens appended to the title. As with TV, worst case for Sports will append lair_date YYYY-mm-dd.
I may have to make title handling a bit more configurable… This means one more beta at least.
@cjcox, i’ve got an interesting problem here.
When I use the normal MP4 transcoder I get my file just fine. When I try and use the new MP4zap1 transcoder it runs but I get a 0 size file as output. Any suggestions on where to look? I don’t get any errors and it processes just fine as far as I can see.
What platform OS? What version of ffmpeg? When you say it “runs”… does it seem to be doing something? (that is, how many seconds do the steps take).
I’ve got it running in a FreeNAS jail under FreeBSD:
[root@tablorip_1 /]# uname -a
FreeBSD tablorip_1 10.3-STABLE FreeBSD 10.3-STABLE #0 r295946+07c41cd(9.10-STABLE): Wed Nov 9 00:19:25 UTC 2016 root@gauntlet:/freenas-9.10-releng/_BE/objs/freenas-9.10-releng/_BE/os/sys/FreeNAS.amd64 amd64
Latest FFMPEG from repository:
[root@tablorip_1 /]# ffmpeg
ffmpeg version 2.8.8 Copyright © 2000-2016 the FFmpeg developers
It runs for about 5-8 mins and does all the extraction and such as I see the .ts file getting created. According to the log file I have it took about 259 seconds to get the file from the Tablo, 41 seconds to generate the first MP4 file and then 0 seconds when it goes to perform the commercial zap and generate the -z.MP4 file. But I am not seeing any error getting thrown.
Might have found it but not 100% sure. Found the line below buried in my log file. Thinking this might be causing the issue.
[aac @ 0x80a896400] The encoder ‘aac’ is experimental but experimental codecs are not enabled, add ‘-strict -2’ if you want to use it.
Not sure if that’s it or not since it does actually generate the file just with 0 size.
I had to do something similar with a previous version of SurLaTablo on Mac OS X.
yep… since it requires a full transcode and not mere recontainerization, you’ll need to either create your own transcoder in surlatablo2.conf (recommended) or edit the code and add the corrected option (see below for what you would put in your surlatablo2.conf):
TRANSCODER_OPTS['Mp4zap1'] = {
'help': [ 'Ultrafast Mp4 re-transcode that force -z 1.' ],
'command': [ FFMPEG ],
'threads': [ '-threads', '0' ],
'inputfile': [ '-i', '${ts_filename}' ],
'subtitlefile': [ '-f', 'srt', '-i', '${srt_filename}' ],
'options': [ '-preset', 'ultrafast' ],
'subtitleoptions': [ '-c:s', 'mov_text', '-metadata:s:s:0', 'language=${lang3}' ],
'audiooptions': [ '-strict', '-2', '-metadata:s:a:0', 'language=${lang3}' ],
'metadata': [],
'zap': [ '1' ],
'ext': [ '-z.mp4' ]
}
Can’t remember if the options go at the front of audioptions or at the rear… but you can test.
Alternatively, you could try a newer ffmpeg.