I doubt that Amazon - [quote=“StuTomato, post:66, topic:20869”]
configured to auto-scale to adapt to heavy load by adding additional server instances in the cloud with suitable loadbalancers in front
[/quote] - for free. And CS processing is now free.
And I did some testing as soon as tablo announced the new CS update. And some 30 minute episodes that use to complete CS processing 45-60 minutes after the end of the recording were now at the 1 hour 30 minute to 1 hour 45 minute time frame.
AWS does not give additional instances for free. However, autoscaling allows for both scaling increased capacity and reduced capacity. If there are currently N instances running 24 hours a day, autoscaling could allow N+4 servers during peak 4 hours (perhaps 8pm-midnight CDT), N instances for 16 hours, and N-4 during lighter 4 hour period (perhaps 6-10am) with the same total ‘daily instance hours’ currently being utilized.
I live on the West coast. So there is 3 hours difference to the East coast.
So shows that end at 11PM PDT (2AM EDT) are still taking longer.
So just when is the universal peak 4 hours?
Recorded 6 shows yesterday from morning to nite and CS failed on all 6. But 2 shows recorded this morning had CS, but it was not correct. Its going to be a lonnnngggg time before I pay for CS! 
Well something is working again, mostly. 8 shows yesterday and this morning they all show CS ready. Monday however 6 shows and all have Upload Failed. I don’t get this so called “retry” thing. I have yet to see any show change status, whatever error message shows up it’s stuck with forever from what I can tell. Shame really…
Hi folks -
With so many great shows returning to the airwaves and many new Tablo customers coming online, our Automatic Commercial Skip server has been struggling to meet the demand, especially during Prime Time.
Unfortunately this has resulted in some unexpected hiccups for this beta feature, and some recordings are not being processed as they should.
We’re working right now to increase the robustness and scalability of the feature and appreciate your patience as we do so.
Thanks for the update and openess to provide information. I think most of us had jumped to the conclusion that it might be load related given all the details but confirmation is good. And given it’s a beta feature and no one is currently paying for it, I would hope everyone sees it as a growing pain. Right now I consider it a cool feature when it works but I don’t expect any guarantees while it’s in beta.
Yes, thanks for the update. We kind of figured it was a capacity issue. Hopefully you guys have more than one server. The ACS is the #1 feature my wife likes, without it…well it’s just another DVR from her point of view and the lack of thumbnails on live TV makes it inferior to what she’s use to. So hopefully the ACS issues will be stable soon and she’ll be happy. I’m trying to win a battle over here.
We do… Just didn’t feel the need to get into a longwinded server architecture discussion 
This sounds like a pretty good strategy. But I am certain this is not what is actually happening. If it were, it would take 2 days before I/we saw the “upload failed” message, but in reality I am seeing it within a few hours of the show being recorded (The past few days in particular).
I know the Tablo didn’t reboot, because I was watching other previously recorded shows with it at the time. My Tablo is hard wired to my gigabit switch, so I know there wasn’t any wifi issues. My Comcast service should be more than adequate bandwidth wise to support the upload, and while I may have only been web-surfing with a tablet while watching those previously recorded shows, I didn’t experience anything that would lead me to believe my network had any issues. My Tablo and internal drive are both new, I’m still in a 30 day evaluation period, so while there could be a problem with the drive, the probability of that seems quite low to me and should be discounted unless further evidence presents itself. Plus, with so many other people seeing the same problem, it seems to point to a software issue with the Tablo, either the firmware in the device, or on the server end.
So I guess I have to ask Tablo to revisit their code, to see what bug is causing this quick failure instead of performing these retries over the course of several nightly maintenances as suggested.
I agree, I know the unit isn’t rebooting or lost power, etc. and yet shows that should have retried and processed along with others that did work yet this didn’t happen.
I really appreciate you being open and upfront about it. Other companies would try and deflect. I can live with that feature not being perfect for a while.
Hey folks -
Just a follow-up to our earlier post regarding the issues some folks are experiencing with Automatic Commercial Skip.
Over the past few weeks we’ve seen a 3x increase in the volume of recordings being sent to the server for processing.
While we did anticipate and plan for a bump from the new Fall TV season, this happened to coincide with a huge number of brand new Tablo users coming online.
While we work to adjust our back-end systems to handle this demand spike, we will be temporarily placing NFL Football games on the ‘filtered’ list.
By not uploading files from these popular (and lengthy) football recordings, we hope to free up enough horsepower to successfully process the remaining recordings for everyone.
Once we’ve made the adjustments to account for the increased traffic, we’ll remove NFL football from the filtered list.
Unfortunately we don’t have a specific ETA to share for that, but we hope it will only be a week or two.
Stay tuned for further updates and please keep us posted on your ongoing experience with this feature in this thread. Your feedback during this open beta is invaluable!
I assume you guys are de-duplicating your work load taking a few samples of a show from a given market , processing it and then skipping processing for others who clearly have the exact same content short a couple pixels. In reality they wouldn’t even need to upload.
With 5 different recording qualities I wonder how that works. If I’m recording at 3 Mbps and the only previous upload of the file is 10 Mbps is there a one-to-one mapping? Or if in the middle of a show an emergency news report comes on indicating that my neighborhood is on fire.
Since I’m PDT time my one hour recording that ended at 11PM(PDT) completed CS at 1:05AM.
But the two that completed at 12:30AM and 1:30AM had pretty speedy CS processing. To bad the server doesn’t have some unsophisticated response back to the tablo software indicating that the AWS is too busy and that process needs to be delayed
In the same market the same report would come on for everyone in that market. i.e Same TV station. The quality doesn’t matter as you just flag the start and end time the commercials occupy. Once you have enough samples of a given show from a given market you tell the other Tablo’s with that same show on that same TV station the results. No need to upload. This would produce far faster results with less work. Hopefully they are already doing this. Football games would work the same way, on the same TV station.
Doesn’t Tablo transcode raw OTA mpeg into HLS H.264 files controlled by a m3u8 playlist.
Currently doesn’t CS work by processing the thumbnails and returning a updated playlist for that specific HLS file. So the playlists are the same between different recording qualities. And they are also the same if you start the recording early and end it late.
Ok, and how much would de-duplicating the work load help even if that’s all true? I suspect greatly.
You can probably find a number of posts about copy forward when the open beta was announced at the end of April.
Maybe it will come true. But if copy forward has failures what does a user do to obtain proper CS processing?
I’m not talking about Copy forward. I’m talking about reducing server load. They sample XX samples and if they agree they send that to the remaining requests. Looks like a duck, looks like a duck, looks like a duck, guess what it’s a duck.