File transfer benchmarks / advice

[jpmarto]

I'm transferring many years of programs from old to new drives, and reorganizing my saved programs (which is proving difficult given the inability to manage folders/rename files on EHDs, but I digress...). Before starting this project, all existing USB 2.0 harware was retired/recycled. Still, it seems to me that transferring files between my Hopper3 and EHDs are going much, much slower than I expected. I'm using a powered hub, 2.5"/7200 rpm EHDs, and what are labeled as USB 3.0/3.1 enclosures. Before using the drives with the Hopper they exhibited expected performance under Windows 10/11 formatted in NTFS. I am using the lower of the two rear Hopper3 USB ports.

Almost all of my files are HD, but I don't know if all one hour HD shows are close to the same size. Does anyone have any good benchmarks as to how long a typical 1 hour HD TV show (assuming there is a standard size) should take to transfer to and/or from the Hopper3 so I can determine if I'm just impatient or have hardware issues? Or any other way to benchmark file transfer performance?

As an aside, it seems to me that trying to transfer too many files at once causes issues. I've begun to limit the number to 10 or 12, which causes me to waste a lot of time when I'm not at home to keep checking progress. Any advice on that as well?

Thanks in advance for any help!

 

[KAB]

As an aside, it seems to me that trying to transfer too many files at once causes issues. I've begun to limit the number to 10 or 12, which causes me to waste a lot of time when I'm not at home to keep checking progress. Any advice on that as well?

Thanks in advance for any help!

It ususally takes 15-20 minutes to transfer three movies. Even 10 or 12 is to too much to transfer at a time, IMO I would keep it at 6-8. And quit checking the progress. It will finish when it finishes.

 

[jpmarto]

Thanks for the advice KAB - I will follow it. They say "a watched kettle never boils", and checking transfers is probably the same concept. Tranferrring hundreds of programs/movies/concerts may take a months of calendar time in increments of 6-8! I'm in the ballpark of 15-20 minutes for 3 movies, so at least my hardware is seems to be working as best as can be expected. Thanks again!

 

[bookworm370]

Before using the drives with the Hopper they exhibited expected performance under Windows 10/11 formatted in NTFS. I am using the lower of the two rear Hopper3 USB ports.

Sorry for you spending the extra money unless your old drives were in peril of failing. No matter what port you use on the Hopper, EHD's are only supported as USB 2 devices. So you'll get no more than USB 2.0 speeds. They did perform better on Windows because they were used at USB 3 speeds.

But Hopper = USB 2 no matter what you do. (Hey, that rhymes!)

 

[jpmarto]

The lower of the two rear USB ports is blue, and labeled "3.0". Silly me thinking it made a difference! I even moved my OTA adapter out of the 3.0 to the 2.0 to make room for the EHD, thinking it would optimize the configuration. I wonder if the 3.0 port is actually 3.0, and if it is, what Dish planned to use it for - as near as I can tell the OTA adapter works fine at 2.0.

 

[TheKrell]

Silly me thinking it made a difference!

Silly me too! I'm shocked that a so-called USB3 port isn't actually a USB3 port.

Perhaps Bookworm is not referring to the ultimate wire speed, but rather the speed the receiver can transfer programs given all the encryption or decryption going on.

 

[ppappas]

I just went through this process, moving about 450 movies from a 2TB drive to a 4TB drive. It took me about a week. Quite frankly, the Hopper 3's support for EHDs is nowhere near as solid as my old VIP 722K. The only advantage of the Hopper is support for multiple EHDs and greater than 2TB of storage. In my experience, transferring a single movie took between 5 and 7 minutes and the direction (to Hopper or EHD) didn't seem to make any difference. I did not limit myself to less than 10 at a time. I usually moved 20 to 40 at a time, sometimes more. Typical time was around 5 to 6 hours for each bulk transfer. On a couple of occasions, the transfer would hang part way through requiring the Hopper to be rebooted. Also, after moving a bunch of recordings to the Hopper from the old EHD, I could not move any from the Hopper to the other EHD. It would give a "failed to transfer error" and rebooting was the only way to continue. The other thing I noticed during the time I had both EHDs connected, sometimes after it's nightly reboot, only one EHD would show as connected the next morning. Powering off and back on the affected EHD usually brought it back. Once I figured all this out, the process was straight forward and just took time.

Even though the Hopper UI has a lot of nice features compared to the VIP series, the UI for recordings leaves a lot to be desired.

- Tiles suck, please give us the option for a list view!
- No count for the number of recordings either on the Hopper or EHD, seriously?
- No really useful information on disk space utilization. Normally just shows percentage used with no reference to the size. You only see this information before committing to a transfer. The VIP UI was so much better than this!
- When transferring a bunch of recordings, the duration is only given as Minutes and Seconds instead of Hours, Minutes and Seconds. Very lame!

Whatever engineering manager approved this stuff should be fired. This is simple programming and should never have been release this way. I was a software engineer and engineering manager for over 40 years and would never agree to remove useful features.

Rant off

 

[Almighty1]

I think the Hopper 2000 probably had better EHD support with the SATA controllers than the Hopper 3 did. Yes, it may support larger than 2TB drives but there is another issue that you or someone else presented as Hopper will not display more than 999 recordings so assuming you had a drive bigger than 2TB, you can record a bigger amount of recordings but what good is that if you can only see the first 999 recordings so if you somehow got like 1020 recordings as a example, you will not even be able to see 21 of them unless you deleted atleast 21 after watching them and not sure how they decide on which 999 recordings to show, is it by the datestamp or is it done randomly each time. It does show total number of recordings only if it was in a folder which excludes EHDD's or by title. Because of the inaccurate reporting of capacity even as a %, the Hopper 3 can say HD is full even when it shows 80% if it was rebooted more than 24 hours ago that only a reboot will fix, I usually move a few 1.5 hour SD recordings daily and yesterday with software v358 for example, I was moving a total of 2 recordings and the transfer hanged in the background without notice as I was trying to select the EHDD and the progress showed like 75% completed with 0 minutes and 43 seconds to go so I stopped it since it was already 30 minutes later and the EHDD light was not showing activity, it acted like it accepted the stop but when I goto the Hopper 3 HDD and select the same EHDD again another 30 minutes later, it showed the same 75% completed with 0 minutes and 43 seconds to go so basically it's like the connection was lost and the only way to fix it is to reboot the Hopper 3. Sometimes the Hopper 3 does not show all the drives after rebooting, no matter which method is used so the only fix is to first unplug the powered USB 3.0 hub, reboot and then plug the hub back in after the Hopper 3 has successfully rebooted. Sometimes the Hopper 3 will detect all 3 EHDD's but will only show 2 because the last one is detected as External Device 4 instead of 3 which was something I saw only with software v358 even though I have only 3 of the same WD Passport 2.5" EHDD's connected to the USB 3.0 hub and the only way fix it is with a reboot as unplugging and plugging will only have it detect as External Device 4 instead of 3 since it appears the Hopper 3 can only mount devices if it's name 1-3 and not 4.

I am as shocked as others as I always thought because of what others said since 2016 that the USB 3.0 port was in fact faster on the Hopper 3 than the other ports being 2.0.

 

[ppappas]

The "not detecting all drives" after a reboot was very annoying, especially since the Hopper 3 reboots every night. That was once of the reasons I moved all my recordings to a larger drive instead of just adding a second drive. Also, I wanted everything in one place which is very useful when sorting by title so I can find a specific movie.

 

[arlo]

I use several different USB interfaces for storage transfers. Using the native interface in my bench pc is just always a lot faster. It handily has a few external sata and power ports. Depending on the file system I'll boot to Linux or use one of the Windows methods to read the Linux file system. And moving chunks of files, folders instead of a ctrl-a and move it all doesn't engulf the resources.
It might be a good idea also to perform whatever "defrag" or "compact" function is available for the specific file system.

 

[bookworm370]

Silly me too! I'm shocked that a so-called USB3 port isn't actually a USB3 port.

Perhaps Bookworm is not referring to the ultimate wire speed, but rather the speed the receiver can transfer programs given all the encryption or decryption going on.

Yes, the connector is USB3, and by the conventions, you don't put in a blue receptile unless it can transfer at least at USB 3.0 speeds. But the transfer differs from what the hardware and software behind the USB port can handle.

I'm sure that at the time it was introduced there was some plan on using the port for a new USB 3 device but it never materialized as the H+ came along. So the software just treats both ports and feeds them at USB 2.1 speeds.

 

[bookworm370]

It would give a "failed to transfer error" and rebooting was the only way to continue. The other thing I noticed during the time I had both EHDs connected, sometimes after it's nightly reboot, only one EHD would show as connected the next morning. Powering off and back on the affected EHD usually brought it back. Once I figured all this out, the process was straight forward and just took time.

The Hopper has been doing this since they introduced EHDs. It's related to the size of the installed EHD (s). The larger the storage, the more often it will fail with a failed-to-transfer message. I have a 6TB and a 1TB for a total of 7TB, and it will work fine on the first transfer of the day for maybe one or two times. Then, it will give the fail message.

Before I figured out a workaround, a reboot to fix it (because sometimes I had 2-3 hours of active DVR recording being recorded, and a reboot would lose sections of the program, I used to wait until nothing was in the queue and just do a Home-Home-Home-Tools-Reset Hopper to get it back to health.

But I did find a workaround if you are patient.

• Select one of the hard drives if you have two;
  if not, just select the EHD as the external drive.
• Play something, anything from it, for a few seconds
• If you have two EHDs, now select the other one
• Play again from that.
• Now select the internal drive.
• Again, play (maybe that part can be taken out)
• But the moving around the input drives seems to clear out what's stuck in the transfer buffers.
• After a few selections/plays/selections, go to the internal drive and start a transfer. 8 out of 10 times, it will, as if by magic, begin to transfer correctly again. If not, just try another select round, and I've always been able to get it to come back.

I only do this if I know that the H3 is recording something currently or within the time the H3 will restart. Sometimes, it will restart, do a systems check, and continue, so you must allow that. But a restart is much easier on the button pushes than mucking around with reselecting drives.

End of the story from an actual engineer, a software developer of 15 years and current CIO, the H3 has a buffer overflow problem from day one that gets the EHD transfer buffer in a gooned-up state. No one fixes it because people just put up with it and restart their H3s and don't report it back to Dish support.

Again, the total size connected and what has to be internally stored for directories does have a direct relationship with the amount of time the hangs will occur. If I just have a 500Med or 1TB drive, I can't remember the last time the transfer errors occurred. Add a 2TB to the 1TB for three; then it happens more often. At the 1+6 configuration, it's just about every day.
That's why I had to figure out a workaround.

I believe that all H3 developers must be forced to take home and use the Hopper as their primary unit configured at the maximum a customer could use. If they started to get these butthole errors at home, I bet they would be in the office the next day with the code all out on multiple monitors pouring over it because their family is pissed off with all the reboots they have to do.

Even though the Hopper UI has a lot of nice features compared to the VIP series, the UI for recordings leaves a lot to be desired.

- Tiles suck, please give us the option for a list view!
- No count for the number of recordings either on the Hopper or EHD, seriously?
- No really useful information on disk space utilization. Normally just shows percentage used with no reference to the size. You only see this information before committing to a transfer. The VIP UI was so much better than this!
- When transferring a bunch of recordings, the duration is only given as Minutes and Seconds instead of Hours, Minutes and Seconds. Very lame!

Rant off

I had a one-on-one with the new VP of H3 UI and Dish Anywhere, a few months ago. All four came up, and he kept nodding (Zoom call) that it would be much better if the user could see the recordings in tile or list view.

Also, the ability to arrange the (Home what's playing) screen to do that stuff they are interested in is near the top (drag and drop tiles in the order you want to see the functions). If you wish to see current shows, movies or Netflix at the top, just drag and drop the tile to the location you want in the list. It's not hard, and the entire UI has been developed by Apple already for the Apple TV with all the remote gestures required to unlock a tile (jiggle), move it around the panels and then drop it where you want it. Do it like the Apple TV does, and you now have a defacto industry standard for tile re-organizations.

Again, a look of (holy sh*t!) came over his face, and he started to type furiously. As I said, he's new, so just what pull he has over an established product is unknown. So Dish does do focus groups or personal talks with some of their higher-end users and tries to listen to them.

We also talked a lot about the convenience features. The layout of the text selection screen allows it to look either like the current or the keyboard style so that people know where the next character they want to select would be if they know QWERTY keyboard layouts by heart.

The call went very well and lasted about twice the allotted time. He also had some ghost developers and managers on the call who were sitting in but did not specify who else was on the call. It was one-on-one with him and me; sometimes, he would look at his tablet and ask a question out of thin air. Which I'm sure he probably got from one of the other minions.

So they are committed to making the Hopper and Dish Anywhere (we had about an hour devoted to how Dish could improve their products).

I think the entire Dish world is just not pointed to the Hopper+. But when anything might come out for this, if any, is anyone's guess?

 

[bookworm370]

I am as shocked as others as I always thought because of what others said since 2016 that the USB 3.0 port was in fact faster on the Hopper 3 than the other ports being 2.0.

As I stated before, the hardware board behind the USB 3 receptacle supports USB 3 speeds, but the Hopper will only deliver data at USB 2 rates. Why? Maybe it was easier to code until they figured out what the USB 3 port could be used for. But at least hardware engineering was looking ahead.

 

[bookworm370]

The "not detecting all drives" after a reboot was very annoying, especially since the Hopper 3 reboots every night. That was once of the reasons I moved all my recordings to a larger drive instead of just adding a second drive. Also, I wanted everything in one place which is very useful when sorting by title so I can find a specific movie.

The EXT3 file system used on the EHDs does not support compact or defrag.

The most you can do is manually move shows between the multiple partitions to spread them out (even though it's recommended to bunch them together to allow indexing to run faster and not bounce the heads around the platter).

Dish starts by recording all shows started inside out (from the center platter to the outer), which is a great idea because of the increased rotational axis speeds in the center versus the edges.

But moving the programs requires a real Linux machine and a good working knowledge of a Linux file manager like Dolphin or one of the others.

 

[Almighty1]

bookworm370 - I'm trying to figure out when to do the workaround, is it immediately after the H3 starts up because in my case with 3 x 2TB drives, what I noticed with mines is I don't see the error message but basically it's like if I was transferring and I mean only 2 x 1.5Hr SD recordings, sometimes it does not finish and when I select the EHDD from the DVR menu after 30 minutes later or basically longer than it normally takes which is probably 5 minutes or less, all I get is the progress screen but the bar that normally turns green does not have any progress while the remaining time would be 0 hr and 43 seconds. The only choices I have is Stop and I forgot what else was on that screen but I have not found a way to actually go into that EHDD to watch a recording because even if I press stop, I think it goes back to whatever I was doing before and if I select the same EHDD again, I get that progress screen with the stop button and the same 0 hr and 43 seconds.

 

[Almighty1]

As I stated before, the hardware board behind the USB 3 receptacle supports USB 3 speeds, but the Hopper will only deliver data at USB 2 rates. Why? Maybe it was easier to code until they figured out what the USB 3 port could be used for. But at least hardware engineering was looking ahead.

Yes, it's more of an assumption since others seem to imply the USB3 port was faster but the only thing is unless we actually can see the transfer speeds using each of them for a comparison, it would not be valid. And what you said makes sense since the hardware is only part of the equation, the actual throughput is another issue as for all we know, it might not even deliver full USB 2 speeds either since it's not like we had a way to see what the actual speeds were.

 

[charlesrshell]

Should the Plus be plugged into tie USB3 port?

 

[Almighty1]

Should the Plus be plugged into tie USB3 port?

Especially when the cable shows the Blue color on the connector. But if what bookworm370 says is correct then even if it was in the USB3 port, it will not do speeds faster than USB2.0.

 

[Bobby]

Should the Plus be plugged into tie USB3 port?

No

 

[ppappas]

The Hopper has been doing this since they introduced EHDs. It's related to the size of the installed EHD (s). The larger the storage, the more often it will fail with a failed-to-transfer message. I have a 6TB and a 1TB for a total of 7TB, and it will work fine on the first transfer of the day for maybe one or two times. Then, it will give the fail message.

Before I figured out a workaround, a reboot to fix it (because sometimes I had 2-3 hours of active DVR recording being recorded, and a reboot would lose sections of the program, I used to wait until nothing was in the queue and just do a Home-Home-Home-Tools-Reset Hopper to get it back to health.

But I did find a workaround if you are patient.

• Select one of the hard drives if you have two;
  if not, just select the EHD as the external drive.
• Play something, anything from it, for a few seconds
• If you have two EHDs, now select the other one
• Play again from that.
• Now select the internal drive.
• Again, play (maybe that part can be taken out)
• But the moving around the input drives seems to clear out what's stuck in the transfer buffers.
• After a few selections/plays/selections, go to the internal drive and start a transfer. 8 out of 10 times, it will, as if by magic, begin to transfer correctly again. If not, just try another select round, and I've always been able to get it to come back.

I only do this if I know that the H3 is recording something currently or within the time the H3 will restart. Sometimes, it will restart, do a systems check, and continue, so you must allow that. But a restart is much easier on the button pushes than mucking around with reselecting drives.

End of the story from an actual engineer, a software developer of 15 years and current CIO, the H3 has a buffer overflow problem from day one that gets the EHD transfer buffer in a gooned-up state. No one fixes it because people just put up with it and restart their H3s and don't report it back to Dish support.

Again, the total size connected and what has to be internally stored for directories does have a direct relationship with the amount of time the hangs will occur. If I just have a 500Med or 1TB drive, I can't remember the last time the transfer errors occurred. Add a 2TB to the 1TB for three; then it happens more often. At the 1+6 configuration, it's just about every day.
That's why I had to figure out a workaround.

I believe that all H3 developers must be forced to take home and use the Hopper as their primary unit configured at the maximum a customer could use. If they started to get these butthole errors at home, I bet they would be in the office the next day with the code all out on multiple monitors pouring over it because their family is pissed off with all the reboots they have to do.

I had a one-on-one with the new VP of H3 UI and Dish Anywhere, a few months ago. All four came up, and he kept nodding (Zoom call) that it would be much better if the user could see the recordings in tile or list view.

Also, the ability to arrange the (Home what's playing) screen to do that stuff they are interested in is near the top (drag and drop tiles in the order you want to see the functions). If you wish to see current shows, movies or Netflix at the top, just drag and drop the tile to the location you want in the list. It's not hard, and the entire UI has been developed by Apple already for the Apple TV with all the remote gestures required to unlock a tile (jiggle), move it around the panels and then drop it where you want it. Do it like the Apple TV does, and you now have a defacto industry standard for tile re-organizations.

Again, a look of (holy sh*t!) came over his face, and he started to type furiously. As I said, he's new, so just what pull he has over an established product is unknown. So Dish does do focus groups or personal talks with some of their higher-end users and tries to listen to them.

We also talked a lot about the convenience features. The layout of the text selection screen allows it to look either like the current or the keyboard style so that people know where the next character they want to select would be if they know QWERTY keyboard layouts by heart.

The call went very well and lasted about twice the allotted time. He also had some ghost developers and managers on the call who were sitting in but did not specify who else was on the call. It was one-on-one with him and me; sometimes, he would look at his tablet and ask a question out of thin air. Which I'm sure he probably got from one of the other minions.

So they are committed to making the Hopper and Dish Anywhere (we had about an hour devoted to how Dish could improve their products).

I think the entire Dish world is just not pointed to the Hopper+. But when anything might come out for this, if any, is anyone's guess?

Wow, how did you manage to get a 1-on-1 video call with their VP of H3 UI? I would love to be on that call! I have only been on the Hopper 3 for a couple of months, and some things really suck compared to the VIPs (I am a 25 year customer).