Uncategorized

HDD Large file transfer problem

Asked 
Viewed 34 times
2

My setup is:

Raspberry Pi Model 3B V1.2

No USB HUB

External 3.5 inch HDD:

  • Powered Enclosure: Sabrent USB 2.0/ESATA to 3.5 Inch IDE or SATA/SATA II Aluminum Hard Drive Enclosure Case with Cooling Fan.
  • HDD: HGST Ultrastar 7K4000 HUS724030ALA640 3TB 64MB Cache

I can mount the drive just fine in fstab and read, write/ make subdirectories just fine. The disk will stay mounted for days if I dont mess with it. Writing small files to the disk is just fine too.

The issue comes when I try to write a large file (~1 GB movie). It will go well for the first few hundred MB and then stall and then quit with the following message: “Error splicing file: Read-only file system” What has happened is that the drive is just spontaneously ejected during this process.

Resolution steps attempted:

3 different power supplies

  • 5.25V/2.5A (KANO)
  • 5V/2A (Apple 10W)
  • 5V/2.1A (Generic)

file transfer method

  • GUI drag and drop
  • cp command
  • mv command
  • > command

deprecating USB speed

  • add dwc_otg.speed=1 to /boot/cmdline.txt

increasing USB power

  • add max_usb_current=1 to /boot/config.txt

At this point I’m out of ideas to try, because now I’m at my limit of what root cause could be.

I’m new to linux and RPi. I started this whole endeavor to set up a low-cost low-power home NAS, Time Machine target and PLEX server, so this is kind of a dealbreaker.

OS info:

pi@raspberrypi:~ $ uname -a

Linux raspberrypi 4.19.97-v7+ #1294 SMP Thu Jan 30 13:15:58 GMT 2020 armv7l GNU/Linux

pi@raspberrypi:~ $ lsb_release -irc

Distributor ID: Raspbian

Release: 10

Codename: buster


Edit 1:

dmesg output:


[ 13.453145] Bluetooth: RFCOMM socket layer initialized
[ 13.453171] Bluetooth: RFCOMM ver 1.11
[ 15.550139] fuse init (API version 7.27)
[ 20.854707] logitech-hidpp-device 0003:046D:101B.0006: HID++ 1.0 device connected.
[ 122.206592] usb 1-1.3: reset high-speed USB device number 6 using dwc_otg
[ 128.767440] usb 1-1.3: USB disconnect, device number 6
[ 128.806611] print_req_error: I/O error, dev sda, sector 2147938544
[ 128.806750] print_req_error: I/O error, dev sda, sector 2147938784
[ 128.806899] print_req_error: I/O error, dev sda, sector 2147939024
[ 128.806930] print_req_error: I/O error, dev sda, sector 2147939264
[ 128.807002] print_req_error: I/O error, dev sda, sector 2147939504
[ 128.807027] print_req_error: I/O error, dev sda, sector 2147939744
[ 128.808494] print_req_error: I/O error, dev sda, sector 2147939984
[ 128.808520] print_req_error: I/O error, dev sda, sector 2147940224
[ 128.808539] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 83886080 size 7344128 starting block 268492544)
[ 128.808554] Buffer I/O error on device sda2, logical block 56576
[ 128.808572] Buffer I/O error on device sda2, logical block 56577
[ 128.808583] Buffer I/O error on device sda2, logical block 56578
[ 128.808593] Buffer I/O error on device sda2, logical block 56579
[ 128.808604] Buffer I/O error on device sda2, logical block 56580
[ 128.808614] Buffer I/O error on device sda2, logical block 56581
[ 128.808625] Buffer I/O error on device sda2, logical block 56582
[ 128.808635] Buffer I/O error on device sda2, logical block 56583
[ 128.808646] Buffer I/O error on device sda2, logical block 56584
[ 128.808656] Buffer I/O error on device sda2, logical block 56585
[ 128.809861] print_req_error: I/O error, dev sda, sector 2147940352
[ 128.809892] print_req_error: I/O error, dev sda, sector 2147940592
[ 128.812013] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 83886080 size 8388608 starting block 268492800)
[ 128.822685] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 83886080 size 8388608 starting block 268493056)
[ 128.824078] JBD2: Detected IO errors while flushing file data on sda2-8
[ 128.824114] Aborting journal on device sda2-8.
[ 128.824299] Buffer I/O error on dev sda2, logical block 134250496, lost sync page write
[ 128.824314] JBD2: Error -5 detected when updating journal superblock for sda2-8.
[ 128.826029] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 2101248 starting block 268493312)
[ 128.829484] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 3149824 starting block 268493568)
[ 128.832403] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 4198400 starting block 268493824)
[ 128.835197] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 5246976 starting block 268494080)
[ 128.838160] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 6295552 starting block 268494336)
[ 128.841046] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 7344128 starting block 268494592)
[ 128.843859] EXT4-fs error (device sda2) in ext4_do_update_inode:5371: Journal has aborted
[ 128.843977] Buffer I/O error on dev sda2, logical block 0, lost sync page write
[ 128.844010] EXT4-fs warning (device sda2): ext4_end_bio:323: I/O error 10 writing to inode 13631490 (offset 92274688 size 8388608 starting block 268494848)
[ 128.845111] EXT4-fs (sda2): I/O error while writing superblock
[ 128.845125] EXT4-fs error (device sda2): mpage_map_and_submit_extent:2587: comm kworker/u8:5: Failed to mark inode 13631490 dirty
[ 128.845191] Buffer I/O error on dev sda2, logical block 0, lost sync page write
[ 128.845221] EXT4-fs (sda2): I/O error while writing superblock
[ 128.845242] EXT4-fs error (device sda2) in ext4_writepages:2897: IO failure
[ 128.845305] Buffer I/O error on dev sda2, logical block 0, lost sync page write
[ 128.845333] EXT4-fs (sda2): I/O error while writing superblock
[ 128.847108] JBD2: Detected IO errors while flushing file data on sda2-8
[ 128.848603] Buffer I/O error on dev sda2, logical block 0, lost sync page write
[ 128.848641] EXT4-fs (sda2): I/O error while writing superblock
[ 128.848650] EXT4-fs error (device sda2): ext4_journal_check_start:61: Detected aborted journal
[ 128.848675] EXT4-fs (sda2): Remounting filesystem read-only
[ 128.848743] Buffer I/O error on dev sda2, logical block 0, lost sync page write
[ 128.848765] EXT4-fs (sda2): I/O error while writing superblock
[ 128.986890] Buffer I/O error on dev sda3, logical block 97550336, lost sync page write
[ 128.986901] JBD2: Error -5 detected when updating journal superblock for sda3-8.
[ 128.986905] Aborting journal on device sda3-8.
[ 128.986914] Buffer I/O error on dev sda3, logical block 97550336, lost sync page write
[ 128.986919] JBD2: Error -5 detected when updating journal superblock for sda3-8.
[ 128.990918] Buffer I/O error on dev sda3, logical block 0, lost sync page write
[ 128.990931] EXT4-fs (sda3): I/O error while writing superblock

Output of lsblk -fa:

sda
├─sda1
│ ext4 1tb_tm_vol
│ 10b12fb6-e691-4276-a8bf-1665ea136b9a 955.7G 0% /mnt/tm
├─sda2
│ ext4 1tb_plex_vol
│ 67f5d5a5-f34e-44c2-9788-b3a08e2e5cb8 955G 0% /mnt/plex
└─sda3
ext4 760gb_nas_vol
345202d2-d4a7-4901-bebd-5de50d5bad27 696.4G 0% /mnt/nas

Output of fsck:

fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

          24 inodes used (0.00%, out of 67108864)
           0 non-contiguous files (0.0%)
           0 non-contiguous directories (0.0%)
             # of inodes with ind/dind/tind blocks: 0/0/0
             Extent depth histogram: 15/1
     4664356 blocks used (1.74%, out of 268435456)
           0 bad blocks
           1 large file

           3 regular files
          12 directories
           0 character device files
           0 block device files
           0 fifos
           0 links
           0 symbolic links (0 fast symbolic links)
           0 sockets
------------
          15 files
 New contributor
  • 1
    when it fails, can you see anything regarding the HDD in the output of dmesg or in any of the logs in /var/log? by the way “increasing USB power” would make zero difference, since the enclosure is self powered (according to you) – also a large file … how many bytes is large? – Jaromanda X 7 hours ago
  • 1
    @JaromandaX output added to question looks pretty sus for a new HDD – Matsumura_Fishworks 7 hours ago
  • 1
    ~1GB mp4 but it never makes it past 700M and usually around 300. – Matsumura_Fishworks 6 hours ago
  • 1
    If it were me, I would be inclined to try another USB drive. That will help you isolate the problem; i.e. is it RPi, or is it USB drive/HDD. – Seamus 6 hours ago
  • I guess the root cause is what you said: “Error splicing file: Read-only file system”. I have a couple of 500GB, 1TB SSD and 2T HDD in hand. I can try to reproduce your situation. Please give me the weblink of your ~1GB file, so that I can download and try it. Cheers. PS – My SSD/HDD is described below: raspberrypi.stackexchange.com/questions/103784/…. – tlfong01 6 hours ago    
  • 1
    @tlfong01 – do you think the content of the file is important? it’s easy to create a 1GB file you know – Jaromanda X 5 hours ago
  • @Jaromanda X, I didn’t catch you. Do you mean I can easily create a large file to do the experiment? Actually I have more than 100 large mp4 files in my SSD. Of course I can also try eg, the 2GB+ Rpi image file as the standard file to compare and contrast. But I do think the “bench mark” file must be the “same”, ie contents sometimes matter. I agree size is more important, ie the root cause, and contents is irrelevant. The usual case is that I tried my files OK, the OP might say, OK your file is OK, why my files not OK. I gave my config just to point later that my HDD is OK, not guarantee, … – tlfong01 5 hours ago    
  • @tlfong01 – if the question stated a problem with one particular file, then the content may indeed matter, but considering the question states “a large file” I assumed the file itself is not the issue, rather, it’s the size of the file. Also, the error log suggests the drive is failing in some way – Jaromanda X 5 hours ago
  • Actually there are always “things that we don’t know that we don’t know”. So I try to reproduce the situation as closely as possible. And also test as comprehensive as possible. Eg, I usually have a USB stick size voltage and current meter to check if PSU, RAM etc are getting too hot after 700MB and 10 minutes: raspberrypi.stackexchange.com/questions/100353/…. – tlfong01 5 hours ago   
  • And I actually have in hand two 2TB HDD, three 500GB/1TB SSD and four Rpi4Bs, 2 x 1GB, 2 x 2GB RAM to do swap test, as suggested by @Seamus. – tlfong01 5 hours ago    
  • @Jaromanda, Ah, I did not read the error log and therefore guess the HDD is “failing in some way”. I need to look at that later, if my HHD OK, and the OP’s HDD not OK. – tlfong01 5 hours ago   
  • How is the drive formatted? Did you try to repair the file system? – Milliways 4 hours ago
  • @Milliways added lsblk output – Matsumura_Fishworks 4 hours ago
  • @Milliways the same behavior happens on all volumes. Adding fsck output. – Matsumura_Fishworks 4 hours ago
  • @tlfong01 – four Rpi4B — too bad the OP has a rpi3b — no comparison possible – Jaromanda X 39 mins ago
  • I do have four Rpi3B+ and another four Rpi3B to compare. But I think the OS, buster or stretch etc, is more important. I vaguely remember that os like wheeze cannot recognize NTFS but only exFAT, and I need to install driver to recognize NTFS. Other thing include disk size. I only remember eFAT cannot handle big disks, but I forgot the difference between exFAT and NTFS. I am doing the partioning on my 5 year old host machine, evil WinTel PC Core i5, 6GB RAM. – tlfong01 31 mins ago    
  • I just installed buster release 2020feb05. I am experiencing some teething problems. Anyway I will try drag and drop some >1GB files to repeat the OP’s problem.Just now I physically move my newly formatted SamSung T5 SSD from my PC to Rpi4B and as usual, Rpi4B recognizes it without any problem, but teething problems include: (1) The Win10 assigned drive numbers disappeared, and I don’t know how to use CLI commands to rename the drives. (2) buster now uato mounts the SSD. Usually only HDD need to mount and unmount, USB SSD is transparent to buster just like an ordinary USB 16GB stick. – tlfong01 26 mins ago   
  • The buster release 2020feb05 has a major upgrade to the file manger, so I am getting lost now and then, here and there. But I did the following test OK. (1) drag and drop one 2.5GB file (buster image .zip) and 7.5GB buster unzipped image file (.img) from one 100GB partition to another 100GB partition. Everything went smoothly as I usually did it. I wonder the OP can use Win10 to test out disk formatting errors etc. One other possibility is that the OP might be crossing a partition boundary, ie his big files spans across two neighbouring partitions. – tlfong01 19 mins ago   
  • Just to correct some typos above. (1) I mean new buster auto mounts SSD, Other things I forget is how to do fschk (in Rpi, not in Win10 SuperShell), I have not used for a long while, and I tired it just now but found no log fschk -F -f file in the … log directory. I guess new os releases change too fast than I can catch up. (2) I do remember once I NTFS formatted a Segate 2T HDD but found it not recognized by the OS I forgot which. So I annoyly reformat it from NTFS to exFAT and it worked again. In short too many version inconsistencies wastes my time:( – tlfong01 10 mins ago   
  • PS – I also forgot what is ext4. I also forgot the difference between volume and drive. I now assume a hard disk can be partitioned into many volumes, each of which has an unique drive name. – tlfong01 48 secs ago   Edit   
  • It is only a couple of days ago I forgot the SD card after formatting has only one volume/drive, but after flashing with buster image becomes two drives (boot and root something) I did not pay too much attention, the result is that I lost count the drive number and FLASHED MY 500GB SSD, mistakenly thing that I was flashing the 16GB sd card. The result is that I saw the infamous TWINKLE TWINKLE TWINKLE TWINKLE littler green star. – tlfong01 33 secs ago   Edit   
  • It is only then I symthesize why some of the sd card format/flash newbies see the blinking green LED and swear half of the Galaxt. Need to google and wiki, but will forget and confuse very soon:) PS – Just thinking aloud, sorry for all the typos. – tlfong01 27 secs ago    Edit   

Categories: Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: