BBB eMMC physical failure?

Hi,

One of my BBB has been having problems with its filesystem for a while now and I think it originates from a physical problem on the eMMC but badblocks didn’t find any error. It’s running on Linux kernel 4.2.0-1-ARCH (Arch Linux) which is installed on the eMMC (no µSD card).

The behavior is that after a while (generally a few days) the journal fails and the FS is remounted as read-only. The latest event’s dmesg output is shown below. I starts with a “mmcqd/1: page allocation failure: order:3, mode:0xc020”, follows with an aborted journal and the FS being remounted RO.

I ran fsck.ext4 manually with the system unmounted (booted with a µSD card) and fixed several errors each time. As this event kept appearing afterwards I concluded that the must be something wrong with the eMMC. I then backed-up the partition (dd) and ran fsck with non-destructive badblocks (https://wiki.archlinux.org/index.php/Badblocks#read-write_Test_.28non-destructive.29)) but no error was found apart from filesystem issues which fsck fixed (no bad sector).

Am I right in assuming that this is most likely a physical problem with the eMMC? Is there a more appropriate test I should run to test the eMMC?

Thanks,

[274268.642991] mmcqd/1: page allocation failure: order:3, mode:0xc020 [274268.649397] CPU: 0 PID: 92 Comm: mmcqd/1 Not tainted 4.2.0-1-ARCH #1 [274268.655895] Hardware name: Generic AM33XX (Flattened Device Tree) [274268.662150] [<c001704c>] (unwind_backtrace) from [<c0014224>] (show_stack+0x20/0x24) [274268.670047] [<c0014224>] (show_stack) from [<c08b4f58>] (dump_stack+0x24/0x28) [274268.677415] [<c08b4f58>] (dump_stack) from [<c01221f8>] (warn_alloc_failed+0xec/0x128) [274268.685478] [<c01221f8>] (warn_alloc_failed) from [<c0125304>] (__alloc_pages_nodemask+0x778/0x8f4) [274268.694667] [<c0125304>] (__alloc_pages_nodemask) from [<c01256c4>] (alloc_kmem_pages+0x40/0x110) [274268.703667] [<c01256c4>] (alloc_kmem_pages) from [<c013e4e0>] (kmalloc_order+0x20/0x38) [274268.711809] [<c013e4e0>] (kmalloc_order) from [<c013e524>] (kmalloc_order_trace+0x2c/0x94) [274268.720215] [<c013e524>] (kmalloc_order_trace) from [<c016454c>] (__kmalloc+0x1e8/0x24c) [274268.728450] [<c016454c>] (__kmalloc) from [<c04febf0>] (edma_prep_slave_sg+0xa4/0x2fc) [274268.736515] [<c04febf0>] (edma_prep_slave_sg) from [<c0717258>] (omap_hsmmc_request+0x430/0x528) [274268.745448] [<c0717258>] (omap_hsmmc_request) from [<c06f9f4c>] (__mmc_start_request+0x3c/0x58) [274268.754288] [<c06f9f4c>] (__mmc_start_request) from [<c06fa0c8>] (mmc_start_request+0x160/0x298) [274268.763201] [<c06fa0c8>] (mmc_start_request) from [<c06fae3c>] (mmc_start_req+0x2a4/0x3d8) [274268.771611] [<c06fae3c>] (mmc_start_req) from [<c070b568>] (mmc_blk_issue_rw_rq+0xb8/0xaac) [274268.780100] [<c070b568>] (mmc_blk_issue_rw_rq) from [<c070c1b8>] (mmc_blk_issue_rq+0x25c/0x4ec) [274268.788943] [<c070c1b8>] (mmc_blk_issue_rq) from [<c070cbf0>] (mmc_queue_thread+0xa8/0x170) [274268.797470] [<c070cbf0>] (mmc_queue_thread) from [<c00659b8>] (kthread+0xe4/0xfc) [274268.805097] [<c00659b8>] (kthread) from [<c0010588>] (ret_from_fork+0x14/0x2c) [274268.812435] Mem-Info: [274268.814827] active_anon:4121 inactive_anon:22 isolated_anon:0 active_file:34640 inactive_file:71649 isolated_file:0 unevictable:4153 dirty:31 writeback:0 unstable:0 slab_reclaimable:5820 slab_unreclaimable:1640 mapped:7397 shmem:146 pagetables:234 bounce:0 free:1223 free_pcp:176 free_cma:1 [274268.848553] Normal free:4892kB min:2836kB low:3544kB high:4252kB active_anon:16484kB inactive_anon:88kB active_file:138560kB inactive_file:286596kB unevictable:16612kB isolated(anon):0kB isolated(file):0kB present:524288kB managed:504308kB mlocked:16612kB dirty:124kB writeback:0kB mapped:29588kB shmem:584kB slab_reclaimable:23280kB slab_unreclaimable:6560kB kernel_stack:712kB pagetables:936kB unstable:0kB bounce:0kB free_pcp:704kB local_pcp:704kB free_cma:4kB writeback_tmp:0kB pages_scanned:68 all_unreclaimable? no [274268.894477] lowmem_reserve[]: 0 0 0 [274268.898089] Normal: 813*4kB (UEMC) 203*8kB (UM) 1*16kB (M) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 4892kB [274268.911028] 108200 total pagecache pages [274268.915069] 0 pages in swap cache [274268.918482] Swap cache stats: add 0, delete 0, find 0/0 [274268.923810] Free swap = 0kB [274268.926796] Total swap = 0kB [274268.929771] 131072 pages RAM [274268.932744] 0 pages HighMem/MovableOnly [274268.936691] 4294966147 pages reserved [274268.940451] 6144 pages cma reserved [274268.944061] edma-dma-engine edma-dma-engine.0: edma_prep_slave_sg: Failed to allocate a descriptor [274268.953147] omap_hsmmc 481d8000.mmc: prep_slave_sg() failed [274268.958842] omap_hsmmc 481d8000.mmc: MMC start dma failure [274268.968506] mmcblk0: unknown error -1 sending read/write command, card status 0x900 [274268.976451] blk_update_request: I/O error, dev mmcblk0, sector 1630840 [...] [274269.037533] blk_update_request: I/O error, dev mmcblk0, sector 1630912 [274269.072885] Aborting journal on device mmcblk0p1-8. [274269.085261] EXT4-fs error (device mmcblk0p1): ext4_journal_check_start:56: Detected aborted journal [274269.094508] EXT4-fs (mmcblk0p1): Remounting filesystem read-only [274269.101813] EXT4-fs error (device mmcblk0p1): ext4_journal_check_start:56: Detected aborted journal [274269.113530] systemd-journald[138]: Failed to create new system journal: Read-only file system [274269.133427] systemd-journald[138]: Failed to rotate /var/log/journal/8a327183d5764238aefe1f3086448cb9/user-1000.journal: Read-only file system [360758.287562] EXT4-fs (mmcblk0p1): error count since last fsck: 3 [360758.293621] EXT4-fs (mmcblk0p1): initial error at time 1444708807: ext4_journal_check_start:56 [360758.302360] EXT4-fs (mmcblk0p1): last error at time 1444708807: ext4_journal_check_start:56

This thread seems to indicate that a physical problem within the eMMC is quite unlikely and that badblocks should’ve been marked automatically: https://groups.google.com/forum/embed/?place=forum/beagleboard&showsearch=true&showpopout=true&showtabs=true&hideforumtitle=true&parenturl=http%3A%2F%2Fbeagleboard.org%2FCommunity%2FForums#!searchin/beagleboard/emmc/beagleboard/6dJHbqgE7-c/uTSkbqCCJgAJ

Should I format/reinstall?

Thanks.

Michel,

In your place, I would probably make an entire backup of the eMMC. Then reinstall to double check it’s not something that accidentally got messed up by . I’ve had this happen to me once, or twice, and I could not tell you what went wrong. Also, for the record, I’m usually very meticulous when it comes to taking notes - So I have exact steps to reproduce things I’ve already done. But I think in both cases, the problem cleared up. The difference for me is that I develop my images using an sdcard, and not the eMMC.

Ok thanks I’ll try that. I would’ve liked to know whether it’s a physical or software failure before reinstalling but it won’t take very long, it’s mostly a file server. Perhaps there were too many power interruptions for its taste.

Hi Michel,

Were you able to find out if that was eMMC issue(or eMMC interface issue?) or software issue ?
Because me too getting similar error(mentioned below) in our BBB based custom board

mmcqd/1: page allocation failure: order:3, mode:0x200020

Thank you,

Regards,
Ankur

No sorry I wasn’t. I formated the eMMC and installed Debian and haven’t had this problem since.