Read-only file system

Speedster

Speedster

Honorary Master
Joined
May 2, 2006
Messages
32,032
Reaction score
21,384
Location
Hatfield
Not quite sure where to post this, so I hope this will do :)

I've got a linux box hosting a file server on a 1TB internal at our small office. Recently the HDD has begun to spontaneously become read-only. For the life of me I can't figure out why that would be. A reboot fixes the issue temporarily but, after a while, we're back to read-only.

Anybody have any idea what it might be?
 
Run 'ls -l' to check for the file permissions in the folder that is having issues.

Also run like 'ps faux' to see under what user the process is running that is trying to modify the files. The user should be in the first column.
 
Check the system logs, but I suspect that your disk may be failing.

Typically they are mounted with the "errors=remount-ro" option, which, from the docs does "Remount the filesystem read-only on an error."
 
Pada said:
Run 'ls -l' to check for the file permissions in the folder that is having issues.

Also run like 'ps faux' to see under what user the process is running that is trying to modify the files. The user should be in the first column.
Click to expand...

ls -l gives:
drwxrwxrwx 30 root root 1088 Apr 16 16:34

I've rebooted the server after everyone else left so ps faux only shows me as the user.

RoganDawes said:
Check the system logs, but I suspect that your disk may be failing.

Typically they are mounted with the "errors=remount-ro" option, which, from the docs does "Remount the filesystem read-only on an error."
Click to expand...

I have a feeling you may be right, which would really suck. Anyway, how would I go about confirming this to be the case?
 
Could some of the clever people around here help interpret this smartctl output?

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 091 090 006 Pre-fail Always - 95293757
3 Spin_Up_Time 0x0003 096 095 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 168
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 066 060 030 Pre-fail Always - 4919869
9 Power_On_Hours 0x0032 070 070 000 Old_age Always - 27134
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 111
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 414
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 4295032833
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 058 052 045 Old_age Always - 42 (Min/Max 35/46)
194 Temperature_Celsius 0x0022 042 048 000 Old_age Always - 42 (0 15 0 0)
195 Hardware_ECC_Recovered 0x001a 044 015 000 Old_age Always - 95293757
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 1
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 1
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 246599842283824
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 3863647291
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 763889780

SMART Error Log Version: 1
ATA Error Count: 414 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 414 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:58.503 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:58.502 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:58.494 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:58.487 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:58.462 READ NATIVE MAX ADDRESS EXT

Error 413 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:55.355 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:55.354 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:55.346 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:55.340 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:55.314 READ NATIVE MAX ADDRESS EXT

Error 412 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:52.198 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:52.197 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:52.189 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:52.184 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:52.158 READ NATIVE MAX ADDRESS EXT

Error 411 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:49.110 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:49.109 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:49.101 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:49.096 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:49.069 READ NATIVE MAX ADDRESS EXT

Error 410 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:45.954 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:45.953 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:45.945 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:45.940 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:45.921 READ NATIVE MAX ADDRESS EXT

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Self-test routine in progress 90% 27134 -
# 2 Extended offline Completed: read failure 90% 27134 1690828883
# 3 Extended offline Completed: read failure 90% 27133 1690828883
# 4 Extended offline Completed: read failure 90% 27133 1690828883
# 5 Short offline Completed: read failure 90% 27133 1690828883
Click to expand...
 
Check bios - randomly toggled the write protect on an old box of mine. Freaked me out because the OS doesn't even boot without write.

tbh haven't seen something similar in recent BIOSs though...
 
ponder said:
Check the S.M.A.R.T. status.
Click to expand...
The above is the SMART data. There is no such thing as a SMART status - thats determined by the program that you're using to view the data & each will have its own idea about what constitutes an OK status.
 
HavocXphere said:
The above is the SMART data. There is no such thing as a SMART status - thats determined by the program that you're using to view the data & each will have its own idea about what constitutes an OK status.
Click to expand...

Sorry I missed that post. The rest is kuk though, the programs don't change the values reported by SMART, they are what they are.
 
It seems like the long test is running now after I dealt with the initial error position. I'll post the results of that in the morning.
 
SMART parameter 5 RAW non-zero value usually indicate bad sectors, you had one UNC LBA location. Also parameter 187 is suspicious to me.

Stop running SMART tests. It is not for the drive with such condition. Zero-fill entire surface (or secure erase). After that scan for delays. Of course, backup first.
 
When I looked at the SMART results this morning there had been another read failure during the long test last night. I did the dd fix thing on that LBA and am running the test again.

How will I know if I should replace the drive?
 
Finally completed a long SMART scan with no errors. Here are the results. Anything in there I should be concerned about?

== START OF INFORMATION SECTION ===
Model Family: Seagate Barracuda 7200.12
Device Model: ST31000528AS
Firmware Version: CC38
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: 8
ATA Standard is: ATA-8-ACS revision 4
Local Time is: Wed Apr 17 11:42:49 2013 SAST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: ( 609) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 1) minutes.
Extended self-test routine
recommended polling time: ( 184) minutes.
Conveyance self-test routine
recommended polling time: ( 2) minutes.
SCT capabilities: (0x103f) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x000f 091 090 006 Pre-fail Always - 99910463
3 Spin_Up_Time 0x0003 096 095 000 Pre-fail Always - 0
4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 168
5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0
7 Seek_Error_Rate 0x000f 066 060 030 Pre-fail Always - 4932816
9 Power_On_Hours 0x0032 070 070 000 Old_age Always - 27150
10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0
12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 111
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0
187 Reported_Uncorrect 0x0032 001 001 000 Old_age Always - 414
188 Command_Timeout 0x0032 100 100 000 Old_age Always - 4295032833
189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0
190 Airflow_Temperature_Cel 0x0022 057 052 045 Old_age Always - 43 (Min/Max 35/46)
194 Temperature_Celsius 0x0022 043 048 000 Old_age Always - 43 (0 15 0 0)
195 Hardware_ECC_Recovered 0x001a 030 015 000 Old_age Always - 99910463
197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0
240 Head_Flying_Hours 0x0000 100 253 000 Old_age Offline - 247192547770688
241 Total_LBAs_Written 0x0000 100 253 000 Old_age Offline - 3866884307
242 Total_LBAs_Read 0x0000 100 253 000 Old_age Offline - 3258271410

SMART Error Log Version: 1
ATA Error Count: 414 (device log contains only the most recent five errors)
CR = Command Register [HEX]
FR = Features Register [HEX]
SC = Sector Count Register [HEX]
SN = Sector Number Register [HEX]
CL = Cylinder Low Register [HEX]
CH = Cylinder High Register [HEX]
DH = Device/Head Register [HEX]
DC = Device Command Register [HEX]
ER = Error register [HEX]
ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 414 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:58.503 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:58.502 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:58.494 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:58.487 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:58.462 READ NATIVE MAX ADDRESS EXT

Error 413 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:55.355 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:55.354 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:55.346 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:55.340 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:55.314 READ NATIVE MAX ADDRESS EXT

Error 412 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:52.198 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:52.197 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:52.189 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:52.184 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:52.158 READ NATIVE MAX ADDRESS EXT

Error 411 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:49.110 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:49.109 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:49.101 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:49.096 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:49.069 READ NATIVE MAX ADDRESS EXT

Error 410 occurred at disk power-on lifetime: 27131 hours (1130 days + 11 hours)
When the command that caused the error occurred, the device was active or idle.

After command completion occurred, registers were:
ER ST SC SN CL CH DH
-- -- -- -- -- -- --
40 51 00 ff ff ff 0f Error: UNC at LBA = 0x0fffffff = 268435455

Commands leading to the command that caused the error were:
CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
-- -- -- -- -- -- -- -- ---------------- --------------------
25 00 08 ff ff ff ef 00 6d+01:59:45.954 READ DMA EXT
27 00 00 00 00 00 e0 00 6d+01:59:45.953 READ NATIVE MAX ADDRESS EXT
ec 00 00 00 00 00 a0 00 6d+01:59:45.945 IDENTIFY DEVICE
ef 03 46 00 00 00 a0 00 6d+01:59:45.940 SET FEATURES [Set transfer mode]
27 00 00 00 00 00 e0 00 6d+01:59:45.921 READ NATIVE MAX ADDRESS EXT

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed without error 00% 27150 -
# 2 Extended offline Completed: read failure 10% 27136 1771987418
# 3 Extended offline Completed: read failure 90% 27134 1690828883
# 4 Extended offline Completed: read failure 90% 27133 1690828883
# 5 Extended offline Completed: read failure 90% 27133 1690828883
# 6 Short offline Completed: read failure 90% 27133 1690828883
5 of 5 failed self-tests are outdated by newer successful extended offline self-test # 1
Click to expand...
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X