Blog
Skip to end of metadata
Go to start of metadata

isfs6_20110113_000000.dat is corrupt for some reason.

When one does an rsync, the following are reported on isfs6:/var/log/isfs/messages

Jan 15 23:15:55 isfs6 rsyncd[854]: rsync: read errors mapping "PCAPS/raw_data/isfs6_20110113_000000.dat" (in data): Input/output error (5) 

The rsync also results in a bunch of these errors in /var/log/isfs/kernel:

Jan 15 23:12:15 isfs6 kernel: attempt to access beyond end of device
Jan 15 23:12:15 isfs6 kernel: sda1: rw=0, want=78103088, limit=15825499

So that it doesn't get in the way of successive rsyncs, I moved the file to isfs6:/media/usbdisk/projects/PCAPS/bad

data_stats on the rsync'd file reports many bad samples, with incorrect types, ids, length, times, etc.

I was able to clear our most of the junk with these commands on the file at EOL:

sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6.dat isfs6_20110113_000000.dat 
nidsmerge -i isfs6.dat -o isfs6x.dat -s "2011 01 13 00:00"  -e "2011 01 13 12:00"
sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6xx.dat isfs6x.dat
bzip2 /scr/tmp/maclean/isfs6xx.dat -c > isfs6_20110113_000000.dat.bz2 

The real-time file for isfs6 for that day looks quite complete, so I don't think we have lost much, if anything.

We'll see if we have any other problems on the usb flash drive at wslope6. I plan to leave that file where it is in case it is covering a bad address on the flash, to avoid writing to that address again.

The glitch doesn't seem to have anything to do with a sudden power down, wslope6 has been up for 23 days.