isfs6_20110113_000000.dat is corrupt for some reason.
When one does an rsync, the following are reported on isfs6:/var/log/isfs/messages
Jan 15 23:15:55 isfs6 rsyncd[854]: rsync: read errors mapping "PCAPS/raw_data/isfs6_20110113_000000.dat" (in data): Input/output error (5)
The rsync also results in a bunch of these errors in /var/log/isfs/kernel:
Jan 15 23:12:15 isfs6 kernel: attempt to access beyond end of device Jan 15 23:12:15 isfs6 kernel: sda1: rw=0, want=78103088, limit=15825499
So that it doesn't get in the way of successive rsyncs, I moved the file to isfs6:/media/usbdisk/projects/PCAPS/bad
data_stats on the rsync'd file reports many bad samples, with incorrect types, ids, length, times, etc.
I was able to clear our most of the junk with these commands on the file at EOL:
sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6.dat isfs6_20110113_000000.dat nidsmerge -i isfs6.dat -o isfs6x.dat -s "2011 01 13 00:00" -e "2011 01 13 12:00" sensor_extract -s 6,30 -s 6,100 -s 6,20 -s 6,50 -s 6,0x8000 -s 6,40 -s 6,80 -l 0 isfs6xx.dat isfs6x.dat bzip2 /scr/tmp/maclean/isfs6xx.dat -c > isfs6_20110113_000000.dat.bz2
The real-time file for isfs6 for that day looks quite complete, so I don't think we have lost much, if anything.
We'll see if we have any other problems on the usb flash drive at wslope6. I plan to leave that file where it is in case it is covering a bad address on the flash, to avoid writing to that address again.
The glitch doesn't seem to have anything to do with a sudden power down, wslope6 has been up for 23 days.