PTB210 at s1 was working up until 2020-04-12,15:23:03, except of course for periods when s1 was probably dead. There are a few messages after that, until 202-04-16,01:29:11, and then they stop. As of the swap of the s9 DSM, nothing has been received from the PTB.
The PTB was set for 7E1, so I've tried connecting to it on s9 with minicom in 7E1, but there is still no response. Maybe a blown fuse, or maybe the PTB itself is dead. The data messages received in 7E1 mode are recoverable with some code changes to NIDAS.
2020 04 12 15:22:59.8795 0.7426 1, 22 10 \xb1\xb1\xb1\xb2.\xb26\x8d\n 2020 04 12 15:23:00.5341 0.6546 1, 22 10 \xb1\xb1\xb1\xb2.\xb26\x8d\n 2020 04 12 15:23:01.3770 0.8429 1, 22 10 \xb1\xb1\xb1\xb2.\xb26\x8d\n 2020 04 12 15:23:02.0969 0.7199 1, 22 10 \xb1\xb1\xb1\xb2.\xb2\xb7\x8d\n 2020 04 12 15:23:02.7773 0.6803 1, 22 10 \xb1\xb1\xb1\xb2.\xb2\xb7\x8d\n 2020 04 12 15:23:03.6169 0.8396 1, 22 10 \xb1\xb1\xb1\xb2.\xb2\xb7\x8d\n
2020-04-29,10:42:57|INFO|opening: s1_20200415_144143.dat 2020 04 15 14:41:44.7493 2.567e+05 1, 22 10 \xb1\xb1\xb13.0\xb2\x8d\n 2020-04-29,10:42:57|INFO|opening: s1_20200415_144322.dat
2020-04-29,10:43:03|INFO|opening: s1_20200416_012833.dat 2020 04 16 01:29:10.0350 3.885e+04 1, 22 10 \xb1\xb109.5\xb2\x8d\n 2020 04 16 01:29:10.5671 0.5322 1, 22 10 \xb1\xb109.5\xb2\x8d\n 2020 04 16 01:29:11.2509 0.6837 1, 22 10 \xb1\xb109.53\x8d\n 2020-04-29,10:43:03|INFO|opening: s1_20200416_012957.dat 2020-04-29,10:43:03|INFO|opening: s1_20200416_013538.dat 2020-04-29,10:43:03|INFO|opening: s1_20200416_013838.dat 2020-04-29,10:43:03|INFO|opening: s1_20200416_120000.dat
When Leila swapped the s9 DSM for s1, I discovered the disdrometer messages were broken. The quick summary is that the eeprom got erased, leading to these questions. The details follow.
- Can we take up this problem with Ott Hydromet?
- Any clues as to what could be causing the Parsivel2 to lose it's memory?
- Is voltage or current supply borderline for reliable operation?
It looks like the first data file at setup for s1 is s1_20200325_120000.dat. Disdrometer was working as of 2020-03-25,17:49:13. I'm guessing the site was setup that day, and for whatever reason the disdrometer data messages start there, without the boot messages. Probably the time was not synchronized until then.
[isfs@barolo raw_data]$ data_stats -a -i 1,-1 s1_20200325_120000.dat 2020-04-29,09:39:54|NOTICE|parsing: /h/eol/isfs/isfs/projects/SWEX/ISFS/config/swex.xml Exception: EOFException: s1_20200325_120000.dat: open: EOF sensor dsm sampid nsamps |------- start -------| |------ end -----| rate minMaxDT(sec) minMaxLen s1:/dev/ttyDSM1 1 8 371 2020 03 25 17:49:13.254 03 25 23:59:10.090 0.02 57.007 60.021 4142 4142 s1:/dev/gps_pty0 1 10 44573 2020 03 25 17:48:16.892 03 25 23:59:59.024 2.00 -2.133 1.253 59 147 s1:/var/log/chrony/tracking.log 1 18 2654 2020 03 25 17:48:16.958 03 25 23:59:59.913 0.12 0.000 2578.788 100 100 s1:/dev/ttyDSM2 1 20 22436 2010 02 01 00:00:57.185 03 25 23:59:59.275 0.00-1946506.446 1.342 42 44 s1:/dev/ttyDSM3 1 22 0 *********************** ****************** nan nan nan nan nan s1:/dev/ttyDSM4 1 40 446122 2010 02 01 00:00:57.159 03 25 23:59:59.974 0.00-1946507.325 0.493 60 60 s1:/dev/ttyPWRMONV 1 60 445799 2010 02 01 00:00:56.308 03 25 23:59:59.422 0.00-1946506.527 1.172 3 90 s1:/dev/ttyDSM5 1 100 223047 2020 03 25 17:48:17.032 03 25 23:59:59.976 10.00 -2.889 0.428 32 32 s1:/dev/ttyDSM7 1 0x8000 4467 2020 03 25 17:48:20.498 03 25 23:59:57.411 0.20 1.616 5.689 26 60
The messages appeared to be fine, including reporting the serial number 450620:
2020 03 25 17:49:13.2549 0 1, 8 4142 450620;0000.000;00;20000;024;27025;00000;0;000;...
It booted up again 2020-03-26,01:35:11, for reasons unknown. It reported one good message, then started rebooting and reporting "POWERSUPPLY TEST FAILED !!!".
2020-04-29,09:13:52|INFO|opening: s1_20200326_013511.dat 2020 03 26 01:35:11.6336 121.6 1, 8 3 \r\n 2020 03 26 01:35:11.6346 0.001042 1, 8 22 BOOTLOADER PARSIVEL\r\n 2020 03 26 01:35:12.9390 1.304 1, 8 3 \r\n 2020 03 26 01:35:12.9400 0.001042 1, 8 21 *** PARSIVEL 2 ***\r\n 2020 03 26 01:35:12.9518 0.01182 1, 8 20 OTT HYDROMET GMBH\r\n 2020 03 26 01:35:12.9631 0.01129 1, 8 22 COPYRIGHT (C) 2019 \r\n 2020 03 26 01:35:12.9755 0.01238 1, 8 18 VERSION: 2.11.6\r\n 2020 03 26 01:35:12.9843 0.008857 1, 8 17 BUILD: 2112151\r\n 2020 03 26 01:35:13.0045 0.02015 1, 8 3 \r\n 2020 03 26 01:35:13.3090 0.3045 1, 9 40 0 0 20000 15 0 0 3 0 0 0 2020 03 26 01:35:13.3090 0 1, 8 4142 450620;0000.000;00;.....;\r\n 2020 03 26 01:35:28.0125 14.7 1, 8 4 \x80\r\n 2020 03 26 01:35:29.6423 1.63 1, 8 22 BOOTLOADER PARSIVEL\r\n 2020 03 26 01:35:30.9507 1.308 1, 8 3 \r\n 2020 03 26 01:35:30.9517 0.001042 1, 8 21 *** PARSIVEL 2 ***\r\n 2020 03 26 01:35:30.9635 0.01181 1, 8 20 OTT HYDROMET GMBH\r\n 2020 03 26 01:35:30.9748 0.01131 1, 8 22 COPYRIGHT (C) 2019 \r\n 2020 03 26 01:35:30.9858 0.01094 1, 8 18 VERSION: 2.11.6\r\n 2020 03 26 01:35:30.9961 0.01028 1, 8 17 BUILD: 2112151\r\n 2020 03 26 01:35:31.0044 0.008336 1, 8 3 \r\n 2020 03 26 01:35:35.0523 4.048 1, 8 31 \xf8POWERSUPPLY TEST FAILED !!!\r\n
Eventually it starts repeating the messages "ERROR: No Valid Serial Number found !!!" and "ERROR: No Valid Hardware info found !!!".
2020 03 26 01:38:57.7429 0.001042 1, 8 22 BOOTLOADER PARSIVEL\r\n 2020 03 26 01:38:59.0512 1.308 1, 8 3 \r\n 2020 03 26 01:38:59.0522 0.001042 1, 8 21 *** PARSIVEL 2 ***\r\n 2020 03 26 01:38:59.0641 0.01184 1, 8 20 OTT HYDROMET GMBH\r\n 2020 03 26 01:38:59.0740 0.009899 1, 8 22 COPYRIGHT (C) 2019 \r\n 2020 03 26 01:38:59.0879 0.01393 1, 8 40 VERSION: \xf8POWERSUPPLY TEST FAILED !!!\r\n 2020-04-29,09:13:52|INFO|opening: s1_20200326_120000.dat 2020 03 26 19:49:41.7501 6.544e+04 1, 8 42 ERROR: No Valid Serial Number found !!!\r\n 2020 03 26 19:49:41.7597 0.009589 1, 8 43 ERROR: No Valid Hardware info found !!!\r\r\n 2020 03 26 19:49:58.6922 16.93 1, 8 22 BOOTLOADER PARSIVEL\r\n 2020 03 26 19:50:00.0337 1.341 1, 8 42 ERROR: No Valid Serial Number found !!!\r\n 2020 03 26 19:50:00.0577 0.02407 1, 8 43 ERROR: No Valid Hardware info found !!!\r\r\n
It keeps reporting the "No Valid Hardware info" messages until 2020-03-27,01:50, then some noise, then nothing until 20:13, when it starts reporting the default messages with the serial number of XXXXXXXX:
2020 03 27 01:50:38.3070 0.001042 1, 8 22 BOOTLOADER PARSIVEL\r\n 2020 03 27 01:50:39.0419 0.7349 1, 8 42 \x00\xe0\x00\x00\x00\x00\x00\xff\x00\x00\xfe\x00POWERSUPPLY TEST FAILED !!!\r\n 2020 03 27 01:50:42.5119 3.47 1, 8 33 \x00\x00\x00POWERSUPPLY TEST FAILED !!!\r\n 2020 03 27 01:50:44.8753 2.363 1, 8 44 \x00\x00\x00\x00\x00\x00\x00\x0e\x00\xff\x00\xfe\x00\x00POWERSUPPLY TEST FAILED !!!\r\n 2020 03 27 01:50:47.5648 2.69 1, 8 63 \x00\x00\xff\x02\x00\xfe\x00\xff\x00\x00\x00\x00\x9d(\x80\x00\xff\x00\xff\x00\xfd\xff\x00\x00\xfe\x00\xff\x00\x00\xff\xff\x00\x00POWERSUPPLY TEST FAILED !!!\r\n 2020 03 27 01:50:51.8081 4.243 1, 8 49 \x00\x00\x00\x00\x00@\x14\x00\x00\x00\xfe\x00\x00\x00\x0e\x00\x00\xff\x00POWERSUPPLY TEST FAILED !!!\r\n 2020-04-29,09:13:56|INFO|opening: s1_20200327_120000.dat 2020 03 27 20:13:06.5367 0 1, 8 71 XXXXXXXX;0000.000;0000.00;00;-9.999;20000;0000.00;025;27028;00000;0;\r\n 2020 03 27 20:14:06.5447 0 1, 8 71 XXXXXXXX;0000.000;0000.00;00;-9.999;19320;0000.00;025;27018;00000;0;\r\n
There are still some reboot messages later on and more error messages, so it's not like the disdrometer is stabilized again but just missing eeprom. Either way it's in a broken state, and I don't think this is the only one to have had this kind of problem.
For the moment, I have modified the NIDAS config to parse the messages but skip the serial number field. However, that is not a fix since the whole configuration of the data messages has been lost, and we don't know if losing the hardware info and any other eeprom settings makes the data useless.
A week ago, Leila and Charles visited Site 1 to find the DSM inop. For simplicity sake, this past Sunday (4/26) I had them replace the S1 DSM with the unused S9 DSM. Nothing was swapped between the two, so S9 has it's original cell modem and SD card.
S9 Comms
m59
eth0- 192.168.1.209 (DSM address)
eth1- 166.255.144.36 (cell modem address)
--
Dan
Just a quick note that s8 has routinely seen winds >20 m/s, with the highest 5-min average of 27 m/s on 17 Mar.
s15 is the next windiest, up to 17 m/s, though I suspect that s17 would have been high as well if it were being recorded.
The pattern is not surprising, though the magnitudes are rather high for what ISFS normally sees.
It seems that the s8 CSAT sometimes misses these events – is this the high-wind-speed-error that Sean saw several years ago?
Still in idling mode after being pulled from set-up last month for COVID.
Sites deployed:
s1 - Barometer not reporting (configuration?). Station only comes up when batteries fully charged. (Had worked ok initially after setup.). Sometimes, files are not opened. Most of the time the time stamp is bad.
P.S. Site was visited by Leila and Charles on 18 Apr. They replaced batteries and Victron, which brought power up to DSM and sensors, but DSM still hasn't come up. This suggests either a fault or wrong setting (mode 4 instead of mode 3?) of the old Victron. Dan things the next step is a DSM change, but the DSM was swaged closed so the PIs couldn't get into the box. A DSM issue is odd, though, since it was working last week.
s3 - TRH died in rain on 6 Apr. Otherwise ok
s4 - ok
s8 - EC150 never installed. GPS not receiving most messages as of 14 Mar(!). (At that time, about 13 Mar 23:30 nsat drops from 10 to 7, then further drops to 0 about 14 Mar 02:30.) Otherwise ok
s10 - ok
s14 - Mote data never worked properly, last data on 3 Apr. Cable from DSM to mote found to have water during site visit by PIs on 11 Apr, but didn't have spare. Barometer highly intermittent (but Pirga ok). Otherwise, ok.
s15 - ok
s17 - Site pretty much never worked. Just a few hours of data early on. Last data 26 Mar. From log snapshot taken when station was last up, seemed to be a DSM USB issue.