Blog from June, 2021

Lidar networking 6/30

Went to Marshall today to get the lidar on the instrument network in iss1 via a ubiquiti link.

At the lidar, installed a waterproof box w/ built in plugs that Bill found. When I got the lidar plugged in to the outlet in the box the fans in the lidar didn't turn back on, and after a few minutes I smelled burning and saw some smoke. So I don't think we should be using the outlets in that box. Instead, I added a power strip I found in MISS to the box and plugged the lidar and the ubiquiti POE injector into that. I didn't find any good spots on the neck of the trailer to mount the ubiquiti (the hose clamps they come with for mounting are very small), so mounted it on a separate pole near the neck of the trailer.

At iss1, I mounted the ubiquiti directly below the existing ubiquiti that connects to the Marshall network. I was worried about interference between the two nanobeams based on some info I had found on the internet, but it doesn't seem to be a problem. The lidar link is configured as point-to-point and I set a different SSID, so the lidar link shouldn't be able to connect to the Marshall network.


With the link working I still couldn't connect to the lidar, so I went back to troubleshoot. At one point when I was closing up the waterproof box I had bumped the reset button on the power strip, and I think that tripped something on the computer in the lidar because the lidar didn't come up again after that. The fans were working and the external power light was on, but the head wasn't moving and I couldn't ping or browse to the lidar. With some help from Bill (thanks Bill!) we found that pushing the reset and power buttons on the computer box in the lidar brought it back up. After that I verified that I could ping the lidar from iss1, as well as connect from the lidar laptop (now set up on the instrument network in iss1) and get live data.

As a side note, the UPS in the lidar doesn't seem to be working. Presumably the battery is dead.

Though now as I write this blog post I see that the lidar (and the ubiquiti at the lidar) are no longer reachable. I suspect the power strip tripped again, so I'm heading out to try replacing that with another one.



1 Comment  · 
ISFS Data Check

Reviewed the measurements on NCharts since Friday 25 June. 

  • TRH - ok
  • P - ok
  • Radiometer/Wetness - It had been raining everyday since Friday, 25 June. I’m surprised the wetness sensor remains flatlined. Shouldn’t there be spikes or a saturated level recorded? Was a disdrometer was installed? 
  • Soils
    • Gsoil - ok range -40 - 40 W/m^2 
    • Lamdasoil - ok range 0.25 - 0.325 W/m
    • Qsoil - are now positive but values are very low (< 1% vol)  for all the precipitation we’ve been getting. This is compared with Relampago and CHEESEHEAD where values are > 14 %vol
    • Tau 63 - ok range 6-10 s 
    • Tsoil - Note very high values (> 45degC on 25 June that relax to more representative values 25-28 degC max)
    • Vheat - ok 
  • Sonic - ok - w range +/- 0.5m/s, u &v range -10 - +5 m/s
  • co2/h2o
    • Crazy co2.27m values >> 1500 g/m3
    • h2o - ok-ish h2o.27m also show high-ish values > 40 g/m3


  • iload range on the high end - 1820 - 1950 mA
  • icharge range -190 to -80 mA (shouldn’t this be positive and in the thousands??)
  • Vbatt - ok
  • l3mote - without the spikes range is 50-62 mA
  • lmote - with the spikes range is 15-18 mA
  • Lamda - no values on Ncharts
  • Pirga - ok
  • Tirga - ok
  • Rfan - ok
  • Vcharge - ~27.93 V
Quick data scan

Just some data notes:

  • About 3 June, co2.27m became very large (and h2o.27m became negative) and spikey.  On 23 June, the levels of both magically became much more normal.  We didn't do anything and I don't think it even rained on 23 June.  Could be related to birds liking to perch on this top level.
  • differences in co2 levels still range over mg/m^3 which is horrible.  (P.S. units are wrong – just fixed config)
  • Soils are happier now that they are back in the dirt, though Qsoil is still negative
  • Pirgas have biases of a few mb – not unexpected.  They show the tower raising yesterday (just before losing the data connection).
Site visit today

Many tasks:

  • Met with Electrician Tom and showed him the plug we need for CLAMPS.  He should be able to get it installed before CLAMPS arrives on 12 July.
  • Fully extended TT around 11:00 and left it (guyed) that way.  Noted that the tower near the balloon inflation shed is the same height, so not worried about aircraft
  • Reinstalled soil sensors about 11:30 – also training for Chris
  • Removed the previous extension cords that initially were providing power to TT
  • Matt did Leica scans of all the sensors, using a GPS-based coordinate system!!!
  • Isabel added ISS to the Ubiquiti network, which broke the ISFS link temporarily.  She'll probably be adding a separate blog post about this.

TODOS:

  • shoot boom angle the old way with a compass, just as a check on the Leica data
  • work up the Leica data
  • recheck soil data coming in
  • at some point, take some soil cores?
  • keep on thinking about whether to lower TT
  • place a warning label on the power supply in the job box

John S and Liz also were on site working to get 449 working.  They thought they'd get it running before they left today.

P.S. Since tt wasn't up, Isabel rebooted, but data weren't coming in.  When typing "pio -v" to see the setting, pio killed power to the Ubiquiti, so it is off the net again.  I'll return to the site at the end of the day to restore the pio settings - we really need to fix this command...

ISS networking setup 6/24

I got both ISS sites at Marshall on the net today.

Of the existing ISFS ubiquiti setup, I had to pivot the pedestal ubiquiti a bit so it was more directed at the ISS sites. That doesn't seem to have any affect on the link to the ISFS tower. I also changed the link type of the two existing ubiquitis to 'point to multipoint' from 'point to point', so I could add new links to the same network.

At MISS I mounted the ubiquiti on the rail on the roof of the trailer:

I asked John about interference with the sounding antenna and he thinks they're far enough apart it should be fine. I guess we'll know better once we turn the sounding system on.

At ISS1 I happened to have a pole that would fit where the railings would mount on the trailer platform that isn't currently set up. Feel free to improve upon my mounting techniques...

 

Both ubiquitis are aimed at the ubiquiti on the mar-m05ped pedestal, between the RAL trailers (behind MISS) and some of the bigger snow gauges:

The distance is short and line-of-sight is good, so there's no need for them to be as high up as they are. Anything above 4 or 5 feet and with line of sight should work.

Both data managers are directly on the Marshall network on their WAN interfaces, no router anymore. Details on how to access them on the networking page. It just occurs to me that I didn't remove or turn off the cell modem at MISS, but hopefully that won't matter if we're not sending any data over the cell link.

While I was still at Marshall Steve told me tt was no longer on the net, so I went to investigate. Lights were on and fans were running but I couldn't log in over ethernet and didn't have the console cable to try that, so I rebooted. Seems to have come back up OK (except that now it's down again with the same PIO problem we had last week). I suspect the cause may have been that I unplugged and replugged the power side of POE injector in the dsm, to power cycle the ubiquiti so I could get to its wifi interface, and it seems like sometimes unplugging/replugging other cables will cause similar dsm problems. Haven't had a chance to check the logs about it, but I will when it's back up again.


Site visit yesterday

I had found a few detail mistakes in the LOTOS configuration and asked Isabel to implement them.  As this happened, some sensors were turned off and, in the process of powering them up, we lost internet connection to the site.  Thus, I made a last-minute site visit...

  • Indeed, pio reported that power was off for 28V (the Ubiquiti), bank1, bank2 (the FTDI board and sensor front-panel), and aux.  I logged in and used pio to enable all of these (except aux).  The DSM came back online.
  • Also noticed that Facilities had installed a power drop to our trailer.  Switched the power to our power supply to this new outlet.
  • Checked on why soil data were all strange, including Tsoils that were way offscale.  Found that the maintenance staff had disinterred all soil sensors (disconnecting the Tsoil probe from its mote in the process) during their weed control operations 2 weeks ago.  Unfortunate, since the soil is now quite dry.  Connected the Tsoil probe, but didn't rebury since I didn't have the soil tools.
  • Also looked for a power drop for CLAMPS by the CP3 pad.  There isn't a 14-50 outlet available; indeed I saw no free 220V outlets.  We'll need Facilities to install one.

FYI - noticing that the GPS and the CHRONY variables are missing in the DSM dashboard and NCharts. I've been told data are being collected but not monitored by NIDAS (question)

LOTOS data is now on Ncharts on datavis: http://datavis.eol.ucar.edu/ncharts/projects/LOTOS/noqc_instrument

datavis is accessible from everywhere, while datavis-dev is internal only.

Data look okay

Didn't do a complete look at everything, but did look at perhaps half of the variables, both with ncharts and dashboard today.  Everything looked fine, though a few notes:

  • The sonics didn't appreciate the weekend's rain, with several level shifts
  • Even so, the sonic tc values have some biases with respect to their colocated TRH's.  Made me wonder if we could characterize these biases by serial number, not just array position, to track them through different projects...
  • The batteries look like they <finally> charged up about 11AM today
  • The EC150 barometers (Pirga) have biases of a few mb with respect to each other and with respect to the (one) nanobarometer
  • The EC150 thermometers (Tirga) have a radiation error of up to 2 degrees.  The error only tracks Rsw.in less than half the time, but likely also depends on wind speed (that I didn't look at).

None of this is especially surprising...