Blog

The last two days have been filled with putting up CSAT3 & CSAT3b sonics. We loaded T0 with all of its sensors minus the hotfilms. I believe Gary went out there today to install and test those. We have about 15 more sonics to hang before we are finished at the array. We just received the EC100s and plates needed to finish. Thanks for shipping those out, Steve! We took a break and watched ISS launch a balloon in 22mph winds. The team went out to the trailer tower today to lay plywood at the feet, fill the generator, hang sensors and the ubiquity, lay cables, measure guy wires, install the lightning & ground rods, and install the light at the top. We did a practice raise to measure and mark the height to extend to. Tomorrow we will finish up at the trailer tower and hopefully the array since we have 7 sonics built up and ready to go. Good progress so far!

d 

Just a scratch space for us to use for burn-in.  Granted, it is still early for this, as things are being installed, but I did want to note one item:

Adding what I can see from logging directly into dsms (there are some problems I'm still looking into that show more data being missing on qctables...)

• tt: 7m csat3a/ec150 (ttb port 0) not reporting (wasn't plugged in)
• tt: 28m csat3a/ec150 (ttt port 0) all nans for co2/h2o (wasn't plugged in)
• t0: 2m TRH (t0b port 4) not reporting (bad port 4, moved to t0t port 7)
• t0: 3m TRH (t0t port 1) reports all zeros, far less than once a second
• t2: 4m TRH (port 3) not reporting
• t2: 4m csat3a/ec150 (port 2) not reporting
• t7: 4m csat3 (port 0) ldiag always nonzero
• t8: issues with dsm serial card (this may be the reason for all the t7/8/9 sensor problems) (dsm replaced, seems fine)
• t8: 4m csat3a/ec150 (port 2) not reporting (bad dsm)
• t8: 4m TRH (port 3) not reporting (bad dsm)
• t8: 4m nano (port 4) reporting init string only (bad dsm)
• t9: 4m csat3a (port 5) reporting in ascii mode, not being parsed (taken back to site to configure)
• t10: 4m csat3 (port 0) not reporting (tried other ports, probably not a port problem)
• t15: 4m csat3a (port 5) not reporting (wasn't installed)
• t17: 4m TRH (port 3) not reporting (it was earlier) (restarted dsm)
• t17: 4m nano (port 4) not reporting (it was yesterday) (restarted dsm)
• t18: 4m csat3b (port 5) not reporting (it was yesterday) (restarted dsm)
• t19: 4m csat3 (port 0) not reporting (box wasn't installed)
• t20: 4m TRH (port 3) bad rfan (replaced housing)
• t21: 4m csat3a (port 5) not reporting (needs data cable)
• t22: 4m csat3 (port 0) not reporting (box wasn't installed)
• t23: soils not reporting (rescanned mote)
• t23: 4m TRH (port 3) not reporting (added fuse to port 3)
• t24: 4m csat3b (port 5) not reporting (wasn't installed)
• t25: 4m csat3 (port 0) not reporting (box wasn't installed)
• t26: 4m csat3a/ec150 (port 2) not reporting (plugged in power inside box)
• t26: 4m csat3a/ec150 (port 2) set in ascii mode, taking box back to base
• t27: 4m csat3a (port 5) not reporting (added data cable)
• t28: 4m csat3 (port 0) not reporting (box wasn't installed)
• t29:  TRH T is low.  RH is fine.  Maybe just needs to have new cal re-uploaded?
• t30: 4m csat3b (port 5) not reporting (wasn't installed)
• t31: 4m csat3 (port 0) not reporting (box wasn't installed)
• t33: 4m csat3b (port 5) not reporting (updated sonic type in config)
• t34: 4m csat3 (port 0) not reporting (box wasn't installed)
• t36: 4m csat3b (port 5) not reporting (wasn't installed)
• t37: 4m csat3 (port 0) not reporting (box wasn't installed)
• t39: 4m csat3 (port 5) not reporting (box wasn't installed)
• t40: 4m csat3 (port 0) not reporting (box wasn't installed)
• t42: 4m csat3b (port 5) not reporting (wasn't installed)
• t43: 4m csat3 (port 0) not reporting (box wasn't installed)
• t45: 4m csat3 (port 5) not reporting (box wasn't installed)
• t46: 4m csat3 (port 0) not reporting (box wasn't installed)
• t47: 4m nano (port 5) not reporting, most likely ec100 box wiring issue? taking box back to base
• t48: 4m csat3b (port 5) not reporting (wasn't installed)
• t49: 4m csat3 (port 6) not reporting (box wasn't installed)
• t49: tsoil not reporting (rest of mote sensors are fine) (replaced)

I think I've (finally!) gotten this right...

As allocated in the config is based on the changes I made earlier today, when we thought we had more CSAT3Bs and fewer CSAT3As than we actually do.

CSAT3:

At M2HATS:

21. One has a short cable, just about exactly the length of the boom, so we'd need to figure out a way to mount the electronics box closer than just on the tower.

Allocated in the config:

20

CSAT3A/EC150:

At M2HATS:

24 CSAT3As, 24 EC150/EC100s already built as combination booms and already mounted on towers, or at least staged

3 new-style CSAT3A heads (2 in CSAT3B boxes) and 3 old-style CSAT3A heads, all still in trailer

1 EC100 box in trailer (no Nano), plus 2 EC100 boxes (and EC150s?) currently en route to the site.

Steve says there are 2 more EC100/EC150s at EOL. One has a bad sonic connector so we can't use, other has no nano (which wouldn't be a problem for this use)

Allocated in config:

24 CSAT3A/EC150 combination booms

4 CSAT3A + EC100 (no EC150) booms

CSAT3B, CSAT3BH:

At M2HATS:

7 CSAT3Bs and 1 CSAT3BH currently in trailer.

1 more CSAT3B en route to the site. It will need to be configured.

Allocated in the config:

9

So, once the shipment from EOL gets here, we will have exactly 58 usable sonics (including the CSAT3B that needs to be set up and the CSAT3 with a short cable).

Depending on which is easier to deal with (different mount for CSAT3 box vs. setting up new CSAT3B), I could change one more CSAT3B in the config to a CSAT3.

After a well needed rest day to escape the heat, we hit the ground running. We finished putting DSMs and batteries at all the towers except T0 (batteries but no DSMs). We then put solar panels, victrons, and batteries at the Trailer Tower and at the Chem Shelter. The legs of the TT are extended and attached but still need plywood for added surface area before leveling. Next we built up 4 booms with CSATs, quad-disks, and EC150s and put them at T2,5,8 & 47 along with EC100s and Ubiquitys facing the base trailer. The hope is that we can get enough signal from there without using the TT as the main antenna. I am not sure if the connection worked, it seemed that we were having trouble connecting to T5. Will have to troubleshoot tomorrow morning. Towers 2,5,8 & 47 still need data cables for the EC100s. The process of hanging the booms worked well with one climber positioned off and someone on the ladder directly upwind of the tower helping to guide the boom in.

It started getting really hot outside (103°C?) so we spent the rest of the time adding the equipment locations to AssetPanda and updating the CSAT3As with the gasket washers and bored cassettes. All of the updated cassettes have been swapped in so the rest without the gasket washer will have to wait for the update until after M2HATS. 6 more booms with CSATs, quad-disks, and EC150s were made and are ready to be hung tomorrow on the towers with DSMs. 

Today we started out where we left off yesterday adding CSATs, EC150s, and Quad-disks to every DSM tower. There was actual cloud cover today and a strong breeze to keep us cool. Justin and Isabel did a fantastic job building up the booms. Tony added all the equipment to their prospective locations on Asset Panda. We then built up some TRHs and started hanging them on the towers. We got half way through the array before stopping for the day. We'll finish hanging them first thing tomorrow.  

One thing we found out was that we didn't have enough Quad-disks for the Trailer Tower, EC100s for the CSAT 3As, nor plates to hang the CSAT3 electronic boxes. Steve graciously got everything packed up for UCAR logistics to pickup and ship tomorrow. Thank you Steve.

All ubiquiti links are on the net except to the trailer tower (not set up) and t0 and t35 (need troubleshooting). We decided to mount the omni on the base trailer rather than the trailer tower, so we could get links up before we get around to raising the trailer tower. So far links seem to be working, despite the omni being considerably lower. Other than the ones at tt, t0, and t35, all DSMs are on the net. t35, t0b, and t0t are likely powered and working, just not networked. Ustar is online in the base trailer and is running dsm_server and statsproc. As of this morning raw and netcdf data is being transferred back to EOL. Data is starting to show up on qctables!!

Learning lesson: If we have a field project using all of our sensors, pack everything we have in stock related to those sensors. You never know what you might need...

Another warm day out in the sun setting up towers. Gary and Justin came out to help and we put up another 20 towers on Thursday and finished the array today (Friday). There are boom clamps on all the towers at 4.1m and the ratchet straps are set at ~4m. This afternoon we started hanging DSMs and setting up the solar systems. DSMs for T2-T32 are hanging as of today and are powered on. Looking to finish up after a deserved rest day. The internet has been setup at the base trailer as of this evening thanks to Gary, Tony, and the contractor. Great work team! We also spotted a Long-nosed Leopard Lizard at the site today. 

Another hot day out there.

Started early again this morning out at the tower array. We did more prep for raising towers in the morning, with Tony, Will, and Chris working on siting and placing anchors for the towers, as well as measuring the height of the sonic booms (will be 4.1 meters to be able to fit the boom clamps on the tower) and adding boom clamps. Meanwhile I took the cart and fastened on and laid out all the ratchet strap guy wires for the towers. Done with that around 11 and decided to head back to base for a bit of a break.

At this point I left to join Bill working on ISS stuff, while Tony, Will, and Chris set up the porch on the base trailer, turned on the power (and the AC!), and did some general reorganization of the base. Later in the afternoon they headed back out to the tower array and kept working on erecting towers. Will says we're up to 16ish towers installed!

The base site is still busy with people working on ISS, REAL, and the MPDs.

Another early start, and another hot day out there.

Started putting up towers along the tower array. We started with the one farthest from the base site, then doubled back to the closest to the base site and worked outward from there (Bill wanted to have both ends set up so he could sight off of them to position MISS). We had trouble with some of the auger anchors, and have ended up using rebar stakes for some of the guy points. After trying to align the first couple, we decided a string line would be helpful for keeping the towers in between the flagged ones in line, so took a trip to the hardware store.

At this point I wasn't that helpful to the crew, so I went back to base and worked on ISS stuff for most of the day (so I might miss something in this blog post). Will, Tony, and Chris got back to it at the tower array and got 7ish towers up, out in the hot sun.

The electrician was on site all day getting the power panel set up. We now have power to ISS1, MISS, the base trailer, REAL, and MPD. The electrician will be back once he gets the outlets he ordered (currently trailers are hardwired into the panel), but I think everything is done and working for now.

Chenning and 2 of his students are here, starting the process of setting up posts to run fiber. Today they drilled mounting holes in their pipes and distributed them along the array, not yet pounded in. Chris took Chenning on a tour of the array site so he knows where the fiber will run. Shane, along with Bruce and Scott, is working on setting up REAL now that they have power.

The MPDs arrived early this afternoon, and Ben, Dexter, and Robert are working on getting those set up.

The base trailer showed up late this afternoon. It's sited but not yet leveled, and we've put together the back stairs but not the porch. It does have power. 

Happy birthday Will!

Decided to get an earlier start today to avoid some of the heat, which seems to be working.

Today we worked one long chunk, 7 to 1:30:

• assembled all the rest of the towers! They are all in one piece now, still lying flat. We've staged auger anchors and stakes by each one. We improved the process by having Chris, Tony, and Will work on fitting tower sections together, again done standing up with the use of the truck, while David and I got the bolts in while the tower sections were on the ground. Thanks David for helping out today!

After that long in the sun we decided to quit for the day and start erecting towers tomorrow when we're fresh.

REAL arrived today around 1:30 and is now in position, pretty much exactly within the flags. Tony talked to the electrician, who is coming tomorrow. Hopefully he'll be able to finish the panel so we can power up the trailers and turn on the AC. The base trailer has left Boulder and should be here sometime tomorrow, hooray!

Morning:

• finished staging baseplates, 5' tower sections, and 10' tower sections along the tower array (we found two more baseplates in the container, so we're set). We've loaded up the stakes, augers, and ratchet straps in the truck and have started distributing those as we work on assembling towers.
• started putting tower sections together, starting from the far end of the tower array. After some trial and error, it seems like the best way to do this is to stand the pieces up, so the weight of the tower sections helps them slide together. We've been doing this at the truck, with someone standing in the bed to hold the tower sections while the others work on lining the pieces up and adding hardware. Got about 4 towers done before lunch, plus the very far end one, which we discovered has a baseplate that needs some filing before it'll fit on the tower section.

Afternoon:

• more of the same. Thanks to ISS for the loan of a hammer and a big channel lock pliers, which makes assembly much easier. Got another 10 (?? 12??) assembled this afternoon before calling it quits.
• helped ISS move the 449 panels onto their frame.

Will and I are here for ISFS.  Bill, Isabel, and David are here for ISS.

Morning:

• Orientation for everyone
• Added RF4 flag (far end of DTS fiber, though with no offset from the REAL line)
• Created 2-track with turnarounds at the end of the DTS line and the end of the tower line
• Dropped off chem shelter near DTS start
• Dropped off 4 solar panel racks with coolers (so far) along tower line
• Dropped off feet/generator/job box at TT, showed Will how to drive there.

Afternoon (Tony and Josh arrived halfway through):

• Dropped off the rest of the solar racks and coolers along the tower line – might need a second rack at t0.  Still need to drop off a rack or two at TT.  Also need to drop any extras at chem shelter
• Dropped off all the base plates we had – we are short 2, but Tony thinks there are a few more in the base trailer.
• At the same time, dropped off about half of the 5' sections (all we could load in one pickup load).  THANKS ISS staff for helping with this load – it went really quickly!

I guess I should explain the layout plan (hopefully the diagram below helps).  This is my understanding from various discussions:

• My "reference line" starts with the REAL mirror position, when it is scanning at an azimuth angle 80 degrees and goes out on a bearing of 80 degrees (true).  REAL will scan at 4m AGL.
• I think the plan for the REVEAL is to be along the REAL reference line at an altitude of 2m AGL.
• I think the plan for the lidar on top of MISS is to make a parallel scan also at 4m AGL, but offset to the north by 2m.
• The line of sonics is to be also at 4m AGL, but offset an additional 2m (4m north of the REAL reference line.  The line of tower base plates are centered 2.17m further to the north (6.17m from the REAL line).
• The line of the DTS fiber is to be parallel at an altitude of 2m AGL, 2m to the north of the REAL reference line. 

Left the motel before 6am to try the Leica before the heat set in:

• The optics were fine, but even with pairing with the TS at the site and driving out, I lost and couldn't get back connectivity to the handheld.  I ended up setting reference point 2 manually – set the GPS pole, walk/drive back a mile to the TS to view where it was relative to the radial/distance, going back out to move the GPS pole a few meters, then back to the TS.  After about 4 iterations, I think I got to within 10cm.
• Then moved the TS all the way to the midpoint of the array.  I resectioned with only 2 points (R10, R11) and it seemed to be happy.
• Finally, I could start setting the line!  I defined the line REALSCAN2 from TSC3 (total station, central site, third attempt) to RF1 (REAL scan line, in the field, first position, which is at the distance out where the DTS fiber will start).  Not quite in this order, I now have flagged RF2 (along the REAL scan line at the distance where t0 will be), RF3 (along the REAL scan line at the distance of t49), and the base plates for t0, t2, t4, t6.  I put a flag for t49, but it will have to be adjusted by 34cm due to the alternating base plate rotations.
• At that point, all my batteries died, so I returned to the motel.

All of the above was interrupted several times:

• About 7am, the ISS1 trailer was dropped off.  It is now parked just outside the gate.
• About 10am, the trailer tower was dropped off.  I transferred it to the silver truck and took it all the way to its final location in the field, in the correct orientation.  In the process, I did manage to get stuck in soft sand, but powered through in 4low.  It should be better coming out on the track I'm beginning to establish.  I don't have the outrigger feet yet (coming soon), so didn't even try to set them.  Cell phone GPS: 38d 02.147'N, 117d 05.407'W.  Using a handheld compass, the orientation seemed to be close to 80-12deg declination = 68deg.
• Ran into Town wastewater people unfortunately adding to the pond.  Ralph also was the guy who delivered our porta potty, but to the wrong location.  He'll move it Monday.

Not too bad for a morning.-- went to town with the batteries and got lunch (and gas).

Afternoon:

• 3rd trucker arrived about 4:15.  To place the 2 seatainers, he first dumped both off of the trailer, then picked one up and placed it, then came back to pick and place the other.  The whole process took somewhat over an hour, but now looks exactly as Bill drew in his diagram.  The time to do this was comparable to loading with a crane, or using the manual jacks, though in between the two in flexibility.  BTW, he says that the trailer was $130K for an 8-year-old used one and that his previous one was $180K new 20 years ago.  A pricey bit of equipment!
• Now back at motel where batteries are ready.  Will, Bill, Isabel, and David are now making their way up from Vegas.  Company, at last!

Evening:

• A final run to the site, and Will was able to join up!  Together, we flagged all the even-numbered towers.  We lost the TS-handheld connection again at about 500m while trying to flag the end of the DTS fiber.  As I did last night, we left the TS in place to keep its orientation/position and will do a 2-person sighting of this last point tomorrow.

Plan:

• Blaze a track to the tower line for vehicle access
• Finish the last flagging point (just one)
• Drop the chem shelter for the DTS stuff
• Empty out the seacontainer, distributing tower sections/base plates/solar panels to their respective positions
• If time, grab TT feet and set the TT legs
• Sometime, could flag the odd-numbered towers as well (and check the tower line with a measuring tape)

Just Steve, due to several factors...

Started with a visit to Town Hall, where I got the keys to access the trailer compound and had a brief chat with the Tourism/Events Coordinator about possible outreach activities.

Stopped in to check in with Mark Pederson.

Next, flagged most trailer locations (just using measuring tape):

• Drove out to a possible site for tt.  Had to "break trail" through brush the first 100m or so and there is one downslope place, but the rest was easy.  After later studying the route, I think it can be improved a bit.  Also, I need to move my flagged location about 50m south to be within the 50:1 glideslope
• Flagged locations for the storage seatainers, ISS1, ISFS base, and REAL trailers.  I felt pretty good about all of these, including REAL.

Spent the rest of the day hassling with the Leica.

• Everything seemed okay with connections and set-up
• Placed TS exactly at the REAL lidar mirror, to establish one end of the reference line to (later) stake to.
• Started the process to locate the TS using multiple GPS pole sitings.  Found that it took 2500-3000 GPS scans to get even to 0.200m accuracy (about 10min).  Talking with Matt, tried to clear the ephemeris, but didn't help.  Went through 4 points, and it couldn't calculate a position, tried deleting one (that it did permanently, by mistake) and could get a crude position.  Decided to add more points, but still couldn't calculate TS position.  With 7 points, it came up with a location, but clearly was WAY off.  I had the idea that the position IDs were out of sync between the TS and the GS, but couldn't figure out how to edit this.  While messing with this, managed to get out of the TS position mode and couldn't figure out how to get back in to use the points (GPCx) that it had saved.  Eventually, had to start from scratch (GPDx) collecting another 3 or 4 points.  It still complained about CQ being 1.3, rather than 1.0, but did manage to compute a reasonable position.  This process took about 4 hours!
• I still needed a second point on the reference line, so I took the GPS pole out to the rough location of the closest end of the DTS.  In the process, I oriented myself to the BLM land boundary.  During the drive out to the field, I turned off the GPS and handheld device, but left the TS on (obviously!).  When at the site, I couldn't connect through the long-range antenna to the TS.  Drove back to the TS and could roughly site back to the pole, but "power scan" couldn't find the prism, presumably due to all of the optical shimmer (clearly obvious through the TS telescope) – at this point, it was 3:30pm.  Manually siting through the telescope, it found that the distance was only about 680m.  I could see the power LED on on the LR antenna, but had no indication that it was connected.  Decided to shut everything down and come back in cooler conditions.  To avoid the hassle of re-locating the TS, I put the bag over the head and left it in place.

Learned late in the day that the land use agreement was finally signed!

Stopped by the local urgent care, to learn about their services.

I did go back to the site at sunset, having used Google Earth to estimate the lat/lon of a few points along the reference line.  Just using my phone GPS, to 12' accuracy, I flagged each of these locations.  It was obvious that the shimmer was gone even then.

Plan for tomorrow:

My plan is to go back at dawn to get at least the second reference point flagged (adjusting the flags that I just installed).  I'll pair the handheld with the TS when close by, then leave it on as I drive out.  If this works, I'll try to flag the other points.  If even one of these works, I'll go back and reposition (and resection again) the TS closer to the tower array and flag the DTS and tower locations, using offsets (2m and 6.17m, respectively) from the reference line.  If not, its back to measuring tape and compass!

Also, throughout the day, receive 3 truckers to the site.

Typical sunset

My evening stroll companion!

Only set up to have tower top plate at 27m (plus the 5' Rohn section).  This is 4m between tower sections, rather than the normal 5m.  I think this can be made repeatable by repositioning the top "micro"switch.

The top sonic and TRH will be on the 5' Rohn section.  The beacon also will be on this section.

The sensor heights thus are 7m (top of fixed section), 15m (top of 3rd section), and 28m (on Rohn 5' section).

I would use two cell clamps directly attached to the "blocks" along the block's top edge to get it a bit higher (the TRH inlet still will be lower than the sonic).

Sonic booms will point (approximately) south (160 degrees – the same as the tower array), TRH booms will point approximately northeast (40 degrees).  Because the TRH boom can't poke into the bed of the trailer, this sets the trailer orientation as the tongue needs to point approximately to the west (250 degrees).

We plan to use the omni Ubiquiti on this tower.  We have a mount that will work with the special top-of-section clamps.  Alternately, we could attach (u-bolt or hose clamp) to the top Rohn section.  I actually think the reception may be better lower down due to the elevation angle response of the antenna – perhaps 11m or 19m?

Possible tt locations are red dots.  I prefer the middle two, but driving access may be problematic.

CSAT3: Need 19, have 21 (now that all have returned from Campbell), all have been set to 30sps, 19200baud

CSAT3A (old style): Need 3, have 3 (now that all 3 have returned from Campbell), no reconfiguring needed.  We are currently retesting two of them to double-check that they have come back okay.

CSAT3AW (new style): Need 26, have 27.  Many (17, 18?) are still in SOS.  Will bring directly to M2HATS with no change.

CSAT3B: Need 8, have 9 (Still waiting for the 9th to be delivered new from Campbell).  The new one and the one CSAT3BH now in SOS will need to be set to 50sps (needs AINU).  The other 7 are now set to 50.

EC150: Need 22, have 30.  16 are now in SOS and should have postcals done before being shipped to M2HATS.  2 are in the Volleyball court and could stay there.  Others should be around somewhere.

EC100: Need 26, have 30.  Again a mix of stuff here and at SOS.  Only one here has been set to 60sps.  Need to do this for all the rest! (needs AINU).