Another glorious hot day in the desert. Dust devils all over, including one that went right between our storage seatainers.
ISFS staff: Chris, Tony, Will, Justin, me, Isabel and Gary
- Gary spent time working with hot-film data. At first, had some data gaps when the LabJack buffer overflowed, but later seemed to work fine. We might split the digitizing load between two LabJacks, maybe using 2 DSMs if this problem recurs.
- Most of the crew finished set-up and erected TT. Mark at the FBO was unconcerned about it as long as it was past the 50:1 glideslope and had a beacon (which it does). We invited him to visit (hasn't happened yet) and he invited us to his apricot tree (which we did take advantage of!). The TT erection process went well despite our need this time to stop it early. We just got close and finished up with turnbuckle tension on the guys.
- Then, most of the crew worked on clean-up and setting up spare sensors (specifically, getting the new CSAT3B wired up – thanks, Tony!).
- Chris helped Chenning with power for the DTS fiber controller. Chenning's students will try to pick up a better inverter during a day off in Vegas tomorrow. Chris is optimistic that, with the right inverter, we have sufficient power for the controller. If not, we can resort to hand-swapping batteries.
- Isabel and I worked on burn-in tasks:
- Made some edits to qctables and co2 limits in project_init that got rid of a lot of false alarms
- Also hand edited scalar flux groups in stats_5min.xml to get a couple of pressures reporting (but also now has the correct grouping of nanos and sonics)
- In the afternoon, went to the field with a LARGE punch list and worked through more than half:
- Soils:
- Tsoil.t47 had to be replaced (including reinstallation done by hand in the sand). Strange, but we had a spare.
- All soils at t23 were working. Again, stats_5min needed an edit to get most of these running. Also found an issue calculating Iload/I3load.motes. Still need to add braid around the cables.
- TRH:
- t23 a fuse was missing in the DSM, now working
- t20 we replaced the housing to get Rfan working
- Nanobarometers:
- tried a bunch of stuff to get t47 working (including swapping the nano, DSM ports, cables, etc.). Concluded that it has to be the internal wiring inside the EC100, so took it down. This of course also killed the CSAT.
- Sonics:
- t27 just needed a serial cable installed. Done.
- t26 had the power connector inside the EC100 pulled. Easy to fix, but then was reporting in ASCII. Pulled the EC100 and fixed it in the base (probably could have done on the tower using minicom). Need to reinstall.
- t21 also needs a serial cable, but we didn't have one long enough – will do tomorrow
- t10 is an old CSAT3 that isn't reporting at all. Might return with another data cable and/or open up the box and check power, at least.
- t9 also was reporting in ASCII. Again, pulled the EC100, fixed in the base, and will reinstall tomorrow.
- Soils:
Still have some issues at t7, t2, t0, and tt to resolve.
Thanks, everyone for a great job getting M2HATS on track!! Chris, Tony, Will, and Isabel leave tomorrow. We celebrate with dinner at the Mizpah tonight.
Back to sunny and hot, with a few scattered Cu. I arrived to mess up everyone's workflow....
Spent a 10-hour day addressing (almost) everything on the tower line:
- All sonics now mounted and connected to their appropriate DSMs.
- Installed the last Quad Disk Probe at T0 1m sonic.
- TRHs are all up. We found that TRH.2m.t0 wasn't coming in due to a DSM port issue. We moved it from DSM4.t0b to DSM7.t0t.
- Isabel had issues with T8's serial board, so she replaced the entire DSM with a spare. Seems much better.
- All soil sensors and radiometers installed. Still need to dress the cables and flag the sample area for soils at T23.
- Installed the short CSAT3 at T33 and the CSAT3BH at T48 to complete the array.
- Installed a single solar panel/battery/victron at the end of the DTS fiber for the calibration bath.
- Ran into issues with the inverter at the chem shelter. Isn't strong enough to power the DTS.
So, a productive day. Finishing up the tt and getting it erected is on the docket for tomorrow. I'll probably peel off to work on burn-in issues.
The view down the entire line of sonics at 4m, with REAL in the center of the image, 1.4km away.
The last two days have been filled with putting up CSAT3 & CSAT3b sonics. We loaded T0 with all of its sensors minus the hotfilms. I believe Gary went out there today to install and test those. We have about 15 more sonics to hang before we are finished at the array. We just received the EC100s and plates needed to finish. Thanks for shipping those out, Steve! We took a break and watched ISS launch a balloon in 22mph winds. The team went out to the trailer tower today to lay plywood at the feet, fill the generator, hang sensors and the ubiquity, lay cables, measure guy wires, install the lightning & ground rods, and install the light at the top. We did a practice raise to measure and mark the height to extend to. Tomorrow we will finish up at the trailer tower and hopefully the array since we have 7 sonics built up and ready to go. Good progress so far!
d
Just a scratch space for us to use for burn-in. Granted, it is still early for this, as things are being installed, but I did want to note one item:
Adding what I can see from logging directly into dsms (there are some problems I'm still looking into that show more data being missing on qctables...)
- tt: 7m csat3a/ec150 (ttb port 0) not reporting (wasn't plugged in)
- tt: 28m csat3a/ec150 (ttt port 0) all nans for co2/h2o (wasn't plugged in)
- t0: 2m TRH (t0b port 4) not reporting (bad port 4, moved to t0t port 7)
- t0: 3m TRH (t0t port 1) reports all zeros, far less than once a second
- t2: 4m TRH (port 3) not reporting
- t2: 4m csat3a/ec150 (port 2) not reporting
- t7: 4m csat3 (port 0) ldiag always nonzero
- t8: issues with dsm serial card (this may be the reason for all the t7/8/9 sensor problems) (dsm replaced, seems fine)
- t8: 4m csat3a/ec150 (port 2) not reporting (bad dsm)
- t8: 4m TRH (port 3) not reporting (bad dsm)
- t8: 4m nano (port 4) reporting init string only (bad dsm)
- t9: 4m csat3a (port 5) reporting in ascii mode, not being parsed (taken back to site to configure)
- t10: 4m csat3 (port 0) not reporting (tried other ports, probably not a port problem)
- t15: 4m csat3a (port 5) not reporting (wasn't installed)
- t17: 4m TRH (port 3) not reporting (it was earlier) (restarted dsm)
- t17: 4m nano (port 4) not reporting (it was yesterday) (restarted dsm)
- t18: 4m csat3b (port 5) not reporting (it was yesterday) (restarted dsm)
- t19: 4m csat3 (port 0) not reporting (box wasn't installed)
- t20: 4m TRH (port 3) bad rfan (replaced housing)
- t21: 4m csat3a (port 5) not reporting (needs data cable)
- t22: 4m csat3 (port 0) not reporting (box wasn't installed)
- t23: soils not reporting (rescanned mote)
- t23: 4m TRH (port 3) not reporting (added fuse to port 3)
- t24: 4m csat3b (port 5) not reporting (wasn't installed)
- t25: 4m csat3 (port 0) not reporting (box wasn't installed)
- t26: 4m csat3a/ec150 (port 2) not reporting (plugged in power inside box)
- t26: 4m csat3a/ec150 (port 2) set in ascii mode, taking box back to base
- t27: 4m csat3a (port 5) not reporting (added data cable)
- t28: 4m csat3 (port 0) not reporting (box wasn't installed)
- t29: TRH T is low. RH is fine. Maybe just needs to have new cal re-uploaded?
- t30: 4m csat3b (port 5) not reporting (wasn't installed)
- t31: 4m csat3 (port 0) not reporting (box wasn't installed)
- t33: 4m csat3b (port 5) not reporting (updated sonic type in config)
- t34: 4m csat3 (port 0) not reporting (box wasn't installed)
- t36: 4m csat3b (port 5) not reporting (wasn't installed)
- t37: 4m csat3 (port 0) not reporting (box wasn't installed)
- t39: 4m csat3 (port 5) not reporting (box wasn't installed)
- t40: 4m csat3 (port 0) not reporting (box wasn't installed)
- t42: 4m csat3b (port 5) not reporting (wasn't installed)
- t43: 4m csat3 (port 0) not reporting (box wasn't installed)
- t45: 4m csat3 (port 5) not reporting (box wasn't installed)
- t46: 4m csat3 (port 0) not reporting (box wasn't installed)
- t47: 4m nano (port 5) not reporting, most likely ec100 box wiring issue? taking box back to base
- t48: 4m csat3b (port 5) not reporting (wasn't installed)
- t49: 4m csat3 (port 6) not reporting (box wasn't installed)
- t49: tsoil not reporting (rest of mote sensors are fine) (replaced)
I think I've (finally!) gotten this right...
As allocated in the config is based on the changes I made earlier today, when we thought we had more CSAT3Bs and fewer CSAT3As than we actually do.
CSAT3:
At M2HATS:
21. One has a short cable, just about exactly the length of the boom, so we'd need to figure out a way to mount the electronics box closer than just on the tower.
Allocated in the config:
20
CSAT3A/EC150:
At M2HATS:
24 CSAT3As, 24 EC150/EC100s already built as combination booms and already mounted on towers, or at least staged
3 new-style CSAT3A heads (2 in CSAT3B boxes) and 3 old-style CSAT3A heads, all still in trailer
1 EC100 box in trailer (no Nano), plus 2 EC100 boxes (and EC150s?) currently en route to the site.
Steve says there are 2 more EC100/EC150s at EOL. One has a bad sonic connector so we can't use, other has no nano (which wouldn't be a problem for this use)
Allocated in config:
24 CSAT3A/EC150 combination booms
4 CSAT3A + EC100 (no EC150) booms
CSAT3B, CSAT3BH:
At M2HATS:
7 CSAT3Bs and 1 CSAT3BH currently in trailer.
1 more CSAT3B en route to the site. It will need to be configured.
Allocated in the config:
9
So, once the shipment from EOL gets here, we will have exactly 58 usable sonics (including the CSAT3B that needs to be set up and the CSAT3 with a short cable).
Depending on which is easier to deal with (different mount for CSAT3 box vs. setting up new CSAT3B), I could change one more CSAT3B in the config to a CSAT3.
After a well needed rest day to escape the heat, we hit the ground running. We finished putting DSMs and batteries at all the towers except T0 (batteries but no DSMs). We then put solar panels, victrons, and batteries at the Trailer Tower and at the Chem Shelter. The legs of the TT are extended and attached but still need plywood for added surface area before leveling. Next we built up 4 booms with CSATs, quad-disks, and EC150s and put them at T2,5,8 & 47 along with EC100s and Ubiquitys facing the base trailer. The hope is that we can get enough signal from there without using the TT as the main antenna. I am not sure if the connection worked, it seemed that we were having trouble connecting to T5. Will have to troubleshoot tomorrow morning. Towers 2,5,8 & 47 still need data cables for the EC100s. The process of hanging the booms worked well with one climber positioned off and someone on the ladder directly upwind of the tower helping to guide the boom in.
It started getting really hot outside (103°C?) so we spent the rest of the time adding the equipment locations to AssetPanda and updating the CSAT3As with the gasket washers and bored cassettes. All of the updated cassettes have been swapped in so the rest without the gasket washer will have to wait for the update until after M2HATS. 6 more booms with CSATs, quad-disks, and EC150s were made and are ready to be hung tomorrow on the towers with DSMs.
Today we started out where we left off yesterday adding CSATs, EC150s, and Quad-disks to every DSM tower. There was actual cloud cover today and a strong breeze to keep us cool. Justin and Isabel did a fantastic job building up the booms. Tony added all the equipment to their prospective locations on Asset Panda. We then built up some TRHs and started hanging them on the towers. We got half way through the array before stopping for the day. We'll finish hanging them first thing tomorrow.
One thing we found out was that we didn't have enough Quad-disks for the Trailer Tower, EC100s for the CSAT 3As, nor plates to hang the CSAT3 electronic boxes. Steve graciously got everything packed up for UCAR logistics to pickup and ship tomorrow. Thank you Steve.
All ubiquiti links are on the net except to the trailer tower (not set up) and t0 and t35 (need troubleshooting). We decided to mount the omni on the base trailer rather than the trailer tower, so we could get links up before we get around to raising the trailer tower. So far links seem to be working, despite the omni being considerably lower. Other than the ones at tt, t0, and t35, all DSMs are on the net. t35, t0b, and t0t are likely powered and working, just not networked. Ustar is online in the base trailer and is running dsm_server and statsproc. As of this morning raw and netcdf data is being transferred back to EOL. Data is starting to show up on qctables!!
Learning lesson: If we have a field project using all of our sensors, pack everything we have in stock related to those sensors. You never know what you might need...
Another warm day out in the sun setting up towers. Gary and Justin came out to help and we put up another 20 towers on Thursday and finished the array today (Friday). There are boom clamps on all the towers at 4.1m and the ratchet straps are set at ~4m. This afternoon we started hanging DSMs and setting up the solar systems. DSMs for T2-T32 are hanging as of today and are powered on. Looking to finish up after a deserved rest day. The internet has been setup at the base trailer as of this evening thanks to Gary, Tony, and the contractor. Great work team! We also spotted a Long-nosed Leopard Lizard at the site today.
Another hot day out there.
Started early again this morning out at the tower array. We did more prep for raising towers in the morning, with Tony, Will, and Chris working on siting and placing anchors for the towers, as well as measuring the height of the sonic booms (will be 4.1 meters to be able to fit the boom clamps on the tower) and adding boom clamps. Meanwhile I took the cart and fastened on and laid out all the ratchet strap guy wires for the towers. Done with that around 11 and decided to head back to base for a bit of a break.
At this point I left to join Bill working on ISS stuff, while Tony, Will, and Chris set up the porch on the base trailer, turned on the power (and the AC!), and did some general reorganization of the base. Later in the afternoon they headed back out to the tower array and kept working on erecting towers. Will says we're up to 16ish towers installed!
The base site is still busy with people working on ISS, REAL, and the MPDs.
Another early start, and another hot day out there.
Started putting up towers along the tower array. We started with the one farthest from the base site, then doubled back to the closest to the base site and worked outward from there (Bill wanted to have both ends set up so he could sight off of them to position MISS). We had trouble with some of the auger anchors, and have ended up using rebar stakes for some of the guy points. After trying to align the first couple, we decided a string line would be helpful for keeping the towers in between the flagged ones in line, so took a trip to the hardware store.
At this point I wasn't that helpful to the crew, so I went back to base and worked on ISS stuff for most of the day (so I might miss something in this blog post). Will, Tony, and Chris got back to it at the tower array and got 7ish towers up, out in the hot sun.
The electrician was on site all day getting the power panel set up. We now have power to ISS1, MISS, the base trailer, REAL, and MPD. The electrician will be back once he gets the outlets he ordered (currently trailers are hardwired into the panel), but I think everything is done and working for now.
Chenning and 2 of his students are here, starting the process of setting up posts to run fiber. Today they drilled mounting holes in their pipes and distributed them along the array, not yet pounded in. Chris took Chenning on a tour of the array site so he knows where the fiber will run. Shane, along with Bruce and Scott, is working on setting up REAL now that they have power.
The MPDs arrived early this afternoon, and Ben, Dexter, and Robert are working on getting those set up.
The base trailer showed up late this afternoon. It's sited but not yet leveled, and we've put together the back stairs but not the porch. It does have power.
Happy birthday Will!
Decided to get an earlier start today to avoid some of the heat, which seems to be working.
Today we worked one long chunk, 7 to 1:30:
- assembled all the rest of the towers! They are all in one piece now, still lying flat. We've staged auger anchors and stakes by each one. We improved the process by having Chris, Tony, and Will work on fitting tower sections together, again done standing up with the use of the truck, while David and I got the bolts in while the tower sections were on the ground. Thanks David for helping out today!
After that long in the sun we decided to quit for the day and start erecting towers tomorrow when we're fresh.
REAL arrived today around 1:30 and is now in position, pretty much exactly within the flags. Tony talked to the electrician, who is coming tomorrow. Hopefully he'll be able to finish the panel so we can power up the trailers and turn on the AC. The base trailer has left Boulder and should be here sometime tomorrow, hooray!
Morning:
- finished staging baseplates, 5' tower sections, and 10' tower sections along the tower array (we found two more baseplates in the container, so we're set). We've loaded up the stakes, augers, and ratchet straps in the truck and have started distributing those as we work on assembling towers.
- started putting tower sections together, starting from the far end of the tower array. After some trial and error, it seems like the best way to do this is to stand the pieces up, so the weight of the tower sections helps them slide together. We've been doing this at the truck, with someone standing in the bed to hold the tower sections while the others work on lining the pieces up and adding hardware. Got about 4 towers done before lunch, plus the very far end one, which we discovered has a baseplate that needs some filing before it'll fit on the tower section.
Afternoon:
- more of the same. Thanks to ISS for the loan of a hammer and a big channel lock pliers, which makes assembly much easier. Got another 10 (?? 12??) assembled this afternoon before calling it quits.
- helped ISS move the 449 panels onto their frame.
Will and I are here for ISFS. Bill, Isabel, and David are here for ISS.
Morning:
- Orientation for everyone
- Added RF4 flag (far end of DTS fiber, though with no offset from the REAL line)
- Created 2-track with turnarounds at the end of the DTS line and the end of the tower line
- Dropped off chem shelter near DTS start
- Dropped off 4 solar panel racks with coolers (so far) along tower line
- Dropped off feet/generator/job box at TT, showed Will how to drive there.
Afternoon (Tony and Josh arrived halfway through):
- Dropped off the rest of the solar racks and coolers along the tower line – might need a second rack at t0. Still need to drop off a rack or two at TT. Also need to drop any extras at chem shelter
- Dropped off all the base plates we had – we are short 2, but Tony thinks there are a few more in the base trailer.
- At the same time, dropped off about half of the 5' sections (all we could load in one pickup load). THANKS ISS staff for helping with this load – it went really quickly!
I guess I should explain the layout plan (hopefully the diagram below helps). This is my understanding from various discussions:
- My "reference line" starts with the REAL mirror position, when it is scanning at an azimuth angle 80 degrees and goes out on a bearing of 80 degrees (true). REAL will scan at 4m AGL.
- I think the plan for the REVEAL is to be along the REAL reference line at an altitude of 2m AGL.
- I think the plan for the lidar on top of MISS is to make a parallel scan also at 4m AGL, but offset to the north by 2m.
- The line of sonics is to be also at 4m AGL, but offset an additional 2m (4m north of the REAL reference line. The line of tower base plates are centered 2.17m further to the north (6.17m from the REAL line).
- The line of the DTS fiber is to be parallel at an altitude of 2m AGL, 2m to the north of the REAL reference line.
Left the motel before 6am to try the Leica before the heat set in:
- The optics were fine, but even with pairing with the TS at the site and driving out, I lost and couldn't get back connectivity to the handheld. I ended up setting reference point 2 manually – set the GPS pole, walk/drive back a mile to the TS to view where it was relative to the radial/distance, going back out to move the GPS pole a few meters, then back to the TS. After about 4 iterations, I think I got to within 10cm.
- Then moved the TS all the way to the midpoint of the array. I resectioned with only 2 points (R10, R11) and it seemed to be happy.
- Finally, I could start setting the line! I defined the line REALSCAN2 from TSC3 (total station, central site, third attempt) to RF1 (REAL scan line, in the field, first position, which is at the distance out where the DTS fiber will start). Not quite in this order, I now have flagged RF2 (along the REAL scan line at the distance where t0 will be), RF3 (along the REAL scan line at the distance of t49), and the base plates for t0, t2, t4, t6. I put a flag for t49, but it will have to be adjusted by 34cm due to the alternating base plate rotations.
- At that point, all my batteries died, so I returned to the motel.
All of the above was interrupted several times:
- About 7am, the ISS1 trailer was dropped off. It is now parked just outside the gate.
- About 10am, the trailer tower was dropped off. I transferred it to the silver truck and took it all the way to its final location in the field, in the correct orientation. In the process, I did manage to get stuck in soft sand, but powered through in 4low. It should be better coming out on the track I'm beginning to establish. I don't have the outrigger feet yet (coming soon), so didn't even try to set them. Cell phone GPS: 38d 02.147'N, 117d 05.407'W. Using a handheld compass, the orientation seemed to be close to 80-12deg declination = 68deg.
- Ran into Town wastewater people unfortunately adding to the pond. Ralph also was the guy who delivered our porta potty, but to the wrong location. He'll move it Monday.
Not too bad for a morning.-- went to town with the batteries and got lunch (and gas).
Afternoon:
- 3rd trucker arrived about 4:15. To place the 2 seatainers, he first dumped both off of the trailer, then picked one up and placed it, then came back to pick and place the other. The whole process took somewhat over an hour, but now looks exactly as Bill drew in his diagram. The time to do this was comparable to loading with a crane, or using the manual jacks, though in between the two in flexibility. BTW, he says that the trailer was $130K for an 8-year-old used one and that his previous one was $180K new 20 years ago. A pricey bit of equipment!
- Now back at motel where batteries are ready. Will, Bill, Isabel, and David are now making their way up from Vegas. Company, at last!
Evening:
- A final run to the site, and Will was able to join up! Together, we flagged all the even-numbered towers. We lost the TS-handheld connection again at about 500m while trying to flag the end of the DTS fiber. As I did last night, we left the TS in place to keep its orientation/position and will do a 2-person sighting of this last point tomorrow.
Plan:
- Blaze a track to the tower line for vehicle access
- Finish the last flagging point (just one)
- Drop the chem shelter for the DTS stuff
- Empty out the seacontainer, distributing tower sections/base plates/solar panels to their respective positions
- If time, grab TT feet and set the TT legs
- Sometime, could flag the odd-numbered towers as well (and check the tower line with a measuring tape)
Just Steve, due to several factors...
Started with a visit to Town Hall, where I got the keys to access the trailer compound and had a brief chat with the Tourism/Events Coordinator about possible outreach activities.
Stopped in to check in with Mark Pederson.
Next, flagged most trailer locations (just using measuring tape):
- Drove out to a possible site for tt. Had to "break trail" through brush the first 100m or so and there is one downslope place, but the rest was easy. After later studying the route, I think it can be improved a bit. Also, I need to move my flagged location about 50m south to be within the 50:1 glideslope
- Flagged locations for the storage seatainers, ISS1, ISFS base, and REAL trailers. I felt pretty good about all of these, including REAL.
Spent the rest of the day hassling with the Leica.
- Everything seemed okay with connections and set-up
- Placed TS exactly at the REAL lidar mirror, to establish one end of the reference line to (later) stake to.
- Started the process to locate the TS using multiple GPS pole sitings. Found that it took 2500-3000 GPS scans to get even to 0.200m accuracy (about 10min). Talking with Matt, tried to clear the ephemeris, but didn't help. Went through 4 points, and it couldn't calculate a position, tried deleting one (that it did permanently, by mistake) and could get a crude position. Decided to add more points, but still couldn't calculate TS position. With 7 points, it came up with a location, but clearly was WAY off. I had the idea that the position IDs were out of sync between the TS and the GS, but couldn't figure out how to edit this. While messing with this, managed to get out of the TS position mode and couldn't figure out how to get back in to use the points (GPCx) that it had saved. Eventually, had to start from scratch (GPDx) collecting another 3 or 4 points. It still complained about CQ being 1.3, rather than 1.0, but did manage to compute a reasonable position. This process took about 4 hours!
- I still needed a second point on the reference line, so I took the GPS pole out to the rough location of the closest end of the DTS. In the process, I oriented myself to the BLM land boundary. During the drive out to the field, I turned off the GPS and handheld device, but left the TS on (obviously!). When at the site, I couldn't connect through the long-range antenna to the TS. Drove back to the TS and could roughly site back to the pole, but "power scan" couldn't find the prism, presumably due to all of the optical shimmer (clearly obvious through the TS telescope) – at this point, it was 3:30pm. Manually siting through the telescope, it found that the distance was only about 680m. I could see the power LED on on the LR antenna, but had no indication that it was connected. Decided to shut everything down and come back in cooler conditions. To avoid the hassle of re-locating the TS, I put the bag over the head and left it in place.
Learned late in the day that the land use agreement was finally signed!
Stopped by the local urgent care, to learn about their services.
I did go back to the site at sunset, having used Google Earth to estimate the lat/lon of a few points along the reference line. Just using my phone GPS, to 12' accuracy, I flagged each of these locations. It was obvious that the shimmer was gone even then.
Plan for tomorrow:
My plan is to go back at dawn to get at least the second reference point flagged (adjusting the flags that I just installed). I'll pair the handheld with the TS when close by, then leave it on as I drive out. If this works, I'll try to flag the other points. If even one of these works, I'll go back and reposition (and resection again) the TS closer to the tower array and flag the DTS and tower locations, using offsets (2m and 6.17m, respectively) from the reference line. If not, its back to measuring tape and compass!
Also, throughout the day, receive 3 truckers to the site.
Typical sunset
My evening stroll companion!