Blog

Ethernet work on tower

June 1 

Gordon and Tom worked on improving the ethernet connectivity.  Gordon opened the low DSM and discovered that the ethernet surge protectors for the connections from the mid and high DSMs were extremely hot.

12:13 Gordon pulled the fuse on the eth1 port in the mid DSM

12:17 Gordon removed the fuse on the eth1 port in the high DSM

(I removed the fuses because I didn't know which jumper controlled the output voltage on these eth ports.)

From the low DSM, Tom pinged, in succession, the high DSM, mid DSM, the router (192.169.0.5), and google.com.  In all cases, 50 pings were returned with no missing returns.  At the same time the ethernet surge protecters in the low DSM cooled appreciably.  Vmote (3,19) at the high DSM now reads a voltage of 12.22 V.

Gordon added an ethernet surge protecter to the cable between the ethernet switch and the router now at the top of the tower.

Previously we saw about 10% loss of packets on pings to high. The CBR400 router was also not stable, frequently it would stop responding to http requests. Data samples from the high DSM over the network were being discarded because the TCP connection would not keep up.

The networking to high and to the cbr400 router on the tower now seems to be much improved. It appears that the DSMs were putting a +V on the ethernet cable and the surge protectors were trying to discharge it, getting very hot, and having an adverse effect on the comm signsls.

Guy wire tensions

May 31

Kurt measured guy wire tension, first while Gordon was climbing tower, and second after Gordon was finished on the tower:

Direction

height

number

first

second

6/13/2013(cg)

6/21/13

7/11/2013(cg/jm)

Direction

first

second

5/13/2013(cg)

6/21/13

7/11/2013(cg/jm)

Direction

first

second

6/13/2013(cg)

6/21/13

7/11/2013(cg/jm)

South

low

1

400

410

375

400

405

NNE

490

500

465

460

470

WNW

440

470

425

410

440

 

 

2

350

345

330

340

345

 

480

500

475

470

495

 

510

540

525

470

550

 

 

3

400

400

375

340

345

 

540

580

510

500

525

 

490

500

500

470

500

 

 

4

425

425

350

450

460

 

390

380

340

350

360

 

360

370

325

325

320

 

 

5

370

360

340

375

410

 

410

390

400

410

405

 

260

270

295*

350#

270

 

 

6

350

350

340

440

325

 

370

360

330

420

390

 

370

350

310

350

355

*This turnbuckle is maxed out.  You will have to pull a link.

#3 links taken in on 6/21.

Tower service visit

May 30, ~ 10:30 - 1:30 CDT

Gordon and Tom serviced the sensors and DSMs on the profile tower:

44m: Cleaned Licor

38m: Cleaned EC150

~36m: Cleaned K&Z 4-component net radiometer

32m: Cleaned Licor; grounded DSM; cleaned PAR; Tsfc @ -1.5 degC

26m: Cleaned Licor, PAR, and wetness sensor (splattered w/ bird crap)

20m: Licor and PAR cleaned; PAR covered in bird crap

14m: Grounded DSM and fixed usbdisk; cleaned PAR, KH2O (0.35V) and wetness sensor; no Tsfc

8m: Cleaned PAR, Licor, wetness sensor; typical splatter of bird crap

2m: Cleaned KH2O (0.35V)

There were questions about the PAR at 32 m (1036 W/m^2) and 14m (50 W/m^2), but these seem to fit into the profile of the other PAR's.

Cellular network

Gordon, May 29

The cradlepoint CBR400 router and the Verizon modems have been working, but have some issues.

On May 27 and 28, the router would go offline periodically, and not be pingable from the local network at address 192.168.0.5 (cbr400). From the web interface (http://cbr400), System Settings -> System Control -> Advanced, I turned off "Enable Watchdog Reboot" to see if that would help. The CBR400 stayed up all night. When we arrived at the trailer (9:20) it was pingable, but the web interface would not respond.

The 3G modem has been working fine. On the web interface for the CBR400, the signal strength is typically -75 dBm, with 4 bars.

The 4G modem does not always connect. The status page shows a signal level of -79 to -84 dBm, which I presume is acceptable.

The CBR400 sends log messages to the syslog on "flux". The 4G modem is known as "WAN:9d7c85c4". The logs show "carrier reject", and that it reverts back to 1XRTT (2G):

fgrep CBR /var/log/messages | fgrep 5c4

May 28 16:45:12 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4: Connection Change : None -> EVDO, %
May 28 16:45:12 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4: Connect Event: carrier reject
May 28 16:45:12 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4.CPPM: state=connecting  result=carrier reject
May 28 16:45:19 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4: Connect Event: carrier reject
May 28 16:45:19 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4.CPPM: state=connecting  result=carrier reject
May 28 16:45:44 cbr400.isfs.ucar.edu (CBR400-afa) WAN:9d7c85c4: Connection Change : EVDO -> 1XRTT, %

I plan to contact Verizon and see if they can help from their end concerning the 4G modem.

Morning of May 29:
Upgraded firmware on CBR400 to version 4.3.0
Speed tests: http://www.dslreports.com/speedtest?flash=1

state

download

upload

latency

3G connected (EVDO), 4G disabled

413,444

380,309

121,107

4G connected (1xRTT), 3G disabled

58

127

485

Until we can get the 4G modem to do better than 1xRTT, we'll use the 3G and disable the 4G.

Here is the display of Status -> Internet Connections -> "Modem: Novatel Wireless CDMA" with an EVDO connection:

Here is the display of Status -> Internet Connections -> "Modem: Novatel Wireless 4G", in the disabled state:

May 30, Gordon
The cellular worked quite well yesterday, but not so hot today, frequent disconnects. I tried enabling the watchdog reset on the CBR but then disabled it again after several hours since it didn't help. The router_check.sh script on the "high" dsm is our own watchdog reset.

While on the tower, I checked the antenna cable connections at the modems, and the usb connections at the hug. All looked good.

TODO: look at the logs. Perhaps the signal levels were lower today? Contact verizon about 4G.

WIFI up

Gordon, May 28

AP24 is on a pipe at the NE corner of the base trailer.

eante (192.168.0.80) is mounted on a T-post at pond DSM (192.168.0.101)

Gary setup the etherants at the sodar and lidar.

eantf (192.168.0.81) is at the sodar (192.168.0.91)

eantg (192.168.0.82) is at the lidar (192.168.0.92). There is also a camera at the lidar (lidarcam=192.168.0.253).

All links OK.

To view the signal levels, point a browser to eante, f or g, login to admin, usual password about sonics.

To view the signal levels on the ap24, point a browser to ap24, admin, sonics. Click on "RegTable" link on the left-hand side, or you can ssh to it and view the registration table:

"int wire reg p" is shorthand for "interface wireless registration-table print" (or use TABS to expand commands):

ssh admin@ap24
admin@192.168.0.70's password:
...
[admin@Alico] > int wire reg p
 # INTERFACE     RADIO-NAME       MAC-ADDRESS       AP  SIGNAL... TX-RATE UPTIME
 0 wlan1-Int-Ant                  00:20:F6:05:24:7F no  -36dBm... 11Mbps  3h52m55s
 1 wlan1-Int-Ant                  00:20:F6:05:24:5A no  -32dBm... 11Mbps  2h4m56s
 2 wlan1-Int-Ant                  00:20:F6:05:1E:D5 no  -36dBm... 11Mbps  1h8m14s

Gordon, May 28

From candlewood this morning around 6:45 was able to ping isfs1.dyndns.org. Later, around 7:15 was not able to ping.

On arrival at base, could not ping tower DSMs. SDF was on, but no RXF on fiber transceiver.

At tower, 10:05 CDT

Battery terminals: 12.29V
Charge controller, "solar" input: 13.21
Charge controller, load: 0V
I believe the load disconnect LED was on

TRH fans were off

As I was probing the charge controller, the TRH fans started up. Perhaps the battery/controller/charged cooled a bit since the cooler lid was open?

Measured voltages again:
Battery terms: 12.69
Charger battery: 12.88
Charger solar: 13.07
Charger load: 12.84
DSM 12.16

By looking at the archive files on the DSMs, the system has been up and down this morning:

 

UTC

CDT

died

11:18

06:18

up

12:32

07:32

died

14:15

09:25

up

15:08

10:08

Left cooler lid open.

Fiber network up

The fiber network is up.

When the fiber was strung, we connected all 4 fibers, blue, green, brown and orange at each junction.

The first test was with the blue and green fibers, but the network would not come up. In the trailer, the SDF (signal detect fiber) would go green, but SDF at the tower was out.

Switched to brown and orange, and it is working now.

Base trailer: TX: orange, RX: brown
Tower: TX: brown, RX: orange

At each transceiver (media converter)

LED

color

Power

green

SDC (signal detect copper)

solid green

RXC (receive copper)

flashing green

SDF (signal detect fiber)

solid green

RXF (receive fiber)

flashing green

Switches (see BEACHON/Manitou logbook for info on these)

1

up

2

up

3

down

4

down

May 27.

Installed the soil sensors at site A (near the radiation stand) this morning.  Also buried a little more of the cables near the sensors at site B (near the rain gauge).

Heights of top of each bracket WRT tower base plate. Brackets at 13.92 and 43.86 meters had to be offset a bit due to section junctions or guy hardware.

Height (m)

2

8

13.92

20

26

32

38

43.86

Setup with Power

The whole tower is now dressed and running...kinda.  We have all tubes and instruments running (Gordon saw 100% coming in) then we shut it down due to power.  We just did a test with the batteries then we let the batteries charge off the transformer.  The TRAM Trailer does have the air conditioning working but no racks are set up yet so no power has been laid out.  Soil probe B is in w/ the tipping bucket and the radiation stand has been installed with no soils.  Fiber has been run but not hooked up.  

Kurt, Gordon and Tom will remain on site until start of project.

The radiation stand is within +/- 2 degrees of true E/W and has 4 Kipp & Zonen broadband radiometers (incoming and outgoing long and shortwave radiation), the SPN net radiometer, and a leaf wetness sensor.  Radiometer levels have their bubbles within the 'bullseyes'.

Setup TOWER Dressed

The whole tower has all instruments on it and the cables/tubes are dressed.  We have a couple of tubes that are low that need to be strung and the lower dsm to be hooked up and then we can have Gordon test all instruments while I am here.  We still do not have power to the tower but now that ACDs gear is loaded and the trailer is put into it's spot we can start hooking up transformers and run power to tower.  We can do testing off of batteries for today but we should have power going soon to the tower.  

Gordon and Tom have arrived and they found the tower.  

The inlet on the tubes are hanging just by the Licor mounting bolt.  It seems to be far enough away from sonic/licor and pressure port.  Tom will have the final say on location today.  

Things to do:

Soils/radiation stand (tom)

lower dam (chris)

tower power(kurt)

fiber (john/gordon)

pond & lidar communications (john)

Setup at SOAS

We have instrumented the tower!!!  We are all exhausted from todays setup.

We have all levels up except for the four component radiation and the canopy par wand.  These should be done tomorrow.

Trailer power is up and running.  The power to tower is not, but this is waiting on ACD due to where the trailer will reside.  

Things to do:

Four component on tower.

Par wand at canopy height.

ACD tubes installed.

soil probes installed.

power to tower.

ACD loaded.

Setup Log

We had a great day today.

We had a state forest servicer person get a rig to move our Base Trailer to the pole barn.  It is in place and set up.  WAHOO!!!!  This means no Black Warrior or Electrician needed.

The power cable has been drug through the woods and is in place.  No power yet (need breakers) but it's at least there and ready to hook up.

We also have dressed the High dsm levels and looks great.  

Soil and radiation spots have been picked but not installed.  Waiting on tower to be dressed while Ned is here.  

TRAM trailer has been moved to accommodate ACD.  We are having them directly transfer equipment or forklift rental.  I have been talking with Peter and Andrew on this transfer.

Things to do tomorrow:

Dress the rest of the levels on the tower.

Hook up power to trailer and site.

Communications to Pond and LIDAR.

Soils installation

Help ACD with equipment transfer.

GO TEAM!

Setup Update

May 22, 2013

At setup in Marion, AL.  We have re-plumed the tower using a plum-bob for the first three guy points.  Then we had line of site to the tower and used the theodolites for the remaining upper guy points.  The south, outer, guy anchor is fully tensioned and has moved an inch.  It has pulled towers the tower.  The ground is somewhat moist.  Since this anchor is in the tall grass, that seems to keep the ground more wet than the exposed ground near it.  There is still a puddle that is between the anchor and the bridge (going to tower).

The TRAM trailer is fixed.  It was the leaf spring on the right side.  This had broke, making the axle rest on the frame.  Since it was being driven and not seen, the axle then moved back and making the tire rub the wheel well, putting a gouge the tire on the tread near the outside.  The spring and brackets were all replaced and the trailer is now in position.  It is between the outer anchor and bridge.

Brackets are hung but no instruments.

Cable/fiber line has been flagged and it will be pulled the next day.  We have borrowed the AABC's front loader and it's suspending the spool of 1800' wire.  We will just take turns pulling cable through the woods.  It looks like we have long enough cable.

Ned has an area for the dark horse but has not committed to putting soils in yet.

Pond site is up and running.  No communication yet.  Forgot a bracket for the antenna.  

Base trailer will be moved to the pole barn once we find a trucker.  The gov't site is great but we are getting no responses from the power company.  The pole barn site was not used when the trailer was dropped due to moist ground.  It is dry now and looks great for a rig to drive on.

45m Turbulence Tower

Height

Sonic

TRH

Scalar

Radiation

Inlet

Other

44m

1122

030(022)

Li1163

IRT3389-1

 

Pnano-123997

38m

1244

026(029)

EC1313

CNR4-090020, Lwet

 

 

32m

0672

009(065)

1164

PAR 6951, IRT3389-3, Lwet

 

Pnano-123996

26m

0673

007(053)

1167

PAR1892, IRT3577-2, Lwet

 

 

20m

1121

29(051)

1166

PAR1893, IRT3389-1, Lwet

 

Pnano-123998

14m

0671 (0800)

40(030)

krypton 1101

PAR1529, IRT4078-3, Lwet

 

 

8m

0800 (0538)

60(041)

0813

PAR1530, IRT3577-4, Lwet

 

Pnan-122850

2m

1119

046(005)

krypton 1392

 

 

 

0m

-----

-----------

---------

IN:050824/940185
DOWN:040740/940187

 

Soil1:ECHO001,H943146,TP200589,TS013.
Soil2:ECHO012,H983211,TP200239,TS012.
Tipping Bucket 12.
Sunshine ID:  465.

For TRH number meaning:  Sensor number(fan ID)

ISS Met. Tower (POND)

Height

Sonic

TRH

Scalar

Radiation

Precip

Other

3m

----------

--------

---------

Sunshine:837
EP16107

---------

LeafW

2m

WXT520:
H30030018

027(035)

---------

---------------------

WXT520:

Pressure(WXT520)

1m

----------

--------

---------

---------------------

tipping: 12-466
OTT: 7021040941

-------------

WXT Settings:

0RU,R=00000000&10010000,I=1,U=M,S=M,M=C,Z=M,X=10000,Y=100

0SU,R=00000000&00100000,I=1,S=N,H=N

0WU,R=00000000&01001000,I=1,A=1,G=1,U=M,D=0,N=W,F=4

0TU,R=00000000&11010000,I=1,P=B,T=C,N=T

0XU,A=0,M=A,T=0,C=2,I=60,B=19200,D=8,P=N,S=1,L=25,N=WXT520,V=2.21

0R0,Dm=053D,Sm=0.1M,Ta=20.7C,Ua=32.6P,Pa=0.8361B,Rc=0.00M,Hc=0.0M,Vs=14.2V

0R0,Dm=087D,Sm=0.1M,Ta=20.7C,Ua=32.5P,Pa=0.8361B,Rc=0.00M,Hc=0.0M,Vs=14.2V

SPARE:

0674, 0537(Serializer),TS011, FanID(027,041), TRH056,036,015,022,044,063,048, TRH Assemblies:  062(031), 002(033), 061(032), 016(016).