Upgrade to MacOS 10.15.6 Catalina, More Bookkeeping Notes

./configure --prefix=/opt/sac

On MacOS Catalina 10.15.6, with Xcode Version 11.6  I also needed to update the library path in my .zshrc (the MacOSX.sdk paths are new):

export CPATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include:/opt/local/include'

export LIBRARY_PATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib'

in the  .cshrc file, I added 

setenv CPATH /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include:/opt/local/include

setenv LIBRARY_PATH /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib

but there are more things to change, before sac will compile and run ., . .. 

Upgrade to MacOS 10.15.6 (Catalina), Some Bookkeeping Notes

Sportsfans,

After upgrading my Macs to MacOS X 10.15.6 Catalina last weekend, a number of compiling links broke and also some third-party software was "relocated".  Because I had delayed upgrading the MacOS from Mojave for over 9 months, the third-party software had adjusted to the new situation.  I downloaded a fresh version of "fink" (rev 0.45.1)   and installed it in the directory /opt/sw, because its old location /sw had been expunged by Apple's update.  This required some changes in my .cshrc file, so that I could point to the correct directories.  I had to upgrade XCode upward three versions, from 10.3 to 10.6, install the Xcode command tools, selfupdate fink, and then run an endless installation procedure to install my favorite X11 text editor nedit.    Took some time, but it all worked.

The older gcc5 compilers in fink were relocated by the MacOS upgrade and had been upgraded to gcc10 in fink.  However, I discovered that my fortran codes were compiling just fine, which meant that they were not using the gfortran compiler in fink.  I found the working gfortran compiler in Anaconda, /Users/jjpark/anaconda/bin/gfortran, so I dropped my .cshrc  pointers to the fink directories for gcc and let them utilize the Anaconda gcc compiler.  The X11 libraries also moved, into /opt, but changing the links fixed that.

So far, so good.  Crossed fingers.

Fearless Leader

Running R in Jupyter Notebooks within Anaconda

Running R in Jupyter Notebooks within Anaconda


### you can update packages in the GUI Anaconda Navigator, and create special environment to run R notebooks
#
### to run R in Jupyter notebooks, the Anaconda manual pages suggest that you create an R environment within Anaconda
### see https://docs.anaconda.com/anaconda/navigator/tutorials/r-lang/
### the recommended steps involve clicking the "Environments" tab in the left margin of the Navigator
### click "Create" at the bottom of the second Navigator column, under the list of existing environments
### give your new R environment a recognizable name, such as R-Land or r-tutorial
### click the python and R boxes to include both packages in the environment
### wait a bit until the Create button becomes live in the dialogue box.
### click "create" and go get a cup of coffee.
### When you wish to run or to write an R-based Jupyter notebook, click on this new environment,
### then follow the menu to "Open With Jupyter Notebook"

Running ObsPy in Jupyter Notebooks

Following the guide at

https://seismo.berkeley.edu/wiki_cider/images/1/10/Waite_cider_2019_tutorial.pdf

one sets up the ObsPy environment in Anaconda with the packages needed to make the notebooks go.

To review, setting up the ObsPy environment may have been done already, but if not, FIRST run these commands in a terminal window:

conda config --add channels conda-forge

• The above command sets conda to install packages from a certain place (conda-forge.org). The next command creates an environment called obspy (but you can call it just about anything) with python version 3.7.

conda create -n obspy python=3.7

• Next you will activate that newly created environment, in my installation I use "source" rather than "conda" for the activate command

source activate obspy

• You will notice that the command line prompt now has the name of the environment in parentheses (obspy). Now install obspy, and all the packages it depends on:

conda install obspy

conda will look for the obspy package at conda-forge, download it, and install it. You will be prompted a couple times for confirmation of download/install.

You have installed the obspy package, but there are a few more things that are useful to add. This one adds functionality to jupyter notebook to allow for environment switching

conda install nb_conda

• This should already be installed, but check.  In my case I had already installed basemap in my obspy environment

conda install basemap

• This adds more plotting functions.

conda install ipympl

• And, finally

 conda install jupyter

Using jupyter notebook

• Jupyter notebook is a web-browser tool that allows you to create python scripts and run then in an interactive mode. It a bit easier to use than the command line in python so lets move to that

1. open terminal/xterm (osx or linux)
2. activate the obspy environment on the command line
3. then start jupyter notebook from the command line
4. This will open a new browser window with the jupyter notebook

Now you can create a new notebook or an existing one (they have file extensions (.ipynb for ipython notebook)

cd Desktop/RData/ObsPy
source activate obspy
jupyter notebook

At this point you can cut and paste python commands into the notebook boxes and execute them one by one.  The notebook can be saved as an *.ipynb file for later use.  Be careful which directory you are in, or else you will be storing notebook files in your home directory.

C-compilers that no longer work, because they cannot find stdio.h

In the gcc and cc compilers on my laptop, the stdio.h is sought in the directory /usr/include, which in traditional UNIX/LINUX/MacOS installations has contained a boatload of header files (*.h).  Apple has eliminated this directory.  After much searching on message boards, I found this directory on my laptop:

/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include

I copied the directory into the /usr directory

sudo cp -r /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include /usr

in order to write into a root-owned directory.  Once in place, and after refreshing the shell, I avoided the stdio.h error when compiling a sample C subroutine.  I get a number of other warnings, so Im not eager to write more C code in the future, but the command cc -c refft.c produces an object file refft.o that might work in your application.

Installing ObsPy 1.1.1 in J.Park's MacOS 10.14.6 (Mojave) environment via Anaconda

Installing ObsPy 1.1.1 in J.Park's MacOS 10.14.6 (Mojave) environment via Anaconda

Restarting ObsPy after a hiatus requires that one updates all the packages.  There are many websites out there to advise you, but I didn't find one whose advice did the job, exactly.  So I will report my experience updating to Obspy 1.1.1.

I had Anaconda installed, so the best direction I found was within the Github web pages

https://github.com/obspy/obspy/wiki/Installation-via-Anaconda

but my upgrade/installation deviated from their recipe.  I had to update my version of conda, the program that installs packages within Anaconda.  Follow the web pages on that.  From a terminal window. follow the advice from the GitHub pages.  Add the condo-forge channel to enable downloading software packages.  From the pages, I cut and pasted thusly:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$ conda config --add channels conda-forge

• We strongly recommend to work with separate Anaconda environments, and especially not use the special base environment (formerly named root on conda versions <4 .4="" all="" code="" environment="" for="" is="" nbsp="" style="background-color: rgba(27, 31, 35, 0.0470588); border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 13.600000381469727px; margin: 0px; padding: 0.2em 0.4em;" this="" used="">conda

 commands and environment manipulations, package installations etc.) for anything besides updating the conda package itself (if necessary):

$ conda create -n obspy python=3.7   # or e.g. python=2.7
$ conda activate obspy  # this command used to be 'source activate obspy' on older conda versions < 4.4
(obspy) $ 

• Install pre-compiled ObsPy conda package from Anaconda cloud with:

(obspy) $ conda install obspy

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Two caveats on this recipe.  First, the ObsPy documentation suggests that python=3.7 is not supported by ObsPy versions up to 1.1.1, so I replaced "3.7" with "3.6" for the python version.  Next, the command "conda activate obspy" does not work for me.  I tried the recommended command "conda init bash" to make these commands work, but my computer protested a mis-application of the init-command.  Instead I used the older command for activating the obspy channel: "source activate obspy"  After the obspy channel was activated, I ran "conda install obspy" successfully.

NOTE:  I had installed an earlier version of obspy in Fall 2017, but I was unable to update it with simple commands like "conda update obspy".  Perhaps there were too many packages to update cleanly after 2+ years of neglect.  So I removed the obspy package and did a fresh install "conda install obspy".  Getting the sequence of commands to enact the procedure of updating my installation, then removing it and starting fresh (though without re-installing Anaconda!) took me several days of trouble-shooting.  I replicated the procedure, without the trial-and-error detours, on a desktop iMac.  The procedure was first to update conda, then delete the obspy package, then activate the obspy channel for working with obspy.  Among other features, the obspy channel deprecates the version of python to v3.6.7 to avoid conflicts with python v3.7. Then install the obspy package from scratch.  

Preparing IRIS data files for analysis by R, Take 1

Preparing IRIS data files for analysis by R: converting 2-column ascii to CSV files.  Rdseed and the IRIS DMS produce datafiles in  2-column ASCII format with *.ascii filename suffixes.  These files start with a one=-line header, then a UTC time and a datum in the form of instrument counts

TIMESERIES CU_BCIP_00_LHZ_M, 180001 samples,  1.0 sps, 2016-11-13T10:00:00.000000, TSPAIR, FLOAT, Counts
2016-11-13T10:00:00.000000             3130
2016-11-13T10:00:01.000000             1921

This command filters all the ascii files for a minimum size, to avoid data-window fragments

ls -l *ascii | awk ' $5>NNNN {print $9}' >! infiles

Prepare a shell command file for using awk to strip out the data column and write a CSV file.  First construct the CSV filenames from the key attributes in the IRIS filename: Station.Location.Channel.csv

# construct CSV filenames from long IRIS filenames
sed 's/\./ /g' infiles >! blob
awk '{print  $1 "."  $2 "."  $3 "."  $4 "."  $5 "."  $6 "."  $7 "."  $8 "."  $9 "."  $10 "."  $11 " "  $7 "."  $8 "."  $9 ".csv" }' blob  >! infiles1 

-> head infiles1
2016.318.10.00.00.0000_CU.ANWB.00.LHZ.M.ascii ANWB.00.LHZ.csv
2016.318.10.00.00.0000_CU.BBGH.00.LHZ.M.ascii BBGH.00.LHZ.csv
2016.318.10.00.00.0000_CU.BCIP.00.LHZ.M.ascii BCIP.00.LHZ.csv
2016.318.10.00.00.0000_CU.GRGR.00.LHZ.M.ascii GRGR.00.LHZ.csv
2016.318.10.00.00.0000_CU.GRTK.00.LHZ.M.ascii GRTK.00.LHZ.csv
2016.318.10.00.00.0000_CU.GTBY.00.LHZ.M.ascii GTBY.00.LHZ.csv

Write the shell script that runs awk once per filename, for LHZ files. 

# because I cant figure out how to add a quote to the sed output, the script 
# for generating CSV data files from LHZ ascii data requires a file edit
awk ' {print "awk NR>1 {print NR-1 \", \" $2} " $1 " >! " $2 "; wc "$2 } ' infiles1 >! blob
te blob &
#save as saveblob
source saveblob

note that the NR>1 skips the header line.  The time ordinate is indexed to the record number

-> head blob
awk NR>1 {print NR-1 ,  $2} 2016.318.10.00.00.0000_CU.ANWB.00.LHZ.M.ascii >! ANWB.00.LHZ.csv; wc ANWB.00.LHZ.csv
awk NR>1 {print NR-1 ,  $2} 2016.318.10.00.00.0000_CU.BBGH.00.LHZ.M.ascii >! BBGH.00.LHZ.csv; wc BBGH.00.LHZ.csv
awk NR>1 {print NR-1 ,  $2} 2016.318.10.00.00.0000_CU.BCIP.00.LHZ.M.ascii >! BCIP.00.LHZ.csv; wc BCIP.00.LHZ.csv

note that we edited blob --> saveblob by adding some quote marks around the awk command. 

-> head saveblob

awk 'NR>1 {print NR-1", " $2}' 2016.318.10.00.00.0000_CU.ANWB.00.LHZ.M.ascii >! ANWB.00.LHZ.csv; wc ANWB.00.LHZ.csv

awk 'NR>1 {print NR-1", " $2}' 2016.318.10.00.00.0000_CU.BBGH.00.LHZ.M.ascii >! BBGH.00.LHZ.csv; wc BBGH.00.LHZ.csv

awk 'NR>1 {print NR-1", " $2}' 2016.318.10.00.00.0000_CU.BCIP.00.LHZ.M.ascii >! BCIP.00.LHZ.csv; wc BCIP.00.LHZ.csv

Write the shell script that runs awk once per filename, for VHZ files. Note that the time ordinate is converted to ksec from seconds.  Time-step increments of 0.01 ksec = 10 seconds.

ls -l *ascii | awk ' $5>3700000 {print $9}' >! infiles
# because I cant figure out how to add a quote to the sed output, the script 
# for generating CSV data files from VHZ ascii data requires a file edit
awk ' {print "awk NR>1 {print (NR-1)/100. \", \" $2} " $1 " >! " $2 "; wc "$2 } ' infiles1 >! blob
te blob &
#save as saveblob
source saveblob