Endless Upgrade Notes for MacOS 10.15.6 Catalina: making gfortran work

 From August 18, 2020

I was using the Anaconda version of gfortran, and encountering problems with runtime libraries.  I had used the standard MacOS C-compiler clang as an alias for my gcc, and clang lacks fortran support.   

I switched to gcc-10 and gfortran-10, as of this date.  The runtime library problems I had yesterday afternoon induced me to install a fresh compiler via the Fink project.  http://www.finkproject.org   I am still checking that things are working correctly -- Catalina messed up more of my computing environment than I knew.

some info on this issue from a real programmer is here

https://solarianprogrammer.com/2019/10/12/compiling-gcc-macos/

And my commands were

fink selfupdate

fink install gcc10

The fink command compiled the compilers from scratch, so it too a few hours on a 2019 MacBook Pro.  Dont try this just before you need to carry your laptop someplace.

Upgrade to MacOS 10.15.6 Catalina, Even More Bookkeeping Notes

When trying to install SAC, the Seismic Analysis Code, in MacOS 10.15.6 Catalina, I encountered some problems.  The MacOS upgrade fences off a number of legacy libraries and tools into the Relocated Items folder on the Desktop.  If something stopped working after the upgrade, I discovered that some essential file had been "relocated" to a place on my disk where the application or compiler could not find it.  For SAC, the executable could not find X11 runtime libraries for plotting.  Rather than re-request the software package from IRIS, I decided to recompile the application from the sac-101.6a-source.tar file that I had thankfully never deleted from my Utilities folder.  It is the same version of SAC that I have had from 2018, but I desired a clean (or cleaner) install.  

The SAC compilation is guided by the file Readme.buildsac within the sac-101.6a tarfile, a file unchanged from 2013, but two hurdles were not anticipated in this file.  First, the *.h header files for compilation are placed by Catalina in an obscure place, 

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include

which I had placed into my CPATH environmental variable, and 

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib

which I had placed into my LIBRARY_PATH environmental variable.  The compilation later halted when it could not find the X11 libraries.  The X11 libraries were actually sitting at several locations in my computer, but not where the  SAC Makefile looked for them.  Many software packages install their own versions of X11.  Rather that setting up some Rube-Goldberg linkages that could be broken later inadvertently, I decided to re-install the X11 libraries from scratch by downloading XQuartz, even though I had the latest version already installed, and I could already use XQuartz to open and execute command lines within X-windows.  This gambit succeeded, and the SAC makefile could advance beyond its loading of the X11 libraries.

There was one more Red Sea to part before I could reach the promised land of SAC functionality.  An obscure command within a header file crashed the makefile. The error was

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h:33:18: error: missing binary operator before token "("

 The offending passage looked like this:

# if __has_include (<AvailabilityInternalPrivate.h>) # include <AvailabilityInternalPrivate.h> # endif

This code snippet looks for an auxiliary header file, and inserts it, if it can find it. Easy, right? Not easy. Some Google-sleuthing on the command "__has_include" revealed that not all versions of the gcc compiler have this command in their toolbox. My SAC makefile was crashing on not finding "__has_include"! The solution was to wrap the existing code with another IF statement:

#if defined __has_include # if __has_include (< AvailabilityInternalPrivate.h >) # include < AvailabilityInternalPrivate.h > # endif #endif

This way the Apple SDK checks to see if the C-preprocessor command actually exists before it tries to use it. Thankfully, my SDK does not have AvailabilityInternalPrivate.h, so I am not missing it.

I had to make this correction twice, in SDK files that could only be edited by root.

Upgrade to MacOS 10.15.6 Catalina, More Bookkeeping Notes

./configure --prefix=/opt/sac

On MacOS Catalina 10.15.6, with Xcode Version 11.6  I also needed to update the library path in my .zshrc (the MacOSX.sdk paths are new):

export CPATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include:/opt/local/include'

export LIBRARY_PATH='/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib'

in the  .cshrc file, I added 

setenv CPATH /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include:/opt/local/include

setenv LIBRARY_PATH /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib:/opt/local/lib

but there are more things to change, before sac will compile and run ., . .. 

Upgrade to MacOS 10.15.6 (Catalina), Some Bookkeeping Notes

Sportsfans,

After upgrading my Macs to MacOS X 10.15.6 Catalina last weekend, a number of compiling links broke and also some third-party software was "relocated".  Because I had delayed upgrading the MacOS from Mojave for over 9 months, the third-party software had adjusted to the new situation.  I downloaded a fresh version of "fink" (rev 0.45.1)   and installed it in the directory /opt/sw, because its old location /sw had been expunged by Apple's update.  This required some changes in my .cshrc file, so that I could point to the correct directories.  I had to upgrade XCode upward three versions, from 10.3 to 10.6, install the Xcode command tools, selfupdate fink, and then run an endless installation procedure to install my favorite X11 text editor nedit.    Took some time, but it all worked.

The older gcc5 compilers in fink were relocated by the MacOS upgrade and had been upgraded to gcc10 in fink.  However, I discovered that my fortran codes were compiling just fine, which meant that they were not using the gfortran compiler in fink.  I found the working gfortran compiler in Anaconda, /Users/jjpark/anaconda/bin/gfortran, so I dropped my .cshrc  pointers to the fink directories for gcc and let them utilize the Anaconda gcc compiler.  The X11 libraries also moved, into /opt, but changing the links fixed that.

So far, so good.  Crossed fingers.

Fearless Leader

Running R in Jupyter Notebooks within Anaconda

Running R in Jupyter Notebooks within Anaconda


### you can update packages in the GUI Anaconda Navigator, and create special environment to run R notebooks
#
### to run R in Jupyter notebooks, the Anaconda manual pages suggest that you create an R environment within Anaconda
### see https://docs.anaconda.com/anaconda/navigator/tutorials/r-lang/
### the recommended steps involve clicking the "Environments" tab in the left margin of the Navigator
### click "Create" at the bottom of the second Navigator column, under the list of existing environments
### give your new R environment a recognizable name, such as R-Land or r-tutorial
### click the python and R boxes to include both packages in the environment
### wait a bit until the Create button becomes live in the dialogue box.
### click "create" and go get a cup of coffee.
### When you wish to run or to write an R-based Jupyter notebook, click on this new environment,
### then follow the menu to "Open With Jupyter Notebook"

Running ObsPy in Jupyter Notebooks

Following the guide at

https://seismo.berkeley.edu/wiki_cider/images/1/10/Waite_cider_2019_tutorial.pdf

one sets up the ObsPy environment in Anaconda with the packages needed to make the notebooks go.

To review, setting up the ObsPy environment may have been done already, but if not, FIRST run these commands in a terminal window:

conda config --add channels conda-forge

• The above command sets conda to install packages from a certain place (conda-forge.org). The next command creates an environment called obspy (but you can call it just about anything) with python version 3.7.

conda create -n obspy python=3.7

• Next you will activate that newly created environment, in my installation I use "source" rather than "conda" for the activate command

source activate obspy

• You will notice that the command line prompt now has the name of the environment in parentheses (obspy). Now install obspy, and all the packages it depends on:

conda install obspy

conda will look for the obspy package at conda-forge, download it, and install it. You will be prompted a couple times for confirmation of download/install.

You have installed the obspy package, but there are a few more things that are useful to add. This one adds functionality to jupyter notebook to allow for environment switching

conda install nb_conda

• This should already be installed, but check.  In my case I had already installed basemap in my obspy environment

conda install basemap

• This adds more plotting functions.

conda install ipympl

• And, finally

 conda install jupyter

Using jupyter notebook

• Jupyter notebook is a web-browser tool that allows you to create python scripts and run then in an interactive mode. It a bit easier to use than the command line in python so lets move to that

1. open terminal/xterm (osx or linux)
2. activate the obspy environment on the command line
3. then start jupyter notebook from the command line
4. This will open a new browser window with the jupyter notebook

Now you can create a new notebook or an existing one (they have file extensions (.ipynb for ipython notebook)

cd Desktop/RData/ObsPy
source activate obspy
jupyter notebook

At this point you can cut and paste python commands into the notebook boxes and execute them one by one.  The notebook can be saved as an *.ipynb file for later use.  Be careful which directory you are in, or else you will be storing notebook files in your home directory.

C-compilers that no longer work, because they cannot find stdio.h

In the gcc and cc compilers on my laptop, the stdio.h is sought in the directory /usr/include, which in traditional UNIX/LINUX/MacOS installations has contained a boatload of header files (*.h).  Apple has eliminated this directory.  After much searching on message boards, I found this directory on my laptop:

/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include

I copied the directory into the /usr directory

sudo cp -r /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include /usr

in order to write into a root-owned directory.  Once in place, and after refreshing the shell, I avoided the stdio.h error when compiling a sample C subroutine.  I get a number of other warnings, so Im not eager to write more C code in the future, but the command cc -c refft.c produces an object file refft.o that might work in your application.

Installing ObsPy 1.1.1 in J.Park's MacOS 10.14.6 (Mojave) environment via Anaconda

Installing ObsPy 1.1.1 in J.Park's MacOS 10.14.6 (Mojave) environment via Anaconda

Restarting ObsPy after a hiatus requires that one updates all the packages.  There are many websites out there to advise you, but I didn't find one whose advice did the job, exactly.  So I will report my experience updating to Obspy 1.1.1.

I had Anaconda installed, so the best direction I found was within the Github web pages

https://github.com/obspy/obspy/wiki/Installation-via-Anaconda

but my upgrade/installation deviated from their recipe.  I had to update my version of conda, the program that installs packages within Anaconda.  Follow the web pages on that.  From a terminal window. follow the advice from the GitHub pages.  Add the condo-forge channel to enable downloading software packages.  From the pages, I cut and pasted thusly:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$ conda config --add channels conda-forge

• We strongly recommend to work with separate Anaconda environments, and especially not use the special base environment (formerly named root on conda versions <4 .4="" all="" code="" environment="" for="" is="" nbsp="" style="background-color: rgba(27, 31, 35, 0.0470588); border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 13.600000381469727px; margin: 0px; padding: 0.2em 0.4em;" this="" used="">conda

 commands and environment manipulations, package installations etc.) for anything besides updating the conda package itself (if necessary):

$ conda create -n obspy python=3.7   # or e.g. python=2.7
$ conda activate obspy  # this command used to be 'source activate obspy' on older conda versions < 4.4
(obspy) $ 

• Install pre-compiled ObsPy conda package from Anaconda cloud with:

(obspy) $ conda install obspy

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Two caveats on this recipe.  First, the ObsPy documentation suggests that python=3.7 is not supported by ObsPy versions up to 1.1.1, so I replaced "3.7" with "3.6" for the python version.  Next, the command "conda activate obspy" does not work for me.  I tried the recommended command "conda init bash" to make these commands work, but my computer protested a mis-application of the init-command.  Instead I used the older command for activating the obspy channel: "source activate obspy"  After the obspy channel was activated, I ran "conda install obspy" successfully.

NOTE:  I had installed an earlier version of obspy in Fall 2017, but I was unable to update it with simple commands like "conda update obspy".  Perhaps there were too many packages to update cleanly after 2+ years of neglect.  So I removed the obspy package and did a fresh install "conda install obspy".  Getting the sequence of commands to enact the procedure of updating my installation, then removing it and starting fresh (though without re-installing Anaconda!) took me several days of trouble-shooting.  I replicated the procedure, without the trial-and-error detours, on a desktop iMac.  The procedure was first to update conda, then delete the obspy package, then activate the obspy channel for working with obspy.  Among other features, the obspy channel deprecates the version of python to v3.6.7 to avoid conflicts with python v3.7. Then install the obspy package from scratch.