You have 2 free member-only stories left this month.

My PATH Variable is a Mess

How to clean up your PATH in macOS/Linux

Photo by Lili Popper on Unsplash

Introduction

When you install a program, it often asks you to add a PATH to your shell config file. I do it blindly thinking, “This tool is cool. I want to use it NOW!” After some time you find your PATH variable is a mess. A lot of duplication or triplication of PATHs. It is time to clean it up.

Check your PATH

The following command substitutes colons with new lines, sorts, counts the number of unique PATH occurrences, and displays:

$ echo $PATH | sed 's/:/\n/g' | sort | uniq -c
The part of my output from echo $PATH | sed ‘s/:/\n/g’ | sort | uniq -c. Image by Author.

Do you have any PATH more than once? If so, read on!

macOS users

macOS has the path_helper command. The path_helper is a helper for constructing the PATH environment variable.

Run man path_helper:

The output of man path_helper. Image by Author.

The default PATH and MANPATH values are in /etc/paths and /etc/manpaths. And also the path-helper reads files in the etc/paths.d and /etc/manpaths.d directories.

Let’s check the content of the /etc/paths:

$ cat /etc/paths
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

These are the default paths on macOS. If you have these in your shell config file, it will repeat them in the PATH outputs, echo $PATH.

Now let’s check the content of the /etc/manpaths:

$ cat /etc/manpaths
/usr/share/man
/usr/local/share/man

When you use the man command, it will search the above path for the man pages.

You can output PATH and MANPATH variables:

$ /usr/libexec/path_helper
My output of /usr/libexec/path_helper. Note /Library/Apple/usr/bin in the first line. Image by Author.

You can check what files you have in etc/paths.d and /etc/manpaths.d.

$ ls /etc/paths.d
100-rvictl
$ cat /etc/paths.d/100-rvictl
/Library/Apple/usr/bin
$ ls /Library/Apple/usr/bin
rvictl

It loads the 100-rvictl and it has /Library/Apple/urs/bin. My /usr/libexec/path_helper has /Library/Apple/usr/bin in the first line.

Ubuntu users

Ubuntu uses /etc/environment for the system-wide configuration and you find the default paths in this file.

ubuntu@shin-instance:/$ cat /etc/environment
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
Ubuntu output from /etc/environment. Image by Author.

ZSH users

If you are a ZSH user and there are too many paths in your ~/.zshrc file, you can add the following at the end of your ~/.zshrc:

# .zshrc
typeset -U PATH

This prevents duplicates of PATH variables.

Adding PATH and order of PATH

On UNIX, you can add multiple PATHs using colons:

PATH="/path1/to/file:/path2/to/file:/path3/to/file:$PATH"

If you want to prepend a path (before the default path) use:

PATH="/path/to/file:$PATH"

If you want to append a path (after the default path) use:

PATH="$PATH:/path/to/file"

For example, if you want to add the /usr/local/sbin path:

export PATH="/usr/local/sbin:$PATH"

If your Bash scripts are in the ~/bin directory and you want to prepend it:

export PATH="$HOME/bin:$PATH"

Don’t add $PATH before and after the path you are adding to.

How to share PATHs in .zshrc, .bashrc, .bash_profile

You may want to share the same PATH in the ZSH and Bash shell. One way is to source the ~/.bashrc in your ~/.zshrc file.

# In the ~/.zshrc, sourcing ~/.bashrc 
. ~/.bashrc

If you use iTerm2, iTerm2 has “Login Shell” under Preference > Profiles > General > Command. It will look for ~/.bash_profile before ~/.bashrc.

The iTerm2 uses Login Shell. Image by Author.

This means you need to source ~/.bashrc in ~/bash_profile:

# In ~/.bash_profile
. ~/.bashrc

Then in ~/.bashrc, you add the PATH you want to use in the ZSH and Bash shell. Add PATH or source files that contain PATH.

# I have all the GNU/Linux PATH in ~/.macgnu file
. ~/.macgnu
export PATH="/path/to/file1:$PATH"
# Terminal color
export PS1="\[\033[01;32m\]\w:$ \[\033[00m\]"
# not display errors when using apropos and whatis
alias apropos="apropos 2>/dev/null"
alias whatis="whatis 2>/dev/null"
...

If you are sourcing other PATH files, it can be a cause of repeated PATH. You need to check carefully if you are sourcing the same PATH. Watch out not to add the same PATH in the different shell config files.

Conclusion

If you declare a PATH variable without appending nor prepending, you are not loading PATH from /etc/paths. This will be a cause of the problem in a long run. You always append or prepend new paths to $PATH variable in your config file, such as ~/.zshrc, ~/bashrc , etc. I recommend occasional PATH check-ups for a healthy Shell life. Writing common factors in the ~/.bashrc and source it in the ~/.zshrc is a good idea but be careful about duplication of PATHs. If you use ZSH, adding typeset -U PATH prevents duplicates of PATH variables.

Newsletter sign-up link.

Tools and tips for programmers. Math teacher, programmer, husband, father, Japanese. https://bit.ly/3nEaAfr.

Thanks to Anupam Chugh. 

Sign up for The Variable

Every Thursday, the Variable delivers the very best of Towards Data Science: from hands-on tutorials and cutting-edge research to original features you don't want to miss. Take a look.

You'll need to sign in or create an account to receive this newsletter.

Your home for data science. A Medium publication sharing concepts, ideas and codes.

Give wings to your Excel with Python Xlwings

Learn how to fly with Xlwings (Bavarian sky, image by author)

Motivation:

Oftentimes we conduct impressive Python analysis, but all that our colleagues are asking for is to receive the final results within Excel. Many of our colleagues do not want to work with Jupyter Notebook, Python scripts and the like. They just want to stick to their beloved spreadsheet tool (like Microsoft Excel, to call it by name). In “Excel-Python App for Non-Pythonists” we learned how to build a complete Excel-Python App for those colleagues. In this post we will inspect Xlwings in more depth. Xlwings gives us the power to develop interactive applications using Excel spreadsheets as the GUI (graphical…


Post a quick thought or a long story. It's easy and free.


The average is the most common value presented, yet often misleading

Image by Charles Deluvio on Unsplash

The average response is positive. The average is 3. The general opinion is neutral towards this topic. On average, women agree more than men. You read and hear such statements all the time. What kind of information is it?

The average can be multiple things. It is a typical value in a set of data (as Oxford Languages defines). Using statistical words, the average can be:

  1. Mode is a number that occurs the most often in a data set, meaning an option that is picked the most.
  2. Median divides your data set in the middle when you sort your values…


It’s going to take another step change in analytics capability to realize the trillions of dollars in value being tied to AI

Most of the analysis of AI Value Generation takes it for granted that AI will cross a pretty big chasm. A few years in, the technology has advanced but the techniques for using it are still lacking. Image by Riette Salzmann from Pixabay

Where does AI Value come from?

Recently, the McKinsey Global Institute predicted that AI would generate at least $3.5 Trillion in economic impact by 2022 by valuing a set of 120 AI use cases across 12 industries. Based on my analysis, these use cases focus primarily on problems requiring prescriptive analytics, where the AI must make recommendations on what to do. Prescriptive analytics grew from a progression from Descriptive to Diagnostic to Predictive analytics, where each synthesizes many instances of the prior form to solve a harder problem.

Classifying Problems using the Cynefin Framework


Photo by Chaitanya Tvs on Unsplash

Various still and animated visualizations made from the digits of Pi

Note 1: The graphics are rendered in high resolution to be viewed clearly. Please give some time for the graphics to load.

Note 2: The high resolution pdf version of these graphics can be downloaded from www.pi.aqeel-anwar.com

Note 3: More visualizations will be added to this article over time.

Pi is the ratio of a circle’s circumference to its diameter and is one of the most important numbers in our universe. It’s a non-terminating, non-repeating decimal which means that no group of digits repeat endlessly. Using Google cloud, Emma Haraku set the record of most digits of pi calculated to…


Václav Pluhař via Unsplash

Style Transfer using Pytorch

Neural-style, or Neural-Transfer, allows reproducing a given image with a new artistic style. Here I introduce the Neural-Style algorithm proposed by Leon A. Gatys, Alexander S. Ecker and Matthias Bethge. The algorithm receives a style-image, a content-image and an input image, which can be either an empty white image or a copy of a content-image. Thus, it changes the input image to make it resemble the content of the latter one and the style of the former one. Before I start I would like to thank Alexis Jacq for his article.

Principle

Although the name might make you think that it…