JAFDIP

Just another frakkin day in paradise

Archives for December 2018

Making Friends With The Command Line

Working on the command line can be challenging if your only frame of reference is a touch pad, mouse or similar pointing device. In fact it can be truly frustrating when all you want to do is move around a few directories and maybe once in a while edit a file or two.

One challenge is that if you have a favorite editor that requires more mouse clicking than nano will accommodate then you may need to ensure you know the full path or at least the full command name to launch it form the command line. For this little hack we are going to edit the bash_login script and create new function to make our lives easier.

Normally, when I am on the command line I use vi, vim or nano to modify files however there are times that I know I will be cutting and pasting blocks of code between files and honestly I find that task best suited for mouse gymnastics. In these situations I would use and editor like Coda but since version 2 was released the application includes a spaced file name. This leads to a rather unfriendly command like the following:

/Applications/Coda\ 2.app my-file

I know that it’s not a lot to type but it is annoying to remember the backslash to safely escape the space. Of course I could enclose the command in double quotes but all of this extra typing gets in the way of me doing what I set out to do. Finally there is another option I could always just rename the application to Coda.app thus eliminating the space altogether but I would need to remember this every time I update the application.

Therefore, the smart play is to use built in shell magick to trick the system into doing all of this work for me. This is after all the UNIX way and honestly it is more fun. Keep in mind I am only using Coda as an example that I hope relates to your own particular need.

To start you need to edit the .bash_login file in your home directory. If one does not exist then you will need to create one. Also if you are a zsh user this hack will work for you as well so long as you place it in the right shell startup file. Simple add the following snippet of code to the bottom of the file and save. Then open a new terminal tab to test.

function coda() {
     if [[ -n $1 ]]; then
         open ${1} -a "Coda 2"
      fi
     open -a "Coda 2"
 }

Let’s take a moment to deconstruct this. Basically I have added a function named coda to my shell thus obfuscating the location and name of the actual Coda 2 application. I am launching this with the built-in Mac OS X ‘open’ command via -a flag, which coincidentally stands for application.

Now many of you who are command line savvy may be wondering why go this route at all. Why not just use the built-in editor statement like the following:

export editor=nano

Simply adding this to the .bash_login would does indeed set the default editor for the shell but as I stated earlier there are times when I want to use a more advanced editor. This is especially true if I want to cut and paste multiple blocks from multiple files.

So where does this leave us. Well if you followed the process correctly in your new terminal tab you will be able to move around the filesystem to any directory you have permission to access and locate a text file to open. For instance, suppose you have a Apache config file you wish to edit you could do the following:

pushd /etc/apache2/sites-available
coda jafdip-com.conf

As you can see the file opens up in the same context as other files already active in the editor, making it simple to cut and paste blocks from one to the other. Obviously one could get carried away but the idea is to make simple single function commands that are specific to a need. There are some down sides to this approach in that this new command will not work with sudo.

Hopefully you found this example of how to create your own commands helpful. I personally find this approach better than cluttering my environment with a bunch of shell scripts. However if your needs are more advance than this example then a shell script is probably the better way to go. Especially if you want to chain it to sudo.

[addthis tool=”addthis_relatedposts_inline”]

RSYNC For Code Deployments

In the past we have looked at several different deployment scenarios from sneaker net file wrangling to SFTP and even git cloning/checkouts. Today we need to look at the next level of deploying code. The next level is rsync and if you are not familiar with or never really delved into rsync then today’s the day we crack this nut open.

While you can effectively use SCP or even SFTP to move files around between hosts there are a number of limitations. For one while scripting can be done it is a bit tedious. Furthermore as with SCP and SFTP you will need to properly setup Passwordless SSH Authentication in order to use rsync for automagick code deployments. One of the big advantages taht Rsync offers is the ability to ship only the blocks of data that have actually changed. In addition it has the ability to keep the target in sync with the changes made in the source which makes it particularly well suited for code deploys.

Because the rsync man page has a huge list of options lets take a look at what a typical command might look like. We shall start by deconstructing the following filesystems backup:

rsync —partial —append —status —avzrp SRC DEST

Let’s start with the partial option. This command line switch allows you to resume failed transfers. Normally rsync will discard partially transferred files however this will flag the system to keep them which can be handy with large binaries like image, video or audio files.

The append option is NOT one I recommend for code deploys, but is fantastic for file backups. Essentially this option will append the changes to the destination file if it already exists. This can have unexpected results for code deploys.

The status or stats option simply displays a section of transfer stats that I personally find very helpful when trouble shooting deployment problems. Feel free to omit.

a   archive mode
v   verbose
z   gzip compress during transfer
r   recurse into subdirectories
p   preserve permissions

These remaining options are relatively self explanatory so there’s little need to dig in deeper. I do think it is important that we take a moment to remember that rsync offers a –dry-run option so you can test the commands before doing any irreparable damage to your system.

The append option is NOT recommended for code deploys

The following are both powerful and very dangerous. They are also essential for us to efficiently use rsync for code deployments therefore we will look at them in greater detail.

--del                   an alias for --delete-during
--delete                delete extraneous files from dest dirs
--delete-before         receiver deletes before xfer, not during
--delete-during         receiver deletes during the transfer
--delete-delay          find deletions during, delete after
--delete-after          receiver deletes after transfer, not during
--delete-excluded       also delete excluded files from dest dirs
--force-delete          force deletion of dirs even if not empty

As previously mentioned you should use the –dry-run option until you feel extremely confident that you will not break things. In addition maintaining good backups is a must.

So starting with the –delete option while it may seem obvious and even logical it is one that get misused more then not. If you delete a file from the source path then it will be deleted from the destination. This applies to directories and files equally. This makes the option a good candidate for code deployments but a bad one for filesystem backups.

Well that was the easy one as each of the remaining delete options are complex. For instance the –delete-delay will delete the files in question during the transfer but after the files are done being shipped. This is probably one of the more confusing aspects of working with rsync. In essence it stores a stack of files marked for deletion that it discovers during the transfer process and once it’s done transferring it deletes them.

Reading that I am certain you are confused as to why or how that is different from the –delete-after option. Well the –delete-after option does not begin the search for the files to delete until after the transfer is complete. This also happens on the receiver side of the equation.

Similarly the –delete-before instructs the receiver to scan for files deletions prior and then remove them prior to transferring the changes. In addition the –delete-during performs this during that actual data transfer essentially it performs a just in time operation.

The –delete-excluded option is potentially problematic for for code deploys as most files systems have a bevy of files that you want excluded from rsync process. This options instructs the receiver to analyze the –exclude option for additional items to remove from the destination. I recommend that you use this one extreme caution. For instance assume you have files like minified JavaScript and CSS in your git exclusions which is the same driver for your code deploy. Using this option means that you would deploy those minified files to the destination and then delete them.

The final option –force-delete is another that I recommend you use with extreme caution. This option has an alias –force so once again use with care. Let’s say for the sake of argument you included a file named cache in your code base under wp-content then deploy your code changes to a live WordPress installation. This option will replace the cache directory with your file and while it may not break your site completely it would render the local caching system useless thus degrading server performance.

Now that you have a basic understanding of how rsync we will in part two we will go into more detail by testing actual scripts. As with everything that is scripto-magick you need to test, test again and then test some more. There is no magickal silver bullet for efficient code deployments and your needs can change over time.

Related Content:

Rsync Logo How to setup rsyncd on Mac OS X