== OS Change ==
Soon we'll be switching our Operating System from CentOS 7 to Rocky Linux 9.

=== Accessing the testbed ===
We have provided two new head nodes <tt>helios</tt> and <tt>clymene</tt>.

Once you have logged into beocat, you can access them with ssh. e.g. <tt>ssh helios</tt>

They will be setup much the same way as our previous head nodes were, with access to the module command. The modules we have available under Rocky Linux 9 are available as a searchable list [here https://modules.beocat.ksu.edu/rocky9/]

To submit jobs to the new operating system, we have provided a new constraint so that you are able to request the OS variant. CentOS 7 hosts have the feature <tt>os_el7</tt>, while the Rocky Linux 9 hosts provide the feature <tt>os_el9</tt>.

You would use the following in a job script to tell the scheduler that you would like the new OS:

<syntaxhighlight lang=bash>
#SBATCH -C os_el9
</syntaxhighlight>

=== Using old software ===
Below is a script that will execute a container with all of the public software we provide under CentOS 7 from the head nodes. There may be versions of GPU-related packages missing.
<syntaxhighlight lang=bash>
#!/bin/bash
# This script is a wrapper for our CentOS 7 based container.
# You would use it something like this:
# sbatch -C os_el9 /opt/beocat/containers/beocat_centos-7.9.wrapper.sh ./R-hello_world.sh

# Note that you would need to provide an appropriate path for the script to to execute
# under the contained environnment (either a full path or a relative path), and the script
# would need to be executable.

# This is meant to be a stopgap measure for those that may be reliant on older software
# that we will not or cannot provide under our new operating system.

singularity exec /opt/beocat/containers/beocat_centos-7.9.sif /bin/bash -l <<EOF
${@}
EOF
</syntaxhighlight>
If you would prefer, you could put the <tt>singularity exec</tt> lines in your script, with the commands you would like to run between the <tt><<EOF</tt> and <tt>EOF</tt> sections.

There will be no good way to utilize these tools with multi-node jobs, so it would be a good idea to migrate away from the CentOS 7 tools as soon as possible.

2023-06-05T21:03:57Z

Mozes: /* Example 2 */

'''Disclaimer:''' This is a ''very'' large topic, and much too broad to be covered on a single support page. There are many other sites (yes, entire sites) which cover the topic in more detail. We'll link so some of them below. This page is meant to be just the essentials.

== Logging in for the first time ==
To login to Beocat, you first need an "SSH Client". [[wikipedia:Secure_Shell|SSH]] (short for "secure shell") is a protocol that allows secure communication between two computers. We recommend the following.
* Windows
** [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY] is by far the most common SSH client, both for Beocat and in the world.
** [http://mobaxterm.mobatek.net/ MobaXterm] is a fairly new client with some nice features, such as being able to SCP/SFTP (see below), and running X (which isn't terribly useful on Beocat, but might be if you connect to other Linux hosts).
** [http://www.cygwin.com/ Cygwin] is for those that would rather be running Linux but are stuck on Windows. It's purely a text interface.
* Macintosh
** OS-X has SSH a built-in application called "Terminal". It's not great, but it will work for most Beocat users.
** [http://www.iterm2.com/#/section/home iTerm2] is the terminal application we prefer.
* Others
** There are [[wikipedia:Comparison_of_SSH_clients|many SSH clients]] for many different platforms available. While we don't have experience with many of these, any should be sufficient for access to Beocat.

You'll need to connect your client (via the SSH protocol, if your client allows multiple protocols) to headnode.beocat.ksu.edu.

For command-line tools, the command to connect is
ssh ''username''@headnode.beocat.ksu.edu

Your username is your [http://eid.ksu.edu K-State eID] name and the password is your eID password.

'''Note:''' When you type your password, nothing shows up on the screen, not even asterisks.

You'll know you are successfully logged in when you see a prompt that says
[''username''@''machinename'' ~]$
where ''machinename'' is the name of the machine you've logged into (currently either 'eos' or 'selene') and ''username'' is your eID username

== Transferring files (SCP or SFTP) ==
Usually, one of the first things people want to do is to transfer files into or out of Beocat. To do so, you need to use [[wikipedia:Secure_copy|SCP]] (secure copy) or [[wikipedia:SSH_File_Transfer_Protocol|SFTP]] (SSH FTP or Secure FTP). Again, there are multiple programs that do this.
* Windows
** Putty (see above) has PSCP and PSFTP programs (both are included if you run the installer). It is a command-line interface (CLI) rather than a graphical user interface (GUI).
** MobaXterm (see above) has a built-in GUI SFTP client that automatically changes the directories as you change them in your SSH session.
** [https://filezilla-project.org/ FileZilla] (client) has an easy-to-use GUI. Be sure to use 'SFTP' mode rather than 'FTP' mode.
** [http://winscp.net/eng/index.php WinSCP] is another easy-to-use GUI.
** Cygwin (see above) has CLI scp and sftp programs.
* Macintosh
** [https://filezilla-project.org/ FileZilla] is also available for OS-X.
** Within terminal or iTerm, you can use the 'scp' or 'sftp' programs.
* Linux
** FileZilla also has a GUI linux version, in addition to the CLI tools.

=== Using a Command-Line Interface (CLI) ===
You can safely ignore this section if you're using a graphical interface (GUI). We highly recommend using a GUI when first learning how to use Beocat.

First test case: transfer a file called myfile.txt in your current folder to your home directory on Beocat. For these examples, I use bold text to show what you type and plain text to show Beocat's response

Using SCP:
'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Note the colon at the end of the 'scp' line.

Using SFTP
'''sftp ''username''@headnode.beocat.ksu.edu'''
Password: '''(type your password here, it will not show any response on the screen)'''
Connected to headnode.beocat.ksu.edu.
sftp> '''put myfile.txt'''
Uploading myfile.txt to /homes/kylehutson/myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''exit'''

SFTP is interactive, so this is a two-step process. First, you connect to Beocat, then you transfer the file. As long as the system gives the <code>sftp> </code> prompt, you are in the sftp program, and you will remain there until you type 'exit'.

Second test case: transfer a file called myfile.txt in your current folder to a diretory named 'mydirectory' under your home directory on Beocat.

Here we run into one of the problems with scp - there is no easy way of creating 'mydirectory' if it doesn't already exist. If it does not already exist, you must login via ssh (as seen above) and create the directory using the 'mkdir' command (see Common Linux Commands) below.

'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:mydirectory'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

An alternative version. If the colon is immediately followed by a slash, the directory name is taken from the root, rather than your home directory. So, given that your home directory on Beocat is /homes/''username'', we could instead type
'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:/homes/''username''/mydirectory'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Using SFTP:
sftp ''username''@headnode.beocat.ksu.edu
Password: '''(type your password here, it will not show any response on the screen)'''
Connected to headnode.beocat.ksu.edu.
sftp> '''mkdir mydirectory'''
[Note, if this directory already exists, you will get the response "Couldn't create directory: Failure"]
sftp> '''cd mydirectory'''
sftp> '''put myfile.txt'''
Uploading myfile.txt to /homes/''username''/mydirectory/myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''quit'''

Third test case: copy myfile.txt from your home directory on Beocat to your current folder.

Using scp:
scp ''username''@headnode.beocat.ksu.edu:myfile.txt .
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Using SFTP:
'''sftp ''username''@headnode.beocat.ksu.edu'''
Password: '''(type your password here, it will not show any response on the screen)'''
Connected toheadnode.beocat.ksu.edu.
sftp> '''get myfile.txt'''
Fetching /homes/''username''/myfile.txt to myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''exit'''

== Basic Linux Commands ==
Again, this guide is very limited, mostly limited to directory navigation and basic file commands. [http://www.ee.surrey.ac.uk/Teaching/Unix/ Here] is a pretty decent tutorial if you want to dig deeper. If you want more, entire books have been written on the subject.

=== The Lingo ===
{| class="wikitable"
!''Term''
!''Definition''
|-
|Directory
|A "Folder" in Windows or OS-X terms. A location where files or other directories are stored. The current directory is sometimes represented as `.` and the parent directory can be referenced as `..`
|-
|Shell
|The interface or environment under which you can run commands. There is a section below on shells
|-
|SSH
|Secure Shell. A protocol that encrypts data and can give access to another system, usually by a username and password
|-
|SCP
|Secure Copy. Copying to or from a remote system using part of SSH
|-
|path
|The list of directories which are searched when you type the name of a program. There is a section below on this
|-
|ownership
|Every file and directory has an user and a group attached to it, called its owners. These affect permissions.
|-
|permissions
|The ability to read, write, and/or execute a file. Permissions are based on ownership
|-
|switches
|Modifiers to a command-line program, usually in the form of -(letter) or --``(word). Several examples are given below, such as the '-a' on the 'ls' command
|-
|pipes and redirects
|Changes the input (often called 'stdin') and/or output (often called stdout) to a program or a file
|}

=== Linux Command Line Cheat Sheet ===
{| class="wikitable"
|+File System Navigation
|-
!''Command''
!''What it does''
!''Example Usage''
!''Example Output''
|-
|pwd
|"Print working directory", Where am I now?
|<code>pwd</code>
|<code>/homes/mozes</code>
|-
|ls
|Lists files and folders
|<code>ls ~/</code>
|<code>NewFile NewFolder</code>
|-
|ls -lh
|Lists files and folders with perms size and ownership
|<code>ls -lh ~/</code>
|<code>-rw-r--r-- 1 mozes mozes_users 1 Jul 13 2011 NewFile
drwxr-xr-x 9 mozes mozes_users 9.0K Apr 12 2010 NewFolder</code>
|-
|ls -a
|Lists all files and folders
|<code>ls -a ~/</code>
|<code>. .. .bashrc .bash_profile .tcshrc NewFile NewFolder</code>
|-
|cd
|Changes directory
|<code>cd NewFolder</code>
|
|-
|cd ..
|Changes to parent directory
|<code>cd ..</code>
|
|-
|cd -
|Changes to the previous directory you were in
|<code>cd -</code>
|
|-
|cd ~
|Changes to your home directory
|<code>cd ~</code>
|
|}

{| class="wikitable"
|+Working with files
|-
!Command'
!What it does
!Example Usage'
!Example Output''
|-
|file
|Identifies the type of object a file is
|<code>file NewFile</code>
|<code>NewFile: a /usr/bin/python script, ASCII text executable</code>
|-
|cat
|Prints the contents of one or more files
|<code>cat NewFile</code>
|<code>This is line one
This is line two</code>
|-
|cp
|copy a file
|<code>cp OldFile NewFile</code>
|
|-
|cp -i
|copy a file, ask to overwrite
|<code>cp -i OldFile NewFile}</code>
|<code>overwrite NewFile? (y/n [n])</code>
|-
|cp -r
|copy a directory, including contents
|<code>cp -r OldFolder NewFolder</code>
|
|-
|mv
|move, or rename, a file
|<code>mv OldFile NewFile</code>
|
|-
|mv -i
|move, or rename, a file, ask to overwrite
|<code>mv -i OldFile NewFile</code>
|<code>overwrite NewFile? (y/n [n])</code>
|-
|rm
|remove a file
|<code>rm NewFile</code>
|
|-
|rm -i
|remove a file, ask to be sure (useful with -r)
|<code>rm -i NewFile</code>
|<code>remove NewFile? (y/n [n])</code>
|-
|rm -r
|remove a direcory and its contents
|<code>rm -r NewFolder</code>
|
|-
|mkdir
|creates a directory
|<code>mkdir TempFolder</code>
|
|-
|rmdir
|removes an empty directory
|<code>rmdir TempFolder</code>
|
|-
|touch
|creates an empty file
|<code>touch TempFile</code>
|
|}

{| class="wikitable"
|+Finding files and directories with [http://linux.die.net/man/1/find find]
|-
!''Command''
!''What it does''
!''Example Usage''
|-
| find <directory>
| finds all files and folders within <directory>
| find ~/
|-
| find <directory> -iname '<filename>'
| finds all files and directories within <directory> that match <filename>
| find ~/ -iname 'hello.qsub'
|-
| find <directory> -iname '*<partialmatch>*'
| finds all files and directories within <directory> that partially match <partialmatch>
| find ~/ -iname '*.qsub*'
|}

Other useful commands include <code>htop</code>, <code>less</code>, and <code>man</code>. <code>man</code> followed by a command name above will give you the manual page for the specified command full of many other useful options for the command. <code>htop</code> will give you an overview of the processes currently being run on the host you are connected to. <code>less</code> allows you to page through files and see their contents using <PgUp> and <PgDn>.

=== Editing Text Files ===
If you're new to Linux, the editor you will probably want to use is 'nano'. It works much the same as 'Notepad' in Windows or 'textedit' on OS-X. Note that you cannot use your mouse to change position within the document as you can with your local computer. You must use the arrow keys, instead.

So, if I wanted to edit my .bashrc (as shown below), and I was already in my home directory (see above), I would type
nano .bashrc

While in nano, there is a list of actions you can take at the bottom of the screen. <Ctrl> is represented by a caret (`^`), so to exit (labeled as `^`X at the bottom of the screen), I would type <ctrl>-x. This action prompts you whether you want to save and exit (Y), lose changes and exit (N), or cancel and go back to editing (<ctrl>-c).

If you do a significant amount of text editing in Linux, you'll probably want to switch to a more powerful editor, such as vim. The usage of vim is beyond the scope of this document. It is not at all intuitive to the beginning user, but with a little practice it becomes a much faster way of editing text files. If you're interested in using vim, [http://www.openvim.com/tutorial.html there is a nice tutorial here].

=== Shells ===
==== What is a Shell? ====
In this case, I don't believe I can do a better job explaining shells than [[wikipedia:Shell_(computing)|this]].
==== tcsh ====
Elsewhere at Kansas State University, the default Shell is set to tcsh. tcsh stands for "TENEX C SHell." It is considered a replacement for csh and uses many of the same features. If you have experience with either csh or tcsh you'll probably feel right at home. This was the default shell until July of 2013. If you had an account before then, it is probably still tcsh.

But what if you don't want or like tcsh, what can you do? Well, we have other shells available of Beocat as well.
==== bash ====
[http://www.gnu.org/software/bash/ Bash] seems to be the defacto standard shell in most Linux installs today. Bash is common and probably what most of you are used to. As of July 2013, bash is our new default shell. All new users will be set to bash initially. [https://software-carpentry.org/ Software Carpentry] teaches classes on several subjects specifically targeting researchers, including the bash shell. Their documentation is all freely available. [http://swcarpentry.github.io/shell-novice/ Here is a link to their excellent tutorial on using BASH.] Most of our documentation assumes you are using BASH.

bash configuration files:

This section gets into some minutiae with the way our job scheduler interacts with bash. If you're trying to solve a problem, read on, otherwise you can probably skip this section.

Bash has 3 user configurable configuration files, <code>~/.bashrc ~/.bash_profile</code> and <code>~/.bash_logout</code>. We'll look at the two more relevant ones <code>~/.bashrc</code> and <code>~/.bash_profile</code>

Bash has 3 ways of looking at things, '''login''', '''interactive''', or '''none'''.

Normally what happens is that shells that are '''interactive''' read <code>~/.bash_profile</code>, shells that are '''login''' read <code>~/.bashrc</code>. '''none''' shells read neither.

sbatch jobs are '''login''', srun jobs are '''login+interactive''', logging into Beocat in a way that you can enter commands is '''login+interactive'''. There are very few cases that you will get '''none'''. For any session that isn't '''interactive''', your sourced files cannot output anything to the screen, or else it can break scp or sftp file transfers.

If they are ''quiet'' statements, and you want them in all shells, you can put them in your <code>~/.bashrc</code>. If they are not ''quiet'' or they output ''anything'' to the screen, you must put them in your <code>~/.bash_profile</code>.

==== zsh ====
[http://zsh.sourceforge.net/ zsh] is an alternative to bash and tcsh. It tends to support more complex features than either of the other two while using a syntax remarkably similar to bash. Unless specifically noted, when we specify '''Change your shell to bash''', <tt>zsh</tt> should work as well.

==== Changing Shells ====
Previously, we gave you the option of using a <code>~/.login</code> to modify your shell. This is no longer supported, if you have issues with your shell/paths/environment variables we will ask you to delete your <code>~/.login</code> file and change your shell via the method below.

You can change your shell is via <code>chsh</code> on either of the headnodes (eos/selene). This does not need to be re-done if you've changed to it to your preferred shell in the past.

Use the appropriate of the following three lines:
<syntaxhighlight lang="bash" line>
/usr/local/bin/chsh -s bash && bash -l
/usr/local/bin/chsh -s tcsh && tcsh -l
/usr/local/bin/chsh -s zsh && zsh -l
</syntaxhighlight>

=== Changing your PATH ===
Typically, you don't have to change your PATH, but it is useful to know what your PATH is and what it does. The PATH is the list of directories which are searched when you type the name of a program. Note that by default the current directory is NOT included in the path, so if you were wanting to run a program called MyProgram in the current directory, you could NOT simply type 'MyProgram', you would instead type <code>'./MyProgram'</code> (where the '.' represents the current directory).

To find your PATH, we need to identify which shell you are using. If you do not know, run the following:
<syntaxhighlight lang="bash" line>
ps | awk '/sh/ {print $4}'
</syntaxhighlight>

==== tcsh ====
You'll need to edit a file in your home directory called .tcshrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
setenv PATH /usr/local/bin:$PATH
</syntaxhighlight>
==== bash ====
You'll need to edit a file in your home directory called .bashrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
export PATH=/usr/local/bin:$PATH
</syntaxhighlight>

==== zsh ====
You'll need to edit a file in your home directory called .zshrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
export PATH="/usr/local/bin:$PATH"
</syntaxhighlight>

=== Ownership and Permissions ===
Every file and directory has a user and group associated with it. You can view ownership information by using the '-l' switch on ls. By default on Beocat, files you create have a user ownership of your username (i.e., your eID) and a group ownership of your username_users. So, if I were logged in as 'myusername' and I had a single file in my home directory called MyProgram, the result of typing 'ls -l' would be something like this:
total 0
-rwxr-x--- 1 myusername myusername_users 79 May 31 2011 MyProgram
This tells us several things.
* The first column ('-rwxr-x---') is permissions (covered below)
* The second column ('1') is the number of links to this file. You can safely ignore this (unless you're both masochistic and interested in filesystem details)
* The third column ('myusername') shows the user ownership
* The fourth column ('myusername_users') shows the group ownership
* The fifth column ('79') gives the size of the file in bytes
* The next columns ('May 31 2011'), as you have probably guessed, gives the date the file was last changed
* The final column ('MyProgram') is the name of the file

So why is this interesting to us? Because whenever things ''don't'' work, it's usually because of file ownership or permissions. Looking at these often gives us some useful diagnostic information.

The permissions field shows us who has permissions to do what with this file. It is always 10 characters. The first character (-) is usually either a '-' for a regular file or a 'd' for a directory. The next 9 characters are broken into three groups of three, with each group showing read (r), write (w), and execute (x) permissions for the owner, group, and world, in that order.
* The first group (rwx) shows permissions for the owner (myusername). The owner here has read, write, and execute permissions
* The next group (r-x) shows permissions for the group (myusername_users). The group here has read and execute permissions, but cannot write.
* The last group (---) shows permissions for the rest of the world. The world has no permissions to read, write, or execute.

When you create a shell script with a text editor, and sometimes when you copy programs to Beocat via SCP, the execute flag is not set. The permissions string may look more like (-rw-r--r--). To change this, you need to give yourself permission to execute this program. This is done with the 'chmod' (change mode) command. 'chmod' can have a long and confusing syntax, but since by far the most common problem is to give yourself execute permissions, here is the command to change that:
chmod u+x MyProgram
This changes the permissions so that the user ('u', i.e., the owner) adds ('+') execute permission ('x').

For more complex ownership or permissions changes, please feel free to contact the Beocat staff.

=== Access Control Lists ===
Access Control Lists build on our knowledge and use of basic Linux permissions, so we'll cover those again:

Linux permissions are typically broken down to ('''r''')ead, ('''w''')rite, and e('''x''')ecute split across 3 classes of accessors.

'''Files'''
; read
: Read the file, pretty straight forward
; write
: Write to the file, including overwrite, truncation, etc.
; execute
: Execute the file, this permission allows you to run the file.

'''Folders'''
; read
: List the directory, (ls)
; write
: Create new files and folders in the directory.
; execute
: Pass through the directory (cd into and through).

Those accessors are ('''u''')ser, ('''g''')roup, and ('''o''')ther.
; user
: The user would typically be the user account that created the file or folder
; group
: The group would be that accounts primary group by default or can be changed by the user to be any group that they are a member of
; other
: Other is special, other is anything that doesn't meet either of the two other critera. We typically refer to them as world permissions, as they match ''everyone'' else.

Unfortunately, it is that "Other" permission that is a frequent problem. You may want to share some data with a colleague, but, from a security standpoint, you also may need to make sure that only that colleague has access to the data. If you aren't in the same group as the colleague, then, under standard Linux permissions, you have no other option except making the file "world" accessible.

This is where <abbr title="Access Control Lists">ACLs</abbr> come into play. ACLs are like the standard Linux permissions, except you can apply many of them, and you can allow individual users and groups to access alongside your own.

ACLs can also do things that standard Linux permissions can't, like setting up "default" permissions for newly created files/folders within a directory.

One big thing to be aware of for any permissions scheme is that permissions are checked at every level in a directory hierarchy.

# /
# /homes
# /homes/$USER
# /homes/$USER/$SHARE

If at any point the accessing user is denied permission, the traversal and access attempt will stop.

==== Example 1 ====
Let's say I have a file in a directory that I want the user billy to be able to read. This file is <tt>/homes/mozes/example/input.file</tt>. We'll look at the current permissions of the directory tree like so:

We'll assume everyone has requisite permissions for <tt>/</tt> and <tt>/homes</tt>.

First we'll check my home directory
$ getfacl -e /homes/mozes
# owner: mozes
# group: mozes_users
user::rwx
group::r-x #effective:r-x
group:beocat_support:r-x #effective:r-x
mask::r-x
other::---
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x
default:other::---

If we make it past that permissions check, we'd go one level deeper.
$ getfacl -e /homes/mozes/example
# owner: mozes
# group: mozes_users
user::rwx
group::r-x
group:beocat_support:r-x #effective:r-x
other::r-x
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x

Finally we'd check if we had permission to access the file itself:
$ getfacl -e /homes/mozes/example/input.file
# owner: mozes
# group: mozes_users
user::rw-
group::r--
group:beocat_support:r-x #effective:r-x
other::r--

There is quite a lot of information contained in the the above output, so lets look at and attempt to understand the contents.

First, in each section, we see the POSIX owner and group as comments prefixed by '#' characters. These are what the respective user:: and group:: lines refer to when viewing the permissions.

Second, we have lines related to the permissions of each accessor. These do what they say, show the permissions that an accessor would be granted. Please note there is a catch here, it seems to be most specific permission wins. This can come into play with granting a certain group access and then a specific member of the group a differering level of access.

Third, on many of them there are lines prefixed with default: and then a permissions set. Default permissions are interesting. They are only set to directories and they define the starting set of acls that should be set when new files or folders are created within that folder.

Finally, there is a mask, I'm not covering it because there are very few cases that people would need to use them.

Back to the task at hand, We want billy to be able to read <tt>/homes/mozes/example/input.file</tt>. Checking <tt>/homes/mozes</tt>, we see that 'other' has no permissions, and billy has not been granted any special access.

So we grant billy access "through" <tt>/homes/mozes</tt>, granting the smallest set of permissions would be this:
$ setfacl -m u:billy:x /homes/mozes

Note, since I didn't give billy read access to my home directory, they wouldn't be able to <tt>ls /homes/mozes</tt>, they can simply cd into it and through it.

Then we check the rest of the permissions, <tt>/homes/mozes/example</tt> has an 'other' permission granting (r)ead and e(x)ecute, so that shouldn't be an issue. <tt>/homes/mozes/example/input.file</tt> allows 'other' to read it, so our job is done. Billy has access to read my file.

If we decide later that billy needs to write to my file, we can grant them specific read/write permissions to just that file with:
$ setfacl -m u:billy:rw /homes/mozes/example/input.file

==== Example 2 ====
That's all well and good, but lets say we want all of my grad students to have read/write access to my example directory.

Looking at the acls that have been set so far:
$ getfacl -e /homes/mozes
# owner: mozes
# group: mozes_users
user::rwx
user:billy:--x
group::r-x #effective:r-x
group:beocat_support:r-x #effective:r-x
mask::r-x
other::---
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x
default:other::---

$ getfacl -e /homes/mozes/example
# owner: mozes
# group: mozes_users
user::rwx
group::r-x
group:beocat_support:r-x #effective:r-x
other::r-x
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x

We now want to grant my group of grad students the correct permissions to <tt>/homes/mozes/example</tt>
$ setfacl -R -m g:my_grad_students:rw -m d:g:my_grad_students:rw -m d:u:mozes:rw /homes/mozes/example

There are a few things to note there:
* We're setting multiple acls at once (note the multiple -m arguments)
* We're setting those permissions recursively (on all files/folders nested anywhere in that directory hierarchy). The <tt>-R</tt> option does this.
* We're setting some default permissions. Default permissions are prefixed with d:. Here we're saying that the (g)roup my_grad_students should be granted read/write permissions, we aslo set a default permission for ourselves. d:u:mozes:rw grants me read/write access to those files as if I were the owner, this is nice in the event that you're not a member of the my_grad_students group, it would make sure that you still retain a reasonable baseline of access.

That all looks good, right? Except my grad students are complaining that they can't access <tt>/homes/mozes/example</tt>. What did we forget?

Permissions are checked at every level of the directory hierarchy, and we forgot to grant my_grad_students access through my home directory.
$ setfacl -m g:my_grad_students:x /homes/mozes

=== Manual (man) pages ===
Most commands have a complex set of switches that will modify the amount or type of information they display. To find out what switches are available, or how a program expects data, you can use the manual pages by typing "`man` ''command''". Using one of the most common Linux commands, take a look the output of 'man ls'. It shows that it has over 50 switches available, ranging from which files to include, to how to display file sizes, to sort order and more. (I'm not pasting it here, because it's over 200 lines long!) To navigate a 'manpage', use the up-arrow and down-arrow keys. Press 'q' to quit.

=== Pipes and Redirects ===
Typically a Linux program takes data from the keyboard and outputs data to the screen. In Unix and Linux terminology, the keyboard is the default 'stdin' (pronounced "standard in") and the screen is the default 'stdout' (pronounced "standard out"). Many times, we want to take data from somewhere else (like a file, or the output of another program) and send it to yet another location. These redirectors are:
{|
|cmd > filename
|Redirect output from cmd to filename ||
|-
|cmd >> filename
|Redirect output from cmd and append to filename
|-
|cmd < filename
|Redirect input from cmd to filename
|-
| cmd1 | cmd2
| Use the output from cmd1 as the input to cmd2
|}
Here is a quick example. Let's say I have a thousands of files in a directory, and I want a list of those that end in '.sh'
'ls' by itself scrolls so far I can't see even a fraction of them. So, I redirect the output to a file
ls > ~/filelist.txt
That gives me all the files in the current folder and saves them in my home directory in 'filelist.txt'.
A quick look through the file in my favorite editor tells me this is still going to take too long, so I need another step. The 'grep' program is a commonly-used program to perform pattern matching. The syntax of 'grep' is beyond the scope of this document, but take my word for it that
grep '\.sh$'
will return all lines that end in .sh.

We can now redirect the input from grep to the file we just created:
grep '\.sh$' < ~/filelist.txt
Great! We now have our list. However, we wanted to save this as filelist.txt, and instead we have another list that we have to copy-and-paste. Instead of redirecting to a file, we'll use the vertical bar '|' (which we often term a "pipe") to send the output of one command to another.
ls | grep '\.sh$' > ~/filelist.txt
This time the output of 'ls' is ''not'' redirected to a file, but is redirected to the next command (grep). The output of grep (which is all our .sh files) instead of being sent to the screen is redirected to the file ~/filelist.txt.

This example is a very simple demonstration of how pipes and redirects work. Many more examples with complex structures can be found at http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-103-4/index.html

LinuxBasics

2023-06-05T20:56:11Z

Mozes: /* Access Control Lists */

'''Disclaimer:''' This is a ''very'' large topic, and much too broad to be covered on a single support page. There are many other sites (yes, entire sites) which cover the topic in more detail. We'll link so some of them below. This page is meant to be just the essentials.

== Logging in for the first time ==
To login to Beocat, you first need an "SSH Client". [[wikipedia:Secure_Shell|SSH]] (short for "secure shell") is a protocol that allows secure communication between two computers. We recommend the following.
* Windows
** [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY] is by far the most common SSH client, both for Beocat and in the world.
** [http://mobaxterm.mobatek.net/ MobaXterm] is a fairly new client with some nice features, such as being able to SCP/SFTP (see below), and running X (which isn't terribly useful on Beocat, but might be if you connect to other Linux hosts).
** [http://www.cygwin.com/ Cygwin] is for those that would rather be running Linux but are stuck on Windows. It's purely a text interface.
* Macintosh
** OS-X has SSH a built-in application called "Terminal". It's not great, but it will work for most Beocat users.
** [http://www.iterm2.com/#/section/home iTerm2] is the terminal application we prefer.
* Others
** There are [[wikipedia:Comparison_of_SSH_clients|many SSH clients]] for many different platforms available. While we don't have experience with many of these, any should be sufficient for access to Beocat.

You'll need to connect your client (via the SSH protocol, if your client allows multiple protocols) to headnode.beocat.ksu.edu.

For command-line tools, the command to connect is
ssh ''username''@headnode.beocat.ksu.edu

Your username is your [http://eid.ksu.edu K-State eID] name and the password is your eID password.

'''Note:''' When you type your password, nothing shows up on the screen, not even asterisks.

You'll know you are successfully logged in when you see a prompt that says
[''username''@''machinename'' ~]$
where ''machinename'' is the name of the machine you've logged into (currently either 'eos' or 'selene') and ''username'' is your eID username

== Transferring files (SCP or SFTP) ==
Usually, one of the first things people want to do is to transfer files into or out of Beocat. To do so, you need to use [[wikipedia:Secure_copy|SCP]] (secure copy) or [[wikipedia:SSH_File_Transfer_Protocol|SFTP]] (SSH FTP or Secure FTP). Again, there are multiple programs that do this.
* Windows
** Putty (see above) has PSCP and PSFTP programs (both are included if you run the installer). It is a command-line interface (CLI) rather than a graphical user interface (GUI).
** MobaXterm (see above) has a built-in GUI SFTP client that automatically changes the directories as you change them in your SSH session.
** [https://filezilla-project.org/ FileZilla] (client) has an easy-to-use GUI. Be sure to use 'SFTP' mode rather than 'FTP' mode.
** [http://winscp.net/eng/index.php WinSCP] is another easy-to-use GUI.
** Cygwin (see above) has CLI scp and sftp programs.
* Macintosh
** [https://filezilla-project.org/ FileZilla] is also available for OS-X.
** Within terminal or iTerm, you can use the 'scp' or 'sftp' programs.
* Linux
** FileZilla also has a GUI linux version, in addition to the CLI tools.

=== Using a Command-Line Interface (CLI) ===
You can safely ignore this section if you're using a graphical interface (GUI). We highly recommend using a GUI when first learning how to use Beocat.

First test case: transfer a file called myfile.txt in your current folder to your home directory on Beocat. For these examples, I use bold text to show what you type and plain text to show Beocat's response

Using SCP:
'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Note the colon at the end of the 'scp' line.

Using SFTP
'''sftp ''username''@headnode.beocat.ksu.edu'''
Password: '''(type your password here, it will not show any response on the screen)'''
Connected to headnode.beocat.ksu.edu.
sftp> '''put myfile.txt'''
Uploading myfile.txt to /homes/kylehutson/myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''exit'''

SFTP is interactive, so this is a two-step process. First, you connect to Beocat, then you transfer the file. As long as the system gives the <code>sftp> </code> prompt, you are in the sftp program, and you will remain there until you type 'exit'.

Second test case: transfer a file called myfile.txt in your current folder to a diretory named 'mydirectory' under your home directory on Beocat.

Here we run into one of the problems with scp - there is no easy way of creating 'mydirectory' if it doesn't already exist. If it does not already exist, you must login via ssh (as seen above) and create the directory using the 'mkdir' command (see Common Linux Commands) below.

'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:mydirectory'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

An alternative version. If the colon is immediately followed by a slash, the directory name is taken from the root, rather than your home directory. So, given that your home directory on Beocat is /homes/''username'', we could instead type
'''scp myfile.txt ''username''@headnode.beocat.ksu.edu:/homes/''username''/mydirectory'''
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Using SFTP:
sftp ''username''@headnode.beocat.ksu.edu
Password: '''(type your password here, it will not show any response on the screen)'''
Connected to headnode.beocat.ksu.edu.
sftp> '''mkdir mydirectory'''
[Note, if this directory already exists, you will get the response "Couldn't create directory: Failure"]
sftp> '''cd mydirectory'''
sftp> '''put myfile.txt'''
Uploading myfile.txt to /homes/''username''/mydirectory/myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''quit'''

Third test case: copy myfile.txt from your home directory on Beocat to your current folder.

Using scp:
scp ''username''@headnode.beocat.ksu.edu:myfile.txt .
Password: '''(type your password here, it will not show any response on the screen)'''
myfile.txt 100% 0 0.0KB/s 00:00

Using SFTP:
'''sftp ''username''@headnode.beocat.ksu.edu'''
Password: '''(type your password here, it will not show any response on the screen)'''
Connected toheadnode.beocat.ksu.edu.
sftp> '''get myfile.txt'''
Fetching /homes/''username''/myfile.txt to myfile.txt
myfile.txt 100% 0 0.0KB/s 00:00
sftp> '''exit'''

== Basic Linux Commands ==
Again, this guide is very limited, mostly limited to directory navigation and basic file commands. [http://www.ee.surrey.ac.uk/Teaching/Unix/ Here] is a pretty decent tutorial if you want to dig deeper. If you want more, entire books have been written on the subject.

=== The Lingo ===
{| class="wikitable"
!''Term''
!''Definition''
|-
|Directory
|A "Folder" in Windows or OS-X terms. A location where files or other directories are stored. The current directory is sometimes represented as `.` and the parent directory can be referenced as `..`
|-
|Shell
|The interface or environment under which you can run commands. There is a section below on shells
|-
|SSH
|Secure Shell. A protocol that encrypts data and can give access to another system, usually by a username and password
|-
|SCP
|Secure Copy. Copying to or from a remote system using part of SSH
|-
|path
|The list of directories which are searched when you type the name of a program. There is a section below on this
|-
|ownership
|Every file and directory has an user and a group attached to it, called its owners. These affect permissions.
|-
|permissions
|The ability to read, write, and/or execute a file. Permissions are based on ownership
|-
|switches
|Modifiers to a command-line program, usually in the form of -(letter) or --``(word). Several examples are given below, such as the '-a' on the 'ls' command
|-
|pipes and redirects
|Changes the input (often called 'stdin') and/or output (often called stdout) to a program or a file
|}

=== Linux Command Line Cheat Sheet ===
{| class="wikitable"
|+File System Navigation
|-
!''Command''
!''What it does''
!''Example Usage''
!''Example Output''
|-
|pwd
|"Print working directory", Where am I now?
|<code>pwd</code>
|<code>/homes/mozes</code>
|-
|ls
|Lists files and folders
|<code>ls ~/</code>
|<code>NewFile NewFolder</code>
|-
|ls -lh
|Lists files and folders with perms size and ownership
|<code>ls -lh ~/</code>
|<code>-rw-r--r-- 1 mozes mozes_users 1 Jul 13 2011 NewFile
drwxr-xr-x 9 mozes mozes_users 9.0K Apr 12 2010 NewFolder</code>
|-
|ls -a
|Lists all files and folders
|<code>ls -a ~/</code>
|<code>. .. .bashrc .bash_profile .tcshrc NewFile NewFolder</code>
|-
|cd
|Changes directory
|<code>cd NewFolder</code>
|
|-
|cd ..
|Changes to parent directory
|<code>cd ..</code>
|
|-
|cd -
|Changes to the previous directory you were in
|<code>cd -</code>
|
|-
|cd ~
|Changes to your home directory
|<code>cd ~</code>
|
|}

{| class="wikitable"
|+Working with files
|-
!Command'
!What it does
!Example Usage'
!Example Output''
|-
|file
|Identifies the type of object a file is
|<code>file NewFile</code>
|<code>NewFile: a /usr/bin/python script, ASCII text executable</code>
|-
|cat
|Prints the contents of one or more files
|<code>cat NewFile</code>
|<code>This is line one
This is line two</code>
|-
|cp
|copy a file
|<code>cp OldFile NewFile</code>
|
|-
|cp -i
|copy a file, ask to overwrite
|<code>cp -i OldFile NewFile}</code>
|<code>overwrite NewFile? (y/n [n])</code>
|-
|cp -r
|copy a directory, including contents
|<code>cp -r OldFolder NewFolder</code>
|
|-
|mv
|move, or rename, a file
|<code>mv OldFile NewFile</code>
|
|-
|mv -i
|move, or rename, a file, ask to overwrite
|<code>mv -i OldFile NewFile</code>
|<code>overwrite NewFile? (y/n [n])</code>
|-
|rm
|remove a file
|<code>rm NewFile</code>
|
|-
|rm -i
|remove a file, ask to be sure (useful with -r)
|<code>rm -i NewFile</code>
|<code>remove NewFile? (y/n [n])</code>
|-
|rm -r
|remove a direcory and its contents
|<code>rm -r NewFolder</code>
|
|-
|mkdir
|creates a directory
|<code>mkdir TempFolder</code>
|
|-
|rmdir
|removes an empty directory
|<code>rmdir TempFolder</code>
|
|-
|touch
|creates an empty file
|<code>touch TempFile</code>
|
|}

{| class="wikitable"
|+Finding files and directories with [http://linux.die.net/man/1/find find]
|-
!''Command''
!''What it does''
!''Example Usage''
|-
| find <directory>
| finds all files and folders within <directory>
| find ~/
|-
| find <directory> -iname '<filename>'
| finds all files and directories within <directory> that match <filename>
| find ~/ -iname 'hello.qsub'
|-
| find <directory> -iname '*<partialmatch>*'
| finds all files and directories within <directory> that partially match <partialmatch>
| find ~/ -iname '*.qsub*'
|}

Other useful commands include <code>htop</code>, <code>less</code>, and <code>man</code>. <code>man</code> followed by a command name above will give you the manual page for the specified command full of many other useful options for the command. <code>htop</code> will give you an overview of the processes currently being run on the host you are connected to. <code>less</code> allows you to page through files and see their contents using <PgUp> and <PgDn>.

=== Editing Text Files ===
If you're new to Linux, the editor you will probably want to use is 'nano'. It works much the same as 'Notepad' in Windows or 'textedit' on OS-X. Note that you cannot use your mouse to change position within the document as you can with your local computer. You must use the arrow keys, instead.

So, if I wanted to edit my .bashrc (as shown below), and I was already in my home directory (see above), I would type
nano .bashrc

While in nano, there is a list of actions you can take at the bottom of the screen. <Ctrl> is represented by a caret (`^`), so to exit (labeled as `^`X at the bottom of the screen), I would type <ctrl>-x. This action prompts you whether you want to save and exit (Y), lose changes and exit (N), or cancel and go back to editing (<ctrl>-c).

If you do a significant amount of text editing in Linux, you'll probably want to switch to a more powerful editor, such as vim. The usage of vim is beyond the scope of this document. It is not at all intuitive to the beginning user, but with a little practice it becomes a much faster way of editing text files. If you're interested in using vim, [http://www.openvim.com/tutorial.html there is a nice tutorial here].

=== Shells ===
==== What is a Shell? ====
In this case, I don't believe I can do a better job explaining shells than [[wikipedia:Shell_(computing)|this]].
==== tcsh ====
Elsewhere at Kansas State University, the default Shell is set to tcsh. tcsh stands for "TENEX C SHell." It is considered a replacement for csh and uses many of the same features. If you have experience with either csh or tcsh you'll probably feel right at home. This was the default shell until July of 2013. If you had an account before then, it is probably still tcsh.

But what if you don't want or like tcsh, what can you do? Well, we have other shells available of Beocat as well.
==== bash ====
[http://www.gnu.org/software/bash/ Bash] seems to be the defacto standard shell in most Linux installs today. Bash is common and probably what most of you are used to. As of July 2013, bash is our new default shell. All new users will be set to bash initially. [https://software-carpentry.org/ Software Carpentry] teaches classes on several subjects specifically targeting researchers, including the bash shell. Their documentation is all freely available. [http://swcarpentry.github.io/shell-novice/ Here is a link to their excellent tutorial on using BASH.] Most of our documentation assumes you are using BASH.

bash configuration files:

This section gets into some minutiae with the way our job scheduler interacts with bash. If you're trying to solve a problem, read on, otherwise you can probably skip this section.

Bash has 3 user configurable configuration files, <code>~/.bashrc ~/.bash_profile</code> and <code>~/.bash_logout</code>. We'll look at the two more relevant ones <code>~/.bashrc</code> and <code>~/.bash_profile</code>

Bash has 3 ways of looking at things, '''login''', '''interactive''', or '''none'''.

Normally what happens is that shells that are '''interactive''' read <code>~/.bash_profile</code>, shells that are '''login''' read <code>~/.bashrc</code>. '''none''' shells read neither.

sbatch jobs are '''login''', srun jobs are '''login+interactive''', logging into Beocat in a way that you can enter commands is '''login+interactive'''. There are very few cases that you will get '''none'''. For any session that isn't '''interactive''', your sourced files cannot output anything to the screen, or else it can break scp or sftp file transfers.

If they are ''quiet'' statements, and you want them in all shells, you can put them in your <code>~/.bashrc</code>. If they are not ''quiet'' or they output ''anything'' to the screen, you must put them in your <code>~/.bash_profile</code>.

==== zsh ====
[http://zsh.sourceforge.net/ zsh] is an alternative to bash and tcsh. It tends to support more complex features than either of the other two while using a syntax remarkably similar to bash. Unless specifically noted, when we specify '''Change your shell to bash''', <tt>zsh</tt> should work as well.

==== Changing Shells ====
Previously, we gave you the option of using a <code>~/.login</code> to modify your shell. This is no longer supported, if you have issues with your shell/paths/environment variables we will ask you to delete your <code>~/.login</code> file and change your shell via the method below.

You can change your shell is via <code>chsh</code> on either of the headnodes (eos/selene). This does not need to be re-done if you've changed to it to your preferred shell in the past.

Use the appropriate of the following three lines:
<syntaxhighlight lang="bash" line>
/usr/local/bin/chsh -s bash && bash -l
/usr/local/bin/chsh -s tcsh && tcsh -l
/usr/local/bin/chsh -s zsh && zsh -l
</syntaxhighlight>

=== Changing your PATH ===
Typically, you don't have to change your PATH, but it is useful to know what your PATH is and what it does. The PATH is the list of directories which are searched when you type the name of a program. Note that by default the current directory is NOT included in the path, so if you were wanting to run a program called MyProgram in the current directory, you could NOT simply type 'MyProgram', you would instead type <code>'./MyProgram'</code> (where the '.' represents the current directory).

To find your PATH, we need to identify which shell you are using. If you do not know, run the following:
<syntaxhighlight lang="bash" line>
ps | awk '/sh/ {print $4}'
</syntaxhighlight>

==== tcsh ====
You'll need to edit a file in your home directory called .tcshrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
setenv PATH /usr/local/bin:$PATH
</syntaxhighlight>
==== bash ====
You'll need to edit a file in your home directory called .bashrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
export PATH=/usr/local/bin:$PATH
</syntaxhighlight>

==== zsh ====
You'll need to edit a file in your home directory called .zshrc, replacing /usr/local/bin with the directory that you want added to your PATH using a text editor as shown above.
<syntaxhighlight lang="bash" line>
export PATH="/usr/local/bin:$PATH"
</syntaxhighlight>

=== Ownership and Permissions ===
Every file and directory has a user and group associated with it. You can view ownership information by using the '-l' switch on ls. By default on Beocat, files you create have a user ownership of your username (i.e., your eID) and a group ownership of your username_users. So, if I were logged in as 'myusername' and I had a single file in my home directory called MyProgram, the result of typing 'ls -l' would be something like this:
total 0
-rwxr-x--- 1 myusername myusername_users 79 May 31 2011 MyProgram
This tells us several things.
* The first column ('-rwxr-x---') is permissions (covered below)
* The second column ('1') is the number of links to this file. You can safely ignore this (unless you're both masochistic and interested in filesystem details)
* The third column ('myusername') shows the user ownership
* The fourth column ('myusername_users') shows the group ownership
* The fifth column ('79') gives the size of the file in bytes
* The next columns ('May 31 2011'), as you have probably guessed, gives the date the file was last changed
* The final column ('MyProgram') is the name of the file

So why is this interesting to us? Because whenever things ''don't'' work, it's usually because of file ownership or permissions. Looking at these often gives us some useful diagnostic information.

The permissions field shows us who has permissions to do what with this file. It is always 10 characters. The first character (-) is usually either a '-' for a regular file or a 'd' for a directory. The next 9 characters are broken into three groups of three, with each group showing read (r), write (w), and execute (x) permissions for the owner, group, and world, in that order.
* The first group (rwx) shows permissions for the owner (myusername). The owner here has read, write, and execute permissions
* The next group (r-x) shows permissions for the group (myusername_users). The group here has read and execute permissions, but cannot write.
* The last group (---) shows permissions for the rest of the world. The world has no permissions to read, write, or execute.

When you create a shell script with a text editor, and sometimes when you copy programs to Beocat via SCP, the execute flag is not set. The permissions string may look more like (-rw-r--r--). To change this, you need to give yourself permission to execute this program. This is done with the 'chmod' (change mode) command. 'chmod' can have a long and confusing syntax, but since by far the most common problem is to give yourself execute permissions, here is the command to change that:
chmod u+x MyProgram
This changes the permissions so that the user ('u', i.e., the owner) adds ('+') execute permission ('x').

For more complex ownership or permissions changes, please feel free to contact the Beocat staff.

=== Access Control Lists ===
Access Control Lists build on our knowledge and use of basic Linux permissions, so we'll cover those again:

Linux permissions are typically broken down to ('''r''')ead, ('''w''')rite, and e('''x''')ecute split across 3 classes of accessors.

'''Files'''
; read
: Read the file, pretty straight forward
; write
: Write to the file, including overwrite, truncation, etc.
; execute
: Execute the file, this permission allows you to run the file.

'''Folders'''
; read
: List the directory, (ls)
; write
: Create new files and folders in the directory.
; execute
: Pass through the directory (cd into and through).

Those accessors are ('''u''')ser, ('''g''')roup, and ('''o''')ther.
; user
: The user would typically be the user account that created the file or folder
; group
: The group would be that accounts primary group by default or can be changed by the user to be any group that they are a member of
; other
: Other is special, other is anything that doesn't meet either of the two other critera. We typically refer to them as world permissions, as they match ''everyone'' else.

Unfortunately, it is that "Other" permission that is a frequent problem. You may want to share some data with a colleague, but, from a security standpoint, you also may need to make sure that only that colleague has access to the data. If you aren't in the same group as the colleague, then, under standard Linux permissions, you have no other option except making the file "world" accessible.

This is where <abbr title="Access Control Lists">ACLs</abbr> come into play. ACLs are like the standard Linux permissions, except you can apply many of them, and you can allow individual users and groups to access alongside your own.

ACLs can also do things that standard Linux permissions can't, like setting up "default" permissions for newly created files/folders within a directory.

One big thing to be aware of for any permissions scheme is that permissions are checked at every level in a directory hierarchy.

# /
# /homes
# /homes/$USER
# /homes/$USER/$SHARE

If at any point the accessing user is denied permission, the traversal and access attempt will stop.

==== Example 1 ====
Let's say I have a file in a directory that I want the user billy to be able to read. This file is <tt>/homes/mozes/example/input.file</tt>. We'll look at the current permissions of the directory tree like so:

We'll assume everyone has requisite permissions for <tt>/</tt> and <tt>/homes</tt>.

First we'll check my home directory
$ getfacl -e /homes/mozes
# owner: mozes
# group: mozes_users
user::rwx
group::r-x #effective:r-x
group:beocat_support:r-x #effective:r-x
mask::r-x
other::---
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x
default:other::---

If we make it past that permissions check, we'd go one level deeper.
$ getfacl -e /homes/mozes/example
# owner: mozes
# group: mozes_users
user::rwx
group::r-x
group:beocat_support:r-x #effective:r-x
other::r-x
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x

Finally we'd check if we had permission to access the file itself:
$ getfacl -e /homes/mozes/example/input.file
# owner: mozes
# group: mozes_users
user::rw-
group::r--
group:beocat_support:r-x #effective:r-x
other::r--

There is quite a lot of information contained in the the above output, so lets look at and attempt to understand the contents.

First, in each section, we see the POSIX owner and group as comments prefixed by '#' characters. These are what the respective user:: and group:: lines refer to when viewing the permissions.

Second, we have lines related to the permissions of each accessor. These do what they say, show the permissions that an accessor would be granted. Please note there is a catch here, it seems to be most specific permission wins. This can come into play with granting a certain group access and then a specific member of the group a differering level of access.

Third, on many of them there are lines prefixed with default: and then a permissions set. Default permissions are interesting. They are only set to directories and they define the starting set of acls that should be set when new files or folders are created within that folder.

Finally, there is a mask, I'm not covering it because there are very few cases that people would need to use them.

Back to the task at hand, We want billy to be able to read <tt>/homes/mozes/example/input.file</tt>. Checking <tt>/homes/mozes</tt>, we see that 'other' has no permissions, and billy has not been granted any special access.

So we grant billy access "through" <tt>/homes/mozes</tt>, granting the smallest set of permissions would be this:
$ setfacl -m u:billy:x /homes/mozes

Note, since I didn't give billy read access to my home directory, they wouldn't be able to <tt>ls /homes/mozes</tt>, they can simply cd into it and through it.

Then we check the rest of the permissions, <tt>/homes/mozes/example</tt> has an 'other' permission granting (r)ead and e(x)ecute, so that shouldn't be an issue. <tt>/homes/mozes/example/input.file</tt> allows 'other' to read it, so our job is done. Billy has access to read my file.

If we decide later that billy needs to write to my file, we can grant them specific read/write permissions to just that file with:
$ setfacl -m u:billy:rw /homes/mozes/example/input.file

==== Example 2 ====
That's all well and good, but lets say we want all of my grad students to have read/write access to my example directory.

Looking at the acls that have been set so far:
$ getfacl -e /homes/mozes
# owner: mozes
# group: mozes_users
user::rwx
user:billy:--x
group::r-x #effective:r-x
group:beocat_support:r-x #effective:r-x
mask::r-x
other::---
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x
default:other::---

$ getfacl -e /homes/mozes/example
# owner: mozes
# group: mozes_users
user::rwx
group::r-x
group:beocat_support:r-x #effective:r-x
other::r-x
default:user::rwx
default:group::r-x #effective:r-x
default:group:beocat_support:r-x #effective:r-x
default:mask::r-x

We now want to grant my group of grad students the correct permissions to <tt>/homes/mozes/example</tt>
$ setfacl -R -m g:my_grad_students:rw -m d:g:my_grad_students:rw -m d:u:mozes:rw /homes/mozes/example

There are a few things to note there:
* We're setting multiple acls at once (note the multiple -m arguments)
* We're setting those permissions recursively (on all files/folders nested anywhere in that directory hierarchy).
* We're setting some default permissions. Default permissions are prefixed with d:. Here we're saying that the (g)roup my_grad_students should be granted read/write permissions, we aslo set a default permission for ourselves. d:u:mozes:rw grants me read/write access to those files as if I were the owner, this is nice in the event that you're not a member of the my_grad_students group, it would make sure that you still retain a reasonable baseline of access.

That all looks good, right? Except my grad students are complaining that they can't access <tt>/homes/mozes/example</tt>. What did we forget?

Permissions are checked at every level of the directory hierarchy, and we forgot to grant my_grad_students access through my home directory.
$ setfacl -m g:my_grad_students:x /homes/mozes

=== Manual (man) pages ===
Most commands have a complex set of switches that will modify the amount or type of information they display. To find out what switches are available, or how a program expects data, you can use the manual pages by typing "`man` ''command''". Using one of the most common Linux commands, take a look the output of 'man ls'. It shows that it has over 50 switches available, ranging from which files to include, to how to display file sizes, to sort order and more. (I'm not pasting it here, because it's over 200 lines long!) To navigate a 'manpage', use the up-arrow and down-arrow keys. Press 'q' to quit.

=== Pipes and Redirects ===
Typically a Linux program takes data from the keyboard and outputs data to the screen. In Unix and Linux terminology, the keyboard is the default 'stdin' (pronounced "standard in") and the screen is the default 'stdout' (pronounced "standard out"). Many times, we want to take data from somewhere else (like a file, or the output of another program) and send it to yet another location. These redirectors are:
{|
|cmd > filename
|Redirect output from cmd to filename ||
|-
|cmd >> filename
|Redirect output from cmd and append to filename
|-
|cmd < filename
|Redirect input from cmd to filename
|-
| cmd1 | cmd2
| Use the output from cmd1 as the input to cmd2
|}
Here is a quick example. Let's say I have a thousands of files in a directory, and I want a list of those that end in '.sh'
'ls' by itself scrolls so far I can't see even a fraction of them. So, I redirect the output to a file
ls > ~/filelist.txt
That gives me all the files in the current folder and saves them in my home directory in 'filelist.txt'.
A quick look through the file in my favorite editor tells me this is still going to take too long, so I need another step. The 'grep' program is a commonly-used program to perform pattern matching. The syntax of 'grep' is beyond the scope of this document, but take my word for it that
grep '\.sh$'
will return all lines that end in .sh.

We can now redirect the input from grep to the file we just created:
grep '\.sh$' < ~/filelist.txt
Great! We now have our list. However, we wanted to save this as filelist.txt, and instead we have another list that we have to copy-and-paste. Instead of redirecting to a file, we'll use the vertical bar '|' (which we often term a "pipe") to send the output of one command to another.
ls | grep '\.sh$' > ~/filelist.txt
This time the output of 'ls' is ''not'' redirected to a file, but is redirected to the next command (grep). The output of grep (which is all our .sh files) instead of being sent to the screen is redirected to the file ~/filelist.txt.

This example is a very simple demonstration of how pipes and redirects work. Many more examples with complex structures can be found at http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-103-4/index.html