linfun Documentation - Read the Docs

linfun DocumentationRelease 0.0.1

rob green

Jul 18, 2018

Contents

1 Chapter 5. PACKAGE MANAGEMENT SYSTEMS 31.1 Chapter 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Chapter 5.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Chapter 5.3 Software Packaging Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Chapter 5.4 Why Use Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Chapter 5.5 Package Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.6 Chapter 5.6 Available Package Management Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 41.7 Chapter 5.7 Packaging Tool Levels and Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.8 Chapter 5.8 Package Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.9 Chapter 5.9 Creating Package Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.10 Chapter 5.10 Revision Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.11 Chapter 5.12 The Linux Kernel and the Birth of git . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.12 Chapter 5.13 How git Works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Chapter 6. THE RED HAT PACKAGE MANAGER RPM 72.1 Chapter 6.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Chapter 6.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Chapter 6.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.4 Chapter 6.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Chapter 7. DPKG 93.1 Chapter 7.3 DPKG Essentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Chapter 7.4 Package File Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.3 Chapter 7.5 Source Packages I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 Chapter 7.5b Source Packages II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.5 Chapter 7.6 DPKG Queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.6 Chapter 7.7 Installing/Upgrading/Uninstalling Packages . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Chapter 10. APT 134.1 Chapter 10.3 What is APT? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2 Chapter 10.4 apt-get . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.3 Chapter 10.5 Queries using apt-cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.4 Chapter 10.6a Installing/ removing/ upgrading I . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.5 Chapter 10.6b Installing/ removing/ upgrading II . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5 Chapter 11. SYSTEM MONITORING 155.1 Chapter 11.3 Available Monitoring Tools I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

i

5.2 Chapter 11.3 Available Monitoring Tools III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155.3 Chapter 11.3 Available Monitoring Tools IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165.4 Chapter 11.4 The /proc and /sys Psuedo-filesystems . . . . . . . . . . . . . . . . . . . . . . . . . . 165.5 Chapter 11.5 /proc basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165.6 Chapter 11.6 A survey of /proc I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.7 Chapter 11.6.b A survey of /proc II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.8 Chapter 11.6.c A survey of /proc III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.9 Chapter 11.6.d A survey of /proc IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.10 Chapter 11.7.a /proc/sys I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.11 Chapter 11.7.b /proc/sys II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.12 Chapter 11.7.c /proc/sys III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.13 Chapter 11.8 /sys Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.14 Chapter 11.9.a A Survey of /sys I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205.15 Chapter 11.9.b A Survey of /sys II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.16 Chapter 11.9.c A Survey of /sys III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.17 Chapter 11.9.c A Survey of /sys IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.18 Chapter 11.10.a sar I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.19 Chapter 11.10.b sar II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.20 Chapter 11.10.c sar III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6 Chapter 12 Monitoring Tools (processes) 256.1 Chapter 12.4 Viewing process states with ps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.2 Chapter 12.5 BSD option format for ps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.3 Chapter 12.6 ps Output fields I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.4 Chapter 12.6.b ps Output fields II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.5 Chapter 12.7.a UNIX Output fields for ps I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.6 Chapter 12.7.b UNIX Option format for ps II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.7 Chapter 12.8 Customizing the ps Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.8 Chapter 12.9 Using pstree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.9 Chapter 12.10 Viewing System Loads with top . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296.10 Chapter 12.11 top Options I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296.11 Chapter 12.11 top Options II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

7 Chapter 13. Memory: Monitoring Usage and Tuning 317.1 Chapter 13.3 Memory Tuning Considerations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.2 Chapter 13.4 Memory Monitoring Tools: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317.3 Chapter 13.5.a /proc/sys/vm I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.4 Chapter 13.5.b /proc/sys/vm II (ENTRIES) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.5 Chapter 13.6.a vmstat I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337.6 Chapter 13.6.b vmstat II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347.7 Chapter 13.6.c vmstat III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347.8 Chapter 13.6.d vmstat IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357.9 Chapter 13.6.e vmstat V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357.10 Chapter 13.6.f vmstat VI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367.11 Chapter 13.6.g vmstat VII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367.12 Chapter 13.7.a /proc/meminfo I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367.13 Chapter 13.7.b /proc/meminfo II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.14 Chapter 13.8.a OOM Killer I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377.15 Chapter 13.8.a OOM Killer II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

8 KNOWLEDGE CHECK Q&A 398.1 CHAPTER 5 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398.2 CHAPTER 6 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398.3 CHAPTER 7 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

ii

8.4 CHAPTER 10 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398.5 CHAPTER 11 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408.6 CHAPTER 12 KNOWLEDGE CHECK Q&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

9 License 43

10 Help 45

11 IndIces and tables 47

iii

linfun Documentation, Release 0.0.1

SYS ADMIN:

Contents 1


2 Contents

CHAPTER 1

Chapter 5. PACKAGE MANAGEMENT SYSTEMS

1.1 Chapter 5.1

1.2 Chapter 5.2

1.3 Chapter 5.3 Software Packaging Concepts

Package management systems supply the tools that allow system administrators to automate installing, upgrading,configuring and removing software packages in a known, predictable and consistent manner.

These systems:

Gather and compress associated software files into a single package (archive), which may require one or more otherpackages to be installed first.

Allow for easy software installation or removal.

Can verify file integrity via an internal database.

Can authenticate the origin of packages.

Facilitate upgrades.

Group packages by logical features.

Manage dependencies between packages.

A given package may contain executable files, data files, documentation, installation scripts and configuration files.Also included are metadata attributes such as version numbers, checksums, vendor information, dependencies, de-scriptions, etc.

Upon installation, all that information is stored locally into an internal database which can be conveniently queried forversion status and update information.

3


1.4 Chapter 5.4 Why Use Packages

Software package management systems are widely seen as one of the biggest advancements Linux brought to enter-prise IT environments. By keeping track of files and metadata in an automated, predictable and reliable way, systemadministrators can use package management systems to make their installation processes scale to thousands of systemswithout requiring manual work on each individual system.

Features include:

Automation: No need for manual installs and upgrades. Scalability: Install packages on one system, or 10,000 systems.Repeatability and predictability. Security and auditing.

1.5 Chapter 5.5 Package Types

Packages come in several different types:

Binary packages contain files ready for deployment, including executable files and libraries. These are architecture-dependent and must be compiled for each type of machine.

Source packages are used to generate binary packages; one should always be able to rebuild a binary package (forexample, by using rpmbuild –rebuild on RPM-based systems) from the source package. One source package can beused for multiple architectures.

Architecture-independent packages contain files and scripts that run under script interpreters, as well as documentationand configuration files.

Meta-packages are essentially groups of associated packages that collect everything needed to install a relativelylarge subsystem, such as a desktop environment, or an office suite, etc. Binary packages are the ones that systemadministrators have to deal with most of the time.

On 64-bit systems that can run 32-bit programs, one may have two binary packages installed for a given program,perhaps one with x86_64 or amd64 in its name, and the other with i386 or i686 in its name.

Source packages can be helpful in keeping track of changes and source code used to come up with binary packages.They are usually not installed on a system by default, but can always be retrieved from the vendor.

1.6 Chapter 5.6 Available Package Management Systems

There are two very common package management systems:

RPM (Red Hat Package Manager) This system is used by all Red Hat-derived distributions, such as Red Hat EnterpriseLinux, CentOS, Scientific Linux and CentOS, as well as by SUSE and its related community openSUSE distribution.dpkg (Debian Package) This system is used by all Debian-derived distributions ,including Debian, Ubuntu and LinuxMint. There are other package management systems, such as portage/emerge used by Gentoo, pacman used by Arch,and specialized ones used by Embedded Linux systems and Android.

Another ancient system is just to supply packages as tarballs without any real management or clean removal strategies;this approach still marks Slackware, one of the oldest Linux distributions.

1.7 Chapter 5.7 Packaging Tool Levels and Variables

There are two levels to packaging systems:

4 Chapter 1. Chapter 5. PACKAGE MANAGEMENT SYSTEMS


1. Low Level Utility This simply installs or removes a single package, or a list of packages, each one of which isindividually and specifically named. Dependencies are not fully handled, only warned about: - If anotherpackage needs to be installed, first installation will fail. - If the package is needed by another package,removal will fail.

The rpm and dpkg utilities play this role for the packaging systems that use them.

2. High Level Utility

This solves the dependency problems:

• If another package or group of packages needs to be installed before software can be installed,such needs will be satisfied.

• If removing a package interferes with another installed package, the administrator will be giventhe choice of either aborting, or removing all affected software.

The yum, dnf, and zypper utilities (and more recently, PackageKit) take care of the dependency resolution for rpmsystems, and apt-get and apt-cache and other utilities take care of it for dpkg systems. In this course, we will onlydiscuss the command line interface to the packaging systems; while the graphical frontends used by each Linuxdistribution can be useful, we would like to be less tied to any one interface and have more flexibility.

1.8 Chapter 5.8 Package Sources

Every distrib has one or more package repo where system utils go to get software or update to new versions. It is thejob of the distib to ensure all packages work well with each other

There are always other external repos which can be added to the standard list. ie: EPEL (Extra Packages for EnterpriseLinux) as their source is Fedora and maintainers are close to Red Hat

However, some ext repos may not be so well maintained or constructed which can lead to complications with depen-dancies, aka Dependancy Hell

1.9 Chapter 5.9 Creating Package Sources

Building your own custom software packages makes it easy to distribute and install your own software. Almost everyversion of Linux has some mechanism for doing this.

Building your own package allows you to control exactly what goes in the software and exactly how it is installed.You can create the package so that installing it runs scripts that perform all tasks needed to install the new softwareand/or remove the old software, such as:

Creating needed symbolic links Creating directories as needed Setting permissions Anything that can be scripted. Wewill not discuss mechanisms of how to build .rpm or .deb packages, as that is a question mostly for developers, ratherthan administrators.

1.10 Chapter 5.10 Revision Control Systems

Software projects become more complex to manage as either the size of it increases or as more devs become involvedIn order to organize updates and allow co-operation many different schemes are available for source control. Standardfeatures should include, accurate history, log, backups, show conflicts et.

RCS fills the role of coordinating cooperative development

1.8. Chapter 5.8 Package Sources 5


1.11 Chapter 5.12 The Linux Kernel and the Birth of git

The Linux kernel development system has special needs in that it is widely distributed throughout the world, withliterally thousands of developers involved. Furthermore it is all done very publicly, under the GPL license.

For a long time, there was no real source revision control system. Then, major kernel developers went over to the useof BitKeeper (see http://www.bitkeeper.com), a commercial project which granted a restricted use license for Linuxkernel development.

However, in a very public dispute over licensing restrictions in the spring of 2005, the free use of BitKeeper becameunavailable for Linux kernel development.

1.12 Chapter 5.13 How git Works. . .

Technically, git is not a source control management system in the usual sense and teh basic units it works with are notfiles. It has two important data structures: an object database and a directory cache.

The object database contains objects of three varieties:

1. Blobs:

• Chunks of binary data containg file contents

2. Trees:

• Sets of blobs including file names and attributes, giving the directory structure

3. Commits:

• Changesets describing tree snapshots

The directory cache captures the state of the directory tree.

By liberating the controls system from a file-by-file-based system, one is better able to handle changesets whichinvolve many files.

6 Chapter 1. Chapter 5. PACKAGE MANAGEMENT SYSTEMS

http://www.bitkeeper.com

CHAPTER 2

Chapter 6. THE RED HAT PACKAGE MANAGER RPM

The Red Hat Package Manager (RPM) is used by a number of major distributions (and their close relatives) to controlthe installation, verification, upgrade, and removal of software on Linux systems. The low-level rpm program canperform all these operations, either on just one package, or on a list of packages. Operations which would causeproblems (such as removing a package that another package depends on, or installing a package when the systemneeds other software to be installed first) are blocked from completion.

2.1 Chapter 6.3

‘’RPM” (the Red Hat Package Manager) was developed (unsurprisingly) by Red Hat. All files related to a specifictask are packaged into a single rpm file, which also contains information about how and where to install and uninstallthe files. New versions of software lead to new rpm files which are then used for updating.

rpm files also contain dependency information. Note that unless given a specific URL to draw from, rpm in itself doesnot retrieve packages over the network and installs only from the local machine using absolute or relative paths.

rpm files are usually distribution-dependent; installing a package on a different distribution than it was created for canbe difficult, if not impossible.

2.2 Chapter 6.4

For sys admins, RPM makes it easy to:

1. Determine what package, if any, any file on the system is part of

2. Determine what version is intalled

3. Install and uninstall (erase) packages without leaving debris behind

4. Verify that a package was installed correctly; this is useful for both troubleshooting and system auditing

5. Distinguish doc files from the rest of the package and optionallly decide not to install them to save space.

6. Use ftp or HTTP to install packages over the internet.

7


For Developers RPM offers advantages as well:

1. Software is often made available on more than one OS. With RPM the original full and unmodified sourceis used as the basis, but a developer can seperate out the changes needed to build on Linux

2. More than one architecture can be built using only one source package.

2.3 Chapter 6.5

RPM package file names are based on fields that represent specific information, as documented in the RPM standard(http://www.rpm.org/)

The standard naming format for a binary package is: <name>-<version>-<release>.<distro>.<architecture>.rpmsed-4.2.1-10.el6.x86_64.rpm The standard naming format for a source package is: <name>-<version>-<release>.<distro>.src.rpm sed-4.2.1-10.el6.src.rpm

2.4 Chapter 6.6

/var/lib/rpm is the default system directory which holds RPM database files in the form of Berkeley DB hash files. Thedatabase files should not be manually modified; updates should be done only through use of the rpm program.

An alternative database directory can be specified with the –dbpath option to the rpm program. One might do this, forexample, to examine an RPM database copied from another system.

You can use the –rebuilddb option to rebuild the database indices from the installed package headers; this is more of arepair, and not a rebuild from scratch.

8 Chapter 2. Chapter 6. THE RED HAT PACKAGE MANAGER RPM

http://www.rpm.org/

CHAPTER 3

Chapter 7. DPKG

The Debian Package Manager (DPKG) is used by all Debian-based distributions to control the installation, verification,upgrade, and removal of software on Linux systems. The low-level dpkg program can perform all these operations,either on just one package, or on a list of packages. Operations which would cause problems (such as removing apackage that another package depends on, or installing a package when the system needs other software to be installedfirst) are blocked from completion.

3.1 Chapter 7.3 DPKG Essentials

DPKG (Debian Package) is the packaging system used to install, remove, and manage software packages under DebianLinux and other distributions derived from it. Like RPM, it is not designed to directly retrieve packages in day-to-dayuse, but to install and remove them locally.

Package files have a .deb suffix and the DPKG database resides in the /var/lib/dpkg directory.

Like rpm, the dpkg program has only a partial view of the universe: it knows only what is installed on the system, andwhatever it is given on the command line, but knows nothing of the other available packages, whether they are in someother directory on the system, or out on the Internet. As such, it will also fail if a dependency is not met, or if one triesto remove a package other installed packages need.

3.2 Chapter 7.4 Package File Names

Debain package file names are based on fileds that represent specific information.The standard naming format for a binary package:<name>_<version>-<revision_number>_<architecture>.deb

as in:

logrotate_3.8.7-1_amd64.deb

on Debian, and

logrotate_3.8.7-1ubuntu1_amd64.deb

9


on Ubuntu. Note that, for historical reasons, the 64-bit x86 platform is called amd64 rather than x86_64, and distribu-tors such as Ubuntu manage to insert their name in the package name.

3.3 Chapter 7.5 Source Packages I

In the Debian packaging system, a source package consists of at least three files:

1. An upstream tarball, ending with .tar.gz. This is the unmodified source as it comes from the package maintainers

2. A description file, ending with .dsc, containing the package name and other metadata, such as architecture anddependencies

3. A second tarball that contains any patches to the upstream source and additional files created for the packageand ends with a name .debian.tar.gz or .diff.gz depending on the distro.

3.4 Chapter 7.5b Source Packages II

For example on a Ubuntu system, download a source package, then see what files are downloaded or created:

1. apt-get source logrotate

2. du -shc logrotate*

Shows all files, file sizes and total size

3.5 Chapter 7.6 DPKG Queries

Table 1: DPKG QUERIEScmd desired outcome$ dpkg -l List all packages installed$ dpkg -L wget List files in the wget package$ dpkg -s wget Show info about an installed package$ dpkg dpkg -I webfs_1.21+ds1-8_amd64.deb

Show info about a package file

$ dpkg -c webfs_1.21+ds1-8_amd64.deb

List files in a package file

$ dpkg -S /etc/init/networking.conf

Show what package owns /etc/init/networking.conf

$ dpkg -s wget Show the status of a package$ dpkg -V package Verify the installed packages integrity

3.6 Chapter 7.7 Installing/Upgrading/Uninstalling Packages

The command:

$ sudo dpkg -i foobar.deb

would be used for either installing or upgrading the foobar package.

10 Chapter 3. Chapter 7. DPKG


If the package is not currently installed then it will be installed. If the package is newer than the one currently installedthen it will be upgraded.

The command:

$ sudo dpkg -r package

is used to remove all of an installed package except for its configuration files.

The command:

$ sudo dpkg -P package

is used to remove all of an installed package including its config files (Note that -P stands for purge)

3.6. Chapter 7.7 Installing/Upgrading/Uninstalling Packages 11


12 Chapter 3. Chapter 7. DPKG

CHAPTER 4

Chapter 10. APT

For use on Debian-based systems, the APT (Advanced Packaging Tool) set of programs provides a higher level ofintelligent services for using the underlying dpkg program, and plays the sam role as yum on Red Hat-based systems.The main utilities are apt-get and apt-cache. It can automatically resolve dependencies when installing, updat-ing and removing packages. It accesses external software repos synchronizing with them and retrieving and installingsoftware as needed

4.1 Chapter 10.3 What is APT?

APT is not a program in itself; it stands for Advanced Packaging Tool, which includes a number of utilities, such asapt-get and apt-cache. These of course, in turn, invoke the lower level dpkg program.

The APT system works with Debian packages whose file have a .deb extension. There are many distros that havedescended from debianwhich have adopted the Debian packaging system with no essential mods. In fact it is notuncommon to use a repo on more than one Debian based Linux Distribution.

4.2 Chapter 10.4 apt-get

apt-get is the main APT command line tool for package management. It can be used to install, manage and upgradeindividual packages or the entire system. It can even upgrade the distribution to a completely new release, which canbe a difficult task.

4.3 Chapter 10.5 Queries using apt-cache

Queries are done using the apt-cache utility:

13


Table 1: APT-CACHE QUERIESQuery desired outcome$ apt-cache search apache2 Search the repo for a package named apache2$ apt-cache show apache2 Display basic information about the apache2 package$ apt-cache showpkg apache2 Display more detailed info on the package$ apt-cache depends apache2 List all dependant packages for apache2$ apt-file search apache2.conf

Search the repo for a file name apache2.conf

$ apt-file list apache2 List all files in the apache2 package

4.4 Chapter 10.6a Installing/ removing/ upgrading I

The apt-get program is the work horse of installing, removing and upgrading packages.

Table 2: apt-getQuery desired effect$ sudo apt-get update Sync the package index with their repo sources$ sudo apt-get install[package]

Install new packages or update and already installed package

$ sudo apt-get remove[package]

Remove a package without removing its config files

$ asudo apt-get --purgeremove [package]

Remove package and its config files

$ sudo apt-get upgrade Apply all available updates to a package already installed

4.5 Chapter 10.6b Installing/ removing/ upgrading II

Table 3: apt-getQuery desired effect$ sudo apt-get dist-upgrade Do a smart upgrade that will do a more thorough dependency resolu-

tion$ sudo apt-get autoremove Get rid of any packages not needed anymore such as old Linux kernels$ sudo apt-get clean Clean out cache files and any archived package files that have been

installed

14 Chapter 4. Chapter 10. APT

CHAPTER 5

Chapter 11. SYSTEM MONITORING

5.1 Chapter 11.3 Available Monitoring Tools I

Linux distros come with many standard performance and profiling tools already installed. Many of them are familiarfrom other UNIX-like operating systems, while some were developed specifically for Linux.

Most of these tools make use of mounted psuedo-filesystems, especially /proc and /sys

While there are also a number of gui monitoring tools we will only consider the cmd line options

Table 1: Available ToolsUtility Purpose Package$ top Process activity, dynamically updated procps$ uptime How long the system is running and the aver-

age loadprocps

$ ps Detailed info about processes procps$ pstree A tree of processes and their connections procps$ mpstat Multiple processor usage psmisc (or pstree)$ iostat CPU utilization and I/O stats sysstat$ sar Display and collect info about system activity psysstat$ numastat Info about NUMA (Non-Uniform Memory

Architecture)numactl

$ strace Information about all system calls a processmakes

strace

5.2 Chapter 11.3 Available Monitoring Tools III

Summary of the main memory and I/O monitoring utility tools:

Memory Monitoring Utilities

15


Table 2: MemoryUtility Purpose Package$ free Brief summary of memory usage procps$ vmstat Detailed virtual memory statistics and block

I/O, dynamically updatedprocps

$ pmap Process memory map procps

I/O Monitoring Utilities

Table 3: I/OUtility Purpose Package$ iostat CPU utilization and I/O statistics sysstat$ sar Display and collect information about system

activitysystat

$ vmstat Detailed virtual memory statistics and blockI/O, dynamically updated

procps

5.3 Chapter 11.3 Available Monitoring Tools IV

Network Monitoring Utilities

Table 4: NetworkUtility Purpose Package$ netstat Detailed networking statistics sysstat$ iptraf Gather information on network interfaces iptraf$ tcpdump Detailed analysis of network packets and traf-

fictcpdump

$ wireshark Detailed network traffic statistics wireshark

5.4 Chapter 11.4 The /proc and /sys Psuedo-filesystems

The /proc and /sys psuedo-filesystems contain a lot of information about the system. Furthermore, many of the entriesin these directory trees are writable and can be used to change system behavior; in most cases this requires a root user.

These are psuedo-filesystems because they exist totally in memory; if you look at the disk partition when the system isnot running, there will be only empty directory which is used as a mount point. Furthermore, the information displayedis gathered only when it is looked at; there is no constant or periodic polling to update entries.

5.5 Chapter 11.5 /proc basics

The /proc psuedo-filesystem has a long history; it has roots in other UNIX operating system variants and originallywas developed to display information about processes on the system, each of which has its own subdirectory in /procwith all important process characteristics available.

Over time, it grew to contain a lot of information about system properties, such as interrupts, memory, networking,etc.

16 Chapter 5. Chapter 11. SYSTEM MONITORING


5.6 Chapter 11.6 A survey of /proc I

What resides in the /proc psuedo-filesystem:

5.7 Chapter 11.6.b A survey of /proc II

First is a subdirectory for each process on the system, whether they are running, sleeping or sheduled out.

Here is the display of ls -F /proc/3589 (a random process).

The -F option will display a ‘/’ immediately after each pathname that is a directory, an asterisk ‘*’ aftereach that is executable, an at sign ‘@’ after each symbolic link, an equals sign ‘=’ after each socket, apercent sign ‘%’ after each whiteout, and a vertical bar’|’ after each that is a FIFO

5.6. Chapter 11.6 A survey of /proc I 17


5.8 Chapter 11.6.c A survey of /proc III

This directory is full of information about the status of the process and the resources it is using.

ie:

5.9 Chapter 11.6.d A survey of /proc IV

Other entries give system wide info. For example, you can see the interrupt statistics here:



For each interrupt, we can see what the type is, how many times it has been handled on each CPU and which devicesare registered to respond to it. We also get global statistics.

5.10 Chapter 11.7.a /proc/sys I

Most of the tunable parameters can be found in the subdirectory tree rooted at /proc/sys:

5.10. Chapter 11.7.a /proc/sys I 19


5.11 Chapter 11.7.b /proc/sys II

Each of these subdirectories contains information, as well as knobs that can be tuned (with care):

• abi/ Contains files with applicatin binary information; rarely used.

• debug/ Debugging parameters; for now, just some control of exception reporting.

• dev/ Device parameters, including subdirectories for cdrom, scsi, raid, and parport.

• fs/ Filesystem parameters, including quota, file handles used, and maximums, inode and directory informationetc.

• kernel/ Kernel parameters. Many important entries here.

• net/ Network parameters, including subdirectories for ipv4, netfilter, etc.

• vm/ Virtual memory parameters, many important entries here.

5.12 Chapter 11.7.c /proc/sys III

Viewing and changing the parameters can be done with simple commands, For example, the maximum number ofthreads allowed on the system can be seen by looking at:

$ ls - l /proc/sys/kernel/threads-max $ cat /proc/sys/kernel/threads-max 129498

We can then modify and verify the change was effected:

$ sudo bash -c 'echo 100000 > /proc/sys/kernel/threads-max' $ cat /proc/sys/kernel/threads-max 100000

Remeber from sysctl the same effect is accomplished by:

$ sudo sysctl kernel.threads-max=100000

Viewing the value can be done as a normal user, while changing it requires superuser privilege.

5.13 Chapter 11.8 /sys Basics

The /sys pseudo-filesystem is an integral part of what is termed the Unified Device Model. Conceptually, it is basedon a device tree and one can walk through it and see the buses, devices, etc. It also now contains information whichmay or may not be strictly related to devices, such as kernel modules.

It has a more tightly defined structure than does /proc. Most entries contain only one line of text, although there areexceptions, unlike its antecedent, which has many multi-line entries whose exact contents have been known to changebetween kernel versions. Thus, the interface is hopefully more stable.

There are system properties which have display entries in both /proc and ‘‘/sys‘; for compatibility with widely usedsystem utilities, the older forms are only gradually being whittled down.

5.14 Chapter 11.9.a A Survey of /sys I

Support for the sysfs virtual filesystem is built into all modern kernels, and it should be mounted under /sys. However,the unified device model does not require mounting sysfs in order to function.



Let’s take a look at what can be found using the 3.18 kernel; we warn you that the exact layout of this filesystem has atendency to mutate. Doing a top level directory command yields:

$ ls -F /sys block/ bus/ class/ dev/ devices/ firmware/ fs/ kernel/ module/power/

which displays the basic device hierarchy. The device model sysfs implementation also includes information notstrictly related to hardware.

5.15 Chapter 11.9.b A Survey of /sys II

Network device can be examined with:

5.16 Chapter 11.9.c A Survey of /sys III

You can view the Ethernet card as shown below, the intention with sysfs is to have one text value per line, althoughthis is not expected to be rigorously enforced.

5.17 Chapter 11.9.c A Survey of /sys IV

The underlying device and driver for the first network interface can be traced through the device and the driversymbolic links. The screenshot here shows us what we can see when looking at the directory corresponding to the firstEthernet card.

5.15. Chapter 11.9.b A Survey of /sys II 21


5.18 Chapter 11.10.a sar I

sar stands for System Activity Reporter. Its an all-purpose tool for gathering system activity and performance dataand creating reports readable by humans.

On Linux systems, the backend to sar is sadc (system activity data collector), which actually accumilates the stats.Its stores info in /var/log/sa directory, with a daily frequency by default, which can be adjusted. Data collectioncan be started from the command line, and regular periodic collection is usually started as a cron jon stored in /etc/cron.d/sysstat

sar then reads in this data and then produces a report.

sar is invoked via:

$ sar [ options ] [ interval ] [ count ]

With no options given a report on CPU usage will be given.

5.19 Chapter 11.10.b sar II

List of the major sar options, each one has its own sub-options:

5.20 Chapter 11.10.c sar III

This screenshot demonstrates how to get paging statistics and the I/O transfer rate stats.



The ksar program is a java-based utility for generating nice graphs of sar data. It can be downloaded from http://sourceforge.net/projects/ksar.

5.20. Chapter 11.10.c sar III 23

http://sourceforge.net/projects/ksar

http://sourceforge.net/projects/ksar

CHAPTER 6

Chapter 12 Monitoring Tools (processes)

Linux administrators make use of many utilities, such as ps, pstree and top, all of which have long histories in UNIX-like operating systems.

6.1 Chapter 12.4 Viewing process states with ps

ps is a workhorse for displaying characteristics and statistics associated with processes, all of which are garnered fromthe /proc directory associated with the process.

This command utility has existed in all UNIX like operating system variants, and that diversity is relected in thecomplicated potpourri of options that the linux version of ps accepts, which fall into 3 categories:

1. UNIX options, which must be preceded by -, and which may be grouped.

2. BSD options, which must not be preceded by -, and which may be grouped.

3. GNU long options, each of which must be preceded by –,

Having all these possible options can be confusing, most sys admins tend to use one or two standard combinations fortheir daily use.

6.2 Chapter 12.5 BSD option format for ps

You can see a typical usage with the BSD option format in the screenshot provided, where the aux option shows allprocesses. Commands which are surrounded by square brackets (as in [ksoftirqd/0]) are threads that exist totally withinthe kernel; if there is one for each CPU, the command is followed by the integer specifying the CPU it is running on.

25


6.3 Chapter 12.6 ps Output fields I

Most of the fields in the preceding example are self-explanatory. Of the others:

VSZ is the process’ virtual memory size in KB. RSS is the resident set size; the non-swapped physical memory a taskis using in KB. STAT describes the state of the process; in our example we see only S for sleeping, or R for running.The additional character in the state (where it exists) can be: - < for high priority (not nice) - N for low priority (nice)- L for having pages locked in memory - s for session leader - l for multi-threaded - + for being in the foregroundprocess group.

6.4 Chapter 12.6.b ps Output fields II

Adding the f option will show how processes connect by ancestry, as in:

26 Chapter 6. Chapter 12 Monitoring Tools (processes)


6.5 Chapter 12.7.a UNIX Output fields for ps I

You can see a typical usage with the UNIX option format in the screenshot provided. Note that it is now showing theParent Process ID (PPID) and the niceness (NI). You may observe that many processes show PPID=2 in this example(taken from RHEL 7 and using systemd) an internal kernel process, kthreadd, which is designed to adopt childrenwhen the parent process dies. In older kernels and systems, you would see PPID=1 for sbin/init, but it is really thesame thing going on.

6.6 Chapter 12.7.b UNIX Option format for ps II

Some common selection options in the UNIX format are:

-A or -e Select all processes

-N Negate selection (means do the opposite)

6.5. Chapter 12.7.a UNIX Output fields for ps I 27


-C Select by command name

-G Select by real group ID (also supports names)

-U Select by real user ID (also supports names).

6.7 Chapter 12.8 Customizing the ps Output

If you use the -o option, followed by a comma-separated list of field identifiers, you can print out a customized list ofps fields:

pid: Process ID number uid: User ID number cmd: Command with all arguments cputime: Cumulative CPU timepmem: Ratio of the process’s resident set size to the physical memory on the machine, expressed as a percentage. Youcan see an example in the screenshot provided. You can consult the ps man page for many other output options.

6.8 Chapter 12.9 Using pstree

pstree gives a visual description of the process ancestry and multi-threaded applications:

$ pstree -aAp 2408

bash,2408 |-emacs,24998 pmonitor.tex | |-{emacs},25002 | ‘-{emacs},25003 |-evince,18036 LFS201-SLIDES.pdf ||-{evince},18040 | |-{evince},18046 | ‘-{evince},18047

Consult the man page for pstree for an explanation of many options; in the above we have chosen just to showinformation for pid=2408.

Note that one of its child processes (evince, pid=18036) has three children of its own. Another way to see that is:

$ ls -l /proc/18036/task

total 0 dr-xr-xr-x 5 coop coop 0 Sep 11 07:15 18036 dr-xr-xr-x 5 coop coop 0 Sep 11 07:15 18040 dr-xr-xr-x 5 coopcoop 0 Sep 11 07:15 18046 dr-xr-xr-x 5 coop coop 0 Sep 11 07:15 18047



6.9 Chapter 12.10 Viewing System Loads with top

When one wants to know what the system is spending its time on, the first tool one often uses is top. The screenshotshows you what you can see when using top without arguments.

By default, top refreshes itself every 3.0 seconds.

6.10 Chapter 12.11 top Options I

top is an ancient utility and has a ton of options, as well as interactive commands triggered when certain keys arepressed. For example, if one hits 1, each CPU is shown separately, and if one hits i only active processes are shown.You can see what doing both gives us in the screenshot.

6.9. Chapter 12.10 Viewing System Loads with top 29


6.11 Chapter 12.11 top Options II

One has a lot of control over how processes are sorted and which fields are displayed; there are many others besidesthe defaults. For example, hitting h or ? gives a brief list of interactive commands and q exits.

Furthermore, one can kill a task by hitting k, or renice it (change its priority) with r.

Doing man top will give you extensive documentation on configuration possibilities, options, and interactive possibil-ities.

Note that there are popular alternatives to the standard top program, some of which have more visual interfaces and/oradditional information, such as htop, ntop and atop. And most Linux distributions have a graphical system monitor(such as gnome-system-monitor or ksysguard) which has a top-like display window that can be shown.


CHAPTER 7

Chapter 13. Memory: Monitoring Usage and Tuning

7.1 Chapter 13.3 Memory Tuning Considerations:

Tuning the memory sub-system can be a complex process. First of all, one has to take note that memory usage and I/Othroughput are intrinsically related, as, in most cases, most memory is being used to cache contents of files on disk.

Thus, changing memory parameters can have a large effect on I/O performance, and changing I/O parameters can havean equally large converse effect on the virtual memory sub-system.

When tweaking parameters in /proc/sys/vm, the usual best practice is to adjust one thing at a time and look foreffects. The primary (inter-related) tasks are:

1. Controlling flushing parameters; ie., how many pages are allowed to be dirty and how often they are flushedout to disk

2. Controlling *swap* behaviour; ie., how much pages that reflect file contents are allowed to remian in memory,as opposed to those that need to be swapped out as they have no other backing store.

3. Controlling how much memory *overcommission* is allowed, since many programs never need the fullamount of memory they request, particularly because of copy on write (COW) techniques.

Memory tuning can often be subtle and what works in one system situation or load may be far from optimal in othercircumstances.

7.2 Chapter 13.4 Memory Monitoring Tools:

Table 1: Memory MonitoringUtility Purpose Package$ free Brief summary of memory usage procps$ vmstat Detailed virtual memory statistics and block

I/O, dynamically updatedprocps

$ pmap Process memory map procps

31


7.3 Chapter 13.5.a /proc/sys/vm I

The /proc/sys/vm directory contains many tunable knobs to control the Virtual Memory system. Exactly whatappears in this directory will depend somewhat on the kernel version. Almost all of the entries are writable (by root).

These values can be changed either by directly writing to the entry, or using the sysctl utility. Furthermore, bymodifying the /etc/sysctl.conf, values can be set at boot time.

You can find full documentation for the /proc/sys/vm directory in the kernel source (or kernel documentationpackage on your distribution), usually under Documentation/sysctl/vm.txt.

7.4 Chapter 13.5.b /proc/sys/vm II (ENTRIES)

Table 2: EntriesEntry Purposeadmin_reserve_kbytes Amount of free memory reserved for privileged usersblock_dump Enables block I/O debuggingcompact_memory Turns on or off memory compaction (essentially defrag-

mentation) when configured into the kerneldirty_background_bytes Dirty memory threshold that triggers writing uncommit-

ted pages to diskdirty_background_ratio Percentage of total pages at which kernel will start writ-

ing dirty data out to diskdirty_bytes The amount of dirty memory a process needs to initiate

writing on its owndirty_expire_centisecs When dirty data is old enough to be written out in hun-

dredths of a second)dirty_ratio Percentage of pages at which a process writing will start

writing out dirty data on its owndirty_writeback_centisecs Interval in which periodic writeback daemons wake up

to flush. If set to zero, there is no automatic periodicwriteback

drop_caches Echo 1 to free page cache, 2 to free dentry and inodecaches, 3 to free all. Note only clean cached pages aredropped; do sync first to flush dirty pages

extfrag_threshold Controls when the kernel should compact memoryhugepages_treat_as_movable Used to toggle how huge pages are treatedhugetlb_shm_group Sets a group ID that can be used for System V huge

pageslaptop_mode Can control a number of features to save power on lap-

topslegacy_va_layout Use old layout (2.4 kernel) for how memory mappings

are displayedlowmen_reserve_ratio Controls how much low memory is reserved for pages

that can only be there; i.e., pages which can go in highmemory instead will do so. Only important on 32-bitsystems with high memory

max_map_count Maximum number of memory mapped areas a processmay have. The default is 64 K

Continued on next page

32 Chapter 7. Chapter 13. Memory: Monitoring Usage and Tuning


Table 2 – continued from previous pageEntry Purposemin_free_kbytes Minimum free memory that must be reserved in each

zonemmap_min_addr How much address space a user process cannot mem-

ory map. Used for security purposes, to avoid bugswhere accidental kernel null dereferences can overwritethe first pages used in an application

nr_hugepages Minimum size of hugepage poolnr_pdflush_hugepages Maximum size of the hugepage pool = nr_hugepages

*nr_overcommit_hugepagesnr_pdflush_threads Current number of pdflush threads; not writeableoom_dump_tasks If enabled, dump information produced when oom-

killer cuts inoom_kill_allocating_task If set, the oom-killer kills the task that triggered the out

of memory situation, rather than trying to select the bestone

overcommit_kbytes One can set either overcommit_ratio or this entry, butnot both

overcommit_memory If 0, kernel estimates how much free memory is leftwhen allocations are made. If 1, permits all allocationsuntil memory actually does run out. If 2, prevents anyovercommission

overcommit_ratio If overcommit_memory = 2 memory commission canreach swap plus this percentage of RAM

page-cluster Number of pages that can be written to swap at once, asa power of two. Default is 3 (which means 8 pages)

panic_on_oom Enable system to crash on an out of memory situationpercpu_pagelist_fraction Fraction of pages allocated for each cpu in each zone for

hot-pluggable CPU machinesscan_unevictable_pages If written to, system will scan and try to move pages to

try and make them reclaimablestat_interval How often vm statistics are updated (default 1 second)

by vmstat swappiness, How aggressively should the ker-nel swap

user_reserve_kbytes If overcommit_memory is set to 2 this sets how low theuser can draw memory resources

vfs_cache_pressure How aggressively the kernel should reclaim memoryused for inode and dentry cache. Default is 100; if 0this memory is never reclaimed due to memory pressure

7.5 Chapter 13.6.a vmstat I

`vmstat is a multi-purpose tool that displays information about memory, paging, I/O, processor activity and pro-cesses. It has many options. The general form of the command is:

`$ vmstat [options] [delay] [count]`

If delay is given in seconds, the report is repeated at that interval count times; if count is not given vmstat will keepreporting statistics forever until it is killed such as Ctl-C.

`vmstat 2 4`

7.5. Chapter 13.6.a vmstat I 33


7.6 Chapter 13.6.b vmstat II

Table 3: Available ToolsField Subfield MeaningProcesses r Number of processes waiting to be

scheduled inProcesses b Number of processess in uninter-

ruptible sleepmemory swpd Virtual memory used (KB)memory free Free, idle memorymemory buff Buffer memoryswap si Memory swapped inswap so Memory swapped outI/O Display and collect info about system activity psysstatI/O Info about NUMA (Non-Uniform Memory

Architecture)numactl

system in interupts/secondsystem cs Context switches/secondCPU st Time stolen from vm %CPU us CPU time running user code %CPU sy CPU time running kernel codeCPU id CPU time idleCPU wa Time waiting for I/O

7.7 Chapter 13.6.c vmstat III

If the option `-S m` is given, memory stats will be given in MB instead of KB.

With the option `-a, *vmstat* displays information about active and inactive memory, where active memory pagesare thos which have been recently used; they may be clean (disk contents are up to date) or dirty (need to be flushed todisk eventually). By contrast, inactive memory pages have not been recently used and are more likely to be cleanandreleasr sooner under memory pressure:

`$ vmstat -a 2 4`



7.8 Chapter 13.6.d vmstat IV

To get a table of memory statistics and certain event counters use the -s option:

7.9 Chapter 13.6.e vmstat V

To get a table of disk statistics use the -d option:

7.8. Chapter 13.6.d vmstat IV 35


7.10 Chapter 13.6.f vmstat VI

Table 4: vmstat Disk FieldsField Subfield Meaningreads total Total reads completed successfullyreads merged Grouped reads - resulting in one I/Oreads ms Milliseconds spent readingwrites total Total writes completed successfullywrites merged Grouped writeswrites ms Ms spent writingI/O cur I/O in progressI/O sec seconds spent for I/O

7.11 Chapter 13.6.g vmstat VII

If you just want to get some quick statistics on only one partition, use the -p option:

7.12 Chapter 13.7.a /proc/meminfo I

As noted earlier, a relatively lengthy summary of memory statistics resides in /proc/meminfo:



7.13 Chapter 13.7.b /proc/meminfo II

7.14 Chapter 13.8.a OOM Killer I

The simplest way to deal with memory pressure would be to permit memory allocations to succedd as long as freememory is available and then fail when all memory is exhausted

7.13. Chapter 13.7.b /proc/meminfo II 37


The second simplest way is to use the swap space on disk to push some of the resident memory out of the core; in thiscase, the total available memory (in theory) is the actual RAM plus the size of the swap space. The hard part of thisis to figure out which pages of memory to swap out when pressure demands. In this approach, once the swap spaceitself is filled, requests for new memory must fail.

Linux, however, goes one better: it permits the system to overcommit memory, so that it can grant memory requeststhat exceed the size of RAM plus swap. While this might seem foolhardy, many (if not most) processess do not useall the requested memory.

An example would be a program that allocates a 1 MB buffer, and then uses only a few pages of the memory. Anotherexample is that every time a child process is forked, it receives a copy of the entire memory space of the parent.Because Linux uses the COW (copy on write) technique, unless one of the processes modifies memory, no actual copyneeds be made. However, the kernel has to assume that the copy might need to be done.

Thus, the kernel permits overcommission of memory, but only for pages dedicated to user processes; pages used withinthe kernel are not swappable and are always allocated at request time.

One can modify, and even turn off this overcommission by setting the value of /proc/sys/vm/overcommit_memory`:

0: (default) Permit overcommission, but refuse obvious overcommits, and give root users somewhat more memoryallocation than normal users. 1: All memory requests are allowed to overcommit. 2: Turn off overcommission.Memory requests will fail when the total memory commit reaches the size of the swap space plus a configurablepercentage (50 by default) of RAM. This factor is modified changing `/proc/sys/vm/overcommit_ratio`.

7.15 Chapter 13.8.a OOM Killer II

If available memory is exhausted, Linux invokes the OOM-killer (Out Of Memory) to decide which process(es) shouldbe exterminated to open up some memory.

There is no precise science for this; the algorithm must be heuristic and cannot satisfy everyone. In the minds of manydevelopers the purpose of the OOM-killer is to permit a graceful shutdown, rather than be a part of normal operations.

An amusing take on this was given by Andries Brouwer (http://lwn.net/Articles/104185/):

“An aircraft company discovered that it was cheaper to fly its planes with less fuel on board. The planes would belighter and use less fuel and money was saved. On rare occasions however the amount of fuel was insufficient, andthe plane would crash. This problem was solved by the engineers of the company by the development of a specialOOF (out-of-fuel) mechanism. In emergency cases a passenger was selected and thrown out of the plane. (Whennecessary, the procedure was repeated.) A large body of theory was developed and many publications were devotedto the problem of properly selecting the victim to be ejected. Should the victim be chosen at random? Or should onechoose the heaviest person? Or the oldest? Should passengers pay in order not to be ejected, so that the victim wouldbe the poorest on board? And if for example the heaviest person was chosen, should there be a special exception incase that was the pilot? Should first class passengers be exempted? Now that the OOF mechanism existed, it wouldbe activated every now and then, and eject passengers even when there was no fuel shortage. The engineers are stillstudying precisely how this malfunction is caused.”

In order to make decisions of who gets sacrificed to keep the system alive, a value called the badness is computed(which can be read from /proc/[pid]/oom_score) for each process on the system and the order of the killing is deter-mined by this value.

Two entries in the same directory can be used to promote or demote the likelihood of extermination. The valueof oom_adj is the number of bits the points should be adjusted by. Normal users can only increase the badness; adecrease (a negative value for oom_adj) can only be specified by a superuser. The value of oom_adj_score directlyadjusts the point value. Note that the use of oom_adj is deprecated.


http://lwn.net/Articles/104185/

CHAPTER 8

KNOWLEDGE CHECK Q&A

8.1 CHAPTER 5 KNOWLEDGE CHECK Q&A

Red Hat Enterprise Linux, SUSE, CentOS and Fedora use the [. . . . . . . . . .] packaging system.

1. RPM

Debian, Ubuntu and Mint use the [. . . . . . . . . .] packaging system.

1. dpkg


‘‘rpm -qa lists all installed packages on the system‘

What rpm command would you use to verify the integrity of /bin/ls?

rpm -V coreutils


dpkg -l lists all installed packages on the system


‘‘apt-get install <package>‘ is used to install a new package.

‘‘apt-get update‘ does not accept a package as argument.

apt-file find <file path> can be used to find which package provides the file specified as argument.

39


‘‘apt-cache search‘ can be used for searching on package name and short description.


1. ___________ is a debug tool that shows how a process makes requests to the operating system.

(a) strace

2. ___________ is a tool that shows for how long the system is running.

(a) uptime

3. ___________ is an interactive tool for monitoring process activity.

(a) top

4. ___________ is a tool that displays the summary of memory usage.

(a) free

5. Given a PID such as 1017, the special file __________ contains the command line used to start the process.

(a) /proc/[PID NUM]/cmdline

6. What information that is not related to processes can be found at /proc? a.

(a) The kernel commandline

(b) CPU model info

(c) Memory utilization stats

(d) Disk Partition info

a2. Also includes more


3 tools to monitor processes:

top, ps, ‘‘pstree‘

What command will show the parent process IDs (PPIDs) for all the processes on the system?

ps -elf

1. Run ps with the options -ef. Then run it again with the options aux. Note the differences in the output.

1. $ ps -ef, $ ps aux‘

2. Run ps so that only the process ID, priority, nice value, and the process command line are displayed.

3. Start a new bash session by typing bash at the command line. Start another bash session using the nice commandbut this time giving it a nice value of 10.

4. Run ps as in step 2 to note the differences in priority and nice values. Note the process ID of the two bashsessions.

5. Change the nice value of one of the bash sessions to 15 using renice. Once again, observe the change in priorityand nice values.

6. Run top and watch the output as it changes. Hit q to stop the program.

Cmds / notes for exam prep.

40 Chapter 8. KNOWLEDGE CHECK Q&A


1. redo all labs from course

2. full filesystem (chapter 2)

To look at: rpm files du, su, dpkg, sar

Taken from LINUX FOUNDATION CERT PREP PDF

Essential Commands - 25% Log into local & remote graphical and text mode consoles Search for files Evaluate andcompare the basic file system features and options Compare and manipulate file content Use input-output redirection(e.g. >, >>, |, 2>) Analyze text using basic regular expressions Archive, backup, compress, unpack, and uncompressfiles Create, delete, copy, and move files and directories Create and manage hard and soft links List, set, andchange standard file permissions Read, and use system documentation Manage access to the root account Operationof Running Systems - 20% Boot, reboot, and shut down a system safely Boot or change system into differentoperating modes Install, configure and troubleshoot bootloaders Diagnose and manage processes Locate and analyzesystem log files Schedule tasks to run at a set date and time Verify completion of scheduled jobs Update softwareto provide required functionality and security Verify the integrity and availability of resources Verify the integrityand availability of key processes Change kernel runtime parameters, persistent and non- persistent Use scriptingto automate system maintenance tasks Manage the startup process and services (In Services Configuration) Listand identify SELinux/AppArmor file and process contexts Manage Software Identify the component of a Linuxdistribution that a file belongs to User and Group Management - 10% Create, delete, and modify local user accountsCreate, delete, and modify local groups and group memberships Manage system-wide environment profiles Managetemplate user environment Configure user resource limits Manage user privileges Configure PAM

Networking - 12% Configure networking and hostname resolution statically or dynamically Configure networkservices to start automatically at boot Implement packet filtering Start, stop, and check the status of network servicesStatically route IP traffic Synchronize time using other network peers Service Configuration - 20% Configure acaching DNS server Maintain a DNS zone Configure email aliases Configure SSH servers and clients Restrict accessto the HTTP proxy server Configure an IMAP and IMAPS service Query and modify the behavior of system servicesat various operating modes Configure an HTTP server Configure HTTP server log files Configure a database serverRestrict access to a web page Manage and configure containers Manage and configure Virtual Machines StorageManagement - 13% List, create, delete, and modify physical storage partitions Manage and configure LVM storageCreate and configure encrypted storage Configure systems to mount file systems at or during boot Configure andmanage swap space Create and manage RAID devices Configure systems to mount file systems on demand Create,manage and diagnose advanced file system permissions Setup user and group disk quotas for filesystems Create andconfigure file systems

8.6. CHAPTER 12 KNOWLEDGE CHECK Q&A 41


42 Chapter 8. KNOWLEDGE CHECK Q&A

CHAPTER 9

License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documen-tation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use,copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whomthe Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of theSoftware.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PAR-TICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHTHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTIONOF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFT-WARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

43


44 Chapter 9. License

CHAPTER 10

Help

you are in the wrong place :/

add commit

45


46 Chapter 10. Help

CHAPTER 11

IndIces and tables

• genindex

• modindex

• search

47

linfun Documentation - Read the Docs

Documents

Transcript of linfun Documentation - Read the Docs