Linux performance analysis and tuning (English) (Virtueel)

Lesmethode :

Virtueel

Algemeen :

The training "Linux performance analysis and tuning" provides an insight into the methods and techniques to measure the performance of a Linux system, to interpret the results, and to implement improvements. Buying additional hardware usually is not an appropriate solution to a performance problem. The course teaches how to measure the load on the four most important hardware resources (CPU, memory, disk, network), and to prevent overloading by rearranging the configuration, by spreading or limiting the load or, as a last resort, to purchase additional hardware. Lab exercises, - under the guidance of the course teacher - will provide the necessary practical skills.

Course attendees receive the following documentation:

• Course attendants will receive a student pack containing copies of the presentations, lab exercises, answers to the exercises, and background information. This course material is in English.
• Shortly after the course the student will receive a certificate as a proof of participation

Doel :

Performance

Improving the performance and responsiveness are topics that appeal to every system administrator in a natural way. System load increases gradually, and performance decreases as a result. This is caused by the multi user and multi tasking aspect of the Linux architecture, and by the fact that newer software exhibits an ever increasing appetite to consume system resources. The system administrator is the first to face the resulting complaints of the users and of the company management.

Measure, understand, improve

This course discusses the components most likely involved in the loss of performance by overloading. The critical ones are: CPUs, disks, memory and network connections.

Many software tools produce endless lists of statistical counters and good looking graphs: the measuring part. Problems usually start when conclusions must be drawn from this hodgepodge of numbers or lines: what is (not) relevant in the given situation, and what is the mutual relation between all those values: the understanding. Well-founded conclusions can only be based upon knowledge about the way a Linux kernel manages a certain system component, and how the capacity is distributed.

For this reason, the course focussus heavily upon the internal mechanisms of a Linux kernel. Once a bottle neck is identified, remedying it often is straightforward: the improve. A large selection of suggestions for improvement will be provided. Software tools for extensive measuring will be discussed, and extra tools will be provided as part of the course package. This will guide the course attendants to a better analysis of Linux system performance, and towards well founded performance improvement.

Doelgroep :

Experienced system administrators who need to manage their systems effectively, and to get the best performance from all software and hardware components.
The teaching language will be English unless the attendants prefer the Dutch language unanimously.

Voorkennis :

You are an experienced administrator of Linux systems.
Basic knowledge may be obtained via our courses "Linux/UNIX part 1", followed by "Linux/UNIX part 2", and augmented by further training for or experience in systems administration
En de volgende cursussen:

• Linux/UNIX Fundamentals (English)

Onderwerpen :

- Plan of action for performance problems: measure, understand and improve.
- Potential bottle necks and rules of thumb for measuring.
- The /proc and /sys file systems.
- Reading statistical counters from the kernel. Categories of counters. Misleading counters.
- Monitoring and adapting kernel settings.
- Process management: Programs and processes. Creation and transformation of processes. Zombie processes. Life cycle of a process. Role of the kernel in a Linux system. Sleep and wakeup mechanism. Processes and threads. Analysis of blocked threads. Stolen time.
- Overview of commonly used measurement tools. Importance and use of the measuring tools atop and atopsar.
- Control groups (cgroups) in general: limiting the use of hardware components.
- Processor:
   - Hardware (multi socket, multi core, hyperthreading, NUMA), instruction execution, variable clock speed (frequency scaling, cache sharing. etc), instructions per clock cycle.
   - Time sharing and real time processes. Thread states.
   - Thread scheduling, policies, nice value, implementation of the the CFS scheduler (classes deadline, realtime and fair).
   - Task groups and autogroup.
   - Scheduling domains.
   - Measuring with atop, atopsar, ltrace, strace, perf and many other commands.
   - Improving with nice, chrt, taskset and numactl (NUMA) commands, and with the cgroups controllers 'cpu' and 'cpuset'.
- Memory:
   - Hardware (Memory Management Unit), virtual versus physical addressing, Translation Lookaside Buffer (TLB), page directories and page tables.
   - Demand paging, virtual and physical process sizes, memory layout, swap space, memory leakage, overcommit, OOM killing, huge pages, samepage merging, nodes and zones, swapout mechanism, NUMA memory policies.
   - Influence of kernel parameters.
   - Measuring with atop, atopsar and many other commands.
   - NUMA improvement with migratepages and numactl commands.
   - Limiting memory with help of the cgroups controller 'memory'.
- Disk:
   - Hardware, disk characteristics of HDD (zones) and SDD (wear leveling, page allocation, trim), thin provisioning, disk controllers.
   - Queueing algorithms and I/O scheduling (CFQ, etc), block layer, RAID techniques, Logical Volume Management (LVM), layout of ext4 and XFS filesystem, filesystem allocation strategy, pathname cache, workings of page cache and cache flushing.
   - Journaling.
   - Influence of kernel parameters.
   - Measuring with atop, atopsar, countcat and many other commands.
   - Improving with ionice, defragmentation tools, fstrim (SSD) and with help of the cgroups controller 'blkio'.
- Network:
   - Hardware (interface controllers, switches, routers).
   - TCP/IP protocol (bandwidth delay product), network delays, SNMP.
   - Measure with atop, atopsar, netstat, traceroute, attract, iftop, iptables, SNMP and many other commands.
   - Use of the kernel module netatop.
   - Improve by partitioning network bandwidth with help of the cgroups controller 'net_cls'.