Christopher Hallinan - Embedded Linux Primer - A Practical, Real-World Approach

cat-6637 0D.h1 10us : redirect_hardirq (__do_IRQ)

cat-6637 0D.h. 12us : handle_IRQ_event (__do_IRQ)

cat-6637 0D.h. 13us : timer_interrupt (handle_IRQ_event)

cat-6637 0D.h. 15us : handle_tick_update (timer_interrupt)

cat-6637 0D.h1 16us : do_timer (handle_tick_update)

...

cat-6637 0D.h. 22us : run_local_timers (update_process_times)

cat-6637 0D.h. 22us : raise_softirq (run_local_timers)

cat-6637 0D.h. 23us : wakeup_softirqd (raise_softirq)

...

cat-6637 0Dnh. 34us : wake_up_process (wakeup_softirqd)

cat-6637 0Dnh. 35us+: rcu_pending (update_process_times)

cat-6637 0Dnh. 39us : scheduler_tick (update_process_times)

cat-6637 0Dnh. 39us : sched_clock (scheduler_tick)

cat-6637 0Dnh1 41us : task_timeslice (scheduler_tick)

cat-6637 0Dnh. 42us+: preempt_schedule (scheduler_tick)

cat-6637 0Dnh1 45us : note_interrupt (__do_IRQ)

cat-6637 0Dnh1 45us : enable_8259A_irq (__do_IRQ)

cat-6637 0Dnh1 47us : preempt_schedule (enable_8259A_irq)

cat-6637 0Dnh. 48us : preempt_schedule (__do_IRQ)

cat-6637 0Dnh. 48us : irq_exit (do_IRQ)

cat-6637 0Dn.. 49us : preempt_schedule_irq (need_resched)

cat-6637 0Dn.. 50us : __schedule (preempt_schedule_irq)

...

<...>-3 0D..2 74us+: __switch_to (__schedule)

<...>-3 0D..2 76us : __schedule (74 62)

<...>-3 0D..2 77us : __schedule (schedule)

<...>-3 0D..2 78us : trace_irqs_on (__schedule)

...

We have trimmed this listing significantly for clarity, but the key elements of this trace are obvious. This trace resulted from a timer interrupt. In the hardirq thread, little is done beyond queuing up some work for later in a softirq context. This is seen by the wakeup_softirqd() function at 23 microseconds and is typical for interrupt processing. This triggers the need_resched flag, as shown in the trace by the n in the third column of the second field. At 49 microseconds, after some processing in the timer softirq, the scheduler is invoked for preemption. At 74 microseconds, control is passed to the actual softirqd-timer/0 thread running in this particular kernel as PID 3. (The process name was truncated to fit the field width and is shown as <...> .)

Most of the fields of Listing 17-7 have obvious meanings. The irqs-off field contains a D for sections of code where interrupts are off. Because this latency trace is an interrupts off trace, we see this indicated throughout the trace. The need_resched field mirrors the state of the kernel's need_resched flag. An n indicates that the scheduler should be run at the soonest opportunity, and a period (.) means that this flag is not active. The hardirq/softirq field indicates a thread of execution in hardirq context with h, and softirq context with s. The preempt-depth field indicates the value of the kernel's preempt_count variable, an indicator of nesting level of locks within the kernel. Preemption can occur only when this variable is at zero.

The DEBUG_DEADLOCKS kernel configuration option enables detection and reporting of deadlock conditions associated with the semaphores and spinlocks in the kernel. When enabled, potential deadlock conditions are reported in a fashion similar to this:

==========================================

[ BUG: lock recursion deadlock detected! |

------------------------------------------

...

Much information is displayed after the banner line announcing the deadlock detection, including the lock descriptor, lock name (if available), lock file and name (if available), lock owner, who is currently holding the lock, and so on. Using this debug tool, it is possible to immediately determine the offending processes. Of course, fixing it might not be so easy!

The DEBUG_RT_LOCKING_MODE option enables a runtime control to switch the real-time mutex back into a nonpreemptable mode, effectively changing the behavior of the real-time (spinlocks as mutexes) kernel back to a spinlock-based kernel. As with the other configuration options we have covered here, this tool should be considered a development aid to be used only in a development environment.

It does not make sense to enable all of these debug modes at once. As you might imagine, most of these debug modes add size and significant processing overhead to the kernel. They are meant to be used as development aids and should be disabled for production code.

• Linux is increasingly being used in systems where real-time performance is required. Examples include multimedia applications and robot, industrial, and automotive controllers.

• Real-time systems are characterized by deadlines. When a missed deadline results in inconvenience or a diminished customer experience, we refer to this as soft real time. In contrast, hard real-time systems are considered failed when a deadline is missed.

• Kernel preemption was the first significant feature in the Linux kernel that addressed system-wide latency.

• Recent Linux kernels support several preemption modes, ranging from no preemption to full real-time preemption.

• The real-time patch adds several key features to the Linux kernel, resulting in reliable low latencies.

• The real-time patch includes several important measurement tools to aid in debugging and characterizing a real-time Linux implementation.

Linux Kernel Development , 2nd Edition

Robert Love

Novell Press, 2005

This is an exact reproduction of the GLP license as authored and published by the Free Software Foundation. An electronic copy can be obtained at www.fsf.org.

Version 2, June 1991

Copyright © 1989, 1991 Free Software Foundation, Inc.

51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free softwareto make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too.

When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.

To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.