Process’s afinity
Using schedutils, one can alter any process’s afinity by running a simple command: taskset. This way, it is feasible to change the GNU/Linux scheduler behaviour when it comes to running processes on a multi-core GNU/Linux box. For example, if we run any command directly, on a dual-core system, GNU/Linux Kernel sets the default afinity for a process this way:
etch32:~/dev/modest.cvs/kmodest-src-2010# taskset -p 2047
pid 2047’s current affinity mask: 3
In this case, taskset is showing us that 2047 process could be executed either on cpu 0 or CPU 1 – affinity mask of 3 -. So, which CPU is running right now this process?
The task_struct * data structure
Thinking back on it, we do know the task_struct * internal data structure, controlled by the GNU/Linux Kernel. We can determine which CPU belongs to a given process at any time thanks to it, no matter if that process is running a system call – Ring 0 – or it is still executing instructions in Ring 3 – user land -. Taking a quick look at this data structure inside the GNU/Linux Kernel sources, it seems very clear how to achieve that:
767 struct task_struct { 768 volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ 769 struct thread_info *thread_info; (...) 27 struct thread_info { 28 struct task_struct *task; /* main task structure */ 29 struct exec_domain *exec_domain; /* execution domain */ 30 unsigned long flags; /* low level flags */ 31 unsigned long status; /* thread-synchronous flags */ 32 __u32 cpu; /* current CPU */ 33 int preempt_count; /* 0 => preemptable, <0 => BUG */ 34 35 36 mm_segment_t addr_limit; /* thread address space: 37 0-0xBFFFFFFF for user-thead 38 0-0xFFFFFFFF for kernel-thread 39 */ 40 void *sysenter_return; 41 struct restart_block restart_block; 42 43 unsigned long previous_esp; /* ESP of the previous stack in case 44 of nested (IRQ) stacks 45 */ 46 __u8 supervisor_stack[0]; 47 }; |
Clearly, the thread_info pointer has a field called “cpu“, just an unsigned int 32 value storing the CPU number running this process. This data structure is declared in the include/asm-i386/thread_info.h header file.
Using kmodest to determine which CPU is running a given PID process.
Some little changes were made to the original modest.c and umodest.c source files so as to be able to find out which CPU is running our PID process at any moment, just by running a simple command through the umodest user-land utility. Thus, I added this piece of code in order to get “on-the-fly” that very CPU for a given process as soon as a user-land process is trying to read this information after sending the F_GETCPU ioctl command, declared in the common-structures.h header file:
204 /* Read just a byte - 0-255- storing on which cpu this process is being executed : */ 205 case F_GETCPU: 206 /* Only one byte to read allowed */ 207 if(count<0 || count>sizeof(char))return -EINVAL; 208 /* Okay, get the current CPU and try to copy to user space memory area: */ 209 /* So, we need to reference this PID inside task_struct list : */ 210 p = find_task_by_pid (pid_cpu); 211 if(p){ 212 k = copy_to_user(bf, &p->thread_info->cpu , sizeof(char)); 213 if(k!=0)return -EFAULT; 214 } else return -EFAULT; 215 /* Update file pointer */ 216 *fp_pos+=1; real_read = sizeof(char); 217 break; |
Then, I added a few code lines inside umodest utility to allow it to deal with this new functionality:
158 /* Get information about on what CPU this process is being executed */ 159 if(strcmp(*(argv+3),"-s")==0){ 160 myargs.command = F_GETCPU; 161 ioctl_return = ioctl(drv,F_GETCPU,&myargs); 162 if(ioctl_return!=0){ 163 perror("Sending command to device"); return errno; 164 } 165 /* Read from "device" and print it out. It's just a char (0-255) : */ 166 syscall_ret = read(drv, &on_which_cpu, sizeof(char)); 167 if(syscall_ret==1)printf("Process with PID: %d is running on CPU: %d.\n" , myargs.pid , on_which_cpu); 168 else printf("Error getting CPU\n"); 169 } |
Using the umodest utility
Now, it is feasible to figure out on which CPU a given process is being executed at any moment this way:
# umodest -p PID -s
For example, running umodest on a dual-core GNU/Linux box to determine syslog’s CPU it gives:
etch32:~/dev/modest.cvs/kmodest-src-2010# taskset -p 2047
pid 2047’s current affinity mask: 3
etch32:~/dev/modest.cvs/kmodest-src-2010# bin/umodest -p 2047 -s
Process with PID: 2047 is running on CPU: 0.
Well, it’s working!. Now, what if we alter that PID’s affinity using taskset? Let’s see what happens:
etch32:~/dev/modest.cvs/kmodest-src-2010# taskset -p 0x2 2047
pid 2047’s current affinity mask: 3
pid 2047’s new affinity mask: 2
etch32:~/dev/modest.cvs/kmodest-src-2010# bin/umodest -p 2047 -s
Process with PID: 2047 is running on CPU: 1.
Now, after using taskset, our syslog process is being executed on CPU number 1, because of its affinity mask, that is, 0x2, i.e 0010, CPU number 1. (Don’t forget the first one is numbered as CPU number zero).
External links and references
Take a look at how to use schedutils and all about process’s affinity by reading this.
The latest kmodest source files with this new functionality can be downloaded from our CVS repository, right here.
There’s a new video showing how this new functionality works right here.