Inefficient process and thread distribution to cores on multicore system

Asked by Jan Lachnitt

I am doing some scientific computations in Ubuntu on a multicore workstation. I will run a multi-threaded program using OpenMP and I want it to run fast. Currently I am testing and debugging this program and meanwhile I have some single-threaded processes running (which are computing auxiliary results). The problem is that Ubuntu distributes the processes and threads very inefficiently between CPU cores.

In the workstation, there is Intel Xeon W3680, which has six physical cores and hyperthreading, so there are 12 logical cores. I can understand if Ubuntu cannot recognize physical cores but I cannot understand why it is using almost only four logical cores. Example - I had five single-thread and one six-thread processes running and they were distributed to the cores this way:
* core 1: two single-thread processes and the six-thread process, i.e. 8 threads running on a single logical core,
* core 3: a single-thread process,
* core 4: a single-thread process,
* core 5: a single-thread process,
* remaining 8 logical cores: almost no load.
My observation is based on Gnome System Monitor and confirmed by the slowness of the processes running on the same core.

As these computations take whole days, it is quite important to run them efficiently. So what can I do? Should I install Debian? ;-) Well, I would appreciate a faster and simpler solution :-) .

Jan

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu linux Edit question
Assignee:
No assignee Edit question
Solved by:
Ubfan
Solved:
Last query:
Last reply:
Revision history for this message
Best Ubfan (ubfan1) said :
#1

Obviously a desktop scheduler is not what you want, but you may exercise some control with the schedutils package. See:
http://www.cyberciti.biz/tips/setting-processor-affinity-certain-task-or-process.html

Revision history for this message
Jan Lachnitt (pepalogik) said :
#2

Thank you very much for the link! With "taskset", I have found out that some of the processes had their affinity masks set to 0x1, while the others to 0xfff. If I set all affinity masks to 0xfff, then everything works fine.

It seems very mysterious to me, who or what set the affinities to 0x1. However, I am glad I can fix it with "taskset" and thank you once again.

Jan