MULTITHREADING AND MULTICORE CPU

After posting the post JAVA CONCURRENCY - PERFORMANCE BOOST, I got a question "how it is going to work in single core CPU and multi core CPU". I was demonstrating the same set of source code for that and I found the program is utilizing only one core of the CPU. I took the help of "htop" tool in Ubuntu to monitor the CPU usage ("htop" is a wrapper around "top" and is having a user friendly UI). My be the purpose of that post was to demonstrate excecuting multiple tasks in parallel and specially the callback approach to save time.


CPU Usage with the program posted in above post:

cpu usage using htop

CPU core no 3 is been utilized with 100% and rest all other cores are hardly used. After lot of google and several coding exercise, I came to the following understaning. I wrote a small program to demonstrate my finding.

Even a single CPU can do "multiple things at the same time" in a loose sense, but they are not truly in parallel. You can start 100 threads to run on a single core and they will get time slices during which each of them can run a few instructions, thus creating the impression that they are all executing at the same time.


The term threads usually covers three abstraction layers:

User threads are threads launched by applications and are mapped N:M to:
Kernel threads, which are threads managed by the operating system, mapped N:M to:
Hardware threads, which are the actual physical resources available.

Java threads are user threads. The 4 cores in your CPU count as hardware threads. Since the mapping is N:M across the layers, you can see that you can have several user threads mapped to a smaller number of hardware threads.

Now, having said this, there are generally two classes of thread activities, each with their own quirks:

I/O threads: these threads spend most of their time waiting on read/write operations from a stream and are blocked in the meantime (they are not scheduled for execution until an event occurs to wake them up). There are light on the CPU and a lot of them can run concurrently even on a single core.

Computational threads: these thread do a lot of number crunching and use the CPU to the maximum. Generally starting more than (2x the number of available cores) such threads is going to degrade performance, because the CPU has a limited number of functional units: ALUs, FPUs, etc.
The second class of threads above lets you really see the benefit or running a multithreaded java program on your quad-core CPU. Here is a simple example of a program that executes squaring of 1.000.000.000 numbers first sequentially and then in parallel using a thread pool with 4 threads:

package org.satish.concurrency;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

class ThreadTask implements Runnable {

    private int total = 0;

    public ThreadTask(int total) {
        this.total = total;
    }

    @Override
    public void run() {
        int value = 0;
        for(int i = 0; i < total; i++) {
            System.out.println(value);
            value = i * i;
        }
    }       
}

public class MultiCoreTester {

    public static void main(String[] args) throws InterruptedException {

        int total = 1000000000;

        long start = System.currentTimeMillis();
        long value = 0;
        for(int i = 0; i < total; i++) {
            value = i * i;
        }       
        long stop = System.currentTimeMillis();

        System.out.println((stop - start) + " ms");

        ExecutorService exec = Executors.newFixedThreadPool(4);
        start = System.currentTimeMillis();
        for(int i = 0; i < 4; i++) {
            exec.submit(new ThreadTask(total / 4));
        }
        exec.shutdown();
        exec.awaitTermination(10, TimeUnit.SECONDS);
        stop = System.currentTimeMillis();

        System.out.println((stop - start) + " ms");     
    }
}

cpu usage before staring the program using htop

cpu usage after staring the program using htop

Before our program starts, all the four cores are hardy used. And once the program get started all the four cores are been used equally.