Difficulty: Beginner | Prerequisites: Basic understanding of what a CPU does.
This material covers how computers manage to run multiple programs at once, even when the hardware only has one or two processor cores. It also introduces virtualisation, where a single physical machine pretends to be several machines at once. These ideas sit at the heart of how every modern computer, phone, and cloud server operates. If you have used a computer while listening to music and browsing the web simultaneously, you have already benefited from concurrency.
Concurrency is the ability of a system to handle multiple tasks in overlapping time periods. On a single-core machine this is faked by rapid switching; on a multi-core machine some tasks can run in true parallel. Hypervisors take this a level further by letting entire operating systems share one physical machine. More cores help, but they do not multiply speed proportionally because tasks have parts that cannot be split up.
Concurrency
The general concept of multiple tasks making progress within overlapping time periods. It does not require that they execute at literally the same instant. In simple terms, this means the computer juggles several jobs at once, even if it can only work on one at any given moment.
Parallelism
Multiple tasks executing at exactly the same instant, each on its own processor core. Think of it as two people each solving a different maths problem at the same time, side by side.
Time-slicing (time-sharing)
A technique where the operating system gives each running process a tiny slice of CPU time, then switches to the next process. The switching happens so fast that it looks simultaneous to the user. In simple terms, the CPU takes turns really quickly, so everything feels smooth.
Process
A running instance of a program, including its code, data, and the resources the operating system has assigned to it. Think of it as one copy of an app that is alive in memory and doing work (or waiting for its turn).
Context switch
The act of saving the state of the currently running process and loading the state of the next one so the CPU can switch between tasks. In simple terms, the CPU bookmarks where it was, picks up a different book, and reads from that bookmark instead.
Operating system (OS)
System software that manages hardware resources and provides services to application programs. It schedules processes, manages memory, and handles input/output. Think of it as the manager of the computer: it decides who gets CPU time, memory, and access to devices.
Hypervisor (virtual machine monitor)
Software that creates and manages virtual machines, allowing multiple operating systems to share a single set of physical hardware. In simple terms, it is a layer that sits beneath (or beside) operating systems and tricks each one into thinking it has its own dedicated computer.
Virtual machine (VM)
A software-emulated computer that runs its own operating system and applications as though it were a separate physical machine. Think of it as a computer inside a computer.
Amdahl's Law (concept)
The principle that the speedup from adding more processors is limited by the portion of the task that must be done sequentially (one step at a time). In simple terms, if 10% of a job cannot be split up, no amount of extra workers will make that 10% go faster.
A single-core CPU can execute only one instruction at a time.
The operating system uses time-slicing to rotate between processes very rapidly (often thousands of times per second).
Each process gets a short burst of CPU time, then the OS performs a context switch to the next process.
To the user, it appears that multiple applications are running simultaneously, but in reality only one is executing at any given instant.
A dual-core (or multi-core) processor has two or more independent processing units on the same chip.
With two cores, two processes can execute at literally the same instant, one on each core.
The OS still time-slices if there are more processes than cores, but overall throughput is higher because true parallelism is possible.
Example: on a dual-core machine running a music player and a web browser, each app could get its own core, meaning neither has to wait for the other.
Many tasks have sequential portions that cannot be divided among multiple workers.
Coordination overhead: the more workers (cores) involved, the more time is spent communicating, synchronising, and dividing up the work.
The puzzle analogy: if 1,000 people each hold one piece of a 1,000-piece puzzle, they cannot all place their piece at the same time because placing a piece depends on knowing where adjacent pieces already are. People block each other, argue over the same section, and spend time coordinating rather than solving.
In computing terms, tasks such as reading a shared file, writing results to one output, or waiting for a previous calculation's result all force sequential execution regardless of how many cores are available.
Amdahl's Law formalises this: if a fraction s of a task is serial, the maximum speedup with N cores approaches 1/s, no matter how large N becomes.
Every running process consumes CPU time, memory (RAM), and potentially disk and network resources.
If dozens of copies of an unwanted application are running, the OS must time-slice among all of them, leaving very little CPU time for the programs the user wants.
RAM fills up, forcing the OS to swap data to the much slower hard drive (thrashing), which makes everything dramatically slower.
This is a common symptom of malware: a malicious program spawns many copies of itself, deliberately or through a bug, consuming resources until the machine is nearly unusable.
Killing the processes frees resources immediately; deleting the offending application prevents it from restarting. However, the user should also run a full malware scan, change passwords, and check for other compromised software, because the unwanted app may have installed additional threats.
An operating system manages processes (individual applications) on one machine.
A hypervisor manages entire virtual machines, each of which runs its own OS and its own set of processes.
Similarities in handling concurrency:
Both allocate CPU time among competing workloads.
Both use scheduling algorithms to decide who runs next.
Both must handle situations where workloads compete for the same physical resources (CPU, memory, I/O).
Differences:
The OS schedules processes; the hypervisor schedules virtual machines (each containing an OS that, in turn, schedules its own processes).
The hypervisor provides hardware-level isolation between VMs: one VM crashing does not bring down another. An OS provides process-level isolation, which is lighter but less robust.
A Type 1 (bare-metal) hypervisor runs directly on the hardware with no host OS beneath it. A Type 2 (hosted) hypervisor runs on top of a conventional OS.
Hypervisors are central to cloud computing: a single physical server in a data centre might host dozens of VMs for different customers, each unaware of the others.
Time-slicing is why your phone can play a podcast while you scroll through messages, even though most phones have only a handful of cores. Hypervisors are the backbone of cloud platforms like AWS and Azure, where thousands of customers share physical servers without seeing each other's data. Amdahl's Law is the reason adding more servers to a database cluster eventually stops improving query speed: the lock on the shared data becomes the bottleneck.
Students often think concurrency and parallelism are the same thing. Concurrency means tasks overlap in time; parallelism means they execute at the same instant. A single-core machine can have concurrency but never true parallelism.
Students often assume that doubling the cores doubles the speed. It does not, because of sequential portions of tasks and coordination overhead.
Students sometimes think a hypervisor is just "another operating system." A hypervisor manages operating systems, not applications directly. The abstraction level is different.
Students may confuse processes with programs. A program is a file on disk. A process is a running instance of that program. You can have many processes from one program.
⚠️ Be ready to explain the difference between concurrency on a single-core vs a multi-core machine. This is a classic exam question.
⚠️ Know why more cores do not scale linearly. The puzzle analogy is a useful way to frame your answer.
⚠️ Understand the hypervisor vs OS distinction and be able to name at least one similarity and one difference in how they handle concurrency.
⚠️ If asked about a slow computer with many mysterious processes, connect it to resource contention and likely malware.
True or false: A single-core processor can run two applications concurrently. Answer: True. It uses time-slicing to alternate between them rapidly.
Fill in the blank: A __________ manages virtual machines, while an operating system manages processes. Answer: Hypervisor.
True or false: If a task is 100% parallelisable, doubling the cores will double the speed. Answer: True, in theory, but real tasks are never 100% parallelisable.
Fill in the blank: The overhead of saving and restoring a process's state when the CPU switches tasks is called a __________. Answer: Context switch.
Q: What does concurrency mean in computing, and how does it differ on a single-core vs a dual-core machine?
A: Concurrency means multiple tasks make progress in overlapping time periods. On a single-core machine, the OS achieves this through time-slicing, rapidly switching between processes so they appear simultaneous. On a dual-core machine, two processes can execute at the same instant (true parallelism), and time-slicing is still used if there are more processes than cores.
Q: Why would dozens of copies of an unknown application cause a computer to become very slow?
A: Each process consumes CPU time and memory. With dozens of extra processes, the OS must divide CPU time among all of them, leaving little for the user's actual applications. Memory fills up, potentially causing disk swapping (thrashing). The likely cause is malware, and the user should run a full security scan after removing the offending app.
Q: Explain one similarity and one difference between how a hypervisor and an OS handle concurrency.
A: Similarity: both use scheduling to allocate CPU time among competing workloads. Difference: an OS schedules individual processes, while a hypervisor schedules entire virtual machines, each of which contains its own OS scheduling its own processes.
Q: Why does having N processor cores not make a computer N times faster?
A: Every real task has some sequential portion that cannot be divided among cores. Additional cores also introduce coordination overhead (synchronisation, communication). Amdahl's Law states that the speedup is limited by the serial fraction of the task, so beyond a certain point, adding cores yields diminishing returns.
This material connects directly to operating system design (process scheduling, memory management) covered in later CS courses. The hypervisor content is foundational for understanding cloud computing and containerisation (e.g., Docker, Kubernetes). Amdahl's Law reappears in algorithm design and distributed systems courses whenever you evaluate whether parallelising a solution is worthwhile.
concurrency, parallelism, time-slicing, time-sharing, context switch, process, thread, multi-core, dual-core, single-core, CPU scheduling, round-robin scheduling, hypervisor, virtual machine, VM, VMware, VirtualBox, Type 1 hypervisor, Type 2 hypervisor, bare-metal hypervisor, Amdahl's Law, parallel speedup, diminishing returns, malware, runaway process, resource contention, thrashing, CS 10100, CS 101, Purdue, Foundations of Computer Science