System calls and OS services
System calls are the controlled interface user programs use to request operating-system services; this note explains how they work, common services, the syscall mechanism, and shows small C programs using syscalls.
Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.
Why it matters
System calls are the bridge between user programs and the operating system kernel. An engineer uses system calls when writing programs that need files, processes, memory, or devices — and when implementing OS features or debugging system-level problems.
Key ideas
Definition: OS service — a functionality provided by the kernel (e.g., file operations, process management, networking). Services are implemented behind system calls.
System-call flow (step-by-step, draw this as a vertical flow diagram):
- Draw a left column box labelled "User program". Inside write "application code".
- To the right draw a small box labelled "libc wrapper" that calls the trap instruction.
- Draw an arrow to a middle box labelled "Trap / syscall instruction (software interrupt)".
- Draw an arrow to a right column big box labelled "Kernel". Inside add: "syscall dispatcher -> validate args -> call syscall handler -> service (VFS, scheduler, driver) -> prepare return".
- Draw arrow back to user via "return from trap". Label transitions with "user->kernel mode switch (trap)" and "kernel->user return (sysret/iret)".
This diagram shows the minimal components: user code, libc wrapper, trap, kernel dispatcher, service implementation, and return path.
Mechanism details (exact sequence when a syscall executes):
- The user program calls a library wrapper function (e.g., the C function open()). The wrapper prepares arguments in registers or on the stack following the system's ABI (Application Binary Interface).
- The wrapper executes a special instruction (called a trap instruction, e.g. syscall/sysenter/int 0x80 depending on architecture).
- The CPU switches to kernel mode, saves the user CPU context (PC, registers) and switches to the kernel stack for that thread/process.
- The kernel syscall dispatcher reads the system call number (a small integer, placed in a register) and arguments, validates pointers and sizes, and looks up the syscall handler in the system call table.
- The syscall handler executes the requested service (which may call the Virtual File System, scheduler, or device driver). The kernel may perform blocking: the calling thread can be put to sleep and another scheduled.
- The kernel prepares the return value or error code, restores the saved user context, and executes the return-from-trap instruction to switch back to user mode.
- The wrapper returns to application code, typically setting errno on error and returning -1 or another sentinel.
Assumptions: CPU and ABI support a defined trap/syscall instruction and a calling convention for syscall numbers and arguments; kernel enforces memory safety.
Types of system calls (definitions):
- Process control: create, terminate, wait (e.g., fork, execve, exit, waitpid).
- File manipulation: open, close, read, write, lseek.
- Device management: ioctl, read/write to device files.
- Information maintenance: getpid, alarm, uname.
- Communication: pipe, socket, send/recv.
- Memory management: brk, mmap, munmap.
Blocking vs non-blocking: Blocking system call causes the calling thread to sleep if the resource is not ready (e.g., read() on empty pipe without data). Non-blocking system call returns immediately with an error (e.g., EAGAIN) if the action would block. Non-blocking mode is enabled with O_NONBLOCK at open() or via fcntl(F_SETFL).
Error reporting: Kernel returns a negative integer in a register for many ABIs; library wrapper maps it to -1 and sets errno (a thread-local integer) with the positive error code. The kernel does not set errno.
Performance note: System calls cross the user/kernel boundary and cost more than library calls. Engineers must minimize unnecessary syscalls (e.g., batch I/O, use readv/writev) in performance-sensitive code.
Formulas
(Syntax and prototypes to remember; parameters explained.)
int open(const char *pathname, int flags, ...); — pathname: NUL-terminated file path; flags: O_RDONLY, O_WRONLY, O_RDWR, O_CREAT, O_TRUNC, O_APPEND; if O_CREAT is specified, a third argument mode_t mode supplies file permission bits; returns a nonnegative file descriptor on success or -1 on error.
ssize_t read(int fd, void *buf, size_t count); — fd: file descriptor; buf: pointer to buffer; count: max bytes to read; returns number of bytes read or -1 on error.
ssize_t write(int fd, const void *buf, size_t count); — returns number of bytes written or -1 on error.
int close(int fd); — release file descriptor; returns 0 on success or -1 on error.
pid_t fork(void); — create child process; returns child's PID to parent, 0 to child, or -1 on error.
int execve(const char *pathname, char *const argv[], char *const envp[]); — replace current process image with a new program; does not return on success.
pid_t waitpid(pid_t pid, int *wstatus, int options); — wait for state change in child process.
void _exit(int status); — terminate calling process immediately; does not flush stdio buffers.
void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset); — map file or memory; returns MAP_FAILED ((void *)-1) on error; offset must be a multiple of the page size.
For each: size_t and ssize_t are counts of bytes (unit: bytes). pid_t is process id (integer). mode_t is file mode (bits). off_t is byte offset.
Worked examples
Example 1 — Copy stdin to stdout using low-level system calls (read/write). C program (save as copy_syscall.c):
#include <unistd.h> #include <errno.h> #include <stdlib.h>
int main(void) { char buf[1024]; ssize_t r; while (1) { r = read(0, buf, sizeof(buf)); if (r == 0) break; // EOF if (r < 0) { if (errno == EINTR) continue; _exit(1); } ssize_t w = 0; while (w < r) { ssize_t n = write(1, buf + w, r - w); if (n < 0) { if (errno == EINTR) continue; _exit(1); } w += n; } } return 0; }
Sample run (exact): Input (stdin): Hello world\n Command: printf "Hello world\n" | ./copy_syscall Output (stdout): Hello world\n
Example 2 — fork, parent waits, child uses write and exits. C program (save as fork_wait.c):
#include <unistd.h> #include <sys/wait.h> #include <stdlib.h> #include <stdio.h>
int main(void) { pid_t pid = fork(); if (pid < 0) return 1; if (pid == 0) { // child const char msg[] = "Child: I am running.\n"; write(1, msg, sizeof(msg)-1); _exit(42); } else { // parent int status; pid_t w = waitpid(pid, &status, 0); if (w == pid) { if (WIFEXITED(status)) { char buf[100]; int n = snprintf(buf, sizeof(buf), "Parent: child %ld exited with %d.\n", (long)pid, WEXITSTATUS(status)); write(1, buf, n); } } } return 0; }
Sample run (exact): Command: ./fork_wait Output (stdout): Child: I am running. Parent: child 12345 exited with 42.
(Note: the PID number 12345 above will be the actual child's PID on the test machine; in an exam answer say "replace 12345 with actual PID".)
Common mistakes
- Forgetting to retry on EINTR for read/write.
- Assuming read(fd,buf,n) always returns n bytes; it can return fewer (short read) even for regular files in some cases — always loop until done if you need exact bytes.
- Mixing stdio buffered I/O (fwrite/printf) with low-level syscalls without flushing; this can duplicate or reorder output.
- Not closing file descriptors; leaking descriptors leads to EMFILE.
For your exam
Kinds of questions you will see:
- Definitions and short notes: Define system call, explain user vs kernel mode, or list syscall types.
- Diagrams: Draw the syscall flow (user program -> libc wrapper -> trap -> kernel dispatcher -> handler -> return). Label registers, stack switch, and syscall table lookup.
- Derivations/explanations: Step-by-step sequence of what happens on a syscall, including context save/restore and argument validation.
- Small programs: Write C code that uses read/write or fork/exec; include sample input and output.
- Error-handling questions: Explain errno, EINTR, and how to robustly use read/write.
What a full-marks answer includes:
- Clear definitions with bolded terms if asked.
- A correct labeled diagram when required.
- Step-by-step description of the syscall mechanism, including where arguments go (registers or stack), the trap instruction, dispatcher, and return path.
- Example syscall prototypes and explanation of parameters.
- For code questions, correct compilation and sample run, checking edge cases (EOF, errors).
Quick check
- What instruction does a user program execute to enter the kernel for a system call?
- How are syscall errors conveyed to user space, and who sets errno?
- True/False: read() always fills the buffer completely unless EOF or error.
Answers: 1) a trap/syscall instruction (e.g., syscall, sysenter, or software interrupt) 2) The kernel returns a negative error code in the return register; the libc wrapper returns -1 and sets errno to the positive code 3) False (read can return fewer bytes than requested)
Finished this topic? Mark it so your progress, study plan and readiness keep up.
Stuck on something here?