From Code to Program: Unveiling the Hidden Stages of Compiler Design

Every time you hit “run”, a remarkable chain of events unfolds behind the scenes. Your high-level source code — readable to humans — gets transformed into binary instructions that a processor can execute directly. The engine behind that transformation is the compiler, and understanding how it works demystifies a huge portion of how software actually runs.

What is a Compiler?

A compiler is a software tool that takes the entire source code written in a high-level programming language and converts it into executable machine code or object code.

The translation process involves several key stages:

• Lexical analysis
• Syntax analysis
• Semantic analysis
• Intermediate code generation
• Code optimisation
• Machine code generation

An important compiler responsibility is detecting and reporting errors in source programs during translation. When you see a type error or an undeclared variable in your IDE, the compiler’s analysis phases are at work.

Types of Language Translators

Not every tool that processes source code is a compiler in the strict sense. It helps to know the family:

Interpreter — Converts and executes source code line by line rather than converting everything beforehand. Commonly used for scripting languages like JavaScript and Python. Slower for repeated execution but faster to get started.

Assembler — Converts assembly language code into machine code or object code for direct computer execution. Works one level below high-level compilers.

Transcompiler — A language processor that translates source code from one high-level language to another. TypeScript-to-JavaScript is a well-known modern example.

Hybrid Compilers — Combine compilation and interpretation. Java exemplifies this: source code is first compiled into platform-independent bytecode, then the Java Virtual Machine executes it — initially by interpreting the bytecode directly, and then by Just-In-Time (JIT) compiling frequently-executed paths into native machine code for speed. Interpretation and JIT compilation are two distinct strategies; the JVM uses both.

In its essence, a compiler is a translator that allows programmers to write code in a language that is more natural to them and then automatically converts it into a language that the computer can understand.

Flow of Transformation

The Language Processor

Language processors enable programmers to write high-level code and translate it into executable machine code. They also coordinate multiple programs needed for execution — for example, linking standard libraries like C’s stdio.h to make printf available.

The Preprocessor

The preprocessor is the first stage to touch source code, before compilation proper begins. It:

• Handles #include directives, inserting header file contents
• Expands macros into source language statements
• Strips or retains comments (most compilers remove them during lexical analysis)

Macros are preprocessor directives that let programmers define reusable code sequences:

// Defines a macro "MAX" that calculates the highest of two values
#define MAX(a, b) ((a) > (b) ? (a) : (b))

int x = 10, y = 20;
int max_val = MAX(x, y);  // expands to: ((10) > (20) ? (10) : (20))

Linker and Loader

Once compilation produces object files, two more components complete the journey to a running program:

Linker — Combines multiple object files into a single executable. It resolves references between files, ensures all symbols are defined, and can strip unused code.

Loader — Loads the final executable into memory. It maps the file from disk into RAM, resolves any remaining external references, and sets up the initial program state so execution can begin.

Modern systems typically merge both roles into a single linker/loader program, enabling the complex multi-module software we take for granted today.

Analysis vs Synthesis: The Two Halves

The compilation pipeline divides cleanly into two halves:

Analysis — The Front End

The analysis phase:

• Breaks the source program into constituent pieces
• Imposes grammatical structure on those pieces
• Creates an intermediate representation of the program
• Collects information stored in a Symbol Table

The symbol table travels through the entire pipeline alongside the intermediate representation, carrying type information, scope, and declarations.

Synthesis — The Back End

The synthesis phase:

• Constructs the target program from the intermediate representation
• Uses the symbol table to resolve names and types
• Produces target machine code ready for assembly or direct execution

Different compiler implementations may group stages differently, and some skip explicit intermediate representations. But the symbol table is universal — every serious compiler maintains one.

The Six Stages of Compilation

Stage 1 — Lexical Analysis

The scanner reads source code character by character and groups characters into tokens — the smallest meaningful units of the language. Tokens represent keywords, identifiers, operators, literals, and punctuation.

int sum = a + b;

The lexer turns that line into a token stream: [KEYWORD: int] [IDENTIFIER: sum] [OPERATOR: =] [IDENTIFIER: a] [OPERATOR: +] [IDENTIFIER: b] [PUNCTUATION: ;]

Stage 2 — Syntax Analysis

Also called parsing, this stage verifies that the token sequence conforms to the programming language’s grammar — the rules that define how tokens combine into valid statements and expressions. When tokens don’t match the grammar, the parser reports a syntax error.

The output is a parse tree or Abstract Syntax Tree (AST) representing the syntactic structure of the program.

Stage 3 — Semantic Analysis

Syntax correctness doesn’t guarantee meaning. The semantic analyser validates that the parse tree makes sense according to the language’s rules:

• Type checking — is a string being added to an int?
• Undeclared variable detection — is foo in scope?
• Function call validation — are the right number and types of arguments passed?

The output is an annotated AST carrying type and meaning information.

Stage 4 — Intermediate Code Generation

After semantic analysis, the compiler generates a machine-architecture-independent intermediate representation (IR). The IR is simpler than the source language but richer than machine code — an ideal target for the optimisation stage.

Common IR forms include three-address code, stack-based code, and bytecode. Java’s .class files are a well-known example of bytecode IR.

// Three-address code for: sum = a + b
t1 = a + b
sum = t1

Stage 5 — Code Optimisation

The optimiser applies transformations to the IR to produce faster or smaller code without changing its observable behaviour:

• Constant folding — evaluate 2 + 3 at compile time rather than runtime
• Loop unrolling — reduce loop overhead by duplicating the loop body
• Function inlining — replace a function call with the function’s body directly
• Dead code elimination — remove code whose results are never used

Stage 6 — Machine Code Generation

The final stage translates the optimised IR into instructions for the target hardware. The output might be assembly language, relocatable object code, or a fully linked executable — depending on the compiler and target.

The resulting binary is what actually runs on your CPU.

Why This Matters

Understanding compiler stages isn’t just academic. It explains:

• Why some errors are caught at compile time (semantic analysis) while others only appear at runtime
• Why compiled languages are generally faster than interpreted ones (optimisation + direct machine code)
• Why cross-compilation is possible — the front end (analysis) is language-specific; the back end (synthesis) is target-specific; only the IR is shared
• How IDEs give you real-time type errors — they run the front end continuously as you type

The next article in this series dives deep into Stage 1 — lexical analysis — the scanning and tokenisation process that every other stage depends on.