python inner working

In this blog , we will be discussing about inner working in Python that how python source code converted into executable code.

Unveiling Python's Magic: How Your Code Actually Runs!

Ever wondered what happens behind the scenes when you hit "run" on your Python script? Many people debate whether Python is an "interpreted" or a "compiled" language. The truth is, it involves a bit of both! Understanding this "inner working" is incredibly insightful and can deepen your appreciation for Python. Let's break down the fascinating steps your Python code takes to execute.

For a more detailed explanation and visual context regarding Python's inner workings, please refer to the following YouTube video link : https://www.youtube.com/watch?v=3HTKc-ZgZbg&list=PLu71SKxNbfoBsMugTFALhdLlZ5VOqCg2s&index=3&ab_channel=ChaiaurCode

1. Your Python Source Code (.py files)

   It all starts with you writing your Python program, often saved as a .py file (like hello.py). This file contains all your instructions, whether it's for printing "Hello, Chai," performing calculations, or even building a webpage.

2. The Hidden Compilation to Bytecode

   When you run your Python program, the first major step is that your source code gets compiled into "bytecode".

   • What is Bytecode? It's a low-level code, but it's NOT machine code. Machine code comprises direct instructions for your computer's hardware, like what assembly language produces. Bytecode, however, is an intermediate step.

   • Why Bytecode? It's platform-independent, meaning the same bytecode can run on different operating systems like macOS or Windows without significant changes. Running bytecode is also generally faster than running the original script directly, as most syntax checks and parsing have already been completed.

   • Python-Specific: This bytecode is optimized for and interpreted by the Python Virtual Machine (PVM). It's not like Java's bytecode; it's Python-specific.

3. Meet the Files and the Folder

   After compilation, this bytecode is often saved in files with a .pyc extension (e.g., hello.cpython-312.pyc). These are essentially compiled Python files.

   • The pycache Folder: These .pyc files don't just sit in your main directory. Python neatly organizes them into a hidden folder named pycache. This keeps your project clean and prevents clutter from these "frozen binaries". The double underscores (__) around pycache are a Python convention, signaling that these are internal-use items that hold special meaning to Python.

   • What's in the .pyc name? The name hello.cpython-312.pyc is quite informative:

   ◦ cpython: This indicates the bytecode was generated using the standard CPython implementation of Python. CPython is the most common implementation, used in about 90% of cases. Other implementations exist, like Jython (for Java integration), IronPython, Stackless, and PyPy (for performance).

   ◦ 312: This denotes the Python version (e.g., Python 3.12) that created the bytecode. This is important because bytecode is version-specific.

   • When are .pyc files created? They are primarily generated for imported files (modules) to optimize loading and execution. For simple, top-level scripts that don't import anything, .pyc files are generally not generated, as the optimization isn't as critical.

   • Handling Code Changes: If you modify your source code, Python doesn't rebuild the entire .pyc file from scratch. It uses "diffing algorithms" (difference-finding algorithms) to compare the old and new source code and update only the necessary parts of the .pyc file. This is similar to how Git manages code changes.

Execution by the Python Virtual Machine (PVM)

The final key step is when the bytecode (either newly generated or loaded from an existing .pyc file) is fed into the Python Virtual Machine (PVM).

• What is the PVM? The PVM is a small software that acts as a runtime engine for Python. Just like a car needs an engine to run, every programming language needs a runtime engine to execute code.

• How it Works: The PVM runs a continuous loop. When it receives bytecode, it executes the instructions from start to end. The PVM is also capable of directly executing Python source files (.py). This is why top-level scripts don't always generate .pyc files; the PVM can process the source code directly.

Summery

So, Python's execution process involves converting your source code into an intermediate bytecode, which is then run by the Python Virtual Machine. This sophisticated yet straightforward mechanism allows Python to be platform-independent and performant, especially when dealing with complex projects involving multiple modules. It's a blend of "compilation" (to bytecode) and "interpretation" (by the PVM), which makes Python truly versatile and efficient.