Process
Process is an instance of program. Processes spawn threads (sub-processes) to handle subtasks.
- Created by the OS to run programs
- Can have multiple threads
- Two processes can execute code simultaneously in the same python program!
- Processes have more overhead than threads as opening and closing processes takes more time
- Sharing information between processes in slower than sharing between thread as processes do not share memory space. In python they share information by pickling data structures like arrays which requires IO time.
Thread
Threads live inside processes and share the same memory space.
- Mini-processes that live inside a process
- They share memory space and efficiently read and write to the same variables
- Two threads cannot execute code simultaneously in the same python program
Processes speed up Python operations that are CPU intensive because they benefit from multiple cores and avoid the GIL
Threads are best for IO tasks, API calls, or tasks involving external systems because threads can combine their work more efficiently. Processes need to pickle their results to combine them which takes time.
Threads run in the same unique memory heap, Processes run in separate memory heaps. This is prevented by GIL being introduced as a mutex!