[Python] Python Multithreading and Multiprocessing

Process

Process is an instance of program. Processes spawn threads (sub-processes) to handle subtasks. 

  • Created by the OS to run programs
  • Can have multiple threads
  • Two processes can execute code simultaneously in the same python program!
  • Processes have more overhead than threads as opening and closing processes takes more time
  • Sharing information between processes in slower than sharing between thread as processes do not share memory space. In python they share information by pickling data structures like arrays which requires IO time.

Thread

Threads live inside processes and share the same memory space.

  • Mini-processes that live inside a process
  • They share memory space and efficiently read and write to the same variables
  • Two threads cannot execute code simultaneously in the same python program

Processes speed up Python operations that are CPU intensive because they benefit from multiple cores and avoid the GIL

Threads are best for IO tasks, API calls, or tasks involving external systems because threads can combine their work more efficiently. Processes need to pickle their results to combine them which takes time.

Threads run in the same unique memory heap, Processes run in separate memory heaps. This is prevented by GIL being introduced as a mutex!

Reference

[link1]
[link2]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.