C global var is equivalent to a python module’s attribute.
Every python object has the following structure at the top of each object:
typedef struct_object{
unsigned int ob_refcnt;
struct _typeobject *ob_type;
} object;
Here 1st on is the object reference count of the object. (number of entities holding on to that object) Second is the reference to another class, which has a lot of info about the object.
#define INCREF(op) ((op)->ob_refcnt++)
#define DECREF(op) \
if (!--(op)->ob_refcnt >0) \
DELREF(op)
Decref: Subtracts one from the reference count, and it the final value is zero, delete the reference.
a=[]
b=a
sys.getrefcount(a)
# answer is 3 because another ref is created when 'a' is passed to the function
« Explained in lecture @7:00 » Example is of two threads, simulateously decrementing reference count. While checking the ref count value parallely both thread will get , say 3, then decrement it by 1, and store the final value 2, back to object. Which is incorrect. Since both the thread have dropped the object, the final value of ref count should be 1 instead of 2.
To overcome this problem, locks were introduced. But this lead to deadlock , which is happening due to Race condition. « Explaine @9:20 »
To overcome deadlock due to race condition, GIL was introduced: Global Interpreter Lock.
static type_lock;
interpreter_lock;
static PyThread_type_lock
interpreter_lock = 0;
/* This is the GIL */
There is only one lock, which needs to be acquired for interaction with C-python interperter for any operation like mem allocation, runnning bytecode etc. Note: that does not include IO operations like socket connection or fileIO, so these can be carried out with other entity has the GIL.
This means, for IO bound task, GIL is fantastic, since it protects from deadlocks etc. But for CPU bound task, like parallel processing, it is bad, because GIL can only be acquired by 1 entity, therefore python interpreter can only be acquired by one thread.
There are macros used for acquiring and releasing locks:
Py_BEGIN_ALLOW_THREADS
/* Lock released */
/* Other threads can now run */
<< In the middle>>
Py_END_ALLOW_THREADS
/* Lock Acquired */
This happens, natively in the C-python using the check interval, which takes a number which is the amount of bytecode executed and then the GIL MUST be passed on to some other entity. The default number in python 2.7 is 100.
« See lec @17:50 »
Becuase CPU bound threads are never happy :P.
« See lec @19:00 »
« See lec @19.20 »
Added a GIL request var (flag), which tell if someone else wanted to acquire the GIL.
static volitile int gil_drop_request = 0;
Further fixing requires better Scheduling (very hard to do).
« See lec @20:50 »
« See lec @22:45 »
« See lec @24:52 »
Note: There are two ways of memory mangement: Pure GC vs Refcounting with GIL.
Pure GC require heavy C api changes. It would be like starting all over again.
« See lec @30:50 »
Its like Databases, but the data is being commited to memory, and if some commits from different thread collide, well, rollback and start over. It is not slower, its just very hard conceptually to get it right.
« See lec @36:10 »