Installation #
Install (or upgrade)memory_graph using pip:
pip install --upgrade memory_graph
Additionally Graphviz needs to be installed.
Highlights #
!vscode_copying.gif Run a live demo in the 👉 Memory Graph Web Debugger 👈 now, no installation required!- learn the right mental model to think about Python data (references, mutability, shallow vs deep copy)
- visualize the structure of your data to more easily understand and debug any data structure
- understand function calls, variable scope, and the complete program state through call stack visualization
Videos
|Memory Graph #
For program understanding and debugging, the memory_graph package can visualize your data, supporting many different data types, including but not limited to:import memory_graph as mg
class My_Class:
def __init__(self, x, y):
self.x = x
self.y = y
data = [ range(1, 2), (3, 4), {5, 6}, {7:'seven', 8:'eight'}, My_Class(9, 10) ]
mg.show(data)
!many_types.png
Instead of showing the graph on screen you can also render it to an output file (see Graphviz Output Formats) using for example:
mg.render(data, "my_graph.pdf")
mg.render(data, "my_graph.svg")
mg.render(data, "my_graph.png")
mg.render(data, "my_graph.gv") # Graphviz DOT file
mg.render(data) # renders to default: 'memory_graph.pdf'
Sharing Values, Aliasing #
In Python, assigning a list from variablea to variable b causes both variables to reference the same list value and thus share it. Consequently, any change applied through one variable will impact the other. This behavior can lead to elusive bugs if a programmer incorrectly assumes that list a and b are independent.
|
a graph showing |
The fact that a and b share the list can not be verified by printing the lists. It can be verified by comparing the identity of both variables using the id() function or by using the is comparison operator as shown in the program output below, but this quickly becomes impractical for larger programs.
``{verbatim}
a: 4, 3, 2, 1
b: 4, 3, 2, 1
ids: 126432214913216 126432214913216
identical?: True
A better way to understand what values are shared is to draw a graph using memory_graph.
Topics #
Author ##
Bas TerwijnInspiration ##
Inspired by Python Tutor.The main differences are that by running memory_graph locally we support Python Tutor’s unsupported features so that it scales to full multi-file programs in many environments and IDEs instead of just code snippets in a webbrowser, and by mirroring the data’s hierarchy we improve graph readability for larger graphs.
Social Media #
Supported by ##
<img src="https://raw.githubusercontent.com/bterwijn/memory_graph/main/images/uva.png" alt="University of Amsterdam" width="600">___ ___
Python Data Model #
Learn the right mental model to think about Python data. The Python Data Model makes a distiction between immutable and mutable types:- immutable: bool, int, float, complex, str, tuple, frozenset, frozendict, bytes
- mutable: list, set, dict, classes, ... (most other types)
Immutable Type ##
In the code below variable a and b both reference the same tuple value (4, 3, 2). A tuple is an immutable type and therefore when we change variable b its value cannot be mutated in place, and thus an automatic copy is made and a and b each reference their own value afterwards.python
import memory_graph as mg
a = (4, 3, 2) b = a mg.render(locals(), 'immutable1.png')
b += (1,) mg.render(locals(), 'immutable2.png')
python import memory_graph as mg| !mutable1.png | !mutable2.png | |:-----------------------------------------------------------:|:-------------------------------------------------------------:| | immutable1.png | immutable2.png |aMutable Type ##
With mutable types the result is different. In the code below variableandbboth reference the samelistvalue [4, 3, 2]. Alistis a mutable type and therefore when we change variablebits value can be mutated in place and thusaandbboth reference the same new value afterwards. Thus changingbalso changesaand vice versa. Sometimes we want this but other times we don't and then we will have to make a copy ourselfs so thataandbare independent.
a = [4, 3, 2] b = a mg.render(locals(), 'mutable1.png')
b += [1] # equivalent to: b.append(1) mg.render(locals(), 'mutable2.png')
| !mutable1.png | !mutable2.png |
|:-----------------------------------------------------------:|:-------------------------------------------------------------:|
| mutable1.png | mutable2.png |
One practical reason why Python makes the distinction between mutable and immutable types is that a value of a mutable type can be large, making it inefficient to copy each time we change it. Values of immutable type generally don't need to change as much, or are small, making copying less of a concern.
Copying Values of Mutable Type ##
Python offers three different "copy" options that we will demonstrate using a nested list:python
import memory_graph as mg
import copy
a = [ [1, 2], ['x', 'y'] ] # a nested list (a list containing lists)
three different ways to make a "copy" of 'a':
c1 = a c2 = copy.copy(a) # for list equivalent to: a.copy() a[:] list(a) c3 = copy.deepcopy(a)mg.show(locals())
python import memory_graph as mg import copy*c1is an assignment, nothing is copied, all the values are sharedc2is a shallow copy, only the first value is copied, all the underlying values are sharedc3is a deep copy, all the values are copied, nothing is shared !copy_mutbale.pngcustom_copy()Or see it in the Memory Graph Web Debugger.
Custom Copy ##
We can write our own custom copy function or method in case the three standard "copy" options don't do what we want. For example, in the code below themethod of My_Class copies thedigitsbut shares thelettersbetween two objects.
class My_Class:
def __init__(self): self.digits = [1, 2] self.letters = ['x', 'y']
def custom_copy(self): """ Copies 'digits' but shares 'letters'. """ c = copy.copy(self) c.digits = copy.copy(self.digits) return c
a = My_Class() b = a.custom_copy()
mg.show(locals())
python import memory_graph as mg!copy_method.pngaOr see it in the Memory Graph Web Debugger.
Name Rebinding ##
Whenandbshare a mutable value, then changing the value ofbchanges the value ofaand vice versa. However, reassigningbdoes not changea. When you reassignb, you only rebind the namebto another value without affecting any other variable.b += [300]In the example below, also note the difference between expressions:
: that changes bothbandac = c + [600]: that first creates a new valuec + [600]and then assigns this new value tocwithout affectingbThis shows thatx += yis not the same asx = x + yfor a valuexof mutable type.
a = [100, 200] b = a mg.render(locals(), 'rebinding1.png')
b += [300] # changes the value of 'b' and 'a' b = [400, 500] # rebinds 'b' to a new value, 'a' is unaffected c = b mg.render(locals(), 'rebinding2.png')
c = c + [600] # rebinds 'c' to new value 'c + [600]', b is unaffected
mg.render(locals(), 'rebinding3.png')
| !rebinding1.png | !rebinding2.png | !rebinding3.png |
|:--------------:|:--------------:|:--------------:|
| rebinding1.png | rebinding2.png | rebinding3.png |
Or see it in the Memory Graph Web Debugger.
Copying Values of Immutable Type ##
Because a value of immutable type will be copied automatically when it is changed, there is no need to copy it beforehand. Therefore, a shallow or deep copy of a value of immutable type will result in just an assignment to save on the time needed to make the copy and the space (=memory) needed to store the values.python import memory_graph as mg import copya = ( (1, 2), ('x', 'y') ) # a nested tuple
three different ways to make a "copy" of 'a':
c1 = a c2 = copy.copy(a) c3 = copy.deepcopy(a)mg.show(locals())
!copy_immutbale.png
Copying a Mix of Mutable and Immutable Values ##
When copying a mix of values of mutable and immutable type, to save on time and space, a deep copy will try to copy as few values of immutable type as possible in order to copy each value of mutable type.python
import memory_graph as mg
import copy
a = ( [1, 2], ('x', 'y') ) # mix of mutable and immutable values
three different ways to make a "copy" of 'a':
c1 = a c2 = copy.copy(a) c3 = copy.deepcopy(a)mg.show(locals())
python import memory_graph as mg!copy_mix.pngmg.stack()Call Stack #
Thefunction retrieves the entire call stack, including the local variables for each function on the stack. This enables us to understand function calls, variable scope, and the complete program state through call stack visualization. By examining the graph, we can see whether local variables from different function calls share data. For instance, consider the functionadd_one()which adds the value1to each of its parametersa,b, andc.
def add_one(a, b, c): a += [1] b += (1,) c += [1] mg.show(mg.stack())
a = [4, 3, 2] b = (4, 3, 2) c = [4, 3, 2] add_one(a, b, c.copy())
print(f"a:{a} b:{b} c:{c}")
a:[4, 3, 2, 1] b:(4, 3, 2) c:[4, 3, 2]!add_one.pngaOr see it in the Memory Graph Web Debugger.
In the printed output we see that only
is changed as a result of the function call:
add_one()`This is becausebis of immutable type 'tuple' so its value gets copied automatically when it is changed. And because the function is called with a copy ofc, its original value is not changed by the function. The value of variableais the only value of mutable type that is shared between the root stack frame '0: \<module>' and the '1: add_one' stack frame of the function call so only that variable is affected as a result of calling the function. The other changes remain confined to the local variables of the
function.
Function Call Changes 'int' Value ##
Even though int is an immutable type, so an int value can not be changed by directly passing it to a function, we can still change it by wrapping it in a mutable container.
import memory_graph as mg
def add_one(a, b):
a += 1 # change remains confined to 'a' in the add_one function
b[0] += 1 # change also affects 'b' outside of the add_one function
mg.show(mg.stack())
a = 10
b = [10] # wrap in a value of mutable type list
add_one(a, b)
print(f"a:{a} b:{b[0]}")
!wap_int.png
a:10 b:11
Or see it in the Memory Graph Web Debugger
The effect of calling
add_one() is that b[0] increases by 1, while a is unaffected.
Data Model Exercises #
Now is a good time to practice with these Python Data Model concepts. Here are some exercises on references, mutability, copies, and function calls. Also see the programming exercises at the end of the Mutability video.
Block #
It is often helpful to temporarily block program execution to inspect the graph. For this we can use the mg.block() function:
mg.block(fun, arg1, arg2, ...)
This function:
- first executes
fun(arg1, arg2, ...)
then prints the current source location in the program
then blocks execution until the <Enter> key is pressed
and returns the return value of the fun() call
Recursion ##
The call stack is also helpful to visualize how recursion works. Here we use mg.block() to show each step of how recursively factorial(4) is computed:
import memory_graph as mg
def factorial(n):
mg.block(mg.show, mg.stack())
if n==0:
return 1
result = n * factorial(n-1)
mg.block(mg.show, mg.stack())
return result
print( factorial(4) )
!factorial.gif
and the result is: 1 x 2 x 3 x 4 = 24
Or see it in the Memory Graph Web Debugger.
Binary Conversion ##
A more interesting recursive example is function binary() that converts a integer from decimal to binary representation.
import memory_graph as mg
mg.config.type_to_horizontal[list] = True # horizontal lists
def binary(value: int) -> list[int]:
mg.block(mg.show, mg.stack())
if value == 0:
return []
quotient, remainder = divmod(value, 2)
result = binary(quotient) + [remainder]
mg.block(mg.show, mg.stack())
return result
print( binary(100) )
!factorial.gif
[1, 1, 0, 0, 1, 0, 0]
Or see it in the Memory Graph Web Debugger.
Power Set ##
A more complex recursive example is function power_set() where lists are shared by different function calls. A power set is the set of all subsets of a collection of values.
import memory_graph as mg
def get_subsets(subsets, data, i, subset):
mg.block(mg.show, mg.stack())
if i == len(data):
subsets.append(subset.copy())
return
subset.append(data[i])
get_subsets(subsets, data, i+1, subset) # do include data[i]
subset.pop()
get_subsets(subsets, data, i+1, subset) # don't include data[i]
mg.block(mg.show, mg.stack())
def power_set(data):
subsets = []
get_subsets(subsets, data, 0, [])
return subsets
print( power_set(['a', 'b', 'c']) )
[['a', 'b', 'c'], ['a', 'b'], ['a', 'c'], ['a'], ['b', 'c'], ['b'], ['c'], []]
Or see it in the Memory Graph Web Debugger.
Invocation Tree ##
The memory_graph package visualizes data at the currect time, but to better understand recursion it can also be helpful to visualize different function calls over time. This is what the invocation_tree package does.
See the power_set example in the Invocation Tree Web Debugger.
Debugging #
For the best debugging experience with memory_graph set for example expression:
mg.render(locals(), "my_graph.pdf")
as a watch in a debugger tool such as the integrated debugger in Visual Studio Code. Then open the "my_graph.pdf" output file to continuously see all the local variables while debugging. This avoids having to add any memory_graph show() or render() calls to your code.
Call Stack in Watch Context ##
The `mg.stack()` doesn't work well in watch context in most debuggers because debuggers introduce additional stack frames that cause problems. Use these alternative functions for various debuggers to filter out these problematic stack frames:
| debugger | function to get the call stack in 'watch' context |
|:---|:---|
| pdb, pudb |
mg.stack_pdb() |
| Visual Studio Code | mg.stack_vscode() |
| Jupyter Notebooks in VS Code | mg.stack_vscode_jupyter() |
| Cursor AI | mg.stack_cursor() |
| PyCharm | mg.stack_pycharm() |
| Wing | mg.stack_wing() |
!vscode_copying.gif
See the Quick Intro (3:49) video for the setup.
Other Debuggers ##
For other debuggers, invoke this function within the watch context. Then, in the "call_stack.txt" file, identify the slice of functions you wish to include as stack frames in the call stack.
mg.save_call_stack("call_stack.txt")
Then to get the call stack use:
mg.stack_slice(begin_functions : list[(str,int)] = [],
end_functions : list[str] = ["<module>"],
stack_index: int = 0)
with these parameters that determine the begin and end index of the slice of stack frames in the call stack:
- begin_functions: list of (function-name, offset), begins at the index of the first 'function-name' that is found in the call stack with additional 'offset', otherwise begins at index 0
- end_functions: list of function-names, ends at the index of the first 'function-name' that is found in the call stack after begin index (inclusive), otherwise ends at the last index
- stack_index: number of frames removed at the beginning
Debugging without Debugger Tool ##
To simplify debugging without a debugger tool, we offer these alias functions that you can insert into your code at the point where you want to visualize a graph:
| alias | purpose | function call |
|:---|:---|:---|
|
mg.sl() | show local variables | mg.show(locals()) |
| mg.ss() | show the call stack | mg.show(mg.stack()) |
| mg.bsl() | block after showing local variables | mg.block(mg.show, locals()) |
| mg.bss() | block after showing the call stack | mg.block(mg.show, mg.stack()) |
| mg.rl() | render local variables | mg.render(locals()) |
| mg.rs() | render the call stack | mg.render(mg.stack()) |
| mg.brl() | block after rendering local variables | mg.block(mg.render, locals()) |
| mg.brs() | block after rendering the call stack | mg.block(mg.render, mg.stack()) |
| mg.l() | same as mg.bsl() | |
| mg.s() | same as mg.bss() | |
For example, executing this program:
from memory_graph as mg
squares = []
squares_collector = []
for i in range(1, 6):
squares.append(i**2)
squares_collector.append(squares.copy())
mg.l() # block after showing local variables
and pressing <Enter> a number of times, results in:
Debugging using Exceptions ##
To get the call stack at the point where exception
e was raised use mg.stack_exception(e). This allows you to graph the trace back for easier debugging, for example:
` python
import memory_graph as mg
def fun3():
d = [0] * 3
for i in range(4):
d[i] = i # raises IndexError when i = 3
def fun2():
fun3()
def fun1():
fun2()
try:
fun1()
except Exception as e:
mg.show(mg.stack_exception(e)) # graph traceback
raise e # reraise to print traceback
$ python exception_example.py
Traceback (most recent call last):
File "exception_example.py", line 18, in
raise e # raise to print traceback
^^^^^^^
File "exception_example.py", line 15, in
fun1()
File "exception_example.py", line 12, in fun1
fun2()
File "exception_example.py", line 9, in fun2
fun3()
File "exception_example.py", line 6, in fun3
d[i] = i # throws IndexError when i = 3
~^^^
IndexError: list assignment index out of range
!exception_example.png
Data Structure Examples #
Package memory_graph can visualize the structure of your data to easily understand and debug data structures, some examples:
Circular Doubly Linked List ##
python
import memory_graph as mg
import random
random.seed(0) # use same random numbers each run
class Linked_List:
""" Circular doubly linked list """
def __init__(self, value=None,
prev=None, next=None):
self.prev = prev if prev else self
self.value = value
self.next = next if next else self
def add_back(self, value):
if self.value == None:
self.value = value
else:
new_node = Linked_List(value,
prev=self.prev,
next=self)
self.prev.next = new_node
self.prev = new_node
linked_list = Linked_List()
n = 100
for i in range(n):
value = random.randrange(n)
linked_list.add_back(value)
mg.block(mg.show, locals()) # <--- draw locals
!linked_list.png
Linked List in Cursor AI ###
Here we show values being added to a Linked List in Cursor AI. When adding the last value '5' we "Step Into" the code to show more of the details.
!linked_list.gif
Or see it in the Memory Graph Web Debugger.
Binary Tree ##
python
import memory_graph as mg
import random
random.seed(0) # use same random numbers each run
class BinTree:
def __init__(self, value=None, smaller=None, larger=None):
self.smaller = smaller
self.value = value
self.larger = larger
def add(self, value):
if self.value is None:
self.value = value
elif value < self.value:
if self.smaller is None:
self.smaller = BinTree(value)
else:
self.smaller.add(value)
else:
if self.larger is None:
self.larger = BinTree(value)
else:
self.larger.add(value)
mg.block(mg.show, mg.stack()) # <--- draw stack
tree = BinTree()
n = 100
for i in range(n):
value = random.randrange(n)
tree.add(value)
!bin_tree.png
Binary Tree in Visual Studio Code ###
Here we show values being inserted in a Binary Tree in Visual Studio Code. When inserting the last value '29' we "Step Into" the code to show the recursive implementation.
!images/bin_tree.gif
See it in the Memory Graph Web Debugger or see the more advanced Multiway Tree with more than two children per node, making the tree less deep and more efficient.
Hash Set ##
python
import memory_graph as mg
import random
random.seed(0) # use same random numbers each run
class HashSet:
def __init__(self, capacity=15):
self.buckets = [None] * capacity
def add(self, value):
index = hash(value) % len(self.buckets)
if self.buckets[index] is None:
self.buckets[index] = []
bucket = self.buckets[index]
bucket.append(value)
mg.block(mg.show, locals()) # <--- draw locals
def contains(self, value):
index = hash(value) % len(self.buckets)
if self.buckets[index] is None:
return False
return value in self.buckets[index]
def remove(self, value):
index = hash(value) % len(self.buckets)
if self.buckets[index] is not None:
self.buckets[index].remove(value)
hash_set = HashSet()
n = 100
for i in range(n):
new_value = random.randrange(n)
hash_set.add(new_value)
!hash_set.png
Hash Set in PyCharm ###
Here we show values being inserted in a HashSet in PyCharm. When inserting the last value '44' we "Step Into" the code to show more of the details.
!images/hash_set.gif
Or see it in the Memory Graph Web Debugger.
Sorting Algorithms #
Visualization of different sorting algorithms in Memory Graph Web Debugger.
!selections_sort.png
Bitwise Operators #
In this configuration example we show the decimal, binary and two's complement representation representation of int values of dictionary subclass Bits to show the result of bitwise operators. The ~ (inverse) operator can be a bit confusing if not shown with two's complement representation.
!bitwise_operators.png
Sliding Puzzle Solver #
A sliding puzzle solver as a challenging example showing how memory_graph deals with large amounts of data. Click "Continue" to step through the breadth-first search generations until a solution path is found:
!sliding_puzzle.png
Flow Control #
Some examples where we focus more on the flow of execution in Python code.
Different Methods ##
This example shows the difference between 'instance', 'class' and 'static' methods:
python
class My_Class:
# class variables:
count = 0
unique_value = 0
def __init__(self):
print("__init__ called, initialize instance variables of object")
self.my_instance_variable = [] # create instance variable
My_Class.count += 1 # also change class variable
def my_instance_method(self):
print("my_instance_method called, access to instance variables via 'self'")
self.my_instance_variable.append(My_Class.unique_value)
My_Class.unique_value += 1 # also change class variable
@classmethod
def my_class_method(cls):
print("my_class_method called, access to class variables via 'cls', no instance variables")
print(f"{cls.count=}")
@staticmethod
def my_static_method():
print("my_static_method called, no 'self' or 'cls'")
print(f"{My_Class.count=}") # but still access to class variables
obj1 = My_Class()
obj2 = My_Class()
obj1.my_instance_method()
obj1.my_class_method()
obj1.my_static_method()
for _ in range(2):
obj1.my_instance_method()
obj2.my_instance_method()
print(f"{obj1.count=}") # reading class variable if no instance variable found
obj1.count = 100 # creating instance variable
print(f"{obj1.count=}") # now finding and reading instance variable
print(f"{My_Class.count}") # class variable still available via class name
print(f"{obj2.count=}") # obj2 still reads the class variable
Run it in the Memory Graph Web Debugger.
Inheritance ##
This example shows the flow of control when using inheritance with:
super()
class variable message_count
python
class Notification_Service:
message_count = 0
def __init__(self, priority):
self.priority = priority
self.log = []
def send(self, sender, receiver, message):
self.log.append((type(self).__name__, self.priority, sender, receiver, message))
Notification_Service.message_count += 1
class Email_Notification(Notification_Service):
def __init__(self, priority, from_email):
super().__init__(priority)
self.from_email = from_email
def send(self, to_email, message):
super().send(self.from_email, to_email, message)
print(f'sending Email from:{self.from_email} to:{to_email}')
print(f'priority: {self.priority} message: "{message}"')
class SMS_Notification(Notification_Service):
def __init__(self, priority, from_nr):
super().__init__(priority)
self.from_nr = from_nr
def send(self, to_nr, message):
super().send(self.from_nr, to_nr, message)
print(f'sending SMS from:{self.from_nr} to:{to_nr}')
print(f'priority: {self.priority} message: "{message}"')
email = Email_Notification(3, '[email protected]')
email.send('[email protected]', 'Your report is ready')
email.send('[email protected]', 'Update to Privacy Policy')
sms = SMS_Notification(3, '0123456789')
sms.send('001122334455', 'Update to Privacy Policy')
Run it in the Memory Graph Web Debugger.
Iterators ##
What actually happens when Python executes a for-loop?
python
for value in container:
print(value)
Behind the scenes, Python uses the iterator protocol:
python
iterator = iter(container)
while True:
try:
value = next(iterator)
print(value)
except StopIteration:
break
- iter(container): creates an iterator.
next(iterator): retrieves one value at a time.
When there are no more values, the iterator raises StopIteration. The for-loop catches this exception automatically and ends the loop. For containers that support backward iteration, Python also provides:
reversed(container): creates a backward iterator.
We can support these operations in our own classes by implementing:
python
def __iter__(self):
def __next__(self):
def __reversed__(self):
This provides a powerful abstraction: an algorithm can process values without needing to know how a container stores them internally. The same algorithm can therefore work with lists, sets, dictionaries, linked lists, trees, and many other containers.
Run a Linked_List iterator example in the Memory Graph Web Debugger.
Decorator ##
This example shows the flow of control when using a decorator. A decorator wraps a function and is active before and after the function is called.
python
def log_call(function):
def wrapper(args, *kwargs):
print(f"Calling {function.__name__} with: {args}, {kwargs}")
returned = function(args, *kwargs)
print(f"Finished {function.__name__} with return value: {returned}")
return returned
return wrapper
@log_call
def calculate_total(price, quantity, rounding=False):
result = price * quantity
return round(result) if rounding else result
@log_call
def send_email(receiver, message, sender="[email protected]"):
print(f"Sending email to:{receiver} from:{sender}, {message}")
total = calculate_total(7.5, 3, rounding=True)
print(total)
send_email("[email protected]", "Your order is ready")
Run it in the Memory Graph Web Debugger.
Exception Handling ##
This example shows the flow of control when using exception handling.
python
def fun2():
try:
d = [0] * 3
for i in range(6):
try:
print(f'{i=}')
d[i] = i # raises IndexError when i>=3
except ZeroDivisionError as e:
print(type(e), e)
except AssertionError as e:
print(type(e), e)
print('fun2() returns')
def fun1():
try:
return fun2()
except NameError as e:
print(type(e), e)
print('fun1() returns')
try:
fun1()
except LookupError as e:
print(type(e), e)
print('program ended cleanly')
Run it in the Memory Graph Web Debugger. In the program an IndexError exception is raised which propagates up the call stack until it reaches an except clause that matches its type where it is handled. Here, it is handled by the LookupError except clause because IndexError is a subclass of LookupError. Exceptions that are not handled terminate the execution of a program while its traceback is shown for analyses.
Lazy Evaluation ##
In the following Eager and Lazy ealuation examples, we use this
pr() function to print in what order things are created and used.
python
def pr(tag, v):
print(tag, v)
return v
With eager evaluation, the function creates all three elements up front and stores them in a list before iteration begins. With lazy evaluation, the function returns a generator that creates each element only when it is needed.
<table>
<tr> <td width="50%" valign="top"><strong>Eager</strong></td> <td width="50%" valign="top"><strong>Lazy</strong></td></tr>
<tr><td width="50%" valign="top">
python
def fun():
result = []
for i in range(3):
result.append(pr('create:', i))
return result
for i in fun():
pr('use:', i)
Run in Memory Graph Web Debugger
</td><td width="50%" valign="top">
python
def fun():
for i in range(3):
yield pr('create:', i)
for i in fun():
pr('use:', i)
Run in Memory Graph Web Debugger
</td></tr>
<tr><td width="50%" valign="top">
python
def fun():
return [
pr('create:', i)
for i in range(3)
]
for i in fun():
pr('use:', i)
Run in Memory Graph Web Debugger
</td><td width="50%" valign="top">
python
def fun():
return (
pr('create:', i)
for i in range(3)
)
for i in fun():
pr('use:', i)
Run in Memory Graph Web Debugger
</td></tr>
<tr><td width="50%" valign="top">
text
create: 0
create: 1
create: 2
use: 0
use: 1
use: 2
</td><td width="50%" valign="top">
text
create: 0
use: 0
create: 1
use: 1
create: 2
use: 2
Lazy evaluation is critical when you want to quickly start to process the first elements of a large stream of data, that may not even fit in your RAM entirely, or process just a section of an infite stream.
</td></tr>
</table>
Configuration #
Different aspects of memory_graph can be configured. The default configuration can be reset by calling 'mg.config_default.reset()'. The Memory Graph Web Debugger gives examples of the most important configurations.
- mg.config.reopen_viewer : bool
- If True the viewer is reopened each time show() is called, this might change window focus, default True.
- mg.config.render_filename : str
- The default filename to render to, default 'memory_graph.pdf'.
- mg.config.type_labels : bool
- If True the type of each node is shown as label, default True.
- mg.config.block_prints_location : bool
- If True the source location is printed in block(), default True.
- mg.config.press_enter_message : str
- Message to ask user to press <Enter> in block(), set to None to disable.
- mg.config.max_string_length : int
- The maximum length of strings shown in the graph. Longer strings will be truncated.
- mg.config.embedded_types : set[type]
- Holds all types for which no separate node is drawn but that are embedded in their parent Node.
- mg.config.embedded_key_types : set[type]
- Holds all types that are embedded as key in a Node_Key_Value node, even when not in 'embedded_types'.
- mg.config.embedding_types : set[type]
- Holds all dictionary types that embed their key-value tuple children.
- mg.config.no_index_types : set[type]
- Holds all types like 'set' and 'frozenset' that should not have indices as Node_Linear.
- mg.config.type_to_node : dict[type, fun(data) -> Node]
- Determines how a data type is converted to a Node subclass for visualization in the graph.
- mg.config.type_to_color : dict[type, color]
- Maps a type to the graphviz color it gets in the graph.
- mg.config.type_to_horizontal : dict[type, bool]
- Maps a type to its orientation for Node_Linear and Node_Key_Value. Use 'True' for horizontal and 'False' for vertical. If not specified these nodes vertical unless they have references to children.
- mg.config.type_to_slicer : dict[type, int]
- Maps a type to a Slicer. A slicer determines how many elements of a data type are shown in the graph to prevent the graph from getting too big. 'Slicer()' does no slicing, 'Slicer(1,2,3)' shows just 1 element at the beginning, 2 in the middle, and 3 at the end.
- mg.config.max_graph_depth : int
- The maxium depth of the graph with default value 1000.
- mg.config.graph_cut_symbol : str
- The symbol indicating where the graph is cut short with default ✂.
- mg.config.type_to_depth : dict[type, int]
- Maps a type to graph depth to limit the graph size.
- mg.config.max_missing_edges : int
- Maximum number of missing edges that are shown with default value 2. Dashed references are used to indicate that there are more references to a node than are shown.
- mg.config.fontname : str
- The font used in the graph, default 'Times-Roman' (widely available on the web).
- mg.config.fontsize : str
- The font size used in the graph, default '14'.
Functions ##
- mg.layout(horizontal: bool = None)
- Set graph layout to 'True' for horizontal, 'False' for vertical, or 'None' to toggle.
- mg.dark_mode(b: bool = None)
- Set dark mode to 'True' or 'False', or 'None' to toggle.
- mg.transparent_background(b: bool = None)
- Set transparent background to 'True' or 'False', or 'None' to toggle.
Simplified Graph ##
Memory_graph simplifies the visualization (and the viewer's mental model) by not showing separate nodes for immutable types like bool, int, float, complex, and str by default. This simplification can sometimes be slightly misleading. As in the example below, after a shallow copy, lists a and b technically share their int values, but the graph makes it appear as though a and b each have their own copies. However, since int is immutable, this simplification will never lead to unexpected changes (changing a's ints won’t affect b) so will never result in bugs.
The simplification strikes a balance: it is slightly misleading but keeps the graph clean and easy to understand to focus on mutable types where unexpected changes can occur. This is why it is the default behavior. If you do want to show separate nodes for
int values, such as for educational purposes, you can simply remove int from the mg.config.embedded_types set:
python
import memory_graph as mg
a = [100, 200, 300]
b = a.copy()
mg.render(locals(), 'embedded1.png')
mg.config.embedded_types.remove(int) # now show separate nodes for int values
mg.render(locals(), 'embedded2.png')
| !embedded1 | !embedded2 |
|:-----------------------------------------------------------:|:-------------------------------------------------------------:|
| embedded1.png — simplified | embedded2.png — technically correct |
Additionally, the simplification hides away the [reuse of small int values \[-5, 256\]](https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong) in the current CPython implementation, an optimization that might otherwise confuse beginner Python programmers. For instance, after executing
a[1]+=1; b[1]+=1 the 201 value is, maybe surprisingly, still shared between a and b, whereas executing a[2]+=1; b[2]+=1 does not result in sharing the 301 value. Similarly CPython uses String Interning to reuse small strings.
Introspection #
Sometimes the introspection fails or is not as desired. For example the bintrees.avltree.Node object doesn't show any attributes in the graph below.
python
import memory_graph as mg
import bintrees
Create an AVL tree
tree = bintrees.AVLTree()
tree.insert(10, "ten")
tree.insert(5, "five")
tree.insert(20, "twenty")
tree.insert(15, "fifteen")
mg.show(locals())
!avltree_fail.png
All attributes using dir() ##
A useful start is to give it some color, show the list of all its attributes using dir()`, and setting an empty Slicer to see the attribute list in full.
python
import memory_graph as mg
import bintrees
Create an AVL tree
tree = bintrees.AVLTree() tree.insert(10, "ten") tree.insert(5, "five") tree.insert(20, "twenty") tree.insert(15, "fifteen")mg.config.type_to_color[bintrees.avltree.Node] = "sandybrown" mg.config.type_to_node[bintrees.avltree.Node] = lambda data: mg.Node_Linear(data, dir(data)) mg.config.type_to_slicer[bintrees.avltree.Node] = mg.Slicer()
mg.show(locals())
!avltree_dir.png
Next figure out what the attributes are you want to graph and choose a Node type, there are four options:
1) Node_Leaf ##
Node_Leaf is a node with no children and shows just a single value.python
import memory_graph as mg
import bintrees
Create an AVL tree
tree = bintrees.AVLTree() tree.insert(10, "ten") tree.insert(5, "five") tree.insert(20, "twenty") tree.insert(15, "fifteen")mg.config.type_to_color[bintrees.avltree.Node] = "sandybrown" mg.config.type_to_node[bintrees.avltree.Node] = lambda data: mg.Node_Leaf(data, f"key:{data.key} value:{data.value}")
mg.show(locals())
!avltree_leaf.png
2) Node_Linear ##
Node_Linear shows multiple values in a line like a list.python
import memory_graph as mg
import bintrees
Create an AVL tree
tree = bintrees.AVLT... (README truncated for length)