Profile
Back to NewsBack
GitHub Trending 16 min
Reader Mode
bterwijn/memory_graph: Teaching tool and debugging aid in context of references, mutable data types, and shallow and deep copy.

bterwijn/memory_graph: Teaching tool and debugging aid in context of references, mutable data types, and shallow and deep copy.

Installation #

Install (or upgrade) memory_graph using pip:
pip install --upgrade memory_graph
Additionally Graphviz needs to be installed.

Highlights #

!vscode_copying.gif Run a live demo in the 👉 Memory Graph Web Debugger 👈 now, no installation required!
  • learn the right mental model to think about Python data (references, mutability, shallow vs deep copy)
  • visualize the structure of your data to more easily understand and debug any data structure
  • understand function calls, variable scope, and the complete program state through call stack visualization
An example Binary Tree data structure: !bin_tree_vs.gif Or see it in the Memory Graph Web Debugger.

Videos

| Quick Intro</a> | Mutability</a> | |:--:|:--:| | Quick Intro (3:49) | Mutability (17:29) |

Memory Graph #

For program understanding and debugging, the memory_graph package can visualize your data, supporting many different data types, including but not limited to:
import memory_graph as mg

class My_Class:

def __init__(self, x, y): self.x = x self.y = y

data = [ range(1, 2), (3, 4), {5, 6}, {7:'seven', 8:'eight'}, My_Class(9, 10) ] mg.show(data)

!many_types.png

Instead of showing the graph on screen you can also render it to an output file (see Graphviz Output Formats) using for example:

mg.render(data, "my_graph.pdf")
mg.render(data, "my_graph.svg")
mg.render(data, "my_graph.png")
mg.render(data, "my_graph.gv") # Graphviz DOT file
mg.render(data) # renders to default: 'memory_graph.pdf'

Sharing Values, Aliasing #

In Python, assigning a list from variable a to variable b causes both variables to reference the same list value and thus share it. Consequently, any change applied through one variable will impact the other. This behavior can lead to elusive bugs if a programmer incorrectly assumes that list a and b are independent.
import memory_graph as mg

create the lists 'a' and 'b'

a = [4, 3, 2] b = a b.append(1) # changing 'b' changes 'a'

print the 'a' and 'b' list

print('a:', a) print('b:', b)

check if 'a' and 'b' share the list

print('ids:', id(a), id(b)) print('identical?:', a is b)

show all local variables in a graph

mg.show( locals() )

!mutable2.png

a graph showing a and b share the list

The fact that a and b share the list can not be verified by printing the lists. It can be verified by comparing the identity of both variables using the id() function or by using the is comparison operator as shown in the program output below, but this quickly becomes impractical for larger programs. ``{verbatim} a: 4, 3, 2, 1 b: 4, 3, 2, 1 ids: 126432214913216 126432214913216 identical?: True

A better way to understand what values are shared is to draw a graph using memory_graph.

Topics #

Python Data Model

Call Stack

Data Model Exercises

Block

Debugging

Data Structure Examples

Sorting Algorithms

Bitwise Operators

Sliding Puzzle Solver

Flow Control

Configuration

Introspection

Graph Depth

Extensions

Memory Graph Web Debugger

Jupyter Notebook

ipython

Google Colab

Marimo

Animated GIF

Troubleshooting

Other Packages

Author ##

Bas Terwijn

Inspiration ##

Inspired by Python Tutor.

The main differences are that by running memory_graph locally we support Python Tutor’s unsupported features so that it scales to full multi-file programs in many environments and IDEs instead of just code snippets in a webbrowser, and by mirroring the data’s hierarchy we improve graph readability for larger graphs.

Social Media #

Supported by ##

<img src="https://raw.githubusercontent.com/bterwijn/memory_graph/main/images/uva.png" alt="University of Amsterdam" width="600">

___ ___

Python Data Model #

Learn the right mental model to think about Python data. The Python Data Model makes a distiction between immutable and mutable types:
  • immutable: bool, int, float, complex, str, tuple, frozenset, frozendict, bytes
  • mutable: list, set, dict, classes, ... (most other types)

Immutable Type ##

In the code below variable
a and b both reference the same tuple value (4, 3, 2). A tuple is an immutable type and therefore when we change variable b its value cannot be mutated in place, and thus an automatic copy is made and a and b each reference their own value afterwards.
python import memory_graph as mg

a = (4, 3, 2) b = a mg.render(locals(), 'immutable1.png')

b += (1,) mg.render(locals(), 'immutable2.png')

| !mutable1.png | !mutable2.png |
|:-----------------------------------------------------------:|:-------------------------------------------------------------:|
| immutable1.png | immutable2.png |

Mutable Type ##

With mutable types the result is different. In the code below variable
a and b both reference the same list value [4, 3, 2]. A list is a mutable type and therefore when we change variable b its value can be mutated in place and thus a and b both reference the same new value afterwards. Thus changing b also changes a and vice versa. Sometimes we want this but other times we don't and then we will have to make a copy ourselfs so that a and b are independent.
python import memory_graph as mg

a = [4, 3, 2] b = a mg.render(locals(), 'mutable1.png')

b += [1] # equivalent to: b.append(1) mg.render(locals(), 'mutable2.png')

| !mutable1.png | !mutable2.png |
|:-----------------------------------------------------------:|:-------------------------------------------------------------:|
| mutable1.png | mutable2.png |

One practical reason why Python makes the distinction between mutable and immutable types is that a value of a mutable type can be large, making it inefficient to copy each time we change it. Values of immutable type generally don't need to change as much, or are small, making copying less of a concern.

Copying Values of Mutable Type ##

Python offers three different "copy" options that we will demonstrate using a nested list:
python import memory_graph as mg import copy

a = [ [1, 2], ['x', 'y'] ] # a nested list (a list containing lists)

three different ways to make a "copy" of 'a':

c1 = a c2 = copy.copy(a) # for list equivalent to: a.copy() a[:] list(a) c3 = copy.deepcopy(a)

mg.show(locals())

* c1 is an assignment, nothing is copied, all the values are shared
  • c2 is a shallow copy, only the first value is copied, all the underlying values are shared
  • c3 is a deep copy, all the values are copied, nothing is shared
!copy_mutbale.png

Or see it in the Memory Graph Web Debugger.

Custom Copy ##

We can write our own custom copy function or method in case the three standard "copy" options don't do what we want. For example, in the code below the
custom_copy() method of My_Class copies the digits but shares the letters between two objects.
python import memory_graph as mg import copy

class My_Class:

def __init__(self): self.digits = [1, 2] self.letters = ['x', 'y']

def custom_copy(self): """ Copies 'digits' but shares 'letters'. """ c = copy.copy(self) c.digits = copy.copy(self.digits) return c

a = My_Class() b = a.custom_copy()

mg.show(locals())

!copy_method.png

Or see it in the Memory Graph Web Debugger.

Name Rebinding ##

When
a and b share a mutable value, then changing the value of b changes the value of a and vice versa. However, reassigning b does not change a. When you reassign b, you only rebind the name b to another value without affecting any other variable.

In the example below, also note the difference between expressions:

  • b += [300]: that changes both b and a
  • c = c + [600]: that first creates a new value c + [600] and then assigns this new value to c without affecting b
This shows that x += y is not the same as x = x + y for a value x of mutable type.
python import memory_graph as mg

a = [100, 200] b = a mg.render(locals(), 'rebinding1.png')

b += [300] # changes the value of 'b' and 'a' b = [400, 500] # rebinds 'b' to a new value, 'a' is unaffected c = b mg.render(locals(), 'rebinding2.png')

c = c + [600] # rebinds 'c' to new value 'c + [600]', b is unaffected mg.render(locals(), 'rebinding3.png')

| !rebinding1.png | !rebinding2.png | !rebinding3.png |
|:--------------:|:--------------:|:--------------:|
| rebinding1.png | rebinding2.png | rebinding3.png |

Or see it in the Memory Graph Web Debugger.

Copying Values of Immutable Type ##

Because a value of immutable type will be copied automatically when it is changed, there is no need to copy it beforehand. Therefore, a shallow or deep copy of a value of immutable type will result in just an assignment to save on the time needed to make the copy and the space (=memory) needed to store the values.
python import memory_graph as mg import copy

a = ( (1, 2), ('x', 'y') ) # a nested tuple

three different ways to make a "copy" of 'a':

c1 = a c2 = copy.copy(a) c3 = copy.deepcopy(a)

mg.show(locals())

!copy_immutbale.png

Copying a Mix of Mutable and Immutable Values ##

When copying a mix of values of mutable and immutable type, to save on time and space, a deep copy will try to copy as few values of immutable type as possible in order to copy each value of mutable type.
python import memory_graph as mg import copy

a = ( [1, 2], ('x', 'y') ) # mix of mutable and immutable values

three different ways to make a "copy" of 'a':

c1 = a c2 = copy.copy(a) c3 = copy.deepcopy(a)

mg.show(locals())

!copy_mix.png

Call Stack #

The
mg.stack() function retrieves the entire call stack, including the local variables for each function on the stack. This enables us to understand function calls, variable scope, and the complete program state through call stack visualization. By examining the graph, we can see whether local variables from different function calls share data. For instance, consider the function add_one() which adds the value 1 to each of its parameters a, b, and c.
python import memory_graph as mg

def add_one(a, b, c): a += [1] b += (1,) c += [1] mg.show(mg.stack())

a = [4, 3, 2] b = (4, 3, 2) c = [4, 3, 2] add_one(a, b, c.copy())

print(f"a:{a} b:{b} c:{c}")

!add_one.png

Or see it in the Memory Graph Web Debugger.

In the printed output we see that only a is changed as a result of the function call:

a:[4, 3, 2, 1] b:(4, 3, 2) c:[4, 3, 2]
This is because b is of immutable type 'tuple' so its value gets copied automatically when it is changed. And because the function is called with a copy of c, its original value is not changed by the function. The value of variable a is the only value of mutable type that is shared between the root stack frame '0: \<module>' and the '1: add_one' stack frame of the function call so only that variable is affected as a result of calling the function. The other changes remain confined to the local variables of the
add_one()
` function.

Function Call Changes 'int' Value ##

Even though
int is an immutable type, so an int value can not be changed by directly passing it to a function, we can still change it by wrapping it in a mutable container.
import memory_graph as mg

def add_one(a, b): a += 1 # change remains confined to 'a' in the add_one function b[0] += 1 # change also affects 'b' outside of the add_one function mg.show(mg.stack())

a = 10 b = [10] # wrap in a value of mutable type list add_one(a, b)

print(f"a:{a} b:{b[0]}")

!wap_int.png
a:10 b:11
Or see it in the Memory Graph Web Debugger

The effect of calling add_one() is that b[0] increases by 1, while a is unaffected.

Data Model Exercises #

Now is a good time to practice with these Python Data Model concepts. Here are some exercises on references, mutability, copies, and function calls. Also see the programming exercises at the end of the Mutability video.

Block #

It is often helpful to temporarily block program execution to inspect the graph. For this we can use the
mg.block() function:
mg.block(fun, arg1, arg2, ...)

This function:

  • first executes fun(arg1, arg2, ...)
  • then prints the current source location in the program
  • then blocks execution until the <Enter> key is pressed
  • and returns the return value of the fun() call

Recursion ##

The call stack is also helpful to visualize how recursion works. Here we use
mg.block() to show each step of how recursively factorial(4) is computed:

import memory_graph as mg

def factorial(n): mg.block(mg.show, mg.stack()) if n==0: return 1 result = n * factorial(n-1) mg.block(mg.show, mg.stack()) return result

print( factorial(4) )

!factorial.gif

and the result is: 1 x 2 x 3 x 4 = 24

Or see it in the Memory Graph Web Debugger.

Binary Conversion ##

A more interesting recursive example is function
binary() that converts a integer from decimal to binary representation.
import memory_graph as mg
mg.config.type_to_horizontal[list] = True  # horizontal lists

def binary(value: int) -> list[int]: mg.block(mg.show, mg.stack()) if value == 0: return [] quotient, remainder = divmod(value, 2) result = binary(quotient) + [remainder] mg.block(mg.show, mg.stack()) return result

print( binary(100) )

!factorial.gif
[1, 1, 0, 0, 1, 0, 0]

Or see it in the Memory Graph Web Debugger.

Power Set ##

A more complex recursive example is function
power_set() where lists are shared by different function calls. A power set is the set of all subsets of a collection of values.
import memory_graph as mg

def get_subsets(subsets, data, i, subset): mg.block(mg.show, mg.stack()) if i == len(data): subsets.append(subset.copy()) return subset.append(data[i]) get_subsets(subsets, data, i+1, subset) # do include data[i] subset.pop() get_subsets(subsets, data, i+1, subset) # don't include data[i] mg.block(mg.show, mg.stack())

def power_set(data): subsets = [] get_subsets(subsets, data, 0, []) return subsets

print( power_set(['a', 'b', 'c']) )

!power_set.gif

[['a', 'b', 'c'], ['a', 'b'], ['a', 'c'], ['a'], ['b', 'c'], ['b'], ['c'], []]

Or see it in the Memory Graph Web Debugger.

Invocation Tree ##

The memory_graph package visualizes data at the currect time, but to better understand recursion it can also be helpful to visualize different function calls over time. This is what the invocation_tree package does.

See the power_set example in the Invocation Tree Web Debugger.

Debugging #

For the best debugging experience with memory_graph set for example expression:

mg.render(locals(), "my_graph.pdf")
as a watch in a debugger tool such as the integrated debugger in Visual Studio Code. Then open the "my_graph.pdf" output file to continuously see all the local variables while debugging. This avoids having to add any memory_graph
show() or render() calls to your code.

Call Stack in Watch Context ##

The
`mg.stack()` doesn't work well in watch context in most debuggers because debuggers introduce additional stack frames that cause problems. Use these alternative functions for various debuggers to filter out these problematic stack frames:

| debugger | function to get the call stack in 'watch' context | |:---|:---| | pdb, pudb | mg.stack_pdb() | | Visual Studio Code | mg.stack_vscode() | | Jupyter Notebooks in VS Code | mg.stack_vscode_jupyter() | | Cursor AI | mg.stack_cursor() | | PyCharm | mg.stack_pycharm() | | Wing | mg.stack_wing() |

!vscode_copying.gif See the Quick Intro (3:49) video for the setup.

Other Debuggers ##

For other debuggers, invoke this function within the watch context. Then, in the "call_stack.txt" file, identify the slice of functions you wish to include as stack frames in the call stack.
mg.save_call_stack("call_stack.txt")
Then to get the call stack use:
mg.stack_slice(begin_functions : list[(str,int)] = [],
               end_functions : list[str] = ["<module>"],
               stack_index: int = 0)
with these parameters that determine the begin and end index of the slice of stack frames in the call stack:
  • begin_functions: list of (function-name, offset), begins at the index of the first 'function-name' that is found in the call stack with additional 'offset', otherwise begins at index 0
  • end_functions: list of function-names, ends at the index of the first 'function-name' that is found in the call stack after begin index (inclusive), otherwise ends at the last index
  • stack_index: number of frames removed at the beginning

Debugging without Debugger Tool ##

To simplify debugging without a debugger tool, we offer these alias functions that you can insert into your code at the point where you want to visualize a graph:

| alias | purpose | function call | |:---|:---|:---| | mg.sl() | show local variables | mg.show(locals()) | | mg.ss() | show the call stack | mg.show(mg.stack()) | | mg.bsl() | block after showing local variables | mg.block(mg.show, locals()) | | mg.bss() | block after showing the call stack | mg.block(mg.show, mg.stack()) | | mg.rl() | render local variables | mg.render(locals()) | | mg.rs() | render the call stack | mg.render(mg.stack()) | | mg.brl() | block after rendering local variables | mg.block(mg.render, locals()) | | mg.brs() | block after rendering the call stack | mg.block(mg.render, mg.stack()) | | mg.l() | same as mg.bsl() | | | mg.s() | same as mg.bss() | |

For example, executing this program:

from memory_graph as mg

squares = [] squares_collector = [] for i in range(1, 6): squares.append(i**2) squares_collector.append(squares.copy()) mg.l() # block after showing local variables

and pressing <Enter> a number of times, results in:

!debugging.png

Debugging using Exceptions ##

To get the call stack at the point where exception e was raised use mg.stack_exception(e). This allows you to graph the trace back for easier debugging, for example:

` python import memory_graph as mg

def fun3(): d = [0] * 3 for i in range(4): d[i] = i # raises IndexError when i = 3 def fun2(): fun3() def fun1(): fun2()

try: fun1() except Exception as e: mg.show(mg.stack_exception(e)) # graph traceback raise e # reraise to print traceback

$ python exception_example.py Traceback (most recent call last): File "exception_example.py", line 18, in raise e # raise to print traceback ^^^^^^^ File "exception_example.py", line 15, in fun1() File "exception_example.py", line 12, in fun1 fun2() File "exception_example.py", line 9, in fun2 fun3() File "exception_example.py", line 6, in fun3 d[i] = i # throws IndexError when i = 3 ~^^^ IndexError: list assignment index out of range
!exception_example.png

Data Structure Examples #

Package memory_graph can visualize the structure of your data to easily understand and debug data structures, some examples:

Circular Doubly Linked List ##

python import memory_graph as mg import random random.seed(0) # use same random numbers each run

class Linked_List: """ Circular doubly linked list """

def __init__(self, value=None, prev=None, next=None): self.prev = prev if prev else self self.value = value self.next = next if next else self

def add_back(self, value): if self.value == None: self.value = value else: new_node = Linked_List(value, prev=self.prev, next=self) self.prev.next = new_node self.prev = new_node

linked_list = Linked_List() n = 100 for i in range(n): value = random.randrange(n) linked_list.add_back(value) mg.block(mg.show, locals()) # <--- draw locals

!linked_list.png

Linked List in Cursor AI ###

Here we show values being added to a Linked List in Cursor AI. When adding the last value '5' we "Step Into" the code to show more of the details. !linked_list.gif

Or see it in the Memory Graph Web Debugger.

Binary Tree ##

python import memory_graph as mg import random random.seed(0) # use same random numbers each run

class BinTree:

def __init__(self, value=None, smaller=None, larger=None): self.smaller = smaller self.value = value self.larger = larger

def add(self, value): if self.value is None: self.value = value elif value < self.value: if self.smaller is None: self.smaller = BinTree(value) else: self.smaller.add(value) else: if self.larger is None: self.larger = BinTree(value) else: self.larger.add(value) mg.block(mg.show, mg.stack()) # <--- draw stack

tree = BinTree() n = 100 for i in range(n): value = random.randrange(n) tree.add(value)

!bin_tree.png

Binary Tree in Visual Studio Code ###

Here we show values being inserted in a Binary Tree in Visual Studio Code. When inserting the last value '29' we "Step Into" the code to show the recursive implementation. !images/bin_tree.gif

See it in the Memory Graph Web Debugger or see the more advanced Multiway Tree with more than two children per node, making the tree less deep and more efficient.

Hash Set ##

python import memory_graph as mg import random random.seed(0) # use same random numbers each run

class HashSet:

def __init__(self, capacity=15): self.buckets = [None] * capacity

def add(self, value): index = hash(value) % len(self.buckets) if self.buckets[index] is None: self.buckets[index] = [] bucket = self.buckets[index] bucket.append(value) mg.block(mg.show, locals()) # <--- draw locals

def contains(self, value): index = hash(value) % len(self.buckets) if self.buckets[index] is None: return False return value in self.buckets[index]

def remove(self, value): index = hash(value) % len(self.buckets) if self.buckets[index] is not None: self.buckets[index].remove(value) hash_set = HashSet() n = 100 for i in range(n): new_value = random.randrange(n) hash_set.add(new_value)

!hash_set.png

Hash Set in PyCharm ###

Here we show values being inserted in a HashSet in PyCharm. When inserting the last value '44' we "Step Into" the code to show more of the details. !images/hash_set.gif

Or see it in the Memory Graph Web Debugger.

Sorting Algorithms #

Visualization of different sorting algorithms in Memory Graph Web Debugger. !selections_sort.png

Bitwise Operators #

In this configuration example we show the decimal, binary and two's complement representation representation of
int values of dictionary subclass Bits to show the result of bitwise operators. The ~ (inverse) operator can be a bit confusing if not shown with two's complement representation. !bitwise_operators.png

Sliding Puzzle Solver #

A sliding puzzle solver as a challenging example showing how memory_graph deals with large amounts of data. Click "Continue" to step through the breadth-first search generations until a solution path is found:

!sliding_puzzle.png

Flow Control #

Some examples where we focus more on the flow of execution in Python code.

Different Methods ##

This example shows the difference between 'instance', 'class' and 'static' methods:

python class My_Class:

# class variables: count = 0 unique_value = 0

def __init__(self): print("__init__ called, initialize instance variables of object") self.my_instance_variable = [] # create instance variable My_Class.count += 1 # also change class variable

def my_instance_method(self): print("my_instance_method called, access to instance variables via 'self'") self.my_instance_variable.append(My_Class.unique_value) My_Class.unique_value += 1 # also change class variable

@classmethod def my_class_method(cls): print("my_class_method called, access to class variables via 'cls', no instance variables") print(f"{cls.count=}")

@staticmethod def my_static_method(): print("my_static_method called, no 'self' or 'cls'") print(f"{My_Class.count=}") # but still access to class variables

obj1 = My_Class() obj2 = My_Class()

obj1.my_instance_method() obj1.my_class_method() obj1.my_static_method()

for _ in range(2): obj1.my_instance_method() obj2.my_instance_method()

print(f"{obj1.count=}") # reading class variable if no instance variable found obj1.count = 100 # creating instance variable print(f"{obj1.count=}") # now finding and reading instance variable print(f"{My_Class.count}") # class variable still available via class name print(f"{obj2.count=}") # obj2 still reads the class variable

Run it in the Memory Graph Web Debugger.

Inheritance ##

This example shows the flow of control when using inheritance with:

  • super()
  • class variable message_count
python class Notification_Service: message_count = 0 def __init__(self, priority): self.priority = priority self.log = []

def send(self, sender, receiver, message): self.log.append((type(self).__name__, self.priority, sender, receiver, message)) Notification_Service.message_count += 1

class Email_Notification(Notification_Service): def __init__(self, priority, from_email): super().__init__(priority) self.from_email = from_email def send(self, to_email, message): super().send(self.from_email, to_email, message) print(f'sending Email from:{self.from_email} to:{to_email}') print(f'priority: {self.priority} message: "{message}"')

class SMS_Notification(Notification_Service): def __init__(self, priority, from_nr): super().__init__(priority) self.from_nr = from_nr def send(self, to_nr, message): super().send(self.from_nr, to_nr, message) print(f'sending SMS from:{self.from_nr} to:{to_nr}') print(f'priority: {self.priority} message: "{message}"') email = Email_Notification(3, '[email protected]') email.send('[email protected]', 'Your report is ready') email.send('[email protected]', 'Update to Privacy Policy') sms = SMS_Notification(3, '0123456789') sms.send('001122334455', 'Update to Privacy Policy')

Run it in the Memory Graph Web Debugger.

Iterators ##

What actually happens when Python executes a for-loop?

python for value in container: print(value)
Behind the scenes, Python uses the iterator protocol:
python iterator = iter(container) while True: try: value = next(iterator) print(value) except StopIteration: break
- iter(container): creates an iterator.
  • next(iterator): retrieves one value at a time.
When there are no more values, the iterator raises StopIteration. The for-loop catches this exception automatically and ends the loop. For containers that support backward iteration, Python also provides:

  • reversed(container): creates a backward iterator.
We can support these operations in our own classes by implementing:
python def __iter__(self): def __next__(self): def __reversed__(self):
This provides a powerful abstraction: an algorithm can process values without needing to know how a container stores them internally. The same algorithm can therefore work with lists, sets, dictionaries, linked lists, trees, and many other containers.

Run a Linked_List iterator example in the Memory Graph Web Debugger.

Decorator ##

This example shows the flow of control when using a decorator. A decorator wraps a function and is active before and after the function is called.

python def log_call(function): def wrapper(args, *kwargs): print(f"Calling {function.__name__} with: {args}, {kwargs}") returned = function(args, *kwargs) print(f"Finished {function.__name__} with return value: {returned}") return returned return wrapper

@log_call def calculate_total(price, quantity, rounding=False): result = price * quantity return round(result) if rounding else result

@log_call def send_email(receiver, message, sender="[email protected]"): print(f"Sending email to:{receiver} from:{sender}, {message}")

total = calculate_total(7.5, 3, rounding=True) print(total) send_email("[email protected]", "Your order is ready")

Run it in the Memory Graph Web Debugger.

Exception Handling ##

This example shows the flow of control when using exception handling.

python def fun2(): try: d = [0] * 3 for i in range(6): try: print(f'{i=}') d[i] = i # raises IndexError when i>=3 except ZeroDivisionError as e: print(type(e), e) except AssertionError as e: print(type(e), e) print('fun2() returns')

def fun1(): try: return fun2() except NameError as e: print(type(e), e) print('fun1() returns')

try: fun1() except LookupError as e: print(type(e), e) print('program ended cleanly')

Run it in the Memory Graph Web Debugger. In the program an IndexError exception is raised which propagates up the call stack until it reaches an except clause that matches its type where it is handled. Here, it is handled by the LookupError except clause because IndexError is a subclass of LookupError. Exceptions that are not handled terminate the execution of a program while its traceback is shown for analyses.

!exception_tree.png

Lazy Evaluation ##

In the following Eager and Lazy ealuation examples, we use this pr() function to print in what order things are created and used.

python def pr(tag, v): print(tag, v) return v
With eager evaluation, the function creates all three elements up front and stores them in a list before iteration begins. With lazy evaluation, the function returns a generator that creates each element only when it is needed.

<table> <tr> <td width="50%" valign="top"><strong>Eager</strong></td> <td width="50%" valign="top"><strong>Lazy</strong></td></tr> <tr><td width="50%" valign="top">

python def fun(): result = [] for i in range(3): result.append(pr('create:', i)) return result

for i in fun(): pr('use:', i)

Run in Memory Graph Web Debugger

</td><td width="50%" valign="top">

python def fun(): for i in range(3): yield pr('create:', i)

for i in fun(): pr('use:', i)

Run in Memory Graph Web Debugger

</td></tr> <tr><td width="50%" valign="top">

python def fun(): return [ pr('create:', i) for i in range(3) ]

for i in fun(): pr('use:', i)

Run in Memory Graph Web Debugger

</td><td width="50%" valign="top">

python def fun(): return ( pr('create:', i) for i in range(3) )

for i in fun(): pr('use:', i)

Run in Memory Graph Web Debugger

</td></tr> <tr><td width="50%" valign="top">

text create: 0 create: 1 create: 2 use: 0 use: 1 use: 2
</td><td width="50%" valign="top">
text create: 0 use: 0 create: 1 use: 1 create: 2 use: 2
Lazy evaluation is critical when you want to quickly start to process the first elements of a large stream of data, that may not even fit in your RAM entirely, or process just a section of an infite stream.

</td></tr> </table>

Configuration #

Different aspects of memory_graph can be configured. The default configuration can be reset by calling 'mg.config_default.reset()'. The Memory Graph Web Debugger gives examples of the most important configurations.
  • mg.config.reopen_viewer : bool
- If True the viewer is reopened each time show() is called, this might change window focus, default True.
  • mg.config.render_filename : str
- The default filename to render to, default 'memory_graph.pdf'.
  • mg.config.type_labels : bool
- If True the type of each node is shown as label, default True.
  • mg.config.block_prints_location : bool
- If True the source location is printed in block(), default True.
  • mg.config.press_enter_message : str
- Message to ask user to press &lt;Enter&gt; in block(), set to None to disable.
  • mg.config.max_string_length : int
- The maximum length of strings shown in the graph. Longer strings will be truncated.
  • mg.config.embedded_types : set[type]
- Holds all types for which no separate node is drawn but that are embedded in their parent Node.
  • mg.config.embedded_key_types : set[type]
- Holds all types that are embedded as key in a Node_Key_Value node, even when not in 'embedded_types'.
  • mg.config.embedding_types : set[type]
- Holds all dictionary types that embed their key-value tuple children.
  • mg.config.no_index_types : set[type]
- Holds all types like 'set' and 'frozenset' that should not have indices as Node_Linear.
  • mg.config.type_to_node : dict[type, fun(data) -> Node]
- Determines how a data type is converted to a Node subclass for visualization in the graph.
  • mg.config.type_to_color : dict[type, color]
- Maps a type to the graphviz color it gets in the graph.
  • mg.config.type_to_horizontal : dict[type, bool]
- Maps a type to its orientation for Node_Linear and Node_Key_Value. Use 'True' for horizontal and 'False' for vertical. If not specified these nodes vertical unless they have references to children.
  • mg.config.type_to_slicer : dict[type, int]
- Maps a type to a Slicer. A slicer determines how many elements of a data type are shown in the graph to prevent the graph from getting too big. 'Slicer()' does no slicing, 'Slicer(1,2,3)' shows just 1 element at the beginning, 2 in the middle, and 3 at the end.
  • mg.config.max_graph_depth : int
- The maxium depth of the graph with default value 1000.
  • mg.config.graph_cut_symbol : str
- The symbol indicating where the graph is cut short with default
.
  • mg.config.type_to_depth : dict[type, int]
- Maps a type to graph depth to limit the graph size.
  • mg.config.max_missing_edges : int
- Maximum number of missing edges that are shown with default value 2. Dashed references are used to indicate that there are more references to a node than are shown.
  • mg.config.fontname : str
- The font used in the graph, default 'Times-Roman' (widely available on the web).
  • mg.config.fontsize : str
- The font size used in the graph, default '14'.

Functions ##

  • mg.layout(horizontal: bool = None)
- Set graph layout to 'True' for horizontal, 'False' for vertical, or 'None' to toggle.
  • mg.dark_mode(b: bool = None)
- Set dark mode to 'True' or 'False', or 'None' to toggle.
  • mg.transparent_background(b: bool = None)
- Set transparent background to 'True' or 'False', or 'None' to toggle.

Simplified Graph ##

Memory_graph simplifies the visualization (and the viewer's mental model) by not showing separate nodes for immutable types like
bool, int, float, complex, and str by default. This simplification can sometimes be slightly misleading. As in the example below, after a shallow copy, lists a and b technically share their int values, but the graph makes it appear as though a and b each have their own copies. However, since int is immutable, this simplification will never lead to unexpected changes (changing a's ints won’t affect b) so will never result in bugs.

The simplification strikes a balance: it is slightly misleading but keeps the graph clean and easy to understand to focus on mutable types where unexpected changes can occur. This is why it is the default behavior. If you do want to show separate nodes for int values, such as for educational purposes, you can simply remove int from the mg.config.embedded_types set:

python import memory_graph as mg

a = [100, 200, 300] b = a.copy() mg.render(locals(), 'embedded1.png')

mg.config.embedded_types.remove(int) # now show separate nodes for int values

mg.render(locals(), 'embedded2.png')

| !embedded1 | !embedded2 |
|:-----------------------------------------------------------:|:-------------------------------------------------------------:|
| embedded1.png — simplified | embedded2.png — technically correct |

Additionally, the simplification hides away the [reuse of small int values \[-5, 256\]](https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong) in the current CPython implementation, an optimization that might otherwise confuse beginner Python programmers. For instance, after executing a[1]+=1; b[1]+=1 the 201 value is, maybe surprisingly, still shared between a and b, whereas executing a[2]+=1; b[2]+=1 does not result in sharing the 301 value. Similarly CPython uses String Interning to reuse small strings.

Introspection #

Sometimes the introspection fails or is not as desired. For example the
bintrees.avltree.Node object doesn't show any attributes in the graph below.
python import memory_graph as mg import bintrees

Create an AVL tree

tree = bintrees.AVLTree() tree.insert(10, "ten") tree.insert(5, "five") tree.insert(20, "twenty") tree.insert(15, "fifteen")

mg.show(locals())

!avltree_fail.png

All attributes using dir() ##

A useful start is to give it some color, show the list of all its attributes using
dir()`, and setting an empty Slicer to see the attribute list in full.
python import memory_graph as mg import bintrees

Create an AVL tree

tree = bintrees.AVLTree() tree.insert(10, "ten") tree.insert(5, "five") tree.insert(20, "twenty") tree.insert(15, "fifteen")

mg.config.type_to_color[bintrees.avltree.Node] = "sandybrown" mg.config.type_to_node[bintrees.avltree.Node] = lambda data: mg.Node_Linear(data, dir(data)) mg.config.type_to_slicer[bintrees.avltree.Node] = mg.Slicer()

mg.show(locals())

!avltree_dir.png

Next figure out what the attributes are you want to graph and choose a Node type, there are four options:

1) Node_Leaf ##

Node_Leaf is a node with no children and shows just a single value.
python import memory_graph as mg import bintrees

Create an AVL tree

tree = bintrees.AVLTree() tree.insert(10, "ten") tree.insert(5, "five") tree.insert(20, "twenty") tree.insert(15, "fifteen")

mg.config.type_to_color[bintrees.avltree.Node] = "sandybrown" mg.config.type_to_node[bintrees.avltree.Node] = lambda data: mg.Node_Leaf(data, f"key:{data.key} value:{data.value}")

mg.show(locals())

!avltree_leaf.png

2) Node_Linear ##

Node_Linear shows multiple values in a line like a list.
python import memory_graph as mg import bintrees

Create an AVL tree

tree = bintrees.AVLT

... (README truncated for length)

Chat with me