15 Killer Jupyter Hacks That Are Guaranteed To Save you Hours

The Moment You Start Using Them

Published in

Geek Culture

13 min readDec 21, 2022

As Data Scientists we use Jupyter Notebooks almost every day from loading the data to creating and deploying models with it.

I like Jupyter Notebooks because of their simple, streamlined design yet indispensable to any python oriented task. Especially, running and testing scripts with a subset of data as long as the kernel is active is life-changing for me.

However, with the simplicity of this tool, we often tend to make some mistakes that waste time and the cost of computation. Both of them are fundamental pillars of a project.

In this article, we are going to talk about some amazing tips and tricks that every data scientist must know. These could save time and reduce the cost of computation.

Let’s begin! 🔥

1. Magic Commands 🪄

In Jupyter notebooks, “magic commands” are special commands that are not part of the Python language, but make your life easier. These commands are preceded by a % symbol.

Magic commands can be useful and can be embedded directly into python code and solve common problems, such as listing all the files in the current directory or changing the current working directory.

Here are some common magic commands:

%run: Run a Python script within the current kernel.
%load: Load code from a script and run it in the current kernel.
%who: List all the variables.
%timeit: Time the execution of a single line of code.
%debug: Enter the debugger at the point of exception.
%matplotlib inline: Display plots within the notebook.
%load_ext: Load an extension, such as an IPython extension.
%pwd: Print the current working directory.
%ls: Show all the files in the current directory.

View all the magic commands

You can see a list of all magic commands by running %lsmagic in a cell.

lsmagic magic command notebook — Jupyter cell (Image by Author)

To get more information about a specific magic command, you can use the ? operator, for example %run?.

2. Execute another Jupyter notebook file

You can use magic commands to do some interesting things. For example, to execute python code from .py files or jupyter notebook from .ipynb file.

%run will execute a jupyter notebook and show output, it’s not as same as importing python module.

We can run two-histogram notebook and get a output as,

%run ./two-histograms.ipynb

3. View the documentation of a Method or magic command 📓

You can easily view the documentation of a method by highlighting it and pressing Shift + Tab. It will show you the docstring written at the time of writing the function. You can also expand the modal by pressing the + button on the top right.

You can also get more information about each magic command by highlighting it and pressing Shift + Tab.

4. Add Multiple Cursors 🖱

I know someday you faced a situation where you need to rename a variable written in several places or while editing code you wish you had multiple cursors.

In Jupyter notebooks, it is possible to use multiple cursors to edit text simultaneously. This can be useful if you want to make the same change to multiple lines of text at once.

To use multiple cursors in Jupyter notebooks, you can hold down the Alt key and click on the desired locations. This will create a cursor at each clicked location. You can then make edits as usual, and the changes will be applied to all locations simultaneously.

Windows: Hold alt + left-click and drag your cursor.
Mac: Hold option + left-click and drag your cursor.

You can also use the Shift + Alt + Up/Down arrow key combination to select multiple lines of text and create a cursor at the beginning of each selected line.

Keep in mind that using multiple cursors can be a bit tricky, so it’s a good idea to save your notebook before using this feature, in case you make any unintended changes.

5. Insert code from another external python script 🗃

Using this trick you can replace the contents of the cell with external python scripts. You can either use any python file on your computer or from a URL as source.

# Before Running
%load ./hello_world.py

# After Running
# %load ./hello_world.py
if __name__ == "__main__":
 print("Hello World!")

Output
Hello World!

6. Running CMD/Shell Commands ⌨

Do you exit your notebook to run a shell command?

You can run shell commands in a Jupyter notebook by using the ! prefix before the command. For example, to list the files in the current directory, you can use the ls command:

!ls

You can also pass arguments to the command by adding them after the command. For example, to list the files in a specific directory, you can use the -l option to display the files in a long format and specify the directory path as an argument:

!ls -l /path/to/directory

You can also assign the output of the command to a variable and use it in your code. For example:

files = !ls
print(files)

This will assign the list of files in the current directory to the files variable and print it.

You can use this technique to run any shell command in a Jupyter notebook, as long as the command is available on the system where the notebook is being run.

7. Set Alarm for Program Completion ⏰

It’s always helpful to get an notification when your model finishes training or any task.

Windows

We can easily setup it up in windows 10 using the win10toast module.

pip install win10toast

Now you can show completion using this code.

from win10toast import ToastNotifier
toaster = ToastNotifier()

#Your program

toaster.show_toast("Execution complete",
                   "Your calculation completed",
                   duration=10)

You can also ring an alarm when your program has completed execution.

import winsound

# set an alarm of 440HZ for one second (1000ms)
duration = 1000
freq = 440

winsound.Beep(freq, duration)

Mac and Linux

To set the alarm for when a program completes in a Jupyter notebook, you can use the os module to play a sound using the afplay command (on macOS) or the aplay command (on Linux).

Here is an example of how you can do this:

import os

# Run your program here
# Play a sound when the program completes
os.system("afplay /path/to/sound.mp3")  # macOS
# os.system("aplay /path/to/sound.wav")  # Linux

You can replace /path/to/sound.mp3 with the path to the sound file that you want to play. This can be any audio file that is supported by the afplay or aplay command, such as MP3, WAV, or AIFF.

Keep in mind that this method will only work if the afplay or aplay the command is available on the system where the Jupyter notebook is being run.

In Mac you can also use built in command say to say something when your program completes.

import os

os.system('say "Your program has now finished"')

8. Measure Cell Execution Time ⏱️

To measure the execution time of a cell in a Jupyter notebook, you can use the %timeit magic command. This command will execute the cell multiple times and return the average execution time.This will time the execution of the sum function and return the average time taken to execute the function.

Here’s an example of how to use %timeit:

%timeit sum(range(100))

This will time the execution of the sum function and return the average time taken to execute the function.

Jupyter cell (Image by Author)

You can also use the %%timeit cell magic to measure the execution time of an entire cell:

%%timeit

total = 0
for i in range(1000):
    total += i

If you want more detailed information about the execution time, you can use the time module in Python. Here's an example of how to use it:

import time

start_time = time.time()

# code to measure
sum(range(100))

end_time = time.time()

elapsed_time = end_time - start_time
print(f'Execution time: {elapsed_time:.2f} seconds')

Note: that these methods will only measure the execution time of the code in the cell. If the cell depends on other cells or external resources, the execution time will not include the time required to execute those dependencies.

9. Pass variables between notebooks

In a Jupyter notebook, the %store magic command allows you to pass variables between notebooks.

Here’s an example of how to use it:

# In the first notebook
var1 = 10
%store var1

# In the second notebook
%store -r var1
print(var1)

# Now you can use the variable 'var1' in the second notebook as if it was defined in the second notebook

The %store magic command has several options:

%store var1: stores the variable var1
%store -r var1: retrieves the stored variable var1 and assigns it to a variable with the same name in the current notebook
%store -d var1: deletes the stored variable var1
%store -z: deletes all stored variables

You can also store multiple values using a single %store command.

%store var1 var2

Note: that the %store magic command only works within the same Jupyter session, so you'll need to open the notebooks in the same session for the variables to be available.

You can also use store magic command for a unique solution. If you want to store value of a variable even after you restart kernal due to any reason.

The demonstration of store command even after kernal restart.

10. List all Keyboard Shortcuts ⌨

Learning keyboard shortcuts will make you a power user and will save you lots of time. The list of these shortcuts is huge, you can only remember these by adding them to your workflow over time. You can easily check them under the menu at the top: Help > Keyboard Shortcuts, or by pressing the H key in command mode.

Here is a list of some commonly used keyboard shortcuts in Jupyter notebooks:

Enter: enter edit mode for the current cell
Esc: enter command mode for the current cell
Shift + Enter: run the current cell and move to the next cell
Ctrl + Enter: run the current cell
Alt + Enter: run the current cell and insert a new cell below
Shift + Tab: show the documentation for the current function or object
Ctrl + S: save the current notebook
A: insert a new cell above the current cell (in command mode)
B: insert a new cell below the current cell (in command mode)
M: change the current cell to a Markdown cell (in command mode)
Y: change the current cell to a code cell (in command mode)
D + D: delete the current cell (in command mode)
Z: undo the last cell deletion (in command mode)
X: Cut the selected cell (in command mode)
C: Copy the selected cell (in command mode)
V: Paste the selected cell (in command mode)
Ctrl + Shift + -: Will split the current cell into two from where your cursor is. (in command mode)
Esc + F: Find and replace your code but not the outputs. (in command mode)
Esc + O: Toggle cell output (in command mode)
Select Multiple Cells: Shift + Down selects the next cell in a downward direction.
Shift + Up selects the next cell in the upward direction. (in command mode)
Shift + M: Merge multiple selected cells. (in command mode)

You can also access a list of all keyboard shortcuts by going to the Help menu and selecting Keyboard Shortcuts. This will open a dialog box with a list of all available keyboard shortcuts.

To view a list of keyboard shortcuts in a Jupyter notebook, you can go to the Help menu and select Keyboard Shortcuts. This will open a dialog box with a list of all available keyboard shortcuts.

Alternatively, you can use the %shortcuts magic command to view a list of keyboard shortcuts in the output area of the current cell:

%shortcuts

This will display a list of all keyboard shortcuts and their corresponding actions.

11. Hide Unnecessary Output

When you are creating an report from Jupyter notebook, seeing unnecessary memory codes or object id is annoying. To hide this unnecessary output in a notebook cell, you can use the ; character at the end of a line of code. This will suppress the output of that line.

For example, consider the following code:

If you want to suppress the output of matplotlib line, you can use the ; character at the end of plotting statement:

plt.plot(x,y);

12. Writing functions in other languages than python

If you are dealing with a lot of large datasets and speed of numpy is not good enough for then you can directly write some c and fortran code directly inside from your python code.

The principle behind it is when you write a function in cpython or fortain, it can be compiled in the dynamic library and use it as a python wrapper.

Writing a c program in python

In you want to start writing your functions in c then you will need cython library. You can easily install using a new cell,

!pip install Cython

%load_ext Cython

%%cython
def myltiply_by_2(float x):
return 2.0 * x

myltiply_by_2(23.)

Writing a Fortran program in python

To write a fortran function we will need a different library called fortrain-magic. You can easily install using a new cell,

!pip install fortran-magic

%load_ext fortranmagic

%%fortran subroutine compute_fortran(x, y, z)
real, intent(in) :: x(:), y(:)
real, intent(out) :: z(size(x, 1))
z = sin(x + y)
end subroutine compute_fortran

compute_fortran([1, 2, 3], [4, 5, 6])

13. Extend the number of columns and rows shown in pandas output

By default panda’s dataframe only can show a limited number of rows and columns. However we can change the default value.

There are several ways you can extend the number of rows and columns shown in a pandas DataFrame in a Jupyter Notebook.

Option 1:

you can use the pd.options.display.max_rows and pd.options.display.max_columns options.

For example, to display up to 100 rows and 50 columns, you can use the following code:

import pandas as pd

pd.options.display.max_rows = 100
pd.options.display.max_columns = 50

Option 2:

You can also use the pd.set_option function to set these options. For example:

pd.set_option("display.max_rows", 100)
pd.set_option("display.max_columns", 50)

These options will apply to all pandas DataFrames displayed in the Jupyter Notebook.

Alternatively, you can use the head and tail methods to display the first or last few rows of a DataFrame. For example:

df.head(10)  # displays the first 10 rows of the DataFrame
df.tail(5)   # displays the last 5 rows of the DataFrame

Option 3:

You can also use the display function from the IPython.display module to control the display of a DataFrame. For example:

from IPython.display import display

display(df, max_rows=100, max_columns=50)

This will display the DataFrame with up to 100 rows and 50 columns.

14. Extracting Input and Output Cell data

If you have been in a situation when you executed a cell and after it complete execution you realized that you forgot to assign a variable?

I been there and there is a cleaver solution for this. But don’t use this in your pipeline as if your execution number changes. It’s going to break your logic.

When ever we execute a cell in jupyter notebook, it’s going to assign a line number as ln:

In a list and stores all the codes executed. We can easily access it by passing the execution number as a index.

Similarly, we get something call Out when a cell finishes execution.

Out is a python dictionary that stores all the cell outputs. We can access outputs by using execution number as a key.

15. Export the contents of a cell/Show the contents of an external script 📝

You may have faced a situation when you need to create a python file out of a jupyter cell. We can create a .py file and copy, paste the code. But there is a better solution.

%%writefile is a Jupyter Notebook magic command that allows you to save the contents of a cell as a Python file. For example, if you have the following code in a cell:

%%writefile example.py
def add(a, b):
    return a + b
print(add(3, 4))

You can then run the cell and it will create a file named example.py in the same directory as the Jupyter Notebook, with the contents of the cell written to the file.

%pycat is a Jupyter Notebook magic command that displays the contents of a Python file in a cell in the notebook. For example, if you have the following code in a cell:

%pycat example.py

It will display the contents of the example.py file in the cell output. This can be useful for quickly reviewing the contents of a Python file without having to switch to a separate text editor or terminal window.

Conclusion

That’s it for the list and with this we have come to the end of this blog. I hope you found something interesting and new.

Thank you so much for reading the article. 😇

Don’t forget to share this article with all Jupyter notebook users out there; Sharing is Caring.

I’ll love to know you as a person feel free to connect with me on Linkedin at the-anup-das.