Managing a Python development environment in Emacs

Diego Fernández Giraldo

Published in

Analytics Vidhya

6 min readNov 18, 2019

This is the second part of a 2 part post on my Python development. Click here for the 1st part.

The main things I need/want from my IDE are:

Code completion
Documentation
Jump to definition
Send region/buffer to REPL (interpreted languages)
Identification of code issues
Auto format/organize/clean code
Basic refactoring (rename variables) (nice to have)

I’ve tried using a variety of editors an IDEs, but I always end up finding something I don’t like and come back to Emacs. My main issues tend to be around using different environments (one IDE for Java, one for Python, one for Web development, one for editing config files/scripting/etc) and truly learning each tool to take full advantage of it. Instead of having to switch tools, I can just “context switch” in Emacs. I can get most of the functionality I need; although maybe not as “polished” as some alternatives, the extensibility makes it easy to work with almost any type of file/programming language within the same comfortable environment. Here I’ll talk specifically about my setup for Python and how I’ve fulfilled most of the points above.

Elpy

The main tool I use in Emacs for Python development is Elpy. This great tool takes care of most of my requirements: code completion (via jedi), documentation (shows function definitions), jump to def, send to REPL, and basic refactoring. Elpy provides error checking via flymake, but I disable that and use flycheck instead. Here’s my Elpy config:

(use-package elpy
    :straight t
    :bind
    (:map elpy-mode-map
          ("C-M-n" . elpy-nav-forward-block)
          ("C-M-p" . elpy-nav-backward-block))
    :hook ((elpy-mode . flycheck-mode)
           (elpy-mode . (lambda ()
                          (set (make-local-variable 'company-backends)
                               '((elpy-company-backend :with company-yasnippet))))))
    :init
    (elpy-enable)
    :config
    (setq elpy-modules (delq 'elpy-module-flymake elpy-modules))
    ; fix for MacOS, see https://github.com/jorgenschaefer/elpy/issues/1550
    (setq elpy-shell-echo-output nil)
    (setq elpy-rpc-python-command "python3")
    (setq elpy-rpc-timeout 2)))

My most commonly used shortcuts are:

elpy-shell-send-region-or-buffer C-c C-c Lets me send code into the REPL. If a region is selected it’ll send the region, otherwise it’ll send the whole buffer.

elpy-doc C-c C-d Shows the documentation of the function/class under cursor in a new window.

elpy-goto-definition M-. Go to the definition of the function/class under cursor. This helps me see how it’s implemented, even if it’s in a 3rd party package.

elpy-nav-indent-shift-(left/right) M-(left/right) Shifts a selected block either right or left 1 level. This is great for moving blocks of code in/out of if statements or loops.

elpy-nav-(forward/backward)-block C-M-(n/p) Navigate between blocks of code. This is helpful for moving between blocks/functions/classes.

elpy-nav-move-line-or-region-(up/down) M-(up/down) Move an entire block of code up/down. This is helpful for moving code around.

Company

Elpy uses company-mode for code completion. I do a few tweaks to my company-mode config, so I’ll share those here:

(use-package company
  :straight t
  :diminish company-mode
  :init
  (global-company-mode)
  :config
  ;; set default `company-backends'
  (setq company-backends
        '((company-files          ; files & directory
           company-keywords       ; keywords
           company-capf)  ; completion-at-point-functions
          (company-abbrev company-dabbrev)
          ))(use-package company-statistics
    :straight t
    :init
    (company-statistics-mode))(use-package company-web
    :straight t)(use-package company-try-hard
    :straight t
    :bind
    (("C-<tab>" . company-try-hard)
     :map company-active-map
     ("C-<tab>" . company-try-hard)))(use-package company-quickhelp
    :straight t
    :config
    (company-quickhelp-mode))
)

I don’t do much out of the ordinary, other than set the default company-backends (note that when Elpy is enabled, this list of completions is not active by default). However, I also use some other packages to add some nice features. company-statistics helps keep track of which completions I use most often and uses that info the improve the ordering. company-web generates some completions for web modes. company-try-hard lets me cycle through different company backend lists using C-<tab>. And company-quickhelp gives me documentation in a pop-up when suggesting completions.

Format/fix on save

There’s a few tools out there that are very helpful for auto-formatting code, documentation and imports. I use a few of these, and have enabled some to run automatically on save. Here’s my config:

(use-package buftra
    :straight (:host github :repo "humitos/buftra.el"))(use-package py-pyment
    :straight (:host github :repo "humitos/py-cmd-buffer.el")
    :config
    (setq py-pyment-options '("--output=numpydoc")))(use-package py-isort
    :straight (:host github :repo "humitos/py-cmd-buffer.el")
    :hook (python-mode . py-isort-enable-on-save)
    :config
    (setq py-isort-options '("--lines=88" "-m=3" "-tc" "-fgw=0" "-ca")))(use-package py-autoflake
    :straight (:host github :repo "humitos/py-cmd-buffer.el")
    :hook (python-mode . py-autoflake-enable-on-save)
    :config
    (setq py-autoflake-options '("--expand-star-imports")))(use-package py-docformatter
    :straight (:host github :repo "humitos/py-cmd-buffer.el")
    :hook (python-mode . py-docformatter-enable-on-save)
    :config
    (setq py-docformatter-options '("--wrap-summaries=88" "--pre-summary-newline")))(use-package blacken
    :straight t
    :hook (python-mode . blacken-mode)
    :config
    (setq blacken-line-length '88))(use-package python-docstring
    :straight t
    :hook (python-mode . python-docstring-mode))

pyment helps me create docstrings easily. I can highlight my function definition, M-x py-pyment-region and voila, a barebones docstring is created from my function definiton.

isort helps organize my imports, and is done automatically on save. I use some options here to work well with black. I also use autoflake, mainly to expand * imports and remove unused imports.

docformatter helps me format docstrings so they’re within the 88 col limit and helps keep things neat, organized, and standard. This runs automatically on save.

black helps me format code and is auto-enabled on save.

docstring helps highlighting in docstrings when they’re in reStructuredText or Epydoc formats. This isn’t as useful now that I’ve mainly switched to Numpy style, but it’s nice to have when I work on code where I don’t get to choose the format.

Pyenv

In my previous blog I explained how I set up pyenv and pyenv-version-alias. In order to use it in Emacs, I forked and extended a package to meet my needs. I install it and set it up with:

(use-package pyenv
    :straight (:host github :repo "aiguofer/pyenv.el")
    :config
    (setq pyenv-use-alias 't)
    (setq pyenv-modestring-prefix " ")
    (setq pyenv-modestring-postfix nil)
    (setq pyenv-set-path nil)(global-pyenv-mode)
    (defun pyenv-update-on-buffer-switch (prev curr)
      (if (string-equal "Python" (format-mode-line mode-name nil nil curr))
          (pyenv-use-corresponding)))
    (add-hook 'switch-buffer-functions 'pyenv-update-on-buffer-switch))

This gives me a nice display in the mode line showing the currently active pyenv version (using the alias if provided). It also makes sure that the right pyenv is always set up when I switch between buffers. Unfortunately, this hook doesn’t always work so I sometimes have to manually run pyenv-use-corresponding (note that this requires installing switch-buffer-functions).

Misc

I also have a few other adjustments in my Python setup. I described in the other post how I generate a Jupyter kernel per pyenv version. I use some custom code to ensure the right kernel is used when I’m working on some Python code. It will also use django-admin if I’m working on a Django project. I also fix the comint password entry so that passwords are “starred” out when getpass is called in the Python shell. Here’s my config for all of that:

(use-package python
  :hook (inferior-python-mode . fix-python-password-entry)
  :config
  (setq python-shell-interpreter "jupyter-console"
        python-shell-interpreter-args "--simple-prompt"
        python-shell-prompt-detect-failure-warning nil)
  (add-to-list 'python-shell-completion-native-disabled-interpreters
               "jupyter-console")
  (add-to-list 'python-shell-completion-native-disabled-interpreters
               "jupyter")(defun fix-python-password-entry ()
    (push
     'comint-watch-for-password-prompt comint-output-filter-functions))(defun my-setup-python (orig-fun &rest args)
    "Use corresponding kernel"
    (let* ((curr-python (car (split-string (pyenv/version-name) ":")))
           (python-shell-buffer-name (concat "Python-" curr-python))
           (python-shell-interpreter-args (if (bound-and-true-p djangonaut-mode)
                                              "shell_plus -- --simple-prompt"
                                            (concat "--simple-prompt --kernel=" curr-python)))
           (python-shell-interpreter (if (bound-and-true-p djangonaut-mode)
                                         "django-admin"
                                       python-shell-interpreter)))
      (apply orig-fun args)))(advice-add 'python-shell-get-process-name :around #'my-setup-python)
  (advice-add 'python-shell-calculate-command :around #'my-setup-python)
)

Conclusion

This setup does most of what I need. The biggest issue I have is that the pyenv version doesn’t always set correctly when switching between open buffers (because there’s no “real” hooks for switching between buffers). I also have a lot of other emacs config options, so feel free to check out my config in my dotfiles repo.