extending_mypy.rst

Extending and integrating mypy

Integrating mypy into another Python application

It is possible to integrate mypy into another Python 3 application by importing mypy.api and calling the run function with a parameter of type List[str], containing what normally would have been the command line arguments to mypy.

Function run returns a Tuple[str, str, int], namely (<normal_report>, <error_report>, <exit_status>), in which <normal_report> is what mypy normally writes to sys.stdout, <error_report> is what mypy normally writes to sys.stderr and exit_status is the exit status mypy normally returns to the operating system.

A trivial example of using the api is the following

import sys
from mypy import api

result = api.run(sys.argv[1:])

if result[0]:
    print('\nType checking report:\n')
    print(result[0])  # stdout

if result[1]:
    print('\nError report:\n')
    print(result[1])  # stderr

print('\nExit status:', result[2])

Extending mypy using plugins

Python is a highly dynamic language and has extensive metaprogramming capabilities. Many popular libraries use these to create APIs that may be more flexible and/or natural for humans, but are hard to express using static types. Extending the PEP 484 type system to accommodate all existing dynamic patterns is impractical and often just impossible.

Mypy supports a plugin system that lets you customize the way mypy type checks code. This can be useful if you want to extend mypy so it can type check code that uses a library that is difficult to express using just PEP 484 types.

The plugin system is focused on improving mypy's understanding of semantics of third party frameworks. There is currently no way to define new first class kinds of types.

Note

The plugin system is experimental and prone to change. If you want to write a mypy plugin, we recommend you start by contacting the mypy core developers on gitter. In particular, there are no guarantees about backwards compatibility.

Backwards incompatible changes may be made without a deprecation period, but we will announce them in the plugin API changes announcement issue.

Configuring mypy to use plugins

Plugins are Python files that can be specified in a mypy :ref:`config file <config-file>` using one of the two formats: relative or absolute path to the plugin to the plugin file, or a module name (if the plugin is installed using pip install in the same virtual environment where mypy is running). The two formats can be mixed, for example:

[mypy]
plugins = /one/plugin.py, other.plugin

Mypy will try to import the plugins and will look for an entry point function named plugin. If the plugin entry point function has a different name, it can be specified after colon:

[mypy]
plugins = custom_plugin:custom_entry_point

In following sections we describe basics of the plugin system with some examples. For more technical details please read docstrings in mypy/plugin.py in mypy source code. Also you can find good examples in the bundled plugins located in mypy/plugins.

High-level overview

Every entry point function should accept a single string argument that is a full mypy version and return a subclass of mypy.plugins.Plugin:

from mypy.plugin import Plugin

class CustomPlugin(Plugin):
    def get_type_analyze_hook(self, fullname: str):
        # see explanation below
        ...

def plugin(version: str):
    # ignore version argument if the plugin works with all mypy versions.
    return CustomPlugin

During different phases of analyzing the code (first in semantic analysis, and then in type checking) mypy calls plugin methods such as get_type_analyze_hook() on user plugins. This particular method for example can return a callback that mypy will use to analyze unbound types with given full name. See full plugin hook methods list :ref:`below <plugin_hooks>`.

Mypy maintains a list of plugins it gets from the config file plus the default (built-in) plugin that is always enabled. Mypy calls a method once for each plugin in the list until one of the methods returns a non-None value. This callback will be then used to customize the corresponding aspect of analyzing/checking the current abstract syntax tree node.

The callback returned by the get_xxx method will be given a detailed current context and an API to create new nodes, new types, emit error messages etc., and the result will be used for further processing.

Plugin developers should ensure that their plugins work well in incremental and daemon modes. In particular, plugins should not hold global state due to caching of plugin hook results.

Current list of plugin hooks

get_type_analyze_hook() customizes behaviour of the type analyzer. For example, PEP 484 doesn't support defining variadic generic types:

from lib import Vector

a: Vector[int, int]
b: Vector[int, int, int]

When analyzing this code, mypy will call get_type_analyze_hook("lib.Vector"), so the plugin can return some valid type for each variable.

get_function_hook() is used to adjust the return type of a function call. This is a good choice if the return type of some function depends on values of some arguments that can't be expressed using literal types (for example a function may return an int for positive arguments and a float for negative arguments). This hook will be also called for instantiation of classes. For example:

from contextlib import contextmanager
from typing import TypeVar, Callable

T = TypeVar('T')

@contextmanager  # built-in plugin can infer a precise type here
def stopwatch(timer: Callable[[], T]) -> Iterator[T]:
    ...
    yield timer()

get_method_hook() is the same as get_function_hook() but for methods instead of module level functions.

get_method_signature_hook() is used to adjust the signature of a method. This includes special Python methods except __init__() and __new__(). For example in this code:

from ctypes import Array, c_int

x: Array[c_int]
x[0] = 42

mypy will call get_method_signature_hook("ctypes.Array.__setitem__") so that the plugin can mimic the ctypes auto-convert behavior.

get_attribute_hook overrides instance member field lookups and property access (not assignments, and not method calls). This hook is only called for fields which already exist on the class. Exception: if __getattr__ or __getattribute__ is a method on the class, the hook is called for all fields which do not refer to methods.

get_class_decorator_hook() can be used to update class definition for given class decorators. For example, you can add some attributes to the class to match runtime behaviour:

from lib import customize

@customize
class UserDefined:
    pass

var = UserDefined
var.customized  # mypy can understand this using a plugin

get_metaclass_hook() is similar to above, but for metaclasses.

get_base_class_hook() is similar to above, but for base classes.

get_dynamic_class_hook() can be used to allow dynamic class definitions in mypy. This plugin hook is called for every assignment to a simple name where right hand side is a function call:

from lib import dynamic_class

X = dynamic_class('X', [])

For such definition, mypy will call get_dynamic_class_hook("lib.dynamic_class"). The plugin should create the corresponding mypy.nodes.TypeInfo object, and place it into a relevant symbol table. (Instances of this class represent classes in mypy and hold essential information such as qualified name, method resolution order, etc.)

get_customize_class_mro_hook() can be used to modify class MRO (for example insert some entries there) before the class body is analyzed.

get_additional_deps() can be used to add new dependencies for a module. It is called before semantic analysis. For example, this can be used if a library has dependencies that are dynamically loaded based on configuration information.