Warning

You are reading a version of the website built against the unstable main branch. This content is liable to change without notice and may be inappropriate for your use case. You can find the documentation for the current stable release here.

pyop3 package

Subpackages

Submodules

pyop3.arrayref module

class pyop3.arrayref.ArrayReference(base, indices, block_shape=())[source]

Bases: object

Class representing an array that has been indexed.

__getitem__(indices, /)[source]

pyop3.buffer module

class pyop3.buffer.AbstractArrayBuffer[source]

Bases: AbstractBuffer

abstract property ordered
abstract property shape
property size
class pyop3.buffer.AbstractBuffer[source]

Bases: Object

DEFAULT_DTYPE = dtype('float64')
DEFAULT_PREFIX = 'buffer'
copy()[source]
abstract property dtype
abstractmethod duplicate(*, copy=False, constant=None)[source]
abstract property name
abstractmethod nest_shape(nest_indices=())[source]

Return the nest shape at a given level of nesting.

class pyop3.buffer.ArrayBuffer(data, sf=None, *, name=None, prefix=None, constant=False, rank_equal=False, ordered=False)[source]

Bases: AbstractArrayBuffer, ConcreteBuffer

A buffer whose underlying data structure is a lazily-evaluated NumPy/CuPy array.

Parameters:

data – Note that the arrays passed here should be unique to this array. Do not use the same array between buffers as it will disrupt the state tracking done by the buffer.

DEFAULT_PREFIX = 'array'
as_vec(mode, block_shape=())[source]
assemble()[source]

Update roots and leaves.

broadcast_roots_to_leaves()[source]

Unconditionally update leaves.

This will overwrite any existing parallel state tracking.

broadcast_roots_to_leaves_begin()[source]

Start unconditionally updating leaves.

broadcast_roots_to_leaves_end()[source]
collect_buffers(visitor)[source]
property comm
property constant
property data_ro
property data_rw
property data_wo

Have to be careful. If not setting all values (i.e. subsets) should call \(reduce_leaves_to_roots\) first.

When this is called we set roots_valid, claiming that any (lazy) ‘in-flight’ writes can be dropped.

property dtype
duplicate(*, copy=False, constant=None, **kwargs)[source]
classmethod empty(shape, dtype=None, **kwargs)[source]
freeze()[source]

Freeze the buffer, turning any modifying accesses into errors.

classmethod from_scalar(value, *, dtype=None, **kwargs)[source]
classmethod full(size, fill_value, dtype=None, **kwargs)[source]
get_array(intent='ro')[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
inc_state()[source]
property is_frozen
is_nested = False
localize()[source]
property name
nest_shape(nest_indices=())[source]
property ordered
property rank_equal
record_new(**attrs)

Create and initialise a new record from an existing one.

reduce_leaves_to_roots(op)[source]

Unconditionally update roots.

This will overwrite any existing parallel state tracking.

Parameters:

op – The MPI reduction operation to apply when pulling leaves onto roots.

reduce_leaves_to_roots_begin(op)[source]

Start unconditionally updating roots.

reduce_leaves_to_roots_end(op)[source]

Finish unconditionally updating roots.

sf
property shape
property state
sync_leaves()[source]

Update leaves.

sync_leaves_begin()[source]

Start updating leaves.

sync_leaves_end()[source]

Finish updating leaves.

sync_roots()[source]

Update roots.

sync_roots_begin()[source]

Start updating roots.

sync_roots_end()[source]

Finish updating roots.

unfreeze()[source]

Unfreeze the buffer.

vec_ro(block_shape=())[source]
vec_rw(block_shape)[source]
vec_wo(block_shape)[source]
zero()[source]
classmethod zeros(shape, dtype=None, **kwargs)[source]
exception pyop3.buffer.BadOrderingException[source]

Bases: Exception

class pyop3.buffer.ConcreteBuffer[source]

Bases: AbstractBuffer

Abstract class representing buffers that carry actual data.

abstract property constant
abstractmethod inc_state()[source]
abstract property state

Counter used to keep track of modifications.

abstractmethod zero()[source]
exception pyop3.buffer.DataTransferInFlightException[source]

Bases: Exception

class pyop3.buffer.DenseBlockPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(comm)[source]
mult(mat, x, y)[source]
property sizes
class pyop3.buffer.DenseColumnPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(sf)[source]
mult(mat, x, y)[source]
property sizes
class pyop3.buffer.DensePythonMatContext(buffer)[source]

Bases: object

Matrix context for storing narrow and dense (usually Nx1 or 1xN) matrices as PETSc Vecs.

This is important in massively parallel settings where a single dense row would live on a single process and hence be a significant performance bottleneck.

__getitem__(key)[source]
property comm
property data_ro
duplicate(*, copy=False)[source]
getDiagonal(mat, result=None)[source]
abstractmethod mult(mat, x, y)[source]

Set y = self @ x.

multTranspose(mat, x, y)[source]
multTransposeAdd(mat, x, y, z)[source]

z = y + mat^Tx

set_diagonal(value)[source]
zeroEntries(mat)[source]
class pyop3.buffer.DenseRowPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(sf)[source]
mult(mat, x, y)[source]
property sizes
exception pyop3.buffer.IncompatibleStarForestException[source]

Bases: Exception

class pyop3.buffer.IndexedBuffer(buffer, nest_indices)[source]

Bases: Object

Class collecting buffer and nest information.

This class is important during code generation because we want to emit code that matches the wrapped, non-nested types, but we still want to be able to refer to the actual buffer data structure.

buffer
collect_buffers(visitor)[source]
property denested
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property handle

Return the underlying data structure.

This method will introspect the buffer and pull out the object needed by the compiler/executor.

property name
nest_indices
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.buffer.MonolithicPetscMatBufferSpec(mat_type, row_spec, column_spec)[source]

Bases: PetscMatBufferSpec

MonolithicPetscMatBufferSpec(mat_type: ‘str’, row_spec: ‘PetscMatAxisSpec | pyop3.axis_tree.AbstractAxisTree’, column_spec: ‘PetscMatAxisSpec | pyop3.axis_tree.AbstractAxisTree’)

column_spec
property comm
get_instruction_executor_cache_key(visitor)[source]
mat_type
record_new(**attrs)

Create and initialise a new record from an existing one.

row_spec
class pyop3.buffer.MonolithicPetscMatInitBufferSpec(mat_type, block_shape=((), ()))[source]

Bases: PetscMatInitBufferSpec

MonolithicPetscMatInitBufferSpec(mat_type: ‘str’, block_shape: ‘tuple[tuple[int, …], tuple[int, …]]’ = ((), ()))

block_shape = ((), ())
mat_type
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.buffer.NestedPetscMatBufferSpec(submat_specs)[source]

Bases: PetscMatBufferSpec

NestedPetscMatBufferSpec(submat_specs: ‘np.ndarray’)

get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

property shape
submat_specs
class pyop3.buffer.NestedPetscMatInitBufferSpec(submat_specs)[source]

Bases: PetscMatInitBufferSpec

NestedPetscMatInitBufferSpec(submat_specs: ‘np.ndarray’)

mat_type = 'nest'
record_new(**attrs)

Create and initialise a new record from an existing one.

submat_specs
class pyop3.buffer.NullBuffer(shape, dtype=None, *, name=None, prefix=None, ordered=False)[source]

Bases: AbstractArrayBuffer

A buffer that does not carry data.

This is useful for handling temporaries when we generate code. For much of the compilation we want to treat temporaries like ordinary arrays but they are not passed as kernel arguments nor do they have any parallel semantics.

DEFAULT_PREFIX = 'tmp'
collect_buffers(visitor)[source]
property comm
property dtype
duplicate(*, copy=False, constant=None)[source]
get_disk_cache_key(visitor)[source]
instruction_executor_cache_key(buffer_counter)[source]
property name
nest_shape(*args)[source]
property ordered
record_new(**attrs)

Create and initialise a new record from an existing one.

property shape
class pyop3.buffer.PetscMatAxisSpec(size, lgmap, block_shape=())[source]

Bases: Object

PetscMatAxisSpec(size: ‘int’, lgmap: ‘PETSc.LGMap’, block_shape: ‘tuple[int, …]’ = ())

block_shape = ()
property block_size
get_instruction_executor_cache_key(visitor)[source]
lgmap
record_new(**attrs)

Create and initialise a new record from an existing one.

size
class pyop3.buffer.PetscMatBuffer(mat, *, comm, mat_spec=None, name=None, prefix=None, constant=False)[source]

Bases: ConcreteBuffer

A buffer whose underlying data structure is a PETSc Mat.

Parameters:

mat_spec (pyop3.buffer.PetscMatBufferSpec | None) – Only used for preallocation matrices… and actually the only real information is the matrix type, which could be an argument to materialize…

DEFAULT_PREFIX = 'petscmat'
assemble(*, final=True)[source]
assemble_begin(*, final=True)[source]
assemble_end(*, final=True)[source]
collect_buffers(visitor)[source]
property comm
property constant
dtype = dtype('float64')
duplicate(**kwargs)[source]
classmethod empty(mat_spec, *, comm, preallocator=False, **kwargs)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
inc_state()[source]
property insert_mode
property is_nested
property mat
mat_spec
property mat_type
materialize()[source]
maybe_flush_assemble(insert_mode)[source]

Prepare the matrix for a particular insert mode.

maybe_flush_assemble_begin(insert_mode)[source]
maybe_flush_assemble_end(insert_mode)[source]
property name
nest_shape(nest_indices=())[source]
rank_equal = False
record_new(**attrs)

Create and initialise a new record from an existing one.

set_diagonal(value)[source]
property state
zero()[source]
class pyop3.buffer.PetscMatBufferSpec[source]

Bases: Object

A complete specification of how to build a PETSc Mat.

class pyop3.buffer.PetscMatInitBufferSpec[source]

Bases: Object

Abstract class for a ‘partial’ specification of how to build a PETSc Mat.

This class differs from the ‘full’ \(PetscMatBufferSpec\) class where information like the sizes and lgmaps have been determined from the axis trees.

pyop3.buffer.duplicate_mat(mat, copy=False)[source]

Duplicate a PETSc Mat.

This function is temporarily needed because MATNEST matrices do not currently support MatDuplicate.

pyop3.cache module

Provides common base classes for cached objects.

class pyop3.cache.DEFAULT_CACHE[source]

Bases: dict

class pyop3.cache.DictLikeDiskAccess(cachedir, extension='.pickle')[source]

Bases: MutableMapping

A Dictionary like interface for storing and retrieving objects from a disk cache.

__getitem__(key)[source]

Retrieve a value from the disk cache.

open(*args, **kwargs)[source]
read(filehandle)[source]
write(filehandle, value)[source]
class pyop3.cache.WeakHeavyObjectCacheStore[source]

Bases: object

Keys are heavy cache objects, values are caches. If the heavy object dies then the cache must die too.

WeakKeyDictionary doesn’t work because of hashing

obj1 = myfunc() obj2 = myfunc() obj1 == obj2 # True hash(obj1) == hash(obj2) # True obj1 is obj2 # False

weakref.ref(obj1) == weakref.ref(obj2) # True hash(weakref.ref(obj1)) == hash(weakref.ref(obj2)) # True!!!

We need a cache that truly works on identity hashing.

__getitem__(obj)[source]
pyop3.cache.as_hexdigest(*args)[source]

Return args as a hash string.

Notes

This function is relatively expensive to compute so one should avoid calling it wherever possible.

pyop3.cache.cache_filter(comm=None, comm_name=None, alive=False, function=None, cache_type=None)[source]

Filter PyOP2 caches based on communicator, function or cache type.

pyop3.cache.cached_method(key=<function default_hashkey>, make_cache=<class 'dict'>)[source]

TODO

pyop3.cache.cached_method_until(key)[source]

Decorator for a method whose results are cached until some value changes.

This is the counterpart of \(cached_property_until\) but for methods. When a decorated method is called, its results are cached against the set of arguments it was called with. All cached results are discarded when the value of the key changes.

pyop3.cache.cached_on(get_obj, get_key=<function hashkey>, *, multi=False, make_cache=<class 'dict'>, get_version=<function <lambda>>)[source]

Decorator that caches function results on an object.

Parameters:
  • get_obj (Callable) – Function returning the object that holds the cache.

  • get_key (Callable) – Function returning the cache key.

  • multi (bool) – Does get_obj return multiple objects? Default is \(False\).

  • make_cache (Callable) – Function called to create a new cache.

  • get_version (Callable) – Function returning the cache ‘version’. If this changes then previous caches are discarded. Default is to return \(None\).

pyop3.cache.cached_property_until(key)[source]

Decorator for a property that is cached until some value changes.

For example, the expensive_property below will be cached until self.value changes, and will be recomputed with the new self.value and cached when accessed again.

class MyClass:

    def __init__(self, value):
        self.value = value

    @cached_property_until(lambda self: self.value)
    def expensive_property(self):
        # Some expensive computation that depends on self.value
        ...
pyop3.cache.clear_memory_cache(comm)[source]

Completely remove all PyOP2 caches on a given communicator.

pyop3.cache.default_get_comm(*args, **kwargs)[source]

A sensible default comm fetcher for use with \(parallel_cache\).

pyop3.cache.default_hashkey(*args, **kwargs)[source]
pyop3.cache.default_parallel_hashkey(*args, **kwargs)[source]

A sensible default hash key for use with \(parallel_cache\).

pyop3.cache.disk_only_cache(*args, cachedir='/usr/.cache/pyop3', **kwargs)[source]
class pyop3.cache.gc_disabled[source]

Bases: ContextDecorator

Context manager for temporarily disabling the garbage collector.

It may also be used as a function decorator.

pyop3.cache.get_comm_caches(comm)[source]

Return the collection of caches that are stored on a comm.

If a cache stash has not already been created then a new \(dict\) is created and stored.

Parameters:

comm – The communicator to get the caches from.

Returns:

The collection of caches.

Return type:

dict

pyop3.cache.get_method_cache(obj, func, cache_id, make_cache)[source]
class pyop3.cache.heavy_caches(objs)[source]

Bases: object

Context manager that pushes and pops lifetime objects.

For this to be parallel safe, the contract here is that, by using this decorator, you are guaranteeing that all operations within the context manager are at most collective to the level of the the communicator of the lifetime objects.

counter = count(0)
pyop3.cache.memory_and_disk_cache(*args, cachedir='/usr/.cache/pyop3', **kwargs)[source]
pyop3.cache.memory_cache(hashkey=<function default_parallel_hashkey>, get_comm=<function default_get_comm>, make_cache=<function <lambda>>, bcast=False, heavy=False)

Parallel cache decorator.

Parameters:
  • hashkey – Callable taking *args and **kwargs and returning a hash.

  • get_comm – Callable taking *args and **kwargs and returning the appropriate communicator.

  • make_cache – Callable that will build a new cache (if one does not exist). This will be called every time the decorated function is called, and must return an instance of the same type every time it is called.

  • bcast – If \(True\), then generate the new cache value on one rank and broadcast to the others. If \(False\) then values are generated on all ranks. This option can only be \(True\) if the operation can be executed in serial; else it will deadlock.

  • heavy – Do the objects stored in the cache have a large memory footprint? If yes then this cache is only used when a ‘heavy’ cache is set (see the \(heavy_cache\) context manager) and the lifetime of the objects in the cache are tied to the lifetime of the cache.

pyop3.cache.parallel_cache(hashkey=<function default_parallel_hashkey>, get_comm=<function default_get_comm>, make_cache=<function <lambda>>, bcast=False, heavy=False)[source]

Parallel cache decorator.

Parameters:
  • hashkey – Callable taking *args and **kwargs and returning a hash.

  • get_comm – Callable taking *args and **kwargs and returning the appropriate communicator.

  • make_cache – Callable that will build a new cache (if one does not exist). This will be called every time the decorated function is called, and must return an instance of the same type every time it is called.

  • bcast – If \(True\), then generate the new cache value on one rank and broadcast to the others. If \(False\) then values are generated on all ranks. This option can only be \(True\) if the operation can be executed in serial; else it will deadlock.

  • heavy – Do the objects stored in the cache have a large memory footprint? If yes then this cache is only used when a ‘heavy’ cache is set (see the \(heavy_cache\) context manager) and the lifetime of the objects in the cache are tied to the lifetime of the cache.

pyop3.cache.print_cache_stats(*args, **kwargs)[source]

Print cache statistics.

pyop3.cache.serial_cache(hashkey=<function hashkey>, cache_factory=<function <lambda>>)[source]
pyop3.cache.with_heavy_caches(get_obj)[source]

Function decorator that pushes and pops lifetime objects.

pyop3.cache.with_self_heavy_cache(func)

Method decorator that sets self as a heavy cache.

pyop3.cc module

class pyop3.cc.AnonymousCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

Compiler for building a shared library on systems with unknown compiler. The properties of this compiler are entirely controlled through environment variables

class pyop3.cc.Compiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: ABC

A compiler for shared libraries.

Parameters:
  • extra_compiler_flags – A list of arguments to the C compiler (CFLAGS) or the C++ compiler (CXXFLAGS) (optional, prepended to any flags specified as the cflags configuration option). The environment variables PYOP2_CFLAGS and PYOP2_CXXFLAGS can also be used to extend these options.

  • extra_linker_flags – A list of arguments to the linker (LDFLAGS) (optional, prepended to any flags specified as the ldflags configuration option). The environment variable PYOP2_LDFLAGS can also be used to extend these options.

  • version – (Optional) usually sniffed by loader.

  • debug – Whether to use debugging compiler flags.

property bugfix_cflags
property cc
property cflags
property cxx
property cxxflags
property ld
property ldflags
class pyop3.cc.CompilerDiskAccess(cachedir, extension='.pickle')[source]

Bases: DictLikeDiskAccess

open(filename, *args, **kwargs)[source]
read(filename)[source]
setdefault(key, default=None)[source]
write(filename, value)[source]
class pyop3.cc.CompilerOptions(*, include_dirs=(), lib_dirs=(), libs=())[source]

Bases: object

Compiler options associated with a pyop3 object.

include_dirs = ()
lib_dirs = ()
libs = ()
class pyop3.cc.LinuxClangCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

The clang for building a shared library on Linux systems.

class pyop3.cc.LinuxCrayCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

The Cray compiler for building a shared library on Linux systems.

property ldflags
class pyop3.cc.LinuxGnuCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

The GNU compiler for building a shared library on Linux systems.

property bugfix_cflags

Flags to work around bugs in compilers.

class pyop3.cc.LinuxIntelCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

The Intel compiler for building a shared library on Linux systems.

class pyop3.cc.MacClangARMCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: MacClangCompiler

A compiler for building a shared library on ARM based Mac systems.

class pyop3.cc.MacClangCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: Compiler

A compiler for building a shared library on Mac systems.

class pyop3.cc.MacGNUCompiler(extra_compiler_flags=(), extra_linker_flags=(), version=None, debug=False)[source]

Bases: MacClangCompiler

A compiler for building a shared library on Mac systems with a GNU compiler.

pyop3.cc.add_profiling_events(dll, events)[source]

If PyOP2 is in profiling mode, events are attached to dll to profile the local linear algebra calls. The event is generated here in python and then set in the shared library, so that memory is not allocated over and over again in the C kernel. The naming convention is that the event ids are named by the event name prefixed by "ID_".

pyop3.cc.check_source_hashes(compiler, code, extension, comm)[source]

A check to see whether code generated on all ranks is identical.

Parameters:
  • compiler – The compiler to use to create the shared library.

  • code – The code to compile.

  • filename – The filename of the library to create.

  • extension – extension of the source file (c, cpp).

  • comm – Communicator over which to perform compilation.

pyop3.cc.clear_compiler_disk_cache(prompt=False)[source]

Clear the PyOP2 compiler disk cache.

Parameters:

prompt – if True prompt before removing any files

pyop3.cc.expandWl(ldflags)[source]

Generator to expand the \(-Wl\) compiler flags for use as linker flags :arg ldflags: linker flags for a compiler command

pyop3.cc.load(code, extension, cppargs=(), ldargs=(), comm=<mpi4py.MPI.Intracomm object>)[source]

Build a shared library and return a function pointer from it.

Parameters:
  • code – The code to compile.

  • extension – extension of the source file (c, cpp)

  • cppargs – A tuple of arguments to the C compiler (optional)

  • ldargs – A tuple of arguments to the linker (optional)

Keyword Arguments:

comm – Optional communicator to compile the code on (only rank 0 compiles code) (defaults to mpi4py.MPI.COMM_WORLD).

This function is logically collective over MPI ranks, it is an error to call it on fewer than all the ranks in MPI communicator. PYOP3_SPMD_STRICT=1 is in your environment and function calls will be guarded by a barrier where possible.

pyop3.cc.load_hashkey(code, extension, cppargs=(), ldargs=(), comm=None)[source]
pyop3.cc.make_so(compiler, code, extension, comm)[source]

Build a shared library and load it

Parameters:
  • compiler – The compiler to use to create the shared library.

  • code – The code to compile.

  • filename – The filename of the library to create.

  • extension – extension of the source file (c, cpp).

  • comm – Communicator over which to perform compilation.

Returns a ctypes.CDLL object of the resulting shared library.

This function is logically collective over MPI ranks, it is an error to call it on fewer than all the ranks in MPI communicator. PYOP3_SPMD_STRICT=1 is in your environment and function calls will be guarded by a barrier where possible.

pyop3.cc.set_default_compiler(compiler)[source]

Set the PyOP2 default compiler, globally over COMM_WORLD.

Parameters:

compiler – String with name or path to compiler executable OR a subclass of the Compiler class

pyop3.cc.sniff_compiler(exe, comm=<mpi4py.MPI.Intracomm object>)[source]

Obtain the correct compiler class by calling the compiler executable.

Parameters:
  • exe – String with name or path to compiler executable

  • comm – Comm over which we want to determine the compiler type

Returns:

A compiler class

pyop3.cc.sniff_compiler_version(compiler, cpp=False)[source]

Attempt to determine the compiler version number.

Parameters:
  • compiler – Instance of compiler to sniff the version of

  • cpp – If set to True will use the C++ compiler rather than the C compiler to determine the version number.

pyop3.collections module

class pyop3.collections.AbstractOrderedSet[source]

Bases: object

__getitem__(index, /)[source]
index(value, /)[source]
union(*others)[source]
class pyop3.collections.AlwaysEmptyDict[source]

Bases: dict

setdefault(key, default=None, /)[source]
class pyop3.collections.HashableSlice(start, stop, step)[source]

Bases: object

A hashable slice.

This class is needed because in Python <3.12 slices are not hashable.

classmethod from_slice(slice_)[source]
start
step
stop
class pyop3.collections.OrderedFrozenSet(values=(), /)[source]

Bases: AbstractOrderedSet

class pyop3.collections.OrderedSet(values=None, /)[source]

Bases: AbstractOrderedSet

A mutable ordered set.

add(value)[source]
clear()[source]
copy()[source]
count(value)[source]
index(value)[source]
remove(value)[source]
update(*others)[source]
class pyop3.collections.StrictlyUniqueDefaultDict[source]

Bases: defaultdict

class pyop3.collections.StrictlyUniqueDict[source]

Bases: dict

A dictionary where overwriting entries will raise an error.

class pyop3.collections.UniqueList(iterable=(), /)[source]

Bases: list

append(value, /)[source]
extend(other, /)[source]
pyop3.collections.as_tuple(item)[source]
pyop3.collections.is_ordered_mapping(obj)[source]
pyop3.collections.is_ordered_sequence(obj)[source]
pyop3.collections.unique(iterable)[source]

pyop3.config module

class pyop3.config.ConfigOption(typ, default_value, description, from_str=<function ConfigOption.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None)[source]

Bases: object

ConfigOption(typ: ‘Any’, default_value: ‘Any’, description: ‘str’, from_str: ‘Callable’ = <function ConfigOption.<lambda> at 0x70fc926021f0>, default_debug_value: ‘Any’ = <object object at 0x70fcb021f330>, value_getter: ‘Callable | None’ = None, value_setter: ‘Callable | None’ = None)

default_debug_value = <object object>
default_value
description
from_str()
typ
value_getter = None
value_setter = None
class pyop3.config.Pyop3Configuration(**kwargs)[source]

Bases: object

Global configuration options for pyop3.

OPTIONS = immutabledict({'max_static_array_size': ConfigOption(typ=<class 'int'>, default_value=128, description='The maximum size of hard-coded constant arrays.\n\n            Constant arrays that exceed this limit are passed to the kernel as\n            arguments instead.\n\n            ', from_str=<function Pyop3Configuration.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'extra_cflags': ConfigOption(typ=tuple[str, ...], default_value=(), description='Extra flags to be passed to the C compiler.', from_str=<function Pyop3Configuration.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'extra_cxxflags': ConfigOption(typ=tuple[str, ...], default_value=(), description='Extra flags to be passed to the C++ compiler.', from_str=<function Pyop3Configuration.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'extra_ldflags': ConfigOption(typ=tuple[str, ...], default_value=(), description='Extra flags to be passed to the linker.', from_str=<function Pyop3Configuration.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'cache_dir': ConfigOption(typ=<class 'pathlib.Path'>, default_value=PosixPath('/tmp/pyop3-cache-uid0'), description='Location of the generated code (libraries).', from_str=<function ConfigOption.<lambda>>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'node_local_compilation': ConfigOption(typ=<class 'bool'>, default_value=True, description='Compile generated code separately on each node.\n\n            If set it is likely that ``cache_dir`` will have to be set to to a\n            node-local filesystem too.\n\n            ', from_str=<function strtobool>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'log_level': ConfigOption(typ=str | int, default_value='WARNING', description='Level used by the pyop3 logger.', from_str=<function ConfigOption.<lambda>>, default_debug_value='DEBUG', value_getter=None, value_setter=<function _log_level_setter>), 'print_cache_stats': ConfigOption(typ=<class 'bool'>, default_value=False, description='Print cache statistics at the end of the program.', from_str=<function strtobool>, default_debug_value=True, value_getter=None, value_setter=None), 'debug_checks': ConfigOption(typ=<class 'bool'>, default_value=False, description='Enable additional correctness checks.\n\n            This option is enabled in debug mode.\n\n            ', from_str=<function strtobool>, default_debug_value=True, value_getter=None, value_setter=None), 'compiler_use_debug_flags': ConfigOption(typ=<class 'bool'>, default_value=False, description="Pass debugging options (i.e. '-O0' and '-g') to the compiler.\n\n            This option is enabled in debug mode.\n\n            ", from_str=<function strtobool>, default_debug_value=True, value_getter=None, value_setter=None), 'check_src_hashes': ConfigOption(typ=<class 'bool'>, default_value=True, description='Check that generated code is the same on all processes.', from_str=<function strtobool>, default_debug_value=<object object>, value_getter=None, value_setter=None), 'spmd_strict': ConfigOption(typ=<class 'bool'>, default_value=False, description='Turn on additional parallel correctness checks.\n\n            Setting this option will enable barriers for calls marked with @collective\n            and for cache accesses. This adds considerable overhead, but is useful for\n            tracking down deadlocks.\n\n            This option is enabled in debug mode.\n\n            ', from_str=<function strtobool>, default_debug_value=True, value_getter=None, value_setter=None)})
as_dict()[source]
property cache_dir

Location of the generated code (libraries).

property check_src_hashes

Check that generated code is the same on all processes.

property compiler_use_debug_flags

Pass debugging options (i.e. ‘-O0’ and ‘-g’) to the compiler.

This option is enabled in debug mode.

property debug_checks

Enable additional correctness checks.

This option is enabled in debug mode.

property extra_cflags

Extra flags to be passed to the C compiler.

property extra_cxxflags

Extra flags to be passed to the C++ compiler.

property extra_ldflags

Extra flags to be passed to the linker.

property log_level

Level used by the pyop3 logger.

property max_static_array_size

The maximum size of hard-coded constant arrays.

Constant arrays that exceed this limit are passed to the kernel as arguments instead.

property node_local_compilation

Compile generated code separately on each node.

If set it is likely that cache_dir will have to be set to to a node-local filesystem too.

property print_cache_stats

Print cache statistics at the end of the program.

property spmd_strict

Turn on additional parallel correctness checks.

Setting this option will enable barriers for calls marked with @collective and for cache accesses. This adds considerable overhead, but is useful for tracking down deadlocks.

This option is enabled in debug mode.

pyop3.constants module

pyop3.constants.DECIDE = DECIDE

Placeholder indicating that a value should be set by pyop3.

This is important in cases where the more traditional \(None\) is actually meaningful.

class pyop3.constants.DecideType[source]

Bases: object

class pyop3.constants.Intent(*values)[source]

Bases: Enum

INC = 'inc'
MAX_RW = 'max_rw'
MAX_WRITE = 'max_write'
MIN_RW = 'min_rw'
MIN_WRITE = 'min_write'
READ = 'read'
RW = 'rw'
WRITE = 'write'
pyop3.constants.NOTHING = <object object>

Sentinel value indicating nothing should be done.

This is useful in cases where \(None\) holds some meaning.

pyop3.debug module

exception pyop3.debug.TodoWarning[source]

Bases: UserWarning

pyop3.debug.breakpoint_enabled(marker=None)[source]
pyop3.debug.disable_conditional_breakpoints(marker=None)[source]
pyop3.debug.enable_conditional_breakpoints(marker=None)[source]
pyop3.debug.maybe_breakpoint(marker=None)[source]
pyop3.debug.print_if_rank(rank, *args, comm=None)[source]
pyop3.debug.print_with_rank(*args, comm=None)[source]
pyop3.debug.sane_print(array)[source]

Print an array to a fixed precision.

This allows one to compare it by eye without going crazy over irrelevant floating point precision differences.

pyop3.debug.timer(func_name=None)[source]
pyop3.debug.warn_todo(message)[source]

pyop3.device module

class pyop3.device.CPU[source]

Bases: Device

CPU Class, designed to be host object, inheriting Device - Plausible to have multiple CPUs, functionally similar to having GPU

asarray(arr, *, constant=False)[source]

Convert GPU/CuPy/NumPy input array to CPU-compliant NumPy array

name = 'CPU'
zeros_like(arr)[source]
class pyop3.device.CUDAGPU[source]

Bases: Device

GPU class for Nvidia GPUs. inheriting Device. - All offloading will be done through CuPy - Multiple instantiations will be independent of each other

asarray(arr, *, constant=False)[source]
property cp
name = 'CudaGPU'
zeros_like(arr)[source]
pyop3.device.DEVICE_TO_ARRAY_TYPE = {<pyop3.device.CPU object>: <class 'numpy.ndarray'>}

Mapping between device and array type.

When new devices are created they should register themselves here.

Note that this mapping is not invertible. A single array type could map to multiple devices.

class pyop3.device.Device[source]

Bases: ABC

Device - Abstract class - Base for future GPU implementations - All device-specific logic should be kept in here

abstractmethod asarray(arr, *, constant=False)[source]
name
abstractmethod zeros_like(arr)[source]
pyop3.device.flag_constant(array)[source]
pyop3.device.flag_constant(array)

Mark an array as being constant.

pyop3.device.get_current_device()[source]
pyop3.device.offloading(dev)[source]

Context Manager for offloading components to select device This function should be the only way to modfiy the current device variable

Updates current context to the given \(dev\) variable. Former device is stored in stack, to be restored when finished - This also allows for stacking of context windows

Context variables are also async safe.

— Example:

gpu = op3.CUDAGPU()
with op3.offloading(gpu):
    g.dat.assign(23, eager=True, eager_strategy="array")
pyop3.device.on_host(func)[source]

Decorator for components that we want to stay on host device i.e. MPI communications/StarForest

pyop3.dtypes module

class pyop3.dtypes.DTypeLimit(min, max)

Bases: tuple

DTypeLimit(min, max)

max

Alias for field number 1

min

Alias for field number 0

class pyop3.dtypes.OpaqueType(name)[source]

Bases: OpaqueType

pyop3.dtypes.as_cstr(dtype)[source]

Convert a numpy dtype like object to a C type as a string.

pyop3.dtypes.as_ctypes(dtype)[source]

Convert a numpy dtype like object to a ctypes type.

pyop3.dtypes.as_numpy_dtype(dtype)[source]

Convert a dtype-like object into a numpy dtype.

pyop3.dtypes.dtype_limits(dtype)[source]

Attempt to determine the min and max values of a datatype.

Parameters:

dtype – A numpy datatype.

Returns:

a 2-tuple of min, max

Raises:

ValueError – If numeric limits could not be determined.

pyop3.dtypes.get_mpi_dtype(numpy_dtype, cdim=1)[source]

Get an MPI datatype corresponding to a Dat.

This builds (if necessary a contiguous derived datatype of the correct size).

Also returns if it is a builtin type.

pyop3.dtypes.reduction_op(op, invec, inoutvec, datatype)[source]

pyop3.exceptions module

exception pyop3.exceptions.CacheException[source]

Bases: Pyop3Exception

Error during caching.

exception pyop3.exceptions.CommMismatchException[source]

Bases: Pyop3Exception

Exception raised when MPI communicators do not match.

exception pyop3.exceptions.CommNotFoundException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.CompilationException[source]

Bases: Pyop3Exception

Error during compilation.

exception pyop3.exceptions.ContextMismatchException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.EmptyIterableException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.ExpectedLinearAxisTreeException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.ExpressionUnchangedException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.IncompatibleAxisTargetException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.InvalidArrayAccessException[source]

Bases: Pyop3Exception

Exception raised when a data structure cannot be viewed as an array.

exception pyop3.exceptions.InvalidExpressionException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.InvalidIndexCountException[source]

Bases: Pyop3Exception

Exception raised when too few/many indices are used to index an object.

exception pyop3.exceptions.InvalidIndexTargetException[source]

Bases: Pyop3Exception

Exception raised when we try to match index information to a mismatching axis tree.

exception pyop3.exceptions.InvalidIntentException[source]

Bases: Pyop3Exception

Error raised if an argument cannot be accessed using the declared intent.

exception pyop3.exceptions.InvalidMapTargetException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.InvalidScalarTypeException[source]

Bases: Pyop3Exception, TypeError

Exception raised when an operation does not support the scalar type of its data.

exception pyop3.exceptions.LoopContextSensitiveException[source]

Bases: Pyop3Exception

Exception raised when an index is sensitive to the loop index.

exception pyop3.exceptions.MissingVariableException[source]

Bases: Pyop3Exception

Exception raised when information about an axis variable is missing.

exception pyop3.exceptions.MultipleUniqueEntriesException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.NonUnitAxisException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.NonUnitIterableException[source]

Bases: Pyop3Exception

exception pyop3.exceptions.Pyop3Exception[source]

Bases: Exception, ABC

Base class for all pyop3 exceptions.

exception pyop3.exceptions.SizeMismatchException[source]

Bases: Pyop3Exception

Exception raised when the size of an array does not match what is expected.

exception pyop3.exceptions.UnhashableObjectException[source]

Bases: Pyop3Exception, TypeError

exception pyop3.exceptions.UnspecialisedCalledMapException[source]

Bases: Pyop3Exception

Exception raised when an unspecialised map is used in place of a specialised one.

This is important for cases like closure(cell) where the result can be either a set of points, or sets of cells, edges, and vertices. We say that it is ‘unspecialised’ because it cannot be put into an \(IndexTree\) and instead should yield two trees as an \(IndexForest\).

exception pyop3.exceptions.UnsupportedArrayException[source]

Bases: Pyop3Exception, TypeError

exception pyop3.exceptions.ValueMismatchException[source]

Bases: Pyop3Exception

pyop3.labeled_tree module

class pyop3.labeled_tree.AbstractLabeledTreeLike[source]

Bases: Object

abstract property is_linear
exception pyop3.labeled_tree.EmptyTreeException[source]

Bases: Exception

exception pyop3.labeled_tree.InvalidTreeException[source]

Bases: ValueError

class pyop3.labeled_tree.LabeledNodeComponent[source]

Bases: Object, ABC

class pyop3.labeled_tree.LabeledTree[source]

Bases: AbstractLabeledTreeLike

ancestors(node, component_label)[source]

Return the ancestors of a (node_id, component_label) 2-tuple.

property child_to_parent[source]
property depth
find_component(node_label, cpt_label, also_node=False)[source]

Return the first component in the tree matching the given labels.

Notes

This will return the first component matching the labels. Multiple may exist but we assume that they are identical.

classmethod from_iterable(iterable, *args, **kwargs)[source]
classmethod from_nest(nest, *args, **kwargs)[source]
property is_empty
is_leaf(node)[source]
property is_linear
property leaf_node_paths[source]
property leaf_path
property leaf_paths[source]

Return the paths to each leaf of the tree.

property leaves[source]
property node_labels[source]
property nodes[source]
property ordered_leaf_node_paths[source]
parent(node)[source]
property paths[source]

Return all possible paths through the tree.

property root
to_nest()[source]
visited_nodes(path)[source]
class pyop3.labeled_tree.MultiComponentLabeledNode[source]

Bases: Node, Labeled, Object

property component_label
abstract property component_labels
property degree
class pyop3.labeled_tree.MutableLabeledTreeMixin[source]

Bases: object

add_node(path, node)[source]

Return a new tree with node attached at path.

add_subtree(path, subtree)[source]

Attach another tree to a leaf of the current tree.

drop_node(path)[source]
drop_subtree(path, *, allow_empty_subtree=False)[source]
subtree(path)[source]

Return the subtree with path as the root.

class pyop3.labeled_tree.Node[source]

Bases: object

exception pyop3.labeled_tree.NodeNotFoundException[source]

Bases: Exception

exception pyop3.labeled_tree.TreeMutationException[source]

Bases: Pyop3Exception

pyop3.labeled_tree.accumulate_path(path, *, skip_last=False)[source]
pyop3.labeled_tree.as_component_label(component)[source]
pyop3.labeled_tree.as_path(obj)[source]
pyop3.labeled_tree.as_path(path)
pyop3.labeled_tree.as_path(path)
pyop3.labeled_tree.filter_path(orig_path, to_remove)[source]
pyop3.labeled_tree.is_subpath(subpath, full_path)[source]
pyop3.labeled_tree.parent_path(path)[source]
pyop3.labeled_tree.postvisit(tree, fn, path=immutabledict({}), **kwargs)[source]

Traverse the tree in postorder.

Parameters:
  • tree (LabeledTree) – The tree to be visited.

  • fn (collections.abc.Callable) – A function to be applied at each node. The function should take the node to be visited as its first argument, and the results of visiting its children as any further arguments.

pyop3.labeled_tree.previsit(tree, fn, current_node=None, prev=None)[source]

pyop3.log module

The PyOP2 logger, based on the Python standard library logging module.

pyop3.log.log(level, msg, *args, **kwargs)[source]

Print ‘msg % args’ with the severity ‘level’.

Parameters:
  • level – the log level. Valid values: DEBUG, INFO, WARNING, ERROR, CRITICAL

  • msg – the message

pyop3.log.progress(level, msg, *args, **kwargs)[source]

A context manager to print a progress message.

The block is wrapped in msg..., msg...done log messages with an appropriate indent (to distinguish nested message).

Parameters:
  • level – the log level. See log() for valid values

  • msg – the message.

See log() for more details.

pyop3.mpi module

PyOP2 MPI communicator.

pyop3.mpi.decref(comm)[source]

Decrement communicator reference count

pyop3.mpi.incref(comm)[source]

Increment communicator reference count

pyop3.mpi.is_pyop2_comm(comm)[source]

Returns True if comm is a PyOP2 communicator, False if \(comm\) another communicator. Raises exception if comm is not a communicator.

Parameters:

comm – Communicator to query

class pyop3.mpi.temp_internal_comm(comm)[source]

Bases: object

Use a PyOP2 internal communicator and increment and decrement the internal comm. :arg comm: Any communicator

pyop3.node module

class pyop3.node.Node[source]

Bases: Object, ABC

abstract property child_attrs
property children
class pyop3.node.NodeCollector(*, reuse_results=True)[source]

Bases: NodeVisitor, ABC

process(obj, /)[source]
process(tuple_, visited, /)
class pyop3.node.NodeTransformer(*, reuse_results=True)[source]

Bases: NodeVisitor, ABC

reuse_if_untouched(node, visited, **kwargs)[source]

Reuse if untouched.

Parameters:
  • o – Expression to start DAG traversal from.

  • **kwargs – Keyword arguments for the process singledispatchmethod.

Returns:

Processed expression.

class pyop3.node.NodeVisitor(*, reuse_results=True)[source]

Bases: Visitor

children(node, /)[source]
children(node, /)
class pyop3.node.Operator[source]

Bases: Node

A non-terminal.

class pyop3.node.Terminal[source]

Bases: Node, ABC

child_attrs = ()
class pyop3.node.Visitor(*, reuse_results=True)[source]

Bases: ABC

Base class for DAG traversers.

Parameters:
  • compress – If True, result_cache will be used.

  • visited_cache – cache of intermediate results; expr -> r = self.process(expr, …).

  • result_cache – cache of result objects for memory reuse, r -> r.

__call__(node, *args, **kwargs)[source]

Perform memoised DAG traversal with process singledispatch method.

Parameters:
  • node – Expression to start DAG traversal from.

  • *args – Positional arguments for the process singledispatchmethod.

  • **kwargs – keyword arguments for the process singledispatchmethod.

Returns:

Processed Expression.

get_cache_key(node, *args, **kwargs)[source]

Maybe overload this if you want to set some things up

process(o, **kwargs)[source]

Process node by type.

Parameters:
  • o – UFL expression to start DAG traversal from.

  • **kwargs – Keyword arguments for the process singledispatchmethod.

pyop3.node.postorder(method)[source]

Postorder decorator.

It is more natural for users to write a post-order singledispatchmethod whose arguments are (self, o, *processed_operands, **kwargs), while \(DAGTraverser\) expects one whose arguments are (self, o, **kwargs). This decorator takes the former and converts to the latter, processing o.ufl_operands behind the users.

pyop3.obj module

class pyop3.obj.Object[source]

Bases: ABC

Abstract class for all objects that appear in pyop3 operations.

Having a base class for this allows us to have generic traversal operations and set some abstract methods.

collect_buffers(visitor)[source]
property comm[source]

The communicator over which this object is collective.

get_disk_cache_key(renamer)[source]
get_instruction_executor_cache_key(renamer)[source]

pyop3.pyop2_utils module

Common utility classes/functions.

pyop3.pyop2_utils.align(bytes, alignment=16)[source]

Align BYTES to a multiple of ALIGNMENT

pyop3.pyop2_utils.as_tuple(item, type=None, length=None, allow_none=False)[source]
pyop3.pyop2_utils.as_type(obj, typ)[source]

Return obj if it is of dtype typ, otherwise return a copy type-cast to typ.

pyop3.pyop2_utils.flatten(iterable)[source]

Flatten a given nested iterable.

pyop3.pyop2_utils.parse_args(*args, **kwargs)[source]

Return parsed arguments as variables for later use.

ARGS and KWARGS are passed into the parser instantiation. The only recognised options are \(group\) and \(description\).

pyop3.pyop2_utils.parser(description=None, group=False)[source]

Create default argparse.ArgumentParser parser for pyop2 programs.

pyop3.pyop2_utils.strip(code)[source]
pyop3.pyop2_utils.trim(docstring)[source]

Trim a docstring according to PEP 257.

pyop3.pyop2_utils.tuplify(xs)[source]

Turn a data structure into a tuple tree.

pyop3.record module

pyop3.record.attr(attr_name)[source]
pyop3.record.frozenrecord(**kwargs)[source]
pyop3.record.record(**kwargs)[source]

pyop3.sf module

class pyop3.sf.AbstractStarForest[source]

Bases: Object

broadcast(*args)[source]
abstractmethod broadcast_begin(*args)[source]
abstractmethod broadcast_end(*args)[source]
abstract property ileaf
abstractmethod classmethod merge(sfs)[source]
abstract property num_ghost
abstract property num_owned
exception pyop3.sf.BufferSizeMismatchException[source]

Bases: Exception

class pyop3.sf.NullStarForest(size)[source]

Bases: AbstractStarForest

broadcast_begin(*args)[source]
broadcast_end(*args)[source]
property comm
property ileaf
classmethod merge(sfs)[source]
property num_ghost
property num_owned
reduce_begin(*args)[source]
reduce_end(*args)[source]
class pyop3.sf.StarForest(sf, comm)[source]

Bases: AbstractStarForest

Convenience wrapper for a \(petsc4py.SF\).

broadcast(*args)[source]
broadcast_begin(*args)[source]
broadcast_end(*args)[source]
property comm
filter(indices)[source]
classmethod from_graph(size, ilocal, iremote, comm)[source]
property graph
property icore[source]

Return the indices of points that are not roots or leaves.

property ileaf
property ilocal
property iremote
property iroot[source]

Return the indices of roots on the current process.

classmethod merge(sfs)[source]
property nleaves
property num_ghost
property num_owned
record_new(**attrs)

Create and initialise a new record from an existing one.

reduce(*args)[source]
reduce_begin(*args)[source]
reduce_end(*args)[source]
sf
property size
with_section(section)[source]

Create a new star forest via composition with a PETSc section.

pyop3.sf.local_sf(size, comm)[source]
pyop3.sf.single_star_sf(comm, size=np.int32(1), root=0)[source]

Construct a star forest containing a single star.

The single star has leaves on all ranks apart from the “root” rank that point to the same shared data. This is useful for describing globally consistent data structures.

pyop3.types module

pyop3.types.AxisComponentLabelT

Type alias for axis component labels.

pyop3.types.AxisComponentLike = pyop3.axis_tree.tree.AxisComponent | collections.abc.Hashable

Type alias for things that can be parsed into axis components.

pyop3.types.AxisComponentRegionLabelT

Type alias for axis component region labels.

pyop3.types.AxisLabelT

Type alias for axis labels.

pyop3.types.ConcretePathT

Type alias for a tree path.

alias of immutabledict[Hashable, Hashable]

pyop3.types.LabelT

Something that may be used as a label.

pyop3.types.NodeComponentLabelT

Something that may be used as a label for a node component.

pyop3.types.NodeLabelT

Something that may be used as a label for a node.

pyop3.types.PathT

Type alias for things can be parsed into a tree path.

alias of Mapping[Hashable, Hashable]

pyop3.utils module

class pyop3.utils.Atom(item)[source]

Bases: object

Wrapper object indicating that something should not be subdivided.

This class is necessary because the special-casing of tuples in __getitem__ by Python breaks the syntactic sugar we have for slices. For example consider an axis component with (tuple) label ‘(2, 1)’. We would like to be able to take this slice by executing:

dat[(2, 1)]

However, __getitem__ turns this into the very different:

dat[2, 1]

Turning the tuple (2, 1) into an atom resolves this.

item
class pyop3.utils.Labeled[source]

Bases: ABC

classmethod unique_id()[source]
class pyop3.utils.UniqueNameGenerator(existing_names=None, forced_prefix='', forced_suffix='')[source]

Bases: UniqueNameGenerator

Class for generating unique names.

__call__(prefix)[source]
pyop3.utils.apply_at(func, iterable, index)[source]
pyop3.utils.as_dtype(dtype, default)[source]
pyop3.utils.as_numpy_scalar(value)[source]
pyop3.utils.atom(item)[source]
pyop3.utils.ceildiv(a, b, /)[source]
pyop3.utils.debug_assert(predicate, msg=None)[source]
pyop3.utils.deprecated(prefer=None, internal=False)[source]
pyop3.utils.dict_stack(dict_, to_push)[source]
pyop3.utils.expand_collection_of_iterables(compressed)[source]

Expand target paths written in ‘compressed’ form like:

{key1: [item1, item2], key2: [item3]}

Instead to the ‘expanded’ form:

({key1: item1, key2: item3}, {key1: item2, key2: item3})

Valid input types for compressed include ordered mappings and iterables of 2-tuples (i.e. things that can be parsed into a \(dict\)).

pyop3.utils.filter_type(type_, iterable)[source]
pyop3.utils.flatten(iterable)[source]

Recursively flatten a nested iterable.

pyop3.utils.freeze(obj)[source]
pyop3.utils.freeze(tuple_)
pyop3.utils.freeze(slice_)
pyop3.utils.freeze(list_)
pyop3.utils.freeze(immutabledict_)
pyop3.utils.freeze(dict_)
pyop3.utils.freeze(hashable)
pyop3.utils.has_unique_entries(iterable)[source]
pyop3.utils.invert(p)[source]

Return an array s with which np.array_equal(arr[p][s], arr) is True. The array_like argument p must be some permutation of 0, 1, …, len(p)-1.

pyop3.utils.invert_mapping(mapping, *, mapping_type=<class 'dict'>)[source]
pyop3.utils.is_ellipsis_type(obj)[source]
pyop3.utils.is_sequence(item)[source]
pyop3.utils.is_single_valued(iterable)[source]
pyop3.utils.is_sorted(array)[source]

Notes

This function works even for empty arrays, which are reported as being sorted.

pyop3.utils.iterflat(iterable)[source]
pyop3.utils.just_one(iterable)[source]

Return the only entry in an iterable.

Parameters:

iterable – The container with only a single entry.

Returns:

The single item in iterable.

Return type:

Any

Raises:

ValueError – If the iterable does not contain a single entry.

pyop3.utils.map_when(func, when_func, iterable)[source]
pyop3.utils.maybe_generate_name(name, prefix, default_prefix, *, generator=<pyop3.utils.UniqueNameGenerator object>)[source]
pyop3.utils.merge_dicts(dicts)[source]
pyop3.utils.pairwise(iterable, *, final=<object object>)[source]
pyop3.utils.popfirst(dict_)[source]

Remove the first item from a dictionary and return it with its key.

pyop3.utils.popwhen(predicate, iterable)[source]

Pop the first instance from iterable where predicate is True.

pyop3.utils.pretty_type(obj)[source]
pyop3.utils.raise_missing_dispatch_handler(obj)[source]
pyop3.utils.readonly(array)[source]

Return a readonly view of a numpy/cupy array.

pyop3.utils.reduce(func, *args, **kwargs)[source]
pyop3.utils.regexify(pattern)[source]

Convert an expression pattern into a regex pattern.

This is useful for testing.

pyop3.utils.safe_equals(a, b, /)[source]
pyop3.utils.single_valued(iterable)[source]
pyop3.utils.some_but_not_all(iterable)[source]
pyop3.utils.split_at(iterable, index)[source]
pyop3.utils.split_by(condition, items)[source]

Split an iterable in two according to some condition.

Parameters:
  • condition – Callable applied to each item in items, returning True or False.

  • items – Iterable to split apart.

Returns:

A 2-tuple of the form (yess, nos), where yess is a tuple containing the entries of items where condition is True and nos is a tuple of those where condition is False.

pyop3.utils.stack(list_, to_push)[source]
pyop3.utils.steps(sizes, *, drop_last=True, dtype=None)[source]
pyop3.utils.strict_cast(obj, dtype)[source]
pyop3.utils.strict_floordiv(num, fac)[source]
pyop3.utils.strict_int(num)[source]
pyop3.utils.strictly_all(iterable)[source]

Returns all(iterable) but raises an exception if values are inconsistent.

pyop3.utils.strides(sizes, *, drop_last=True)[source]

Examples

# I think… (2, 2) returns (2, 2) - 2i + j (1, 2) returns (2, 1) - 2i + j (2, 1) returns (1, 1) - i + j

pyop3.utils.unique_name(prefix)[source]

Module contents

class pyop3.AbstractLoopIndex[source]

Bases: Object

abstractmethod context_free()[source]
property id
class pyop3.AffineSliceComponent(component, start=0, stop=None, step=1, label=DECIDE)[source]

Bases: SliceComponent

Slice component selecting indices of the form a*i + b.

property component
classmethod from_slice(component, slice_)[source]
property is_full_slice
label
record_new(**attrs)

Create and initialise a new record from an existing one.

start
step
stop
with_size(size=None)[source]
class pyop3.AggregateDat(subdats, axis, *, name=None, prefix=None)[source]

Bases: Object

A dat formed of multiple subdats concatenated together.

DEFAULT_PREFIX = 'aggdat'
assign(other, *, _weakref=True)[source]
property axes[source]
axis
property comm
property dtype
get_instruction_executor_cache_key(visitor)[source]
iassign(other, *, _weakref=True)[source]
materialize()[source]
name
record_new(**attrs)

Create and initialise a new record from an existing one.

property size
subdats
property subtensors
with_context(context)[source]
class pyop3.AggregateMat(submats, row_axis, column_axis, *, name=None, prefix=None)[source]

Bases: Object

A matrix formed of multiple submatrices concatenated together.

DEFAULT_PREFIX = 'aggmat'
assign(other, *, _weakref=True)[source]
property column_axes[source]
column_axis
property comm
property dtype
get_instruction_executor_cache_key(visitor)[source]
iassign(other, *, _weakref=True)[source]
materialize()[source]
name
record_new(**attrs)

Create and initialise a new record from an existing one.

property row_axes[source]
row_axis
submats
property subtensors
with_context(context)[source]
class pyop3.ArrayBuffer(data, sf=None, *, name=None, prefix=None, constant=False, rank_equal=False, ordered=False)[source]

Bases: AbstractArrayBuffer, ConcreteBuffer

A buffer whose underlying data structure is a lazily-evaluated NumPy/CuPy array.

Parameters:

data – Note that the arrays passed here should be unique to this array. Do not use the same array between buffers as it will disrupt the state tracking done by the buffer.

DEFAULT_PREFIX = 'array'
as_vec(mode, block_shape=())[source]
assemble()[source]

Update roots and leaves.

broadcast_roots_to_leaves()[source]

Unconditionally update leaves.

This will overwrite any existing parallel state tracking.

broadcast_roots_to_leaves_begin()[source]

Start unconditionally updating leaves.

broadcast_roots_to_leaves_end()[source]
collect_buffers(visitor)[source]
property comm
property constant
property data_ro
property data_rw
property data_wo

Have to be careful. If not setting all values (i.e. subsets) should call \(reduce_leaves_to_roots\) first.

When this is called we set roots_valid, claiming that any (lazy) ‘in-flight’ writes can be dropped.

property dtype
duplicate(*, copy=False, constant=None, **kwargs)[source]
classmethod empty(shape, dtype=None, **kwargs)[source]
freeze()[source]

Freeze the buffer, turning any modifying accesses into errors.

classmethod from_scalar(value, *, dtype=None, **kwargs)[source]
classmethod full(size, fill_value, dtype=None, **kwargs)[source]
get_array(intent='ro')[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
inc_state()[source]
property is_frozen
is_nested = False
localize()[source]
property name
nest_shape(nest_indices=())[source]
property ordered
property rank_equal
record_new(**attrs)

Create and initialise a new record from an existing one.

reduce_leaves_to_roots(op)[source]

Unconditionally update roots.

This will overwrite any existing parallel state tracking.

Parameters:

op – The MPI reduction operation to apply when pulling leaves onto roots.

reduce_leaves_to_roots_begin(op)[source]

Start unconditionally updating roots.

reduce_leaves_to_roots_end(op)[source]

Finish unconditionally updating roots.

sf
property shape
property state
sync_leaves()[source]

Update leaves.

sync_leaves_begin()[source]

Start updating leaves.

sync_leaves_end()[source]

Finish updating leaves.

sync_roots()[source]

Update roots.

sync_roots_begin()[source]

Start updating roots.

sync_roots_end()[source]

Finish updating roots.

unfreeze()[source]

Unfreeze the buffer.

vec_ro(block_shape=())[source]
vec_rw(block_shape)[source]
vec_wo(block_shape)[source]
zero()[source]
classmethod zeros(shape, dtype=None, **kwargs)[source]
class pyop3.Assignment(assignee, expression, assignment_type)[source]

Bases: AbstractAssignment

Assignment(assignee: ‘Any’, expression: ‘Any’, assignment_type: ‘AssignmentType | str’) -> ‘None’

property assignee
assignment_type
property comm[source]
expression
get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

with_arguments(arguments)[source]
class pyop3.AssignmentType(*values)[source]

Bases: Enum

INC = 'inc'
MAX = 'max'
MIN = 'min'
WRITE = 'write'
class pyop3.Axis(components, label=DECIDE)[source]

Bases: LoopContextFreeAxisTreeLike, MultiComponentLabeledNode

Axis(components, label=DECIDE)

__call__(*args)[source]
__getitem__(indices)[source]
as_tree()[source]

Convert the axis to a tree that contains it.

axes(**kwargs)
property block_shape
buffer_size(**kwargs)[source]
buffer_slice(**kwargs)[source]
collect_buffers(visitor)[source]
property component
component_index(component)[source]
property component_labels
components
property count

Return the total number of entries in the axis across all axis parts. Will fail if axis parts do not have integer counts.

get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property is_linear
property is_nested
iter(**kwargs)[source]
label
linearize(component_label)[source]
property local_size
localize()[source]
matching_component(component_label)[source]
property owned[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

property region_sets[source]
regionless()[source]
property sf[source]
property size
property trees
property unconstrained[source]
property unindexed
with_region_labels(*args, **kwargs)[source]
class pyop3.AxisComponent(regions, label=DECIDE, *, sf=None, size=None)[source]

Bases: LabeledNodeComponent

Parameters:
  • regions (Any) – regions

  • label (NodeComponentLabelT) – label

  • sf (AbstractStarForest | None) – sf

  • size – This is useful if we know a-priori that the region sizes sum to something. For example the number of unconstrained+constrained dofs will always add to 3.

collect_buffers(visitor)[source]
property count
property flat_region_labels
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property has_non_trivial_regions
label
property local_max_size[source]
property local_size[source]
localize()[source]
property rank_equal

Return whether or not this axis component has constant size between ranks.

record_new(**attrs)

Create and initialise a new record from an existing one.

property region_labels
regionless()[source]
regions
sf
property size[source]
class pyop3.AxisComponentRegion(size, label=None)[source]

Bases: Object

AxisComponentRegion(size, label=None) -> ‘None’

collect_buffers(visitor)[source]
property comm[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
label = None
property local_size
record_new(**attrs)

Create and initialise a new record from an existing one.

size
class pyop3.AxisForest(*args, **kwargs)[source]

Bases: LoopContextFreeAxisTreeLike

A collection of equivalent axis trees.

Axis forests are useful to describe circumstances where there are multiple viable axis trees for describing a layout. For instance, one can view the data layout for a function space as a set of DoFs per mesh strata, or as a flat set of nodes. These layouts cannot be transformed between each other and so must coexist.

__getitem__(indices)[source]
property block_shape
blocked(block_shape)[source]
buffer_size(**kwargs)[source]
buffer_slice(**kwargs)[source]
get_instruction_executor_cache_key(visitor)[source]
getitem(indices, *, strict=False)[source]
property global_numbering[source]
property global_size
property is_linear
property is_nested
property local_max_size
property local_size
localize()[source]
materialize()[source]
property nest_labels
property owned
prune()[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

property region_sets[source]
regionless()[source]
restrict_nest(index)[source]
property root
property sf
property size
template_vec(block_shape)[source]
property trees
property unconstrained
property unindexed[source]
with_context(context)[source]
with_region_labels(labels, **kwargs)[source]
class pyop3.AxisTarget(axis, component, expr)[source]

Bases: Object

TODO.

(this is hard to explain)

axis
collect_buffers(visitor)[source]
component
expr
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property path
record_new(**attrs)

Create and initialise a new record from an existing one.

property replace_map
class pyop3.AxisTree(node_map=None, *, comm=None, _validate=True)[source]

Bases: MutableLabeledTreeMixin, AbstractUnindexedAxisTree, AbstractNonUnitAxisTree

Data layout descriptor.

Parameters:
add_axis(path, axis)[source]
append_axis(axis)[source]
blocked(block_shape)[source]
buffer_size(*, include_ghosts)[source]
buffer_slice(*, include_ghosts)[source]
collect_buffers(visitor)[source]
property comm
classmethod from_iterable(iterable, comm=None)[source]
classmethod from_nest(nest, *, comm=None)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property global_numbering[source]
property layout_axes
property layouts[source]
property layouts2
linearize(path, *, partial=False)[source]

Return the axis tree dropping all components not specified in the path.

partial :

If \(True\) then only linearise using a partial path.

localize()[source]
node_map
record_new(**attrs)

Create and initialise a new record from an existing one.

regionless()[source]
section(path, component=<object object>, indices=immutabledict({}))[source]
property sf[source]
property targets[source]
class pyop3.AxisVar(axis)[source]

Bases: TerminalExpression

AxisVar(axis: ‘pyop3.axis_tree.Axis’)

axis
collect_buffers(visitor)[source]
property dtype
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property local_max
property local_min
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.CUDAGPU[source]

Bases: Device

GPU class for Nvidia GPUs. inheriting Device. - All offloading will be done through CuPy - Multiple instantiations will be independent of each other

asarray(arr, *, constant=False)[source]
property cp
name = 'CudaGPU'
zeros_like(arr)[source]
class pyop3.CalledMap(map, index, *, label=DECIDE)[source]

Bases: AbstractCalledMap

A map that has been called against some index.

It represents the object f(i), where f is a \(Map\) and i some other \(Index\).

property axes[source]
index
iter(*, eager=False)[source]
label
map
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Dat(axes, buffer=None, *, data=None, name=None, prefix=None, buffer_kwargs=None, constant=False, transform=None)[source]

Bases: Tensor

Multi-dimensional, hierarchical array.

DEFAULT_PREFIX = 'dat'
__getitem__(indices)[source]
as_array(mode, block_shape=None, *, include_ghosts=False)[source]
as_vec(mode, block_shape=None)[source]
axes
property axis_trees
axpy(alpha, other)[source]

Compute the operation \(y = \alpha x + y\).

In this case, self is y and other is x.

buffer
property comm
concretize(*, linear)[source]

Convert to an expression, can no longer be indexed properly

property context_free
property dat_version
property data_ro_with_halos

Return a read-only view of the data stored by the dat.

This view includes ghost entries.

property data_rw_with_halos

Return a modifiable view of the data stored by the dat.

This view includes ghost entries.

property data_with_halos
property data_wo_with_halos

Return a write-only view of the data stored by the dat.

This view includes ghost entries.

dim = 1
property dtype
duplicate(*, copy=False, constant=None)[source]
classmethod empty(axes, dtype=dtype('float64'), *, buffer_kwargs=immutabledict({}), **kwargs)[source]
classmethod empty_like(dat, **kwargs)[source]
classmethod from_array(array, *, sf=None, comm=<mpi4py.MPI.Intracomm object>, buffer_kwargs=None, constant=False, rank_equal=False, **kwargs)[source]
classmethod from_sequence(sequence, dtype, **kwargs)[source]
classmethod full(axes, fill_value, dtype=dtype('float64'), *, buffer_kwargs=immutabledict({}), **kwargs)[source]
get_instruction_executor_cache_key(visitor)[source]
get_value(indices, path=None, *, loop_exprs=immutabledict({}))[source]
getitem(index, *, strict=False)[source]
property global_data

Return all the data for the Dat gathered onto individual ranks.

This function is logically collective over MPI ranks, it is an error to call it on fewer than all the ranks in MPI communicator. PYOP3_SPMD_STRICT=1 is in your environment and function calls will be guarded by a barrier where possible.

inner(other, /)[source]

Compute the l2 inner product against another dat.

Parameters:

other – The other \(Dat\) to compute the inner product against. Its complex conjugate is taken.

Returns:

The l2 inner product.

Return type:

numpy.number

property local_max
property local_min
localize()[source]
materialize()[source]

Return a new “unindexed” array with the same shape.

property max

Return the global maximum value held by the dat.

maxpy(alphas, dats)[source]

Compute a sequence of axpy operations.

This is equivalent to calling axpy for each pair of scalars and \(Dat\) in the input sequences.

Parameters:
  • alphas – A sequence of scalars.

  • dats – A sequence of \(Dat\).

property min

Return the global minimum value held by the dat.

name
property nbytes

Return an estimate of the size of the data associated with this Dat in bytes. This will be the correct size of the data payload, but does not take into account the (presumably small) overhead of the object and its metadata.

Note that this is the process local memory usage, not the sum over all MPI processes.

property norm

Compute the l2 norm of this \(Dat\).

Note

This acts on the flattened data (see also inner()).

classmethod null(axes, dtype=dtype('float64'), *, buffer_kwargs=immutabledict({}), **kwargs)[source]
null_like(**kwargs)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

reshape(axes, _allow_indexed=False)[source]

Return a reshaped view of the \(Dat\).

TODO

set_value(indices, value, path=None, *, loop_exprs=immutabledict({}))[source]
property sf
property size
property transform
property vec
property vec_ro
property vec_rw
property vec_wo
with_axes(axes)[source]

Return a view of the current \(Dat\) with new axes.

Parameters:

axes – XXX (type?)

Returns:

XXX

Return type:

Dat

with_axis_trees(axis_trees)[source]
classmethod zeros(axes, dtype=dtype('float64'), *, buffer_kwargs=None, **kwargs)[source]
classmethod zeros_like(dat, **kwargs)[source]
class pyop3.DenseBlockPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(comm)[source]
mult(mat, x, y)[source]
property sizes
class pyop3.DenseColumnPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(sf)[source]
mult(mat, x, y)[source]
property sizes
class pyop3.DensePythonMatContext(buffer)[source]

Bases: object

Matrix context for storing narrow and dense (usually Nx1 or 1xN) matrices as PETSc Vecs.

This is important in massively parallel settings where a single dense row would live on a single process and hence be a significant performance bottleneck.

__getitem__(key)[source]
property comm
property data_ro
duplicate(*, copy=False)[source]
getDiagonal(mat, result=None)[source]
abstractmethod mult(mat, x, y)[source]

Set y = self @ x.

multTranspose(mat, x, y)[source]
multTransposeAdd(mat, x, y, z)[source]

z = y + mat^Tx

set_diagonal(value)[source]
zeroEntries(mat)[source]
class pyop3.DenseRowPythonMatContext(buffer)[source]

Bases: DensePythonMatContext

abstractmethod classmethod empty(sf)[source]
mult(mat, x, y)[source]
property sizes
class pyop3.Function(loopy_kernel, access_descrs, *, include_dirs=(), lib_dirs=(), libs=())[source]

Bases: Object

A callable function.

Parameters:
  • loopy_kernel – The loopy kernel that implements the function.

  • access_descrs – The access descriptor of each kernel argument.

  • include_dirs – The include directories that are needed to compile the kernel.

  • lib_dirs – The library directories that are needed to link the kernel.

  • libs – The libraries that are needed to link the kernel.

__call__(*args)[source]
property argspec
code
property comm
property compiler_options
classmethod from_c_string(name, c_code, args, *, preambles=(), **kwargs)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property intents
property name
property num_flops
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Index[source]

Bases: MultiComponentLabeledNode

class pyop3.IndexTree(node_map=None, *, comm=None)[source]

Bases: MutableLabeledTreeMixin, LabeledTree

A tree of indices.

Index trees describe the mapping between axis trees. They are necessary for complex indexing operations like packing multi-field dats (for which simple maps and slices are insufficient).

property comm
classmethod from_iterable(iterable, comm=None)[source]
classmethod from_nest(nest, *, comm=None)[source]
node_map
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.IndexedAxisTree(node_map, unindexed, *, targets, _validate=True)[source]

Bases: AbstractNonUnitAxisTree, AbstractIndexedAxisTree

An axis tree that has been indexed.

Indexed axis trees preserve the original axis tree and the expressions that are needed to address the original tree using indices in the current tree.

blocked(block_shape)[source]

Note: this function assumes that the block shape still exists in the tree.

buffer_size(*, include_ghosts)[source]
buffer_slice(*, include_ghosts)[source]
collect_buffers(visitor)[source]
property comm
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
property global_numbering
property layouts
property layouts2[source]
linearize(path, *, partial=False)[source]

Return the axis tree dropping all components not specified in the path.

localize()[source]
materialize()[source]

Return a new “unindexed” axis tree with the same shape.

node_map
record_new(**attrs)

Create and initialise a new record from an existing one.

regionless()[source]
section(path, component=<object object>, indices=immutabledict({}))[source]
property targets[source]
property unindexed
property unique_markers
class pyop3.Intent(*values)[source]

Bases: Enum

INC = 'inc'
MAX_RW = 'max_rw'
MAX_WRITE = 'max_write'
MIN_RW = 'min_rw'
MIN_WRITE = 'min_write'
READ = 'read'
RW = 'rw'
WRITE = 'write'
class pyop3.LinearDatBufferExpression(buffer_view, layout)[source]

Bases: DatBufferExpression, LinearBufferExpression

A dat with fixed (?) layout.

It cannot be indexed.

This class is useful for describing arrays used in index expressions, at which point it has a fixed set of axes.

buffer_view
child_attrs = ('layout',)
collect_buffers(visitor)[source]
property comm
concretize()[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
layout
property local_max
property local_min
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Loop(*args, **kwargs)[source]

Bases: NonTerminalInstruction

A loop.

child_attrs = ('statements',)
collect_buffers(visitor)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
index
property named_terminal_intents[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

statements
class pyop3.LoopContextFreeLoopIndex(iterset, label=None)[source]

Bases: AbstractLoopIndex, UnitIndex

LoopContextFreeLoopIndex(iterset: ‘AbstractNonUnitAxisTree’, label=None) -> ‘None’

collect_buffers(visitor)[source]
property comm
property component_labels
context_free()[source]
dtype = dtype('int32')
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
is_context_free = True
iterset
label
property leaf_target_paths

Unlike with maps and slices, loop indices are single-component (so return a 1-tuple) but that component can target differently labelled axes (so the tuple entry is an n-tuple).

record_new(**attrs)

Create and initialise a new record from an existing one.

with_context(context, *args)[source]
class pyop3.LoopIndex(iterset, label=None)[source]

Bases: AbstractLoopIndex

LoopIndex(iterset, label=None)

context_free()[source]
get_instruction_executor_cache_key(visitor)[source]
is_context_free()[source]
iterset
label
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Map(connectivity)[source]

Bases: AbstractMap

A map between indices.

Maps are principly defined by their connectivity attribute, which describes the individual mappings between input and output index types. This must be provided as an iterable of mappings because the map can both map from entirely different indices (e.g. multi-component loops that expand to different context-free indices, think looping over mesh.points) and semantically equivalent indices (e.g. a loop over axes[subset].index() has two possible sets of paths and index expressions and the map may map from one or both of these but the result should be the same, think looping over mesh.cells and mesh.points[cell_subset]). Accordingly, the connectivity argument should provide the different indices as different entries in the iterable, and the equivalent indices as different entries in each mapping.

In fact I think to understand the situation we need to consider the following:

closure(mesh.cells.index()) is hard because mesh.cells is an indexed view of mesh.points, and so the loop index carries information on both about. We can feasibly have closure(point) AND closure(cell) being separately valid mappings and we don’t know which we want until we have a target set of axes to make a choice. We therefore want to propagate both for as long as possible. We could similarly imagine a scenario where closure(cell) yields POINTS, not cells, edges and vertices. That is similar in that we get different axis trees that we want to propagate to the end!

With this in mind, connectivity is therefore the map:

{
    input_index_label: [
        [*possible component outputs],
        [*possible component outputs]
    ]
}

for example, closure gives
{
    points: [
        [points],
    ]
    cells: [
        [cells, edges, vertices],
        [points],
    ]
    edges: [
        [edges, vertices],
        [points],
    ]
    ...
}

As a result, indexing things now gives different axis tree, not just different expressions! Indexing therefore must produce an axis forest.

__call__(index, /, **kwargs)[source]
connectivity
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Mat(row_axes, column_axes, buffer, *, name=None, prefix=None, transform=None)[source]

Bases: Tensor

A matrix.

DEFAULT_MAT_INIT_BUFFER_SPEC = MonolithicPetscMatInitBufferSpec(mat_type='aij', block_shape=((), ()))
DEFAULT_PREFIX = 'mat'
as_array(mode)[source]
property axis_trees[source]
buffer
column_axes
property comm
property dim
property dtype
classmethod empty(row_axes, column_axes, *, buffer_spec=None, preallocator=False, buffer_kwargs=immutabledict({}), **kwargs)[source]
classmethod from_sparsity(sparsity, **kwargs)[source]
get_instruction_executor_cache_key(visitor)[source]
getitem(row_index, column_index, *, strict=False)[source]
property handle
property local_max
property local_min
materialize()[source]

Return a new “unindexed” array with the same shape.

name
property ncols[source]

The number of local columns in the matrix (including ghosts).

property nest_indices[source]
property nrows[source]

The number of local rows in the matrix (including ghosts).

classmethod null(row_axes, column_axes, dtype=dtype('float64'), *, buffer_kwargs=immutabledict({}), **kwargs)[source]
null_like(**kwargs)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

reshape(row_axes, column_axes)[source]

Return a reshaped view of the \(Mat\).

TODO

row_axes
property size[source]
classmethod sparsity(row_axes, column_axes, **kwargs)[source]
property transform
property values
with_axes(row_axes, column_axes)[source]

Return a view of the current mat with new axes.

with_axis_trees(axis_trees)[source]
class pyop3.MonolithicPetscMatBufferSpec(mat_type, row_spec, column_spec)[source]

Bases: PetscMatBufferSpec

MonolithicPetscMatBufferSpec(mat_type: ‘str’, row_spec: ‘PetscMatAxisSpec | pyop3.axis_tree.AbstractAxisTree’, column_spec: ‘PetscMatAxisSpec | pyop3.axis_tree.AbstractAxisTree’)

column_spec
property comm
get_instruction_executor_cache_key(visitor)[source]
mat_type
record_new(**attrs)

Create and initialise a new record from an existing one.

row_spec
class pyop3.MonolithicPetscMatInitBufferSpec(mat_type, block_shape=((), ()))[source]

Bases: PetscMatInitBufferSpec

MonolithicPetscMatInitBufferSpec(mat_type: ‘str’, block_shape: ‘tuple[tuple[int, …], tuple[int, …]]’ = ((), ()))

block_shape = ((), ())
mat_type
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.NaN[source]

Bases: TerminalExpression

NaN()

property dtype
classmethod get_custom_comm()[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property local_max
property local_min
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.NestedPetscMatBufferSpec(submat_specs)[source]

Bases: PetscMatBufferSpec

NestedPetscMatBufferSpec(submat_specs: ‘np.ndarray’)

get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

property shape
submat_specs
class pyop3.NestedPetscMatInitBufferSpec(submat_specs)[source]

Bases: PetscMatInitBufferSpec

NestedPetscMatInitBufferSpec(submat_specs: ‘np.ndarray’)

mat_type = 'nest'
record_new(**attrs)

Create and initialise a new record from an existing one.

submat_specs
class pyop3.NullBuffer(shape, dtype=None, *, name=None, prefix=None, ordered=False)[source]

Bases: AbstractArrayBuffer

A buffer that does not carry data.

This is useful for handling temporaries when we generate code. For much of the compilation we want to treat temporaries like ordinary arrays but they are not passed as kernel arguments nor do they have any parallel semantics.

DEFAULT_PREFIX = 'tmp'
collect_buffers(visitor)[source]
property comm
property dtype
duplicate(*, copy=False, constant=None)[source]
get_disk_cache_key(visitor)[source]
instruction_executor_cache_key(buffer_counter)[source]
property name
nest_shape(*args)[source]
property ordered
record_new(**attrs)

Create and initialise a new record from an existing one.

property shape
class pyop3.OpaqueTerminal(buffer_view, *, name=None, prefix=None)[source]

Bases: NamedTerminalExpression

A data object that we don’t know anything about but the local kernel does.

This class is useful for blindly passing arguments into local kernels without doing any packing/unpacking.

DEFAULT_PREFIX = 'opaque'
property buffer
buffer_view
collect_buffers(visitor)[source]
property comm
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
name
nest_indices = ()
record_new(**attrs)

Create and initialise a new record from an existing one.

with_context(ctx)[source]
class pyop3.OutOfPlaceCallableTensorTransform(transform_in, transform_out, prev=None)[source]

Bases: CallableTensorTransform

OutOfPlaceCallableTensorTransform(transform_in: ‘Callable[[Tensor, Tensor], None]’, transform_out: ‘Callable[[Tensor, Tensor], None]’, prev: ‘TensorTransform | None’ = None)

get_instruction_executor_cache_key(visitor)[source]
prev = None
record_new(**attrs)

Create and initialise a new record from an existing one.

transform_in
transform_out
class pyop3.PetscMatBuffer(mat, *, comm, mat_spec=None, name=None, prefix=None, constant=False)[source]

Bases: ConcreteBuffer

A buffer whose underlying data structure is a PETSc Mat.

Parameters:

mat_spec (pyop3.buffer.PetscMatBufferSpec | None) – Only used for preallocation matrices… and actually the only real information is the matrix type, which could be an argument to materialize…

DEFAULT_PREFIX = 'petscmat'
assemble(*, final=True)[source]
assemble_begin(*, final=True)[source]
assemble_end(*, final=True)[source]
collect_buffers(visitor)[source]
property comm
property constant
dtype = dtype('float64')
duplicate(**kwargs)[source]
classmethod empty(mat_spec, *, comm, preallocator=False, **kwargs)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
inc_state()[source]
property insert_mode
property is_nested
property mat
mat_spec
property mat_type
materialize()[source]
maybe_flush_assemble(insert_mode)[source]

Prepare the matrix for a particular insert mode.

maybe_flush_assemble_begin(insert_mode)[source]
maybe_flush_assemble_end(insert_mode)[source]
property name
nest_shape(nest_indices=())[source]
rank_equal = False
record_new(**attrs)

Create and initialise a new record from an existing one.

set_diagonal(value)[source]
property state
zero()[source]
class pyop3.Scalar(value=None, comm=None, *, buffer=None, constant=None, name=None, prefix=None)[source]

Bases: Tensor

A scalar value.

DEFAULT_DTYPE = dtype('float64')
DEFAULT_PREFIX = 'scalar'
as_array(mode)[source]

Return a view of the scalar’s buffer.

Parameters:

mode – The access mode: read-only, write-only, or read-write.

Returns:

A view of the buffer with the requested access mode.

Return type:

numpy.ndarray

axis_trees = ()
buffer
property comm
concretize(*, linear=True)[source]
property constant
copy()[source]
dim = 0
get_instruction_executor_cache_key(visitor)[source]
getitem(*, strict=False)[source]
property local_max
property local_min
loop_axes = immutabledict({})
name
record_new(**attrs)

Create and initialise a new record from an existing one.

shape = (_UnitAxisTree(),)
size = 1
transform = None
property value
with_axis_trees(axis_trees)[source]
class pyop3.ScalarIndex(axis, component, value, label=DECIDE)[source]

Bases: UnitIndex

ScalarIndex(axis, component, value, label=DECIDE) -> ‘None’

axis
component
property component_labels
label
property leaf_target_paths
record_new(**attrs)

Create and initialise a new record from an existing one.

value
class pyop3.ScalarMap(connectivity)[source]

Bases: AbstractMap

An arity 1 map that does not produce an additional axis in the tree.

__call__(index, /, **kwargs)[source]
connectivity

map connectivity. for each input path it can produce multiple equivalent targets (think points vs cells) but never more than one at a time. This differs from other map types where for instance the closure of a cell yields multiple result types.

record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.Slice(axis, components, *, label=DECIDE)[source]

Bases: Index

A slice can be thought of as a map from a smaller space to the target space.

Like maps it can also target multiple outputs. This is useful for multi-component axes.

axis
property component_labels
components
property expanded
label
property leaf_target_paths[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

restrict(paths)[source]
class pyop3.SliceComponent[source]

Bases: LabeledNodeComponent, ABC

abstract property component
abstract property is_full_slice
class pyop3.StarForest(sf, comm)[source]

Bases: AbstractStarForest

Convenience wrapper for a \(petsc4py.SF\).

broadcast(*args)[source]
broadcast_begin(*args)[source]
broadcast_end(*args)[source]
property comm
filter(indices)[source]
classmethod from_graph(size, ilocal, iremote, comm)[source]
property graph
property icore[source]

Return the indices of points that are not roots or leaves.

property ileaf
property ilocal
property iremote
property iroot[source]

Return the indices of roots on the current process.

classmethod merge(sfs)[source]
property nleaves
property num_ghost
property num_owned
record_new(**attrs)

Create and initialise a new record from an existing one.

reduce(*args)[source]
reduce_begin(*args)[source]
reduce_end(*args)[source]
sf
property size
with_section(section)[source]

Create a new star forest via composition with a PETSc section.

class pyop3.SubsetSliceComponent(component, array, *, label=None)[source]

Bases: SliceComponent

A slice component that maps indices using an array.

array
property component
property is_full_slice
label
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.TabulatedMapComponent(target_axis, target_component, array, *, label=DECIDE)[source]

Bases: MapComponent

A map component that maps indices using an array.

arity
array
property data
label
record_new(**attrs)

Create and initialise a new record from an existing one.

target_axis
target_component
class pyop3.Tensor[source]

Bases: TerminalExpression, ABC

DEFAULT_PREFIX = 'array'
__getitem__(indices)[source]
assemble()[source]

Ensure that values are up-to-date.

assign(other, /, mode='write', *, eager=False, eager_strategy=None, compiler_parameters=None, **kwargs)[source]
abstract property axis_trees
property comm
concretize()[source]

Convert to an expression, can no longer be indexed properly

copy(*, constant=None)[source]

Return a copy of the tensor.

Parameters:

constant – Is the copy mutable or not? If \(None\) then default to the const-ness of the original object.

property data
property data_ro

Return a read-only view of the data stored by the dat.

property data_rw

Return a modifiable view of the data stored by the dat.

property data_wo

Return a write-only view of the data stored by the dat.

abstract property dim
property dtype
duplicate(*, copy=False, constant=None)[source]

Return a duplicate of the tensor.

Parameters:
  • copy – Whether to copy values to the new object.

  • constant – Is the duplicate mutable or not? If \(None\) then default to the const-ness of the original object.

abstractmethod getitem(*indices, strict=False)[source]
iassign(other, /, *, eager=False, eager_strategy=None, compiler_parameters=None, **kwargs)[source]
abstract property size
abstract property transform
abstractmethod with_axis_trees(trees)[source]
with_context(context)[source]
zero(**kwargs)[source]
pyop3.as_linear_buffer_expression(obj)[source]
pyop3.atom(item)[source]
pyop3.collect_axis_vars(obj, /)[source]
pyop3.collect_axis_vars(op)
pyop3.collect_axis_vars(var)
pyop3.collect_axis_vars(var)
pyop3.collect_axis_vars(var)
pyop3.collect_axis_vars(var)
pyop3.collect_axis_vars(dat, /)
pyop3.collect_axis_vars(dat, /)
pyop3.evaluate(expr, *, axis_vars=None, loop_indices=None, name_vars=immutabledict({}), allow_missing=False)[source]
pyop3.exscan(*args, eager=False, **kwargs)[source]
pyop3.local_sf(size, comm)[source]
pyop3.loop(*args, eager=False, **kwargs)

Notes

This function has a trailing underscore to avoid clashing with any variables called loop. It is exported as op3.loop.

pyop3.offloading(dev)[source]

Context Manager for offloading components to select device This function should be the only way to modfiy the current device variable

Updates current context to the given \(dev\) variable. Former device is stored in stack, to be restored when finished - This also allows for stacking of context windows

Context variables are also async safe.

— Example:

gpu = op3.CUDAGPU()
with op3.offloading(gpu):
    g.dat.assign(23, eager=True, eager_strategy="array")
pyop3.replace(obj, replace_map, *, assert_modified=False)[source]
pyop3.replace_terminals(obj, /, replace_map, *, assert_modified=False)[source]
pyop3.single_star_sf(comm, size=np.int32(1), root=0)[source]

Construct a star forest containing a single star.

The single star has leaves on all ranks apart from the “root” rank that point to the same shared data. This is useful for describing globally consistent data structures.