Warning

You are reading a version of the website built against the unstable main branch. This content is liable to change without notice and may be inappropriate for your use case. You can find the documentation for the current stable release here.

pyop3.insn package

Submodules

pyop3.insn.base module

class pyop3.insn.base.AbstractAssignment[source]

Bases: AbstractAssignmentLike

property argument_intents[source]
class pyop3.insn.base.AbstractAssignmentLike[source]

Bases: TerminalInstruction

property arguments
property compiler_options[source]
property shape[source]
class pyop3.insn.base.AbstractCalledFunction[source]

Bases: NonEmptyTerminal

property argspec
property argument_intents[source]
property argument_shapes
axis_trees = (_UnitAxisTree(),)
property compiler_options[source]
property function_arguments[source]
property name
shape = ()
with_arguments(arguments)[source]
class pyop3.insn.base.ArgumentSpec(intent, dtype)[source]

Bases: object

ArgumentSpec(intent: ‘Intent’, dtype: ‘np.dtype’)

dtype
intent
class pyop3.insn.base.ArrayAccessType(*values)[source]

Bases: StrEnum

INC = 'inc'
MAX = 'max'
MIN = 'min'
READ = 'read'
WRITE = 'write'
class pyop3.insn.base.Assignment(assignee, expression, assignment_type)[source]

Bases: AbstractAssignment

Assignment(assignee: ‘Any’, expression: ‘Any’, assignment_type: ‘AssignmentType | str’) -> ‘None’

property assignee
assignment_type
property comm[source]
expression
get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

with_arguments(arguments)[source]
class pyop3.insn.base.AssignmentType(*values)[source]

Bases: Enum

INC = 'inc'
MAX = 'max'
MIN = 'min'
WRITE = 'write'
class pyop3.insn.base.CalledFunction(function, arguments)[source]

Bases: AbstractCalledFunction

CalledFunction(function: ‘Function’, arguments: ‘Iterable’) -> ‘None’

function
get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.insn.base.Exscan(_assignee, expression, scan_type, scan_axis, _comm)[source]

Bases: AbstractAssignmentLike

Exscan(_assignee: ‘Any’, expression: ‘ExpressionT’, scan_type: ‘Any’, scan_axis: ‘pyop3.Axis’, _comm: ‘MPI.Comm’)

property argument_intents[source]
property arguments
property assignee
collect_buffers(visitor)[source]
property comm
property compiler_options[source]
expression
property extent[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

scan_axis
scan_type
with_arguments(arguments)[source]
class pyop3.insn.base.Function(loopy_kernel, access_descrs, *, include_dirs=(), lib_dirs=(), libs=())[source]

Bases: Object

A callable function.

Parameters:
  • loopy_kernel – The loopy kernel that implements the function.

  • access_descrs – The access descriptor of each kernel argument.

  • include_dirs – The include directories that are needed to compile the kernel.

  • lib_dirs – The library directories that are needed to link the kernel.

  • libs – The libraries that are needed to link the kernel.

__call__(*args)[source]
property argspec
code
property comm
property compiler_options
classmethod from_c_string(name, c_code, args, *, preambles=(), **kwargs)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
property intents
property name
property num_flops
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.insn.base.FunctionArgument[source]

Bases: ABC

Abstract class for types that may be passed to functions.

class pyop3.insn.base.Instruction[source]

Bases: Node, ABC

__call__(*, compiler_parameters=None, **kwargs)[source]
abstract property named_terminal_intents

Mapping from named terminals to their intents.

For example consider the assignment

dat1[map1] <- 2*dat2[map2] + dat3[map3]

There are:
  • 2 arguments: dat1 and ‘2*dat2 + dat3’

  • 3 named arguments: dat1, dat2 and dat3

  • 6 buffers: dat1, dat2, dat3, map1, map2, map3

property named_terminals
class pyop3.insn.base.InstructionList(*args, **kwargs)[source]

Bases: NonTerminalInstruction

A list of instructions.

child_attrs = ('instructions',)
collect_buffers(visitor)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)
instructions
property named_terminal_intents[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

exception pyop3.insn.base.IntentMismatchError[source]

Bases: Exception

class pyop3.insn.base.KernelArgument[source]

Bases: ABC

Abstract class for types that may be passed as arguments to kernels.

Note that some types that can be passed to functions are not in fact kernel arguments. This is because they either wrap actual kernel arguments (e.g. \(Dat\)), or because no argument is actually passed (e.g. a temporary).

class pyop3.insn.base.Loop(*args, **kwargs)[source]

Bases: NonTerminalInstruction

A loop.

child_attrs = ('statements',)
collect_buffers(visitor)[source]
get_disk_cache_key(visitor)[source]
get_instruction_executor_cache_key(visitor)[source]
index
property named_terminal_intents[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

statements
class pyop3.insn.base.NonEmptyArrayAssignment(assignee, expression, axis_trees, assignment_type, *, comm)[source]

Bases: AbstractAssignment, NonEmptyTerminal

Assignment type.

property assignee
assignment_type
property axis_trees
collect_buffers(visitor)[source]
property comm
expression
get_disk_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

with_arguments(arguments)[source]
class pyop3.insn.base.NonEmptyTerminal[source]

Bases: TerminalInstruction

abstract property axis_trees
abstract property shape
class pyop3.insn.base.NonTerminalInstruction[source]

Bases: Instruction, Operator

class pyop3.insn.base.NullInstruction[source]

Bases: TerminalInstruction

An instruction that does nothing.

arguments = ()
collect_buffers(visitor)[source]
shape = ()
class pyop3.insn.base.StandaloneCalledFunction(function, arguments)[source]

Bases: AbstractCalledFunction

A called function whose arguments do not need packing/unpacking.

collect_buffers(visitor)[source]
function
get_disk_cache_key(visitor)[source]
record_new(**attrs)

Create and initialise a new record from an existing one.

class pyop3.insn.base.TerminalInstruction[source]

Bases: Instruction, Terminal, ABC

abstract property argument_intents
property arguments
abstract property compiler_options

Extra options needed to compile this terminal.

property named_terminal_intents[source]
abstract property with_arguments
exception pyop3.insn.base.UnprocessedExpressionException[source]

Bases: Pyop3Exception

Exception raised when pyop3 expected a preprocessed expression.

pyop3.insn.base.assignment_type_as_intent(assignment_type)[source]
pyop3.insn.base.enlist(insn)[source]
pyop3.insn.base.exscan(*args, eager=False, **kwargs)[source]
pyop3.insn.base.filter_null(iterable)[source]
pyop3.insn.base.fix_intents(tunit, accesses)[source]

The local kernel has underspecified accessors (is_input, is_output). Here coerce them to match the access descriptors provided.

This should arguably be done properly in TSFC.

Note that even if this isn’t done in TSFC we need to guard against this properly as the default error is very unclear.

pyop3.insn.base.loop_(*args, eager=False, **kwargs)[source]

Notes

This function has a trailing underscore to avoid clashing with any variables called loop. It is exported as op3.loop.

pyop3.insn.base.maybe_enlist(instructions)[source]
pyop3.insn.base.non_null(instruction)[source]

pyop3.insn.exec module

Coordinate the execution of instructions.

class pyop3.insn.exec.CompiledCodeExecutor(executable, kernel_name_to_buffer_info, buffer_intents, comm)[source]

Bases: object

Class that executes compiled code.

Parameters:
  • executable – The compiled operation.

  • kernel_name_to_buffer_info – Mapping from local kernel argument names to the buffer and nest indices.

  • buffer_intents – Mapping from buffers to their intents.

Notes

This class has a large number of cached properties to reduce overhead when it is called.

This class is basically executable+buffers. It is useful because we want to cache the executable globally but we don’t want to cache this globally because the buffers are likely to change.

__call__(new_buffers)[source]

Notes

This code is performance critical.

class pyop3.insn.exec.CompilerParameters(*, compress_indirection_maps=False, interleave_comp_comm=False, add_likwid_markers=False, add_petsc_event=False, attach_debugger=False, backend='loopy', propagate_negatives=False, mask_array_accesses=False)[source]

Bases: object

Parameters processed by the pyop3 compiler.

add_likwid_markers = False
add_petsc_event = False
attach_debugger = False
backend = 'loopy'

Option to select ‘loopy’ or ‘mlir’ as code generation backends.

compress_indirection_maps = False
interleave_comp_comm = False
mask_array_accesses = False

Whether to check for and skip expressions like ‘dat[-1]’.

propagate_negatives = False

Whether to propagate negative values in indirections.

class pyop3.insn.exec.Executable(code, comm, *, extra_compiler_options=CompilerOptions(include_dirs=(), lib_dirs=(), libs=()), petsc_events=())[source]

Bases: object

A callable function.

Parameters:

Notes

This class is intentionally distinct from \(CompiledCodeExecutor\) because the executable may be reused by multiple executors (for instance if the buffers are changed) and we want to reuse the work needed to generate the function pointer.

__call__(*args)[source]
code
comm
extra_compiler_options = CompilerOptions(include_dirs=(), lib_dirs=(), libs=())
petsc_events = ()
class pyop3.insn.exec.InstructionExecutionContext(root_insn, compiler_parameters)[source]

Bases: object

Class that coordinates the compilation and execution of an instruction.

__call__(**kwargs)[source]
property comm
compile()[source]
property disk_cache_key[source]

Key used to write the operation to disk.

The returned key should be consistent across ranks and not include overly specific information such as buffer names or array values.

property named_terminal_buffer_intents[source]
preprocess()[source]
property preprocessed_buffers[source]

Data structures that are arguments to the compiled code.

property root_insn
pyop3.insn.exec.META_COMPILER_PARAMETERS = immutabledict({'optimize': {'compress_indirection_maps': True}})

‘Meta’ compiler parameters that set multiple options at once.

class pyop3.insn.exec.ParsedCompilerParameters(*, compress_indirection_maps=False, interleave_comp_comm=False, add_likwid_markers=False, add_petsc_event=False, attach_debugger=False, backend='loopy', propagate_negatives=False, mask_array_accesses=False)[source]

Bases: CompilerParameters

pyop3.insn.exec.cast_loopy_arg_to_ctypes_type(obj)[source]
pyop3.insn.exec.cast_loopy_arg_to_ctypes_type(arg)
pyop3.insn.exec.cast_loopy_arg_to_ctypes_type(arg)
pyop3.insn.exec.parse_compiler_parameters(compiler_parameters)[source]

The process of parsing compiler_parameters is as follows:

  1. Begin with the default options (\(DEFAULT_COMPILER_PARAMETERS\)).

  2. In the order specified in compiler_parameters, parse any ‘macro’ options and tweak the parameters as appropriate.

  3. Lastly, any non-macro options are added.

By setting macro options before individual options the user can make more specific overrides.

pyop3.insn.visitors module

class pyop3.insn.visitors.CompilerOptionsCollector(*, reuse_results=True)[source]

Bases: NodeVisitor

process(obj, /, *args, **kwargs)[source]
process(insn, /, visited)
process(insn)
process(insn, /, visited)
class pyop3.insn.visitors.InstructionTransformer(*, reuse_results=True)[source]

Bases: NodeTransformer

process(insn, /, **kwargs)[source]
process(insn_list, /, *insns, **kwargs)
class pyop3.insn.visitors.LiteralInserter(*, reuse_results=True)[source]

Bases: NodeTransformer

process(obj)[source]
process(insn)
process(insn)
process(insn)
process(insn)
process(insn)
process(assignment, /)
class pyop3.insn.visitors.LoopContextExpander(*, reuse_results=True)[source]

Bases: InstructionTransformer

process(insn, /, **kwargs)[source]
process(loop, /, *, loop_context)
process(func, /, *, loop_context)
process(assignment, /, *, loop_context)
process(insn, /, **kwargs)
pyop3.insn.visitors.collect_compiler_options(insn)[source]
pyop3.insn.visitors.expand_loop_contexts(insn, /)[source]
pyop3.insn.visitors.expand_transforms(obj, /)[source]
pyop3.insn.visitors.expand_transforms(insn_list, /)
pyop3.insn.visitors.expand_transforms(loop, /)
pyop3.insn.visitors.expand_transforms(func, /)
pyop3.insn.visitors.expand_transforms(func, /)
pyop3.insn.visitors.expand_transforms(func, /)
pyop3.insn.visitors.expand_transforms(called_func, /)
pyop3.insn.visitors.expand_transforms(assignment, /)
pyop3.insn.visitors.has_materialized_temporaries(tensor)[source]
pyop3.insn.visitors.insert_literals(insn)[source]

Module contents