Skip to content

Broadcasting

Broadcasting refers to how TML binary operators and assignments behave when their tensor operands differ in number of dimensions or lenght per dimension. TML automatically broadcasts tensors in binary operations, if possible. If broadcasting is not possible due to incompatible dimensions, an error is raised. If dimensions are the same, broadcasting is not performed. TML has different broadcasting rules for logical, relational and arithmetic operations.

Info

Broadcasting rules are inspired by Octave Broadcasting and Numpy Broadcasting.

The difference between TML and Octave/Numpy is support for nested tensor types. For nested types, broadcasting is applied recursively, as 3x3x3 and 3x(3x3) objects are not semantically equal.

Logical, relational and bitwise operations

For broadcasting in binary operation A <op> B to be possible, following conditions must hold:

  1. One operand is tensor and other operand is scalar, or both operands are tensors of the same nesting level and same dimensions on all nesting levels
  2. Scalar/root scalar type must be compatible
    • For relational operations, scalar type must be numeric
    • For logical operations, scalar type must be bool
    • For bitwise operations, scalar type must be uint

Result of broadcasting is a tensor with the same nesting level and same dimensions on all nesting levels as the operands, with bool as root scalar type.

These rules differ from broadcasting rules used in arithmetic operations because adding elements implicitly when doing comparisons or bitwise operations is counterintuitive and provides no benefit.

Following example shows result of broadcasting on a scalar and a tensor:

a = 1

b = [1, 2, 3]
c = [b, b, b]

d = a > b # 1 > [1, 2, 3]
          # [1, 1, 1] > [1, 2, 3]
          # [false, false, false]

d = a > c # 1 > [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
          # [[1, 1, 1], [1, 1, 1], [1, 1, 1]] > [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
          # [[false, false, false], [false, false, false], [false, false, false]]

Following example shows result of broadcasting on two compatible tensors:

a = [1, 2, 3]
b = [4, 5, -1]

c = [a, a, a]
d = [b, b, b]

x = a > b # [1, 2, 3] > [4, 5, -1]
          # [false, false, true]

y = c > d # [[1, 2, 3], [1, 2, 3], [1, 2, 3]] > [[4, 5, -1], [4, 5, -1], [4, 5, -1]]
          # [[false, false, true], [false, false, true], [false, false, true]]

Arithmetic operations

For broadcasting in binary operation A <op> B to be possible, following conditions must hold:

  1. Length of corresponding tensor dimensions must be equal, or one of the corresponding dimensions must be 1
  2. Base type of tensors must be compatible
    • For basic types, type inference must be possible
    • For tensor types, broadcasting must be possible

Broadcasting is performed following these rules:

  1. If number of dimensions is not the same, dimensions with length of 1 are added to the left
  2. Dimensions with length of 1 are broadcast to the greater length
  3. Base type is inferred to a compatible type
    • For basic types, type inference is applied
    • For tensor types, broadcasting is applied

Broadcasting and tensor type order

Tensors can be compatible in binary operations even if they do not meet rules for assignment compatibility. Resulting tensor C maintains the propery that type(C) <= type(A) and type(C) <= type(B)

Following example shows result of broadcasting on two compatible tensors:

a = [1, 2, 3] # with broadcasting applied:
              # a = [
              #     1, 2, 3;
              #     1, 2, 3; <- added
              #     1, 2, 3  <- added
              # ]

b = [         # with broadcasting applied:
    1;        # b = [
    2;        #     1, 1, 1;
    3         #     2, 2, 2;
]             #     3, 3, 3
              #        ^  ^
              #        |  |
              #        |  --added
              #        -----added
              # ]

c = a + b     # c = [
              #     2, 3, 4;
              #     3, 4, 5;
              #     4, 5, 6
              # ]

Following example shows two incompatible tensors:

a = [
    1, 2;
    1, 2
]

b = [
    1, 2, 3;
    1, 2, 3
]

c = a + b # Error: Broadcasting is not possible!

Assignments

Rules for broadcasting in assignments are similar to the rules for broadcasting in binary operations, with the difference that it must be possible to broadcast RHS type to LHS type, without affecting LHS type. This means that, in constrast to binary opearations, no new type is produced.

For broadcasting in assignment A = B to be possible, following conditions must hold:

  1. Length of corresponding A and B tensor dimensions must be equal, or length of corresponding dimension of B must be 1
  2. B must have the same or smaller number of dimensions than A
  3. Base type of A must be the same or greater than B (i.e. base_type(B) <= base_type(A))
    • For basic types, type rules for assignment must be met (i.e. scalar type B must be smaller than scalar type A)
    • For tensor types, broadcasting rules for assignment must be met (ie. tensor type B must be smaller than tensor type A)

Broadcasting is performed following these rules:

  1. If number of dimensions is not the same, dimensions with length of 1 are added to the left
  2. Dimensions with length of 1 are broadcast to the greater length
  3. Base type is inferred to a compatible type
    • For basic types, type inference is applied
    • For tensor types, assignment broadcasting is applied

Following example shows result of broadcasting on two compatible tensors in an assignment:

a = [0, 0, 0]  # type(a) = tensor<int, 3>

a = 1          # broadcasts to:
               # a = [1, 1, 1]

b = [          # type(b) = tensor<int, 3, 3>
    0, 0, 0;
    0, 0, 0;
    0, 0, 0
]

b = [1, 1, 1] # broadcasts to:
              # b = [
              #     1, 1, 1;
              #     1, 1, 1; <- added
              #     1, 1, 1  <- added
              # ]

Following example shows two incompatible tensors in assignment:

tensor<int, 3> a = 0 # a = [0, 0, 0]

a = [1, 3] # Error: Assignment not possible!
           # a has length of 3, and array with length of 2
           # cannot be broadcast

Index access broadcasting (IAB)

TML describes numerical data using two main data types: scalar and tensor types. To support operations that mix scalars and tensors, or tensors of different dimenisons, TML supports broadcasting. Broadcasting rules are explained in detail in the section Broadcasting.

Index access broadcasting rules are based on the fact that scalars and dimensions with the length of 1 are always copied and extended to match dimension of the wider tensor. IAB supports both scalars and tensors with the dimension of 1.

While broadcasting is applied in expressions without any user interaction, IAB enables users to manually index values to the length that would be allowed to be indexed through automatic broadcasting. In practice, only scalars and tensor dimensions with the length of 1 are allowed to be infinitely broadcast, so only such values are allowed to be accessed using IAB.

IAB does not affect type of the expression. Type returned by IAB expressions is always the same as the type that would be returned by ordinary indexing. The only difference is that index expressions are ignored, and same value is allowed to be effectively accessed over infinite length. The only exception to this is handling of scalars, which returns the same value regardless of the index expression nesting depth.

tensor<int, 1> a = 0
b = a[5] # same as a[0], type of the expression is int

tensor<int, 5, 1, 1> c = 0
d = a[4, 5, 5] # same as a[4, 0, 0], type of the expression is int

int e = 0
f = e[0][1, 2] # same as e, type of the expression is int

IAB Rules for Scalars

  • Scalars allow indexing over an arbitrary number of dimensions indefinitely.
  • IAB applies to both lvalues and rvalues.
  • For lvalue indexing, index must be statically proven to be of the value of 0. This prevents writing generic loop logic that iterates over the wrong dimension.
  • If scalar is extracted from ordinary tensor indexing, ordinary scalar IAB rules are applied to the extracted value (i.e. IAB expressions are fully recursive)
  • Conceptually, scalar value a is broadcast across infinite dimensions. In practice, IAB is implemented by ignoring index expressions.
a == 42
a[0] == 42
a[500] == 42
a[4][5] == 42

b == [42]
b[0] == 42
b[0][5] == 42

a = 1

b = a[1]       # 1
c = a[1][2][3] # 1

a[1] = 2       # a = 2
a[1][2][3] = 3 # a = 3

IAB Rules for Tensors

  • Tensors allow indexing over dimensions with the length of 1 indefinitely.
  • IAB applies to both lvalues and rvalues.
  • Unlike scalar IAB, values of indices in lvalue indexing are not statically analyzed. For TML use-cases, it is considered acceptable that users must pay attention to tensor dimension ranges in loops, as IAB is in generic logic mostly used for scalars.
  • Conceptually, values alongside dimensions with the length of one are broadcast infinitely. In practice, IAB is implemented by ignoring index expressions. This means that while out-of-bounds access for non-one dimensions produces undefined behavior, index access for dimensions with the lenght of one always returns the same value.

Strings and IAB

Since strings are used to pass configuration data within the model scope, they follow these IAB rules:

  • The str type is treated as a tensor, meaning indexing a str returns a char.
  • A char is treated as a scalar, meaning infinite indexing is allowed.
a == "test"
a[0] == "t"
a[0][5] == "t"

b == "t"
b[0] == "t"
b[0][5] == "t"

Usual component patterns

Examples of index access broadcasting usage in practice:

fn test():
    # signs has values such as "-" and "+-+"
    for i=0:signs.len:
        # if signs == "+", signs is a char and IAB is applied
        # if signs == "+-+", signs is a char and normal indexing is applied
        if signs[i] == "+":
            # if lenght of signs does not correspond to x being a scalar,
            # error is reported
            x[i] += y[i]
        end
    end
end

Narrowcasting

When working with tensors, it is common to have tensors with some (or even all) dimensions equal to 1. One common type of this problem is having a single input terminal in input terminal group. Those dimensions are effectively useless and can make tensor usage harder in code. To overcome this problem, TML has support for a mechanism opposite to broadcasting that is called narrowcasting.

Narrowcasting is performed only on the top level type and is not performed recursively. Narrowcasting expressions can be nested arbitrarily. In contrast to broadcasting, narrowcasting never produces an error.

Narrowcasting is performed following these rules:

  1. If a scalar is passed, type of the result is unchanged (e.g. int stays int)
  2. If a tensor is passed, dimensions equal to 1 are removed (e.g. tensor<T, 1, 5, 1> becomes tensor<T, 5>)
  3. If a tensor is passed, and no dimensions are equal to 1, type of the result is unchanged (e.g. tensor<T, 5> stays tensor<T, 5>)
  4. If a tensor is passed, and all dimensions are equal to 1, type of the result is tensor base type (e.g. tensor<T, 1, 1, 1> becomes T)

Narrowcasting syntax and examples are shown below.

int a = 1
b = narrow(b) # int

tensor<int, 1> c = 1
d = narrow(c) # int

tensor<int, 1, 5, 1> e = 1
f = narrow(e) # tensor<int, 5>

tensor<int, 5, 5> g = 1
h = narrow(g) # tensor<int, 5, 5>

tensor<tensor<int, 1>, 5, 5> i = 1
j = narrow(i) # tensor<tensor<int, 1>, 5, 5>