Skip to content

Broadcasting

Broadcasting refers to how TML binary operators and assignments behave when their tensor operands differ in number of dimensions or lenght per dimension. TML automatically broadcasts tensors in binary operations, if possible. If broadcasting is not possible due to incompatible dimensions, an error is raised. If dimensions are the same, broadcasting is not performed. TML has different broadcasting rules for logical, relational and arithmetic operations.

Info

Broadcasting rules are inspired by Octave Broadcasting and Numpy Broadcasting.

The difference between TML and Octave/Numpy is support for nested tensor types. For nested types, broadcasting is applied recursively, as 3x3x3 and 3x(3x3) objects are not semantically equal.

Logical, relational and bitwise operations

For broadcasting in binary operation A <op> B to be possible, following conditions must hold:

  1. One operand is tensor and other operand is scalar, or both operands are tensors of the same nesting level and same dimensions on all nesting levels
  2. Scalar/root scalar type must be compatible
    • For relational operations, scalar type must be numeric
    • For logical operations, scalar type must be bool
    • For bitwise operations, scalar type must be uint

Result of broadcasting is a tensor with the same nesting level and same dimensions on all nesting levels as the operands, with bool as root scalar type.

These rules differ from broadcasting rules used in arithmetic operations because adding elements implicitly when doing comparisons or bitwise operations is counterintuitive and provides no benefit.

Following example shows result of broadcasting on a scalar and a tensor:

a = 1

b = [1, 2, 3]
c = [b, b, b]

d = a > b # 1 > [1, 2, 3]
          # [1, 1, 1] > [1, 2, 3]
          # [false, false, false]

d = a > c # 1 > [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
          # [[1, 1, 1], [1, 1, 1], [1, 1, 1]] > [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
          # [[false, false, false], [false, false, false], [false, false, false]]

Following example shows result of broadcasting on two compatible tensors:

a = [1, 2, 3]
b = [4, 5, -1]

c = [a, a, a]
d = [b, b, b]

x = a > b # [1, 2, 3] > [4, 5, -1]
          # [false, false, true]

y = c > d # [[1, 2, 3], [1, 2, 3], [1, 2, 3]] > [[4, 5, -1], [4, 5, -1], [4, 5, -1]]
          # [[false, false, true], [false, false, true], [false, false, true]]

Arithmetic operations

For broadcasting in binary operation A <op> B to be possible, following conditions must hold:

  1. Length of corresponding tensor dimensions must be equal, or one of the corresponding dimensions must be 1
  2. Base type of tensors must be compatible
    • For basic types, type inference must be possible
    • For tensor types, broadcasting must be possible

Broadcasting is performed following these rules:

  1. If number of dimensions is not the same, dimensions with length of 1 are added to the left
  2. Dimensions with length of 1 are broadcast to the greater length
  3. Base type is inferred to a compatible type
    • For basic types, type inference is applied
    • For tensor types, broadcasting is applied

Broadcasting and tensor type order

Tensors can be compatible in binary operations even if they do not meet rules for assignment compatibility. Resulting tensor C maintains the propery that type(C) <= type(A) and type(C) <= type(B)

Following example shows result of broadcasting on two compatible tensors:

a = [1, 2, 3] # with broadcasting applied:
              # a = [
              #     1, 2, 3;
              #     1, 2, 3; <- added
              #     1, 2, 3  <- added
              # ]

b = [         # with broadcasting applied:
    1;        # b = [
    2;        #     1, 1, 1;
    3         #     2, 2, 2;
]             #     3, 3, 3
              #        ^  ^
              #        |  |
              #        |  --added
              #        -----added
              # ]

c = a + b     # c = [
              #     2, 3, 4;
              #     3, 4, 5;
              #     4, 5, 6
              # ]

Following example shows two incompatible tensors:

a = [
    1, 2;
    1, 2
]

b = [
    1, 2, 3;
    1, 2, 3
]

c = a + b # Error: Broadcasting is not possible!

Assignments

Rules for broadcasting in assignments are similar to the rules for broadcasting in binary operations, with the difference that it must be possible to broadcast RHS type to LHS type, without affecting LHS type. This means that, in constrast to binary opearations, no new type is produced.

For broadcasting in assignment A = B to be possible, following conditions must hold:

  1. Length of corresponding A and B tensor dimensions must be equal, or length of corresponding dimension of B must be 1
  2. B must have the same or smaller number of dimensions than A
  3. Base type of A must be the same or greater than B (i.e. base_type(B) <= base_type(A))
    • For basic types, type rules for assignment must be met (i.e. scalar type B must be smaller than scalar type A)
    • For tensor types, broadcasting rules for assignment must be met (ie. tensor type B must be smaller than tensor type A)

Broadcasting is performed following these rules:

  1. If number of dimensions is not the same, dimensions with length of 1 are added to the left
  2. Dimensions with length of 1 are broadcast to the greater length
  3. Base type is inferred to a compatible type
    • For basic types, type inference is applied
    • For tensor types, assignment broadcasting is applied

Following example shows result of broadcasting on two compatible tensors in an assignment:

a = [0, 0, 0]  # type(a) = tensor<int, 3>

a = 1          # broadcasts to:
               # a = [1, 1, 1]

b = [          # type(b) = tensor<int, 3, 3>
    0, 0, 0;
    0, 0, 0;
    0, 0, 0
]

b = [1, 1, 1] # broadcasts to:
              # b = [
              #     1, 1, 1;
              #     1, 1, 1; <- added
              #     1, 1, 1  <- added
              # ]

Following example shows two incompatible tensors in assignment:

tensor<int, 3> a = 0 # a = [0, 0, 0]

a = [1, 3] # Error: Assignment not possible!
           # a has length of 3, and array with length of 2
           # cannot be broadcast

Index access broadcasting (IAB)

Currently, we have two type kinds:

  • Base types (scalars)
  • Tensors

We also support broadcasting, which enables writing generic code like:

c = a + b

Here, a could be a tensor, such as tensor<real, 2, 3>, while b could be a scalar of type real. Broadcasting ensures that b is expanded to match the dimensions of a before performing the element-wise + operation.

Similarly, if b is tensor<real, 2>, its dimensionality is expanded until it becomes compatible with a.

This concept is extended further with Index Access Broadcasting (IAB), which allows indexed access at an arbitrary length along each dimension of size 1.

In simple terms, bounded index access is enforced for dimensions greater than 1, while dimensions of size 1 allow unbounded index access.

Example

Consider a tensor a of type tensor<real, 1, 2>:

a = [3, 4]

As expected, direct indexing works:

a[0][1] == 4

However, following the IAB rules, unbounded indexing is allowed along dimensions of size 1:

a[999][1] == 4

Effectively, a is broadcasted along its first dimension to an arbitrary length.

Indexing can also be extended over additional dimensions indefinitely:

a[999][1][1000][2000] == 4
a[999][0][1000][2000] == 3

This means that a is broadcasted over an arbitrary number of dimensions.

Scalars and Index Access Broadcasting

Although TML distinguishes between scalar and tensor types, treating scalars as tensors can be beneficial in certain cases. This approach allows writing generic code that handles both scalar and tensor model variables consistently, even when element-wise operations are performed manually.

IAB extends to scalars, making each scalar indexable over an arbitrary number of dimensions. For example, if a is an int with a value of 42, the following statements hold:

a == 42
a[0] == 42
a[500] == 42
a[4][5] == 42

Conceptually, a is broadcasted across infinite dimensions.

Indexing Rules for Scalars

  • Scalars allow indexing over an arbitrary number of dimensions indefinitely.
  • Indexing a scalar is ignored and does not change the value.
  • Indexing applies to both lvalues and rvalues.

Strings and Indexing

Since strings are used to pass configuration data within the model scope, they follow these indexing rules:

  • The str type is treated as a tensor, meaning indexing a str returns a char.
  • A char is treated as a scalar, meaning infinite indexing is allowed.

Syntax and Examples

Examples of index access broadcasting are shown below:

a = 1

b = a[1]       # 1
c = a[1][2][3] # 1

a[1] = 2       # a = 2
a[1][2][3] = 3 # a = 3

# signs has values such as "-" and "+-+"
for i=0:signs.len:
    # if signs == "+", signs is a char and IAB is applied
    # if signs == "+-+", signs is a char and normal indexing is applied
    if signs[i] == "+":
        x[i] += y[i]
    end
end

Narrowcasting

When working with tensors, it is common to have tensors with some (or even all) dimensions equal to 1. One common type of this problem is having a single input terminal in input terminal group. Those dimensions are effectively useless and can make tensor usage harder in code. To overcome this problem, TML has support for a mechanism opposite to broadcasting that is called narrowcasting.

Narrowcasting is performed only on the top level type and is not performed recursively. Narrowcasting expressions can be nested arbitrarily. In contrast to broadcasting, narrowcasting never produces an error.

Narrowcasting is performed following these rules:

  1. If a scalar is passed, type of the result is unchanged (e.g. int stays int)
  2. If a tensor is passed, dimensions equal to 1 are removed (e.g. tensor<T, 1, 5, 1> becomes tensor<T, 5>)
  3. If a tensor is passed, and no dimensions are equal to 1, type of the result is unchanged (e.g. tensor<T, 5> stays tensor<T, 5>)
  4. If a tensor is passed, and all dimensions are equal to 1, type of the result is tensor base type (e.g. tensor<T, 1, 1, 1> becomes T)

Narrowcasting syntax and examples are shown below.

int a = 1
b = narrow(b) # int

tensor<int, 1> c = 1
d = narrow(c) # int

tensor<int, 1, 5, 1> e = 1
f = narrow(e) # tensor<int, 5>

tensor<int, 5, 5> g = 1
h = narrow(g) # tensor<int, 5, 5>

tensor<tensor<int, 1>, 5, 5> i = 1
j = narrow(i) # tensor<tensor<int, 1>, 5, 5>